Ceph » cephsqlite

/a/yuriw-2023-03-28_22:43:59-rados-wip-yuri11-testing-2023-03-28-0950-distro-default-smithi/7223995

2023-03-29T04:48:18.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo
2023-03-29T04:48:18.361 INFO:teuthology.task.internal:Restoring /etc/sudoers...
2023-03-29T04:48:18.361 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers
2023-03-29T04:48:18.373 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers
2023-03-29T04:48:18.384 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump
2023-03-29T04:48:18.403 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump
2023-03-29T04:48:18.415 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump
2023-03-29T04:48:18.423 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core
2023-03-29T04:48:18.436 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core
2023-03-29T04:48:18.467 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump
2023-03-29T04:48:18.490 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi052.front.sepia.ceph.com, flagging run as failed
2023-03-29T04:48:18.490 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump
2023-03-29T04:48:18.511 DEBUG:teuthology.orchestra.run:got remote process result: 1
2023-03-29T04:48:18.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive
2023-03-29T04:48:18.524 INFO:teuthology.task.internal:Transferring archived files...
2023-03-29T04:48:18.524 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-03-28_22:43:59-rados-wip-yuri11-testing-2023-03-28-0950-distro-default-smithi/7223995/remote/smithi052
2023-03-29T04:48:18.525 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- .
2023-03-29T04:48:25.303 INFO:teuthology.task.internal:Transferring binaries for coredumps...
2023-03-29T04:48:25.513 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/yuriw-2023-03-28_22:43:59-rados-wip-yuri11-testing-2023-03-28-0950-distro-default-smithi/7223995/remote/smithi052/coredump/1680065090.39304.core.gz: gzip compressed data, was "1680065090.39304.core", last modified: Wed Mar 29 04:48:18 2023, max compression

2023-03-29T04:48:25.514 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ...
2023-03-29T04:48:26.160 INFO:teuthology.task.internal: core looks like: /tmp/tmpsr59t_lu: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_', real uid: 1000, effective uid: 1000, real gid: 1271, effective gid: 1271, execfn: '/usr/bin/sqlite3', platform: 'x86_64'

2023-03-29T04:48:26.190 INFO:teuthology.task.internal: dump_program: sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_
2023-03-29T04:48:26.191 DEBUG:teuthology.orchestra.run.smithi052:> which 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_'
2023-03-29T04:48:26.199 DEBUG:teuthology.orchestra.run:got remote process result: 1
2023-03-29T04:48:26.200 ERROR:teuthology.run_tasks:Manager failed: internal.archive

There is a core file located at /a/yuriw-2023-03-28_22:43:59-rados-wip-yuri11-testing-2023-03-28-0950-distro-default-smithi/7223995/remote/smithi052/coredump/1680065090.39304.core.gz.

The coredump was not generated by any daemon, and there was no executable. So, I checked the details with `file`. Looks like it has something to do with sqlite3. This is also reflected in the above teuthology log.

~$ file 1680065090.39304.core
1680065090.39304.core: ELF 64-bit LSB core file x86-64, version 1 (SYSV), SVR4-style, from 'sqlite3 -cmd .output /dev/null -cmd .load libcephsqlite.so -cmd pragma journal_', real uid: 1000, effective uid: 1000, real gid: 1271, effective gid: 1271, execfn: '/usr/bin/sqlite3', platform: 'x86_64'