Project

General

Profile

Actions

Bug #25106

closed

Ceph-osd coredumps on launch

Added by Michael Jones over 5 years ago. Updated over 5 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

See https://tracker.ceph.com/issues/24993

The problem:
ceph-volume lvm create --bluestore --data /dev/sda <- works
the resulting ceph-osd process that gets triggered by udev/systemd/lvm/whatever coredumps nearly immediately.

As a result:
ceph-disk works great.
ceph-volume is worse than useless, because it screws something up in the kernel that requires I go and press the powerbutton (I'm not joking, a soft reboot hangs) before I can do anything with the disk I tried to use for ceph.

OS: Gentoo, x86_64
Python: 3.6.5
Ceph version : 13.2.0
Using ceph-volume (ceph-disk works just fine)

fenrir ~ # uname -a
Linux fenrir 4.14.52-gentoo #2 SMP Thu Jul 19 02:41:47 CDT 2018 x86_64 AMD E-350D APU with Radeon(tm) HD Graphics AuthenticAMD GNU/Linux

fenrir ~ # emerge --info ceph
Portage 2.3.40 (python 2.7.14-final-0, default/linux/amd64/17.0, gcc-7.3.0, glibc-2.26-r7, 4.14.52-gentoo x86_64) =================================================================
System Settings =================================================================
System uname: Linux-4.14.52-gentoo-x86_64-AMD_E-350D_APU_with_Radeon-tm-_HD_Graphics-with-gentoo-2.4.1
KiB Mem: 16134312 total, 12821152 free
KiB Swap: 10485756 total, 10485756 free
Timestamp of repository gentoo: Tue, 24 Jul 2018 19:05:49 +0000
Head commit of repository gentoo: 78db885c0592cec1fcc7dbadbcf32967275821ce

Head commit of repository jonesmz-public-overlay: c83a2294f2ef3f6c1b4f8bc1086fc9eec1aec37d

Head commit of repository steam-overlay: 1237b523da636a247376b25cd4ec59c16d5b0104

sh bash 4.4_p12
ld GNU ld (Gentoo 2.30 p2) 2.30.0
distcc 3.2rc1 x86_64-pc-linux-gnu [disabled]
app-shells/bash: 4.4_p12::gentoo
dev-lang/perl: 5.24.3-r1::gentoo
dev-lang/python: 2.7.14-r1::gentoo, 3.6.5::gentoo
dev-util/cmake: 3.9.6::gentoo
dev-util/pkgconfig: 0.29.2::gentoo
sys-apps/baselayout: 2.4.1-r2::gentoo
sys-apps/sandbox: 2.13::gentoo
sys-devel/autoconf: 2.69-r4::gentoo
sys-devel/automake: 1.11.6-r3::gentoo, 1.15.1-r2::gentoo
sys-devel/binutils: 2.30-r2::gentoo
sys-devel/gcc: 7.3.0-r3::gentoo
sys-devel/gcc-config: 1.8-r1::gentoo
sys-devel/libtool: 2.4.6-r3::gentoo
sys-devel/make: 4.2.1::gentoo
sys-kernel/linux-headers: 4.13::gentoo (virtual/os-headers)
sys-libs/glibc: 2.26-r7::gentoo
Repositories:

gentoo
location: /usr/portage
sync-type: git
sync-uri: git://anongit.gentoo.org/repo/sync/gentoo.git
priority: -1000

jonesmz-public-overlay
location: /usr/portage-overlays/jonesmz-public-overlay
sync-type: git
sync-uri: https://github.com/jonesmz/gentoo-overlay.git
masters: gentoo

steam-overlay
location: /usr/portage-overlays/steam-overlay
sync-type: git
sync-uri: https://github.com/anyc/steam-overlay.git
masters: gentoo
priority: 50

Installed sets: @archive, @pc-base-system, @portage, @vcs
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=x86-64 -mtune=generic -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -O2 -pipe -march=x86-64 -mtune=generic -O2 -pipe"
DISTDIR="/usr/portage-distfiles"
EMERGE_DEFAULT_OPTS=" --jobs --keep-going --newuse --deep --backtrack=3000 --complete-graph --with-bdeps=y"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildpkg clean-logs compress-build-logs compressdebug config-protect-if-modified distlocks ebuild-locks fixlafiles installsources merge-sync multilib-strict news nostrip parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms split-elog split-log strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en en_US"
MAKEOPTS="-j3"
PKGDIR="/usr/portage-packages"
PORTAGE_COMPRESS="xz"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl amd64 avahi btrfs bzip2 clang crypt cxx dbus gd gudev hardened iconv ipv6 libtirpc lm_sensors multilib ncurses nls nptl openmp pam pcre pie python readline samba seccomp ssl ssp systemd threads udev udisks unicode v4l xattr xtpax zeroconf zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx sse sse2 mmxext" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="coreboot efi-64 emu qemu pc" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en-US" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-0" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" QEMU_SOFTMMU_TARGETS="arm aarch64 x86_64" QEMU_USER_TARGETS="arm aarch64 x86_64" RUBY_TARGETS="ruby23" USERLAND="GNU" VIDEO_CARDS="vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
Package Settings =================================================================

sys-cluster/ceph-13.2.0::gentoo was built with the following:
USE="cephfs fuse mgr radosgw ssl systemd tcmalloc -babeltrace -dpdk -jemalloc -ldap -lttng -mgr-frontend (-static-libs) (-system-boost) -test -xfs -zfs" ABI_X86="(64)" CPU_FLAGS_X86="sse sse2 -sse3 -sse4_1 -sse4_2 -ssse3" PYTHON_TARGETS="python2_7 python3_6 -python3_4 -python3_5"


Files

ceph.audit.log (9.12 KB) ceph.audit.log Michael Jones, 07/25/2018 05:59 PM
ceph.log (8.21 KB) ceph.log Michael Jones, 07/25/2018 05:59 PM
ceph-osd.7.log (635 KB) ceph-osd.7.log Michael Jones, 07/25/2018 05:59 PM
ceph-volume.log (30.6 KB) ceph-volume.log Michael Jones, 07/25/2018 05:59 PM
ceph.audit.log (4.44 KB) ceph.audit.log Michael Jones, 07/26/2018 03:58 PM
ceph.log (2.09 KB) ceph.log Michael Jones, 07/26/2018 03:58 PM
ceph-osd.7.log (361 KB) ceph-osd.7.log Michael Jones, 07/26/2018 03:58 PM
ceph-volume.log (19.1 KB) ceph-volume.log Michael Jones, 07/26/2018 03:58 PM

Related issues 1 (0 open1 closed)

Is duplicate of RADOS - Bug #23492: Abort in OSDMap::decode() during qa/standalone/erasure-code/test-erasure-eio.shResolvedDavid Zafman03/28/2018

Actions
Actions #1

Updated by Josh Durgin over 5 years ago

  • Status changed from New to Duplicate

this will be fixed in 13.2.1

Actions #2

Updated by Josh Durgin over 5 years ago

  • Is duplicate of Bug #23492: Abort in OSDMap::decode() during qa/standalone/erasure-code/test-erasure-eio.sh added

Updated by Michael Jones over 5 years ago

Either the patch here: https://github.com/ceph/ceph/pull/22954

Doesn't fix the bug, or this is not a duplicate issue.

I've rebuilt with the patch, and my OSDs still coredump immediately.

Actions

Also available in: Atom PDF