Bug #23830: rados/standalone/erasure-code.yaml gets 160 byte pgmeta object - RADOS - Ceph

Actions

Copy link

Bug #23830

closed

rados/standalone/erasure-code.yaml gets 160 byte pgmeta object

Added by Sage Weil almost 6 years ago. Updated over 4 years ago.

Status:

Can't reproduce

Priority:

High

Assignee:

Category:

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(RADOS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

2018-04-23T18:48:41.008 INFO:tasks.workunit.client.0.smithi178.stderr:/build/ceph-13.0.2-1662-ge51e976/src/osd/PGLog.h: In function 'static void PGLog::read_log_and_missing(ObjectStore*, ObjectStore::CollectionHandle&, ghobject_t, const pg_info_t&, PGLog::IndexedLog&, missing_type&, std::ostringstream&, bool, bool*
, const DoutPrefixProvider*, std::set<std::__cxx11::basic_string<char> >*, bool) [with missing_type = pg_missing_set<true>; ObjectStore::CollectionHandle = boost::intrusive_ptr<ObjectStore::CollectionImpl>; std::ostringstream = std::__cxx11::basic_ostringstream<char>]' thread 7ff74492a200 time 2018-04-23 18:48:40.9
75924
2018-04-23T18:48:41.008 INFO:tasks.workunit.client.0.smithi178.stderr:/build/ceph-13.0.2-1662-ge51e976/src/osd/PGLog.h: 1294: FAILED assert(st.st_size == 0)
2018-04-23T18:48:41.009 INFO:tasks.workunit.client.0.smithi178.stderr: ceph version 13.0.2-1662-ge51e976 (e51e976fac3ee4ceac2ca977dd449af0d2ac86ec) mimic (dev)
2018-04-23T18:48:41.010 INFO:tasks.workunit.client.0.smithi178.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7ff73bc69f42]
2018-04-23T18:48:41.010 INFO:tasks.workunit.client.0.smithi178.stderr: 2: (()+0x2e3117) [0x7ff73bc6a117]
2018-04-23T18:48:41.010 INFO:tasks.workunit.client.0.smithi178.stderr: 3: (void PGLog::read_log_and_missing<pg_missing_set<true> >(ObjectStore*, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ghobject_t, pg_info_t const&, PGLog::IndexedLog&, pg_missing_set<true>&, std::__cxx11::basic_ostringstream<char, std::c
har_traits<char>, std::allocator<char> >&, bool, bool*, DoutPrefixProvider const*, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, st
d::char_traits<char>, std::allocator<char> > > >*, bool)+0x312a) [0x55f62809971a]

(gdb) p st
$1 = {st_dev = 2049, st_ino = 2228718, st_nlink = 1, st_mode = 33188, st_uid = 1000, st_gid = 1186, __pad0 = 0, st_rdev = 0, st_size = 160, st_blksize = 4096, st_blocks = 8, st_atim = {tv_sec = 1524509271, tv_nsec = 435625664}, st_mtim = {tv_sec = 1524509304, tv_nsec = 570959915}, st_ctim = {tv_sec = 1524509304, 
    tv_nsec = 570959915}, __glibc_reserved = {0, 0, 0}}

/a/sage-2018-04-23_15:07:57-rados-wip-sage3-testing-2018-04-23-0831-distro-basic-smithi/2431028

Actions

Copy link

Updated by Sage Weil almost 6 years ago

Subject changed from rados/standalone/erasure-code.yaml gets 160 by pgmeta object to rados/standalone/erasure-code.yaml gets 160 byte pgmeta object

Actions

Copy link

Updated by Josh Durgin almost 6 years ago

The pg meta object is supposed to be empty since many versions ago. IIRC sage suggested this may be from a race that ends up writing to the wrong fd somewhere.

Actions

Copy link

Updated by Neha Ojha almost 6 years ago

This looks like a similar failure: http://pulpito.ceph.com/nojha-2018-05-30_20:43:02-rados-wip-async-up2-2018-05-30-distro-basic-smithi/2610628/

2018-05-30T21:51:13.367 INFO:tasks.workunit.client.0.smithi033.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.0.0-77-g55b335b/rpm/el7/BUILD/ceph-14.0.0-77-g55b335b/src/osd/PGLog.h: In function 'static void PGLog::read_log_and_missing(ObjectStore*, ObjectStore::CollectionHandle&, ghobject_t, const pg_info_t&, PGLog::IndexedLog&, missing_type&, std::ostringstream&, bool, bool*, const DoutPrefixProvider*, std::set<std::basic_string<char> >*, bool) [with missing_type = pg_missing_set<true>; ObjectStore::CollectionHandle = boost::intrusive_ptr<ObjectStore::CollectionImpl>; std::ostringstream = std::basic_ostringstream<char>]' thread 7fd4390551c0 time 2018-05-30 21:51:13.378551
2018-05-30T21:51:13.381 INFO:tasks.workunit.client.0.smithi033.stderr:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/14.0.0-77-g55b335b/rpm/el7/BUILD/ceph-14.0.0-77-g55b335b/src/osd/PGLog.h: 1294: FAILED assert(st.st_size == 0)
2018-05-30T21:51:13.387 INFO:tasks.workunit.client.0.smithi033.stderr: ceph version 14.0.0-77-g55b335b (55b335bc299c3578e12d7c9ecd2d2f4aeb748b8d) nautilus (dev)
2018-05-30T21:51:13.387 INFO:tasks.workunit.client.0.smithi033.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0xff) [0x7fd43049181f]
2018-05-30T21:51:13.387 INFO:tasks.workunit.client.0.smithi033.stderr: 2: (()+0x287a07) [0x7fd430491a07]
2018-05-30T21:51:13.387 INFO:tasks.workunit.client.0.smithi033.stderr: 3: (void PGLog::read_log_and_missing<pg_missing_set<true> >(ObjectStore*, boost::intrusive_ptr<ObjectStore::CollectionImpl>&, ghobject_t, pg_info_t const&, PGLog::IndexedLog&, pg_missing_set<true>&, std::basic_ostringstream<char, std::char_traits<char>, std::allocator<char> >&, bool, bool*, DoutPrefixProvider const*, std::set<std::string, std::less<std::string>, std::allocator<std::string> >*, bool)+0x3080) [0x55fb6e1f0e50]

Actions

Copy link