Ceph

Run: http://pulpito.ceph.com/teuthology-2016-06-16_18:05:02-upgrade:hammer-x-infernalis-distro-basic-vps/
Job: 264591
Logs: http://qa-proxy.ceph.com/teuthology/teuthology-2016-06-16_18:05:02-upgrade:hammer-x-infernalis-distro-basic-vps/264591/teuthology.log

2016-06-16T19:52:20.358 INFO:tasks.workunit.client.1.vpm157.stdout: got ls 1466131931,1466131933,1466131934,1466131936,1466131937,1466131939,1466131940,0
2016-06-16T19:52:21.364 INFO:tasks.workunit.client.1.vpm157.stdout: got ls 1466131931,1466131933,1466131934,1466131936,1466131937,1466131939,1466131940,0
2016-06-16T19:52:22.381 INFO:tasks.ceph.osd.5.vpm105.stderr:osd/ReplicatedPG.cc: In function 'void ReplicatedPG::hit_set_trim(ReplicatedPG::RepGather*, unsigned int)' thread 7fcabd621700 time 2016-06-17 02:52:22.365989
2016-06-16T19:52:22.381 INFO:tasks.ceph.osd.5.vpm105.stderr:osd/ReplicatedPG.cc: 10514: FAILED assert(obc)
2016-06-16T19:52:22.389 INFO:tasks.ceph.osd.5.vpm105.stderr: ceph version 0.94.7-26-g2e156d7 (2e156d7ad4b9f4ffd6028df3a460b50b30c8b0d3)
2016-06-16T19:52:22.390 INFO:tasks.ceph.osd.5.vpm105.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x8b) [0xbb214b]
2016-06-16T19:52:22.390 INFO:tasks.ceph.osd.5.vpm105.stderr: 2: (ReplicatedPG::hit_set_trim(ReplicatedPG::RepGather*, unsigned int)+0x73f) [0x86c41f]
2016-06-16T19:52:22.390 INFO:tasks.ceph.osd.5.vpm105.stderr: 3: (ReplicatedPG::hit_set_persist()+0xee7) [0x86d427]
2016-06-16T19:52:22.391 INFO:tasks.ceph.osd.5.vpm105.stderr: 4: (ReplicatedPG::do_op(std::tr1::shared_ptr<OpRequest>&)+0xe2e) [0x88eabe]
2016-06-16T19:52:22.391 INFO:tasks.ceph.osd.5.vpm105.stderr: 5: (ReplicatedPG::do_request(std::tr1::shared_ptr<OpRequest>&, ThreadPool::TPHandle&)+0x66a) [0x82b67a]
2016-06-16T19:52:22.391 INFO:tasks.ceph.osd.5.vpm105.stderr: 6: (OSD::dequeue_op(boost::intrusive_ptr<PG>, std::tr1::shared_ptr<OpRequest>, ThreadPool::TPHandle&)+0x3d5) [0x697ea5]
2016-06-16T19:52:22.391 INFO:tasks.ceph.osd.5.vpm105.stderr: 7: (OSD::ShardedOpWQ::_process(unsigned int, ceph::heartbeat_handle_d*)+0x338) [0x6983f8]
2016-06-16T19:52:22.392 INFO:tasks.ceph.osd.5.vpm105.stderr: 8: (ShardedThreadPool::shardedthreadpool_worker(unsigned int)+0x85f) [0xba1c9f]
2016-06-16T19:52:22.392 INFO:tasks.ceph.osd.5.vpm105.stderr: 9: (ShardedThreadPool::WorkThreadSharded::entry()+0x10) [0xba3dc0]
2016-06-16T19:52:22.392 INFO:tasks.ceph.osd.5.vpm105.stderr: 10: (()+0x8184) [0x7fcada4c7184]
2016-06-16T19:52:22.392 INFO:tasks.ceph.osd.5.vpm105.stderr: 11: (clone()+0x6d) [0x7fcad8a3237d]
2016-06-16T19:52:22.393 INFO:tasks.ceph.osd.5.vpm105.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

on vps as well
http://qa-proxy.ceph.com/teuthology/teuthology-2016-06-21_18:05:01-upgrade:hammer-x-infernalis-distro-basic-vps/269104/teuthology.log

teuthology-suite --suite 'upgrade' --suite-branch infernalis --ceph infernalis --machine-type smithi --filter="hammer-x/point-to-point-x/{0-tz-eastern.yaml point-to-point.yaml ubuntu_14.04.yaml}" 

039240418060c9a49298dacc0478772334526dce introduced the fix for naming hitset obj using GMT to hammer. it was cherry-picked from 42f8c5daad16aa849a0b99871d50161673c0c370 in master.

$ git tag --contains 039240418060c9a49298dacc0478772334526dce^C
v0.94.7

0.94.7 has this fix. for the backward compatibility, a boolean "use_gmt_hitset" is added as a property of the pg pool in 0.94.7. pools created by cluster pre-0.94.7 do not have this property, and after they are upgraded, the property of these pools are "false", as we won't assume that user was using GMT for the localtime.

but the upgraded OSD can not survive if a cluster with legacy pools but has its peer OSD in different timezone. because the use_gmt_hitset is still false. i think this behaviour is expected. might want to disable this test for this case.

will be on it.

https://github.com/ceph/ceph-qa-suite/pull/1078