Bug #10911
mon: osd featuers aren't updated correctly
0%
Description
ubuntu@plana41:/var/log/ceph$ ceph osd dump -f json-pretty | grep feature 2015-02-18 09:05:45.559405 7fd9f8261700 0 -- :/1020897 >> 10.214.132.37:6790/0 pipe(0x7fd9f4065010 sd=7 :0 s=1 pgs=0 cs=0 l=1 c=0x7fd9f4026900).fault "features": 1125899906842623, "features": 70368744177663, "features": 1125899906842623, "features": 1125899906842623, ubuntu@plana41:/var/log/ceph$ ceph tell osd.\* version 2015-02-18 09:05:57.659672 7f2cec263700 0 -- :/1020935 >> 10.214.132.37:6790/0 pipe(0x7f2ce8065010 sd=7 :0 s=1 pgs=0 cs=0 l=1 c=0x7f2ce8026900).fault osd.0: { "version": "ceph version 0.92-1117-g90a9ae4 (90a9ae450634a3f2281b93fbfadbc62ad703a0ec)" } osd.1: { "version": "ceph version 0.92-1117-g90a9ae4 (90a9ae450634a3f2281b93fbfadbc62ad703a0ec)" } osd.2: { "version": "ceph version 0.92-1117-g90a9ae4 (90a9ae450634a3f2281b93fbfadbc62ad703a0ec)" } osd.3: { "version": "ceph version 0.92-1117-g90a9ae4 (90a9ae450634a3f2281b93fbfadbc62ad703a0ec)" }
running cluster is on plana41, plana58
reliably reproduced with the below:
interactive-on-error: true overrides: admin_socket: branch: hammer ceph: conf: mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on legacy crush tunables: false osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 20 log-whitelist: - slow request - scrub mismatch - ScrubResult - must scrub before tier agent can activate install: ceph: sha1: 90a9ae450634a3f2281b93fbfadbc62ad703a0ec workunit: sha1: 26dfad798d5193295cbe86902a6bbcf773b03974 roles: - - mon.a - mds.a - osd.0 - osd.1 - - mon.b - mon.c - osd.2 - osd.3 - - client.0 - client.1 - client.2 - client.3 - client.4 suite_branch: wip-10704 targets: ubuntu@plana41.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8wmSjYkK3No1H5yQZL9Yzl3M1nVtiLgScELyN3Duk71HaLRP8wr+Nq98Ylp8N3fK9oSdEtH5I92dEg9MhrEf+FAxyFDIffqpTSdokcJxMrlugALgz2/1OgONIpg15wHQXByJRE1gsecauIJmz+NcucGmfZubWex+/PsyDsDq6o26iXTvl7Bf/cEZ/ya5 k7YNXF5LFx/ekcMkeF1bWHqk4eyQgJLAYPCewBhAZ2EQxBSUQoUJQtokXhe873yiS+Wtzuoo/Z8CX4tzrn+PdK56czySdw9lBTQIAB5V3bX+4aTTTpv9QfvgHKy/5frd9k2Y/LBSn2NIq7jutToRx1qGl ubuntu@plana58.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQingRKteHV641XBd2nFr9XehgercdQI5pXSigO0VqKUckA8HyCNOy62K6W+l3eGGF0A54M3YJGULd+cWNH2YeA4uK/O2VDLYsBbLKL5/2tX0WPp0a2BTRwHBWz+mTNBKJfB4YFb/27SoZKLHp9bij1LQW6tjDGkRCUoJUJ1hkOcVpmYVMH7XZXT/t55a YHoOCl7iyXEjOpB30XhpIzsYLISumDssk/hbVR03L5kiw9YKqA9Rr16sTN+ooTowRql5ktZU0lqCzxyRKs0mVv/rq5yorx91ZU6XUXvR63LfX/1sLwv5u/G2ix5BPd4ZlP5TevfABdUYB2hkwQNv5qe7D ubuntu@plana78.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC8m+86JHGSyRkSWj9p/K6JUbRcPjB7TtLZ9OBudXAGZNgReiOJoCU5kkpwejl0uXXCOHe/DB/bH81JCQbqY3XCJjU5JZ1wBsL/owaErPSfbbaouNV2k1FQjiSXYtPzx+qwEOeOZtEBPQ4p04npai6NzPLX43OGx/UiAwpyEGfVxZedmci0VBtC7QdCQkP 3sNJqSxFYdoVGjU5jv6BarPqV8LM4v00f8TmD1GdP51bfLGSKii6UU1IKXXR78ifb+9QUX4p/Clkl6Qgz8CJ70Iu+mcBZclJaGoAyuoKBhXE2oi2W1cQVquPqloxbN+VbbjoOL5OHbGg2euxyohZhgJaF tasks: - chef: null - clock.check: null - install: branch: wip-ms-dump-on-send-giant - print: '**** done installing giant' - ceph: fs: xfs - print: '**** done ceph' - parallel: - workload - upgrade-sequence - print: '**** done parallel' - rados: clients: - client.1 objects: 50 op_weights: delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 - print: '**** done rados 4-final-workload' - workunit: clients: client.1: - rados/load-gen-mix.sh - print: '**** done rados/load-gen-mix.sh 4-final-workload' - mon_thrash: revive_delay: 20 thrash_delay: 1 - print: '**** done mon_thrash 4-final-workload' - workunit: clients: client.1: - rados/test.sh - print: '**** done rados/test.sh 4-final-workload' - workunit: clients: client.1: - cls/test_cls_rbd.sh - print: '**** done cls/test_cls_rbd.sh 4-final-workload' - workunit: clients: client.1: - rbd/import_export.sh env: RBD_CREATE_ARGS: --new-format - print: '**** done rbd/import_export.sh 4-final-workload' - rgw: - client.1 - print: '**** done rgw 4-final-workload' - swift: client.1: rgw_server: client.1 - print: '**** done swift 4-final-workload' upgrade-sequence: sequential: - install.upgrade: mon.a: null - print: '**** done install.upgrade mon.a to the version from teuthology-suite arg' - ceph.restart: daemons: - mon.a wait-for-healthy: true - sleep: duration: 60 - ceph.restart: daemons: - osd.0 - osd.1 wait-for-healthy: true - sleep: duration: 60 - ceph.restart: - mds.a - sleep: duration: 60 - print: '**** running mixed versions of osds and mons' - exec: mon.b: - ceph osd crush tunables firefly - print: '**** done ceph osd crush tunables firefly' - install.upgrade: mon.b: null - print: '**** done install.upgrade mon.b to the version from teuthology-suite arg' - ceph.restart: daemons: - mon.b - mon.c wait-for-healthy: true - sleep: duration: 60 - ceph.restart: daemons: - osd.2 - osd.3 wait-for-healthy: true - sleep: duration: 60 verbose: true workload: parallel: - rados: clients: - client.0 ec_pool: true objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 4000 - print: '**** done rados ec parallel' - workunit: branch: giant clients: client.1: - cls - print: '**** done cls 2-workload parallel' - workunit: branch: giant clients: client.2: - rados/load-gen-big.sh - print: '**** done rados/load-gen-big.sh 2-workload parallel' - sequential: - exec: client.0: - ceph osd pool create base 4 - ceph osd pool create cache 4 - ceph osd tier add base cache - ceph osd tier cache-mode cache writeback - ceph osd tier set-overlay base cache - ceph osd pool set cache hit_set_type bloom - ceph osd pool set cache hit_set_count 8 - ceph osd pool set cache hit_set_period 3600 - ceph osd pool set cache target_max_objects 250 - rados: clients: - client.0 objects: 500 op_weights: copy_from: 50 delete: 50 evict: 50 flush: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 try_flush: 50 write: 100 ops: 4000 pool_snaps: true pools: - base - print: '**** done test_cache-pool-snaps 2-workload parallel_run' - workunit: branch: giant clients: client.3: - rbd/test_librbd.sh - print: '**** done rbd/test_librbd.sh 2-workload parallel' - workunit: branch: giant clients: client.4: - rbd/test_librbd_python.sh - print: '**** done rbd/test_librbd_python.sh 2-workload parallel'
Related issues
Associated revisions
osd,mon: explicitly specify OSD features in MOSDBoot
We are using the connection features to populate the features field in the
OSDMap, but this is the intersection of mon and osd features, not the
osd features. Fix this by explicitly specifying the features in
MOSDBoot.
Fixes: #10911
Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
osd,mon: explicitly specify OSD features in MOSDBoot
We are using the connection features to populate the features field in the
OSDMap, but this is the intersection of mon and osd features, not the
osd features. Fix this by explicitly specifying the features in
MOSDBoot.
Fixes: #10911
Backport: giant, firefly
Signed-off-by: Sage Weil <sage@redhat.com>
(cherry picked from commit bee91548b07c6a28314dddc50fef8b6a2677e774)
Conflicts:
src/osd/OSD.cc
Minor difference in the MOSDBoot constructor
History
#1 Updated by Sage Weil about 9 years ago
- Status changed from 12 to 7
#2 Updated by Sage Weil about 9 years ago
- Status changed from 7 to Resolved
#3 Updated by David Zafman almost 9 years ago
- Status changed from Resolved to Pending Backport
- Regression set to No
This was never backported. I've create wip-10911-firefly for testing.
#4 Updated by Loïc Dachary almost 9 years ago
- Backport set to firefly
#5 Updated by Nathan Cutler over 8 years ago
- Status changed from Pending Backport to Resolved