Enivronment Variables Already Set ip 127.0.0.1 port 7202 NOTE: hostname resolves to loopback; remote hosts will not be able to connect. either adjust /etc/hosts, or edit this script to use your machine's real IP. creating /home/yuriw/ceph/src/testdir/test-7202/keyring /home/yuriw/ceph/src/monmaptool --create --clobber --add a 127.0.0.1:7202 --add b 127.0.0.1:7203 --add c 127.0.0.1:7204 --print /tmp/ceph_monmap.14893 /home/yuriw/ceph/src/monmaptool: monmap file /tmp/ceph_monmap.14893 /home/yuriw/ceph/src/monmaptool: generated fsid 9d5efff2-4d8d-4c0b-b135-4748dc9d9678 epoch 0 fsid 9d5efff2-4d8d-4c0b-b135-4748dc9d9678 last_changed 2016-06-07 20:15:54.015730 created 2016-06-07 20:15:54.015730 0: 127.0.0.1:7202/0 mon.a 1: 127.0.0.1:7203/0 mon.b 2: 127.0.0.1:7204/0 mon.c /home/yuriw/ceph/src/monmaptool: writing epoch 0 to /tmp/ceph_monmap.14893 (3 monitors) rm -rf /home/yuriw/ceph/src/testdir/test-7202/dev/mon.a mkdir -p /home/yuriw/ceph/src/testdir/test-7202/dev/mon.a /home/yuriw/ceph/src/ceph-mon --mkfs -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf -i a --monmap=/tmp/ceph_monmap.14893 --keyring=/home/yuriw/ceph/src/testdir/test-7202/keyring 2016-06-07 20:15:54.027729 2b0a2dbc8940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:54.027794 2b0a2dbc8940 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/src/ceph-mon: set fsid to 8fedb721-e6d4-42fc-a3ae-49153fde812f /home/yuriw/ceph/src/ceph-mon: created monfs at /home/yuriw/ceph/src/testdir/test-7202/dev/mon.a for mon.a rm -rf /home/yuriw/ceph/src/testdir/test-7202/dev/mon.b mkdir -p /home/yuriw/ceph/src/testdir/test-7202/dev/mon.b /home/yuriw/ceph/src/ceph-mon --mkfs -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf -i b --monmap=/tmp/ceph_monmap.14893 --keyring=/home/yuriw/ceph/src/testdir/test-7202/keyring 2016-06-07 20:15:54.832732 2b7b5d9aa940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:54.832822 2b7b5d9aa940 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/src/ceph-mon: set fsid to 8fedb721-e6d4-42fc-a3ae-49153fde812f /home/yuriw/ceph/src/ceph-mon: created monfs at /home/yuriw/ceph/src/testdir/test-7202/dev/mon.b for mon.b rm -rf /home/yuriw/ceph/src/testdir/test-7202/dev/mon.c mkdir -p /home/yuriw/ceph/src/testdir/test-7202/dev/mon.c /home/yuriw/ceph/src/ceph-mon --mkfs -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf -i c --monmap=/tmp/ceph_monmap.14893 --keyring=/home/yuriw/ceph/src/testdir/test-7202/keyring 2016-06-07 20:15:55.895653 2b0b5e483940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:55.895752 2b0b5e483940 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/src/ceph-mon: set fsid to 8fedb721-e6d4-42fc-a3ae-49153fde812f /home/yuriw/ceph/src/ceph-mon: created monfs at /home/yuriw/ceph/src/testdir/test-7202/dev/mon.c for mon.c /home/yuriw/ceph/src/ceph-mon -i a -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf 2016-06-07 20:15:56.692060 2abc23e9e940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:56.692153 2abc23e9e940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:56.694725 2abc23e9e940 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/src/ceph-mon -i b -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf 2016-06-07 20:15:57.073486 2b20cbf66940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:57.073556 2b20cbf66940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:57.076824 2b20cbf66940 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/src/ceph-mon -i c -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf 2016-06-07 20:15:57.873802 2b0868a5a940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:57.873884 2b0868a5a940 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:57.876515 2b0868a5a940 -1 WARNING: the following dangerous and experimental features are enabled: * ERROR: error accessing '/home/yuriw/ceph/src/testdir/test-7202/dev/osd0/*' add osd0 c8e248f1-e9df-405e-b941-418201377eef 2016-06-07 20:15:58.350756 2af4ae643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:15:58.350760 2af4ae643700 0 lockdep start 2016-06-07 20:15:58.356891 2af4ae643700 -1 WARNING: the following dangerous and experimental features are enabled: * 0 2016-06-07 20:16:18.668346 2af4ae643700 0 lockdep stop 2016-06-07 20:16:18.727256 2b744695a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:18.727263 2b744695a700 0 lockdep start 2016-06-07 20:16:18.737008 2b744695a700 -1 WARNING: the following dangerous and experimental features are enabled: * add item id 0 name 'osd.0' weight 1 at location {host=smithi049,root=default} to crush map 2016-06-07 20:16:20.393299 2b744695a700 0 lockdep stop 2016-06-07 20:16:20.412848 2b4b976f66c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:20.414629 2b4b976f66c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:20.416076 2b4b976f66c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:21.632474 2b4b976f66c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2016-06-07 20:16:28.699151 2b4b976f66c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2016-06-07 20:16:28.748228 2b4b976f66c0 -1 filestore(/home/yuriw/ceph/src/testdir/test-7202/dev/osd0) could not find #-1:7b3f43c4:::osd_superblock:0# in index: (2) No such file or directory 2016-06-07 20:16:43.033287 2b4b976f66c0 -1 created object store /home/yuriw/ceph/src/testdir/test-7202/dev/osd0 for osd.0 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f 2016-06-07 20:16:43.033351 2b4b976f66c0 -1 auth: error reading file: /home/yuriw/ceph/src/testdir/test-7202/dev/osd0/keyring: can't open /home/yuriw/ceph/src/testdir/test-7202/dev/osd0/keyring: (2) No such file or directory 2016-06-07 20:16:43.033542 2b4b976f66c0 -1 created new key in keyring /home/yuriw/ceph/src/testdir/test-7202/dev/osd0/keyring adding osd0 key to auth repository 2016-06-07 20:16:43.094712 2af674d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:43.094722 2af674d71700 0 lockdep start 2016-06-07 20:16:43.107094 2af674d71700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for osd.0 2016-06-07 20:16:43.440885 2af674d71700 0 lockdep stop start osd0 /home/yuriw/ceph/src/ceph-osd -i 0 -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf 2016-06-07 20:16:43.472484 2b17d68d46c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:43.472636 2b17d68d46c0 -1 WARNING: the following dangerous and experimental features are enabled: * starting osd.0 at :/0 osd_data /home/yuriw/ceph/src/testdir/test-7202/dev/osd0 /home/yuriw/ceph/src/testdir/test-7202/dev/osd0/journal ERROR: error accessing '/home/yuriw/ceph/src/testdir/test-7202/dev/osd1/*' add osd1 f2fddc8f-0f00-4b56-aa52-e248880c12f6 2016-06-07 20:16:43.539395 2af3db943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:43.539402 2af3db943700 0 lockdep start 2016-06-07 20:16:43.546734 2af3db943700 -1 WARNING: the following dangerous and experimental features are enabled: * 1 2016-06-07 20:16:44.267779 2af3db943700 0 lockdep stop 2016-06-07 20:16:44.340307 2b10ff3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:44.340313 2b10ff3e3700 0 lockdep start 2016-06-07 20:16:44.351315 2b10ff3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * add item id 1 name 'osd.1' weight 1 at location {host=smithi049,root=default} to crush map 2016-06-07 20:16:45.469422 2b10ff3e3700 0 lockdep stop 2016-06-07 20:16:45.495502 2b3f97c306c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:45.495663 2b3f97c306c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:45.497290 2b3f97c306c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:16:46.279371 2b3f97c306c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2016-06-07 20:17:05.179993 2b3f97c306c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2016-06-07 20:17:05.293254 2b3f97c306c0 -1 filestore(/home/yuriw/ceph/src/testdir/test-7202/dev/osd1) could not find #-1:7b3f43c4:::osd_superblock:0# in index: (2) No such file or directory 2016-06-07 20:17:09.195004 2b3f97c306c0 -1 created object store /home/yuriw/ceph/src/testdir/test-7202/dev/osd1 for osd.1 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f 2016-06-07 20:17:09.195046 2b3f97c306c0 -1 auth: error reading file: /home/yuriw/ceph/src/testdir/test-7202/dev/osd1/keyring: can't open /home/yuriw/ceph/src/testdir/test-7202/dev/osd1/keyring: (2) No such file or directory 2016-06-07 20:17:09.195538 2b3f97c306c0 -1 created new key in keyring /home/yuriw/ceph/src/testdir/test-7202/dev/osd1/keyring adding osd1 key to auth repository 2016-06-07 20:17:09.245267 2b2a7d355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:17:09.265906 2b2a7d355700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for osd.1 2016-06-07 20:17:10.531330 2b2a7d355700 0 lockdep stop start osd1 /home/yuriw/ceph/src/ceph-osd -i 1 -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf 2016-06-07 20:17:10.549183 2ae33e3506c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:17:10.549258 2ae33e3506c0 -1 WARNING: the following dangerous and experimental features are enabled: * starting osd.1 at :/0 osd_data /home/yuriw/ceph/src/testdir/test-7202/dev/osd1 /home/yuriw/ceph/src/testdir/test-7202/dev/osd1/journal ERROR: error accessing '/home/yuriw/ceph/src/testdir/test-7202/dev/osd2/*' add osd2 455c7a54-0424-4e63-af14-378cff9c0ff1 2016-06-07 20:17:10.605981 2b3f26f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:17:10.612289 2b3f26f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2 2016-06-07 20:17:12.050796 2b3f26f7a700 0 lockdep stop 2016-06-07 20:17:12.118353 2b2d460ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:17:12.127781 2b2d460ce700 -1 WARNING: the following dangerous and experimental features are enabled: * add item id 2 name 'osd.2' weight 1 at location {host=smithi049,root=default} to crush map 2016-06-07 20:17:13.072092 2b2d460ce700 0 lockdep stop 2016-06-07 20:17:13.100802 2b02e62276c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:17:13.100886 2b02e62276c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:17:13.102480 2b02e62276c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:17:14.924072 2b02e62276c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2016-06-07 20:19:06.750547 2b02e62276c0 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2016-06-07 20:19:06.786922 2b02e62276c0 -1 filestore(/home/yuriw/ceph/src/testdir/test-7202/dev/osd2) could not find #-1:7b3f43c4:::osd_superblock:0# in index: (2) No such file or directory 2016-06-07 20:19:36.490208 2b02e62276c0 -1 created object store /home/yuriw/ceph/src/testdir/test-7202/dev/osd2 for osd.2 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f 2016-06-07 20:19:36.490235 2b02e62276c0 -1 auth: error reading file: /home/yuriw/ceph/src/testdir/test-7202/dev/osd2/keyring: can't open /home/yuriw/ceph/src/testdir/test-7202/dev/osd2/keyring: (2) No such file or directory 2016-06-07 20:19:36.490311 2b02e62276c0 -1 created new key in keyring /home/yuriw/ceph/src/testdir/test-7202/dev/osd2/keyring adding osd2 key to auth repository 2016-06-07 20:19:36.547551 2b3d4cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:36.558072 2b3d4cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for osd.2 2016-06-07 20:19:37.140953 2b3d4cf79700 0 lockdep stop start osd2 /home/yuriw/ceph/src/ceph-osd -i 2 -c /home/yuriw/ceph/src/testdir/test-7202/ceph.conf 2016-06-07 20:19:37.159272 2b748ca496c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:37.159366 2b748ca496c0 -1 WARNING: the following dangerous and experimental features are enabled: * starting osd.2 at :/0 osd_data /home/yuriw/ceph/src/testdir/test-7202/dev/osd2 /home/yuriw/ceph/src/testdir/test-7202/dev/osd2/journal started. stop.sh to stop. see out/* (e.g. 'tail -f out/????') for debug output. export PYTHONPATH=./pybind:/home/yuriw/ceph/src/pybind:../src/pybind export LD_LIBRARY_PATH=/home/yuriw/ceph/src/.libs export CEPH_CONF=/home/yuriw/ceph/src/testdir/test-7202/ceph.conf export CEPH_KEYRING=/home/yuriw/ceph/src/testdir/test-7202/keyring ++ dirname /home/yuriw/ceph/qa/workunits/cephtool/test.sh + source /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh ++ TIMEOUT=300 ++ PG_NUM=4 ++ : /tmp ++ type xmlstarlet ++ XMLSTARLET=xmlstarlet ++ test --test-mon = TESTS + set -e + set -o functrace + PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:8: main: SUDO=sudo /home/yuriw/ceph/qa/workunits/cephtool/test.sh:35: main: TMPDIR=/tmp/cephtool19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:36: main: mkdir /tmp/cephtool19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:37: main: trap 'rm -fr /tmp/cephtool19083' 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:39: main: TMPFILE=/tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1879: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:37.278645 2ac99375d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:37.285870 2ac99375d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:37.347311 2ac99375d700 0 lockdep stop osd.1 down out weight 0 up_from 0 up_thru 0 down_at 0 last_clean_interval [0,0) :/0 :/0 :/0 :/0 exists,new f2fddc8f-0f00-4b56-aa52-e248880c12f6 osd.2 down out weight 0 up_from 0 up_thru 0 down_at 0 last_clean_interval [0,0) :/0 :/0 :/0 :/0 exists,new 455c7a54-0424-4e63-af14-378cff9c0ff1 waiting for osd(s) to come back up (1/300) *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:38.419261 2b7902b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:38.419265 2b7902b5d700 0 lockdep start 2016-06-07 20:19:38.425009 2b7902b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:38.488901 2b7902b5d700 0 lockdep stop osd.1 down out weight 0 up_from 0 up_thru 0 down_at 0 last_clean_interval [0,0) :/0 :/0 :/0 :/0 exists,new f2fddc8f-0f00-4b56-aa52-e248880c12f6 osd.2 down out weight 0 up_from 0 up_thru 0 down_at 0 last_clean_interval [0,0) :/0 :/0 :/0 :/0 exists,new 455c7a54-0424-4e63-af14-378cff9c0ff1 waiting for osd(s) to come back up (2/300) *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:39.589583 2af97e00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:39.589588 2af97e00d700 0 lockdep start 2016-06-07 20:19:39.595654 2af97e00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:42.678907 2af97e00d700 0 lockdep stop osd.2 down out weight 0 up_from 0 up_thru 0 down_at 0 last_clean_interval [0,0) :/0 :/0 :/0 :/0 exists,new 455c7a54-0424-4e63-af14-378cff9c0ff1 waiting for osd(s) to come back up (3/300) *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:43.749491 2b913cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:43.755167 2b913cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:48.389887 2b913cf79700 0 lockdep stop *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:48.463593 2ab5300a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:48.463598 2ab5300a7700 0 lockdep start 2016-06-07 20:19:48.471036 2ab5300a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:48.532810 2ab5300a7700 0 lockdep stop *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:48.604109 2ad80ae77700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:48.604113 2ad80ae77700 0 lockdep start 2016-06-07 20:19:48.607756 2ad80ae77700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:48.662354 2ad80ae77700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_injectargs /home/yuriw/ceph/qa/workunits/cephtool/test.sh:189: test_mon_injectargs: CEPH_ARGS='--mon_debug_dump_location the.dump' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:189: test_mon_injectargs: ceph tell osd.0 injectargs --no-osd_enable_op_tracker /home/yuriw/ceph/qa/workunits/cephtool/test.sh:190: test_mon_injectargs: check_response 'osd_enable_op_tracker = '\''false'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='osd_enable_op_tracker = '\''false'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'osd_enable_op_tracker = '\''false'\''' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:191: test_mon_injectargs: grep the.dump /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:192: test_mon_injectargs: ceph tell osd.0 injectargs '--osd_enable_op_tracker --osd_op_history_duration 500' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:193: test_mon_injectargs: check_response 'osd_enable_op_tracker = '\''true'\'' osd_op_history_duration = '\''500'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='osd_enable_op_tracker = '\''true'\'' osd_op_history_duration = '\''500'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'osd_enable_op_tracker = '\''true'\'' osd_op_history_duration = '\''500'\''' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:194: test_mon_injectargs: ceph tell osd.0 injectargs --no-osd_enable_op_tracker /home/yuriw/ceph/qa/workunits/cephtool/test.sh:195: test_mon_injectargs: check_response 'osd_enable_op_tracker = '\''false'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='osd_enable_op_tracker = '\''false'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'osd_enable_op_tracker = '\''false'\''' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:196: test_mon_injectargs: ceph tell osd.0 injectargs -- --osd_enable_op_tracker /home/yuriw/ceph/qa/workunits/cephtool/test.sh:197: test_mon_injectargs: check_response 'osd_enable_op_tracker = '\''true'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='osd_enable_op_tracker = '\''true'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'osd_enable_op_tracker = '\''true'\''' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:198: test_mon_injectargs: ceph tell osd.0 injectargs -- '--osd_enable_op_tracker --osd_op_history_duration 600' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:199: test_mon_injectargs: check_response 'osd_enable_op_tracker = '\''true'\'' osd_op_history_duration = '\''600'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='osd_enable_op_tracker = '\''true'\'' osd_op_history_duration = '\''600'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'osd_enable_op_tracker = '\''true'\'' osd_op_history_duration = '\''600'\''' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:200: test_mon_injectargs: expect_failure /tmp/cephtool19083 'Option --osd_op_history_duration requires an argument' ceph tell osd.0 injectargs -- --osd_op_history_duration /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1200: expect_failure: local dir=/tmp/cephtool19083 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1201: expect_failure: shift /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1202: expect_failure: local 'expected=Option --osd_op_history_duration requires an argument' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1203: expect_failure: shift /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1204: expect_failure: local success /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1206: expect_failure: ceph tell osd.0 injectargs -- --osd_op_history_duration /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1209: expect_failure: success=false /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1212: expect_failure: false /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1212: expect_failure: grep --quiet 'Option --osd_op_history_duration requires an argument' /tmp/cephtool19083/out /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1216: expect_failure: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:203: test_mon_injectargs: ceph tell osd.0 injectargs -- '--mon-lease 6' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:204: test_mon_injectargs: check_response 'mon_lease = '\''6'\'' (unchangeable)' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='mon_lease = '\''6'\'' (unchangeable)' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'mon_lease = '\''6'\'' (unchangeable)' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:207: test_mon_injectargs: expect_false ceph tell osd.0 injectargs --osd-scrub-auto-repair-num-errors -1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph tell osd.0 injectargs --osd-scrub-auto-repair-num-errors -1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:56.078874 2ad4d243e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:56.078879 2ad4d243e700 0 lockdep start 2016-06-07 20:19:56.082597 2ad4d243e700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: Parse error setting osd_scrub_auto_repair_num_errors to '-1' using injectargs. 2016-06-07 20:19:56.096952 2ad4d243e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:56.180609 2b03ea252700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:56.187274 2b03ea252700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:56.364193 2b03ea252700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_injectargs_SI //home/yuriw/ceph/qa/workunits/cephtool/test.sh:218: test_mon_injectargs_SI: get_config_value_or_die mon.a mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:111: get_config_value_or_die: local target config_opt raw val //home/yuriw/ceph/qa/workunits/cephtool/test.sh:113: get_config_value_or_die: target=mon.a //home/yuriw/ceph/qa/workunits/cephtool/test.sh:114: get_config_value_or_die: config_opt=mon_pg_warn_min_objects ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: ceph daemon mon.a config get mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: raw='{ "mon_pg_warn_min_objects": "10000" }' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:117: get_config_value_or_die: [[ 0 -ne 0 ]] ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: echo '{' '"mon_pg_warn_min_objects":' '"10000"' '}' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: sed -e 's/[{} "]//g' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: raw=mon_pg_warn_min_objects:10000 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: echo mon_pg_warn_min_objects:10000 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: cut -f2 -d: //home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: val=10000 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:125: get_config_value_or_die: echo 10000 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:126: get_config_value_or_die: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:218: test_mon_injectargs_SI: initial_value=10000 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:219: test_mon_injectargs_SI: ceph daemon mon.a config set mon_pg_warn_min_objects 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** { "success": "mon_pg_warn_min_objects = '10' (unchangeable) " } /home/yuriw/ceph/qa/workunits/cephtool/test.sh:220: test_mon_injectargs_SI: expect_config_value mon.a mon_pg_warn_min_objects 10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:131: expect_config_value: local target config_opt expected_val val /home/yuriw/ceph/qa/workunits/cephtool/test.sh:132: expect_config_value: target=mon.a /home/yuriw/ceph/qa/workunits/cephtool/test.sh:133: expect_config_value: config_opt=mon_pg_warn_min_objects /home/yuriw/ceph/qa/workunits/cephtool/test.sh:134: expect_config_value: expected_val=10 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: get_config_value_or_die mon.a mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:111: get_config_value_or_die: local target config_opt raw val //home/yuriw/ceph/qa/workunits/cephtool/test.sh:113: get_config_value_or_die: target=mon.a //home/yuriw/ceph/qa/workunits/cephtool/test.sh:114: get_config_value_or_die: config_opt=mon_pg_warn_min_objects ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: ceph daemon mon.a config get mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: raw='{ "mon_pg_warn_min_objects": "10" }' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:117: get_config_value_or_die: [[ 0 -ne 0 ]] ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: sed -e 's/[{} "]//g' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: echo '{' '"mon_pg_warn_min_objects":' '"10"' '}' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: raw=mon_pg_warn_min_objects:10 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: echo mon_pg_warn_min_objects:10 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: cut -f2 -d: //home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: val=10 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:125: get_config_value_or_die: echo 10 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:126: get_config_value_or_die: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: val=10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:138: expect_config_value: [[ 10 != \1\0 ]] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:221: test_mon_injectargs_SI: ceph daemon mon.a config set mon_pg_warn_min_objects 10K *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** { "success": "mon_pg_warn_min_objects = '10240' (unchangeable) " } /home/yuriw/ceph/qa/workunits/cephtool/test.sh:222: test_mon_injectargs_SI: expect_config_value mon.a mon_pg_warn_min_objects 10240 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:131: expect_config_value: local target config_opt expected_val val /home/yuriw/ceph/qa/workunits/cephtool/test.sh:132: expect_config_value: target=mon.a /home/yuriw/ceph/qa/workunits/cephtool/test.sh:133: expect_config_value: config_opt=mon_pg_warn_min_objects /home/yuriw/ceph/qa/workunits/cephtool/test.sh:134: expect_config_value: expected_val=10240 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: get_config_value_or_die mon.a mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:111: get_config_value_or_die: local target config_opt raw val //home/yuriw/ceph/qa/workunits/cephtool/test.sh:113: get_config_value_or_die: target=mon.a //home/yuriw/ceph/qa/workunits/cephtool/test.sh:114: get_config_value_or_die: config_opt=mon_pg_warn_min_objects ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: ceph daemon mon.a config get mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: raw='{ "mon_pg_warn_min_objects": "10240" }' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:117: get_config_value_or_die: [[ 0 -ne 0 ]] ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: echo '{' '"mon_pg_warn_min_objects":' '"10240"' '}' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: sed -e 's/[{} "]//g' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: raw=mon_pg_warn_min_objects:10240 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: echo mon_pg_warn_min_objects:10240 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: cut -f2 -d: //home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: val=10240 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:125: get_config_value_or_die: echo 10240 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:126: get_config_value_or_die: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: val=10240 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:138: expect_config_value: [[ 10240 != \1\0\2\4\0 ]] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:223: test_mon_injectargs_SI: ceph daemon mon.a config set mon_pg_warn_min_objects 1G *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** { "success": "mon_pg_warn_min_objects = '1073741824' (unchangeable) " } /home/yuriw/ceph/qa/workunits/cephtool/test.sh:224: test_mon_injectargs_SI: expect_config_value mon.a mon_pg_warn_min_objects 1073741824 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:131: expect_config_value: local target config_opt expected_val val /home/yuriw/ceph/qa/workunits/cephtool/test.sh:132: expect_config_value: target=mon.a /home/yuriw/ceph/qa/workunits/cephtool/test.sh:133: expect_config_value: config_opt=mon_pg_warn_min_objects /home/yuriw/ceph/qa/workunits/cephtool/test.sh:134: expect_config_value: expected_val=1073741824 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: get_config_value_or_die mon.a mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:111: get_config_value_or_die: local target config_opt raw val //home/yuriw/ceph/qa/workunits/cephtool/test.sh:113: get_config_value_or_die: target=mon.a //home/yuriw/ceph/qa/workunits/cephtool/test.sh:114: get_config_value_or_die: config_opt=mon_pg_warn_min_objects ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: ceph daemon mon.a config get mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: raw='{ "mon_pg_warn_min_objects": "1073741824" }' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:117: get_config_value_or_die: [[ 0 -ne 0 ]] ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: echo '{' '"mon_pg_warn_min_objects":' '"1073741824"' '}' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: sed -e 's/[{} "]//g' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: raw=mon_pg_warn_min_objects:1073741824 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: echo mon_pg_warn_min_objects:1073741824 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: cut -f2 -d: //home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: val=1073741824 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:125: get_config_value_or_die: echo 1073741824 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:126: get_config_value_or_die: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: val=1073741824 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:138: expect_config_value: [[ 1073741824 != \1\0\7\3\7\4\1\8\2\4 ]] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:225: test_mon_injectargs_SI: ceph daemon mon.a config set mon_pg_warn_min_objects 10F *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** /home/yuriw/ceph/qa/workunits/cephtool/test.sh:226: test_mon_injectargs_SI: check_response ''\''10F'\'': (22) Invalid argument' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string=''\''10F'\'': (22) Invalid argument' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- ''\''10F'\'': (22) Invalid argument' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:228: test_mon_injectargs_SI: ceph tell mon.a injectargs '--mon_pg_warn_min_objects 10' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:57.283884 2b2bbbe91700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:57.283889 2b2bbbe91700 0 lockdep start 2016-06-07 20:19:57.287610 2b2bbbe91700 -1 WARNING: the following dangerous and experimental features are enabled: * injectargs:mon_pg_warn_min_objects = '10' (unchangeable) 2016-06-07 20:19:59.008047 2b2bbbe91700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:229: test_mon_injectargs_SI: expect_config_value mon.a mon_pg_warn_min_objects 10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:131: expect_config_value: local target config_opt expected_val val /home/yuriw/ceph/qa/workunits/cephtool/test.sh:132: expect_config_value: target=mon.a /home/yuriw/ceph/qa/workunits/cephtool/test.sh:133: expect_config_value: config_opt=mon_pg_warn_min_objects /home/yuriw/ceph/qa/workunits/cephtool/test.sh:134: expect_config_value: expected_val=10 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: get_config_value_or_die mon.a mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:111: get_config_value_or_die: local target config_opt raw val //home/yuriw/ceph/qa/workunits/cephtool/test.sh:113: get_config_value_or_die: target=mon.a //home/yuriw/ceph/qa/workunits/cephtool/test.sh:114: get_config_value_or_die: config_opt=mon_pg_warn_min_objects ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: ceph daemon mon.a config get mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: raw='{ "mon_pg_warn_min_objects": "10" }' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:117: get_config_value_or_die: [[ 0 -ne 0 ]] ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: echo '{' '"mon_pg_warn_min_objects":' '"10"' '}' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: sed -e 's/[{} "]//g' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: raw=mon_pg_warn_min_objects:10 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: echo mon_pg_warn_min_objects:10 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: cut -f2 -d: //home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: val=10 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:125: get_config_value_or_die: echo 10 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:126: get_config_value_or_die: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: val=10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:138: expect_config_value: [[ 10 != \1\0 ]] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:230: test_mon_injectargs_SI: ceph tell mon.a injectargs '--mon_pg_warn_min_objects 10K' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:19:59.201363 2ba643e1d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:19:59.201368 2ba643e1d700 0 lockdep start 2016-06-07 20:19:59.211504 2ba643e1d700 -1 WARNING: the following dangerous and experimental features are enabled: * injectargs:mon_pg_warn_min_objects = '10240' (unchangeable) 2016-06-07 20:20:00.374937 2ba643e1d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:231: test_mon_injectargs_SI: expect_config_value mon.a mon_pg_warn_min_objects 10240 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:131: expect_config_value: local target config_opt expected_val val /home/yuriw/ceph/qa/workunits/cephtool/test.sh:132: expect_config_value: target=mon.a /home/yuriw/ceph/qa/workunits/cephtool/test.sh:133: expect_config_value: config_opt=mon_pg_warn_min_objects /home/yuriw/ceph/qa/workunits/cephtool/test.sh:134: expect_config_value: expected_val=10240 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: get_config_value_or_die mon.a mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:111: get_config_value_or_die: local target config_opt raw val //home/yuriw/ceph/qa/workunits/cephtool/test.sh:113: get_config_value_or_die: target=mon.a //home/yuriw/ceph/qa/workunits/cephtool/test.sh:114: get_config_value_or_die: config_opt=mon_pg_warn_min_objects ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: ceph daemon mon.a config get mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: raw='{ "mon_pg_warn_min_objects": "10240" }' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:117: get_config_value_or_die: [[ 0 -ne 0 ]] ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: sed -e 's/[{} "]//g' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: echo '{' '"mon_pg_warn_min_objects":' '"10240"' '}' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: raw=mon_pg_warn_min_objects:10240 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: echo mon_pg_warn_min_objects:10240 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: cut -f2 -d: //home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: val=10240 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:125: get_config_value_or_die: echo 10240 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:126: get_config_value_or_die: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: val=10240 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:138: expect_config_value: [[ 10240 != \1\0\2\4\0 ]] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:232: test_mon_injectargs_SI: ceph tell mon.a injectargs '--mon_pg_warn_min_objects 1G' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:00.556172 2b14ee4ff700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:00.556184 2b14ee4ff700 0 lockdep start 2016-06-07 20:20:00.568213 2b14ee4ff700 -1 WARNING: the following dangerous and experimental features are enabled: * injectargs:mon_pg_warn_min_objects = '1073741824' (unchangeable) 2016-06-07 20:20:01.789143 2b14ee4ff700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:233: test_mon_injectargs_SI: expect_config_value mon.a mon_pg_warn_min_objects 1073741824 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:131: expect_config_value: local target config_opt expected_val val /home/yuriw/ceph/qa/workunits/cephtool/test.sh:132: expect_config_value: target=mon.a /home/yuriw/ceph/qa/workunits/cephtool/test.sh:133: expect_config_value: config_opt=mon_pg_warn_min_objects /home/yuriw/ceph/qa/workunits/cephtool/test.sh:134: expect_config_value: expected_val=1073741824 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: get_config_value_or_die mon.a mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:111: get_config_value_or_die: local target config_opt raw val //home/yuriw/ceph/qa/workunits/cephtool/test.sh:113: get_config_value_or_die: target=mon.a //home/yuriw/ceph/qa/workunits/cephtool/test.sh:114: get_config_value_or_die: config_opt=mon_pg_warn_min_objects ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: ceph daemon mon.a config get mon_pg_warn_min_objects //home/yuriw/ceph/qa/workunits/cephtool/test.sh:116: get_config_value_or_die: raw='{ "mon_pg_warn_min_objects": "1073741824" }' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:117: get_config_value_or_die: [[ 0 -ne 0 ]] ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: sed -e 's/[{} "]//g' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: echo '{' '"mon_pg_warn_min_objects":' '"1073741824"' '}' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:122: get_config_value_or_die: raw=mon_pg_warn_min_objects:1073741824 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: echo mon_pg_warn_min_objects:1073741824 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: cut -f2 -d: //home/yuriw/ceph/qa/workunits/cephtool/test.sh:123: get_config_value_or_die: val=1073741824 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:125: get_config_value_or_die: echo 1073741824 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:126: get_config_value_or_die: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:136: expect_config_value: val=1073741824 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:138: expect_config_value: [[ 1073741824 != \1\0\7\3\7\4\1\8\2\4 ]] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:234: test_mon_injectargs_SI: expect_false ceph tell mon.a injectargs '--mon_pg_warn_min_objects 10F' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph tell mon.a injectargs '--mon_pg_warn_min_objects 10F' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:01.990949 2b14f7b46700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:01.990954 2b14f7b46700 0 lockdep start 2016-06-07 20:20:01.996850 2b14f7b46700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: injectargs:Parse error setting mon_pg_warn_min_objects to '10F' using injectargs. 2016-06-07 20:20:03.467910 2b14f7b46700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:235: test_mon_injectargs_SI: expect_false ceph tell mon.a injectargs '--mon_globalid_prealloc -1' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph tell mon.a injectargs '--mon_globalid_prealloc -1' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:03.556097 2b5beee22700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:03.556101 2b5beee22700 0 lockdep start 2016-06-07 20:20:03.562224 2b5beee22700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: injectargs:Parse error setting mon_globalid_prealloc to '-1' using injectargs. 2016-06-07 20:20:04.186053 2b5beee22700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:236: test_mon_injectargs_SI: ceph daemon mon.a config set mon_pg_warn_min_objects 10000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** { "success": "mon_pg_warn_min_objects = '10000' (unchangeable) " } /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:04.375215 2b6b8b71d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:04.375221 2b6b8b71d700 0 lockdep start 2016-06-07 20:20:04.382040 2b6b8b71d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:04.750075 2b6b8b71d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_tiering /home/yuriw/ceph/qa/workunits/cephtool/test.sh:285: test_tiering: ceph osd pool create slow 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:04.840852 2ae26ec5e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:04.840868 2ae26ec5e700 0 lockdep start 2016-06-07 20:20:04.860497 2ae26ec5e700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'slow' created 2016-06-07 20:20:06.751745 2ae26ec5e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:286: test_tiering: ceph osd pool create slow2 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:06.842727 2b3046d61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:06.842748 2b3046d61700 0 lockdep start 2016-06-07 20:20:06.849420 2b3046d61700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'slow2' created 2016-06-07 20:20:09.144387 2b3046d61700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:287: test_tiering: ceph osd pool create cache 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:09.239199 2b78cb278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:09.250957 2b78cb278700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache' created 2016-06-07 20:20:10.687514 2b78cb278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:288: test_tiering: ceph osd pool create cache2 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:10.775311 2b2ca0b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:10.775322 2b2ca0b6c700 0 lockdep start 2016-06-07 20:20:10.786428 2b2ca0b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache2' created 2016-06-07 20:20:14.124200 2b2ca0b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:289: test_tiering: ceph osd tier add slow cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:14.210583 2aca28f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:14.210588 2aca28f79700 0 lockdep start 2016-06-07 20:20:14.216555 2aca28f79700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache' is now (or already was) a tier of 'slow' 2016-06-07 20:20:15.043643 2aca28f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:290: test_tiering: ceph osd tier add slow cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:15.126432 2b2f22f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:15.126438 2b2f22f7a700 0 lockdep start 2016-06-07 20:20:15.137651 2b2f22f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache2' is now (or already was) a tier of 'slow' 2016-06-07 20:20:17.925452 2b2f22f7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:291: test_tiering: expect_false ceph osd tier add slow2 cache /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier add slow2 cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:18.005620 2af26b4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:18.005627 2af26b4a4700 0 lockdep start 2016-06-07 20:20:18.015179 2af26b4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: tier pool 'cache' is already a tier of 'slow' 2016-06-07 20:20:18.080989 2af26b4a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:293: test_tiering: ceph osd tier cache-mode cache writeback *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:18.163624 2ae47bc07700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:18.163632 2ae47bc07700 0 lockdep start 2016-06-07 20:20:18.174350 2ae47bc07700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to writeback 2016-06-07 20:20:19.542787 2ae47bc07700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:294: test_tiering: expect_false ceph osd tier cache-mode cache forward /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier cache-mode cache forward *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:19.634808 2b87642b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:19.646448 2b87642b4700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EPERM: 'forward' is not a well-supported cache mode and may corrupt your data. pass --yes-i-really-mean-it to force. 2016-06-07 20:20:19.820533 2b87642b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:295: test_tiering: ceph osd tier cache-mode cache forward --yes-i-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:19.910702 2acacfb46700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:19.910709 2acacfb46700 0 lockdep start 2016-06-07 20:20:19.922095 2acacfb46700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to forward 2016-06-07 20:20:21.853877 2acacfb46700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:296: test_tiering: expect_false ceph osd tier cache-mode cache readonly /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier cache-mode cache readonly *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:21.944089 2ba8eac5e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:21.944095 2ba8eac5e700 0 lockdep start 2016-06-07 20:20:21.953415 2ba8eac5e700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EPERM: 'readonly' is not a well-supported cache mode and may corrupt your data. pass --yes-i-really-mean-it to force. 2016-06-07 20:20:22.009575 2ba8eac5e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:297: test_tiering: ceph osd tier cache-mode cache readonly --yes-i-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:22.098329 2b15fe4ff700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:22.098335 2b15fe4ff700 0 lockdep start 2016-06-07 20:20:22.109911 2b15fe4ff700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to readonly 2016-06-07 20:20:23.838515 2b15fe4ff700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:298: test_tiering: expect_false ceph osd tier cache-mode cache forward /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier cache-mode cache forward *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:23.938407 2af2fec1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:23.938418 2af2fec1e700 0 lockdep start 2016-06-07 20:20:23.949484 2af2fec1e700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EPERM: 'forward' is not a well-supported cache mode and may corrupt your data. pass --yes-i-really-mean-it to force. 2016-06-07 20:20:24.834974 2af2fec1e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:299: test_tiering: ceph osd tier cache-mode cache forward --yes-i-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:24.921228 2b391cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:24.921233 2b391cd71700 0 lockdep start 2016-06-07 20:20:24.929082 2b391cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to forward 2016-06-07 20:20:26.587779 2b391cd71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:300: test_tiering: ceph osd tier cache-mode cache none *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:26.674385 2af30fb5f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:26.674391 2af30fb5f700 0 lockdep start 2016-06-07 20:20:26.684296 2af30fb5f700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to none 2016-06-07 20:20:28.181831 2af30fb5f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:301: test_tiering: ceph osd tier cache-mode cache writeback *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:28.278373 2b56eab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:28.298966 2b56eab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to writeback 2016-06-07 20:20:29.831454 2b56eab5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:302: test_tiering: ceph osd tier cache-mode cache proxy *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:29.932692 2b0bd1355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:29.932698 2b0bd1355700 0 lockdep start 2016-06-07 20:20:29.943306 2b0bd1355700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to proxy 2016-06-07 20:20:30.424214 2b0bd1355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:303: test_tiering: ceph osd tier cache-mode cache writeback *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:30.520179 2ae100b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:30.520185 2ae100b6c700 0 lockdep start 2016-06-07 20:20:30.528846 2ae100b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to writeback 2016-06-07 20:20:31.817537 2ae100b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:304: test_tiering: expect_false ceph osd tier cache-mode cache none /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier cache-mode cache none *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:31.898345 2b60fc3b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:31.909613 2b60fc3b4700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: unable to set cache-mode 'none' on a 'writeback' pool; only 'forward','proxy','readforward','readproxy' allowed. 2016-06-07 20:20:33.097498 2b60fc3b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:305: test_tiering: expect_false ceph osd tier cache-mode cache readonly --yes-i-really-mean-it /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier cache-mode cache readonly --yes-i-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:33.182671 2b1609a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:33.182677 2b1609a1e700 0 lockdep start 2016-06-07 20:20:33.193476 2b1609a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: unable to set cache-mode 'readonly' on a 'writeback' pool; only 'forward','proxy','readforward','readproxy' allowed. 2016-06-07 20:20:33.262124 2b1609a1e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:308: test_tiering: rados -p cache put /etc/passwd /etc/passwd 2016-06-07 20:20:33.307636 2b8f63c429c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:33.307708 2b8f63c429c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:33.309212 2b8f63c429c0 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:309: test_tiering: ceph tell 'osd.*' flush_pg_stats *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:33.741371 2b77fb278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:33.741377 2b77fb278700 0 lockdep start 2016-06-07 20:20:33.746155 2b77fb278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:35.115657 2b77fb278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:311: test_tiering: ceph osd tier cache-mode cache proxy *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:35.203194 2add54b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:35.203199 2add54b6c700 0 lockdep start 2016-06-07 20:20:35.209166 2add54b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to proxy 2016-06-07 20:20:36.679487 2add54b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:312: test_tiering: expect_false ceph osd tier cache-mode cache none /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier cache-mode cache none *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:36.756751 2b4517943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:36.756757 2b4517943700 0 lockdep start 2016-06-07 20:20:36.762784 2b4517943700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EBUSY: unable to set cache-mode 'none' on pool 'cache': dirty objects found 2016-06-07 20:20:36.821516 2b4517943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:313: test_tiering: expect_false ceph osd tier cache-mode cache readonly --yes-i-really-mean-it /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier cache-mode cache readonly --yes-i-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:36.909767 2b5dafc5a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:36.909774 2b5dafc5a700 0 lockdep start 2016-06-07 20:20:36.919394 2b5dafc5a700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EBUSY: unable to set cache-mode 'readonly' on pool 'cache': dirty objects found 2016-06-07 20:20:36.989115 2b5dafc5a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:314: test_tiering: ceph osd tier cache-mode cache writeback *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:37.079521 2b2626e62700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:37.079526 2b2626e62700 0 lockdep start 2016-06-07 20:20:37.085163 2b2626e62700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to writeback 2016-06-07 20:20:38.713871 2b2626e62700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:316: test_tiering: rados -p cache rm /etc/passwd 2016-06-07 20:20:38.755103 2b5611d089c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:38.755198 2b5611d089c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:38.757460 2b5611d089c0 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:317: test_tiering: rados -p cache cache-flush-evict-all 2016-06-07 20:20:39.132039 2ad2cf3039c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:39.132119 2ad2cf3039c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:39.133834 2ad2cf3039c0 -1 WARNING: the following dangerous and experimental features are enabled: * /etc/passwd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:318: test_tiering: ceph tell 'osd.*' flush_pg_stats *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:39.474070 2adf6781e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:39.474079 2adf6781e700 0 lockdep start 2016-06-07 20:20:39.485323 2adf6781e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:40.994663 2adf6781e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:320: test_tiering: ceph osd tier cache-mode cache proxy *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:41.087900 2ac47a47e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:41.087905 2ac47a47e700 0 lockdep start 2016-06-07 20:20:41.094164 2ac47a47e700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to proxy 2016-06-07 20:20:41.773490 2ac47a47e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:321: test_tiering: ceph osd tier cache-mode cache none *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:41.865671 2b07a4233700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:41.871723 2b07a4233700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to none 2016-06-07 20:20:43.761697 2b07a4233700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:322: test_tiering: ceph osd tier cache-mode cache readonly --yes-i-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:43.837989 2b159f03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:43.837995 2b159f03b700 0 lockdep start 2016-06-07 20:20:43.848034 2b159f03b700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to readonly 2016-06-07 20:20:44.409890 2b159f03b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:323: test_tiering: TRIES=0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:324: test_tiering: ceph osd pool set cache pg_num 3 --yes-i-really-mean-it /home/yuriw/ceph/qa/workunits/cephtool/test.sh:331: test_tiering: expect_false ceph osd pool set cache pg_num 4 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set cache pg_num 4 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:45.231596 2b1424f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:45.231601 2b1424f79700 0 lockdep start 2016-06-07 20:20:45.236565 2b1424f79700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EPERM: splits in cache pools must be followed by scrubs and leave sufficient free space to avoid overfilling. use --yes-i-really-mean-it to force. 2016-06-07 20:20:45.410724 2b1424f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:332: test_tiering: ceph osd tier cache-mode cache none *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:45.483481 2b80632b8700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:45.494431 2b80632b8700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache' to none 2016-06-07 20:20:47.333505 2b80632b8700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:333: test_tiering: ceph osd tier set-overlay slow cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:47.427035 2b6a32c5e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:47.427042 2b6a32c5e700 0 lockdep start 2016-06-07 20:20:47.438541 2b6a32c5e700 -1 WARNING: the following dangerous and experimental features are enabled: * overlay for 'slow' is now (or already was) 'cache' (WARNING: overlay pool cache_mode is still NONE) 2016-06-07 20:20:48.745143 2b6a32c5e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:334: test_tiering: expect_false ceph osd tier set-overlay slow cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier set-overlay slow cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:48.821729 2aecd6859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:48.821735 2aecd6859700 0 lockdep start 2016-06-07 20:20:48.827587 2aecd6859700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: pool 'slow' has overlay 'cache'; please remove-overlay first 2016-06-07 20:20:49.347845 2aecd6859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:335: test_tiering: expect_false ceph osd tier remove slow cache /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier remove slow cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:49.434998 2aac46d61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:49.448175 2aac46d61700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EBUSY: tier pool 'cache' is the overlay for 'slow'; please remove-overlay first 2016-06-07 20:20:49.878225 2aac46d61700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:336: test_tiering: ceph osd tier remove-overlay slow *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:49.971056 2b7e54434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:49.982858 2b7e54434700 -1 WARNING: the following dangerous and experimental features are enabled: * there is now (or already was) no overlay for 'slow' 2016-06-07 20:20:51.758923 2b7e54434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:337: test_tiering: ceph osd tier set-overlay slow cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:51.843339 2b23dbe84700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:51.843345 2b23dbe84700 0 lockdep start 2016-06-07 20:20:51.854914 2b23dbe84700 -1 WARNING: the following dangerous and experimental features are enabled: * overlay for 'slow' is now (or already was) 'cache2' (WARNING: overlay pool cache_mode is still NONE) 2016-06-07 20:20:53.195621 2b23dbe84700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:338: test_tiering: ceph osd tier remove-overlay slow *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:53.270228 2b496f7de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:53.270233 2b496f7de700 0 lockdep start 2016-06-07 20:20:53.276134 2b496f7de700 -1 WARNING: the following dangerous and experimental features are enabled: * there is now (or already was) no overlay for 'slow' 2016-06-07 20:20:54.319139 2b496f7de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:339: test_tiering: ceph osd tier remove slow cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:54.402095 2b42bee62700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:54.402100 2b42bee62700 0 lockdep start 2016-06-07 20:20:54.408227 2b42bee62700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache' is now (or already was) not a tier of 'slow' 2016-06-07 20:20:56.373784 2b42bee62700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:340: test_tiering: ceph osd tier add slow2 cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:56.477261 2b97775b2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:56.477269 2b97775b2700 0 lockdep start 2016-06-07 20:20:56.489809 2b97775b2700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache' is now (or already was) a tier of 'slow2' 2016-06-07 20:20:57.261433 2b97775b2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:341: test_tiering: expect_false ceph osd tier set-overlay slow cache /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier set-overlay slow cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:57.344798 2abc700a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:57.344803 2abc700a7700 0 lockdep start 2016-06-07 20:20:57.356234 2abc700a7700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: tier pool 'cache' is not a tier of 'slow' 2016-06-07 20:20:57.415106 2abc700a7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:342: test_tiering: ceph osd tier set-overlay slow2 cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:57.492796 2ae707278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:57.503787 2ae707278700 -1 WARNING: the following dangerous and experimental features are enabled: * overlay for 'slow2' is now (or already was) 'cache' (WARNING: overlay pool cache_mode is still NONE) 2016-06-07 20:20:58.788827 2ae707278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:343: test_tiering: ceph osd tier remove-overlay slow2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:58.867623 2aecbb03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:58.867628 2aecbb03b700 0 lockdep start 2016-06-07 20:20:58.875015 2aecbb03b700 -1 WARNING: the following dangerous and experimental features are enabled: * there is now (or already was) no overlay for 'slow2' 2016-06-07 20:20:59.595502 2aecbb03b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:344: test_tiering: ceph osd tier remove slow2 cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:20:59.691409 2b4a780a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:20:59.691415 2b4a780a7700 0 lockdep start 2016-06-07 20:20:59.697637 2b4a780a7700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache' is now (or already was) not a tier of 'slow2' 2016-06-07 20:21:02.180030 2b4a780a7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:345: test_tiering: ceph osd tier remove slow cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:02.265774 2b650771d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:02.265780 2b650771d700 0 lockdep start 2016-06-07 20:21:02.276400 2b650771d700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache2' is now (or already was) not a tier of 'slow' 2016-06-07 20:21:03.270959 2b650771d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:348: test_tiering: rados -p cache2 put /etc/passwd /etc/passwd 2016-06-07 20:21:03.307342 2ac2a1c549c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:03.307401 2ac2a1c549c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:03.308827 2ac2a1c549c0 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:03.619253 2ba8dd815700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:03.619264 2ba8dd815700 0 lockdep start 2016-06-07 20:21:03.627216 2ba8dd815700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:03.845439 2ba8dd815700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:05.940802 2b69d643e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:05.951285 2b69d643e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:06.019038 2b69d643e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:08.101600 2b6df6212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:08.168247 2b6df6212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:08.232206 2b6df6212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:10.321344 2b481dbf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:10.321355 2b481dbf9700 0 lockdep start 2016-06-07 20:21:10.329966 2b481dbf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:11.712708 2b481dbf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:13.796637 2ae619bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:13.804937 2ae619bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:13.860262 2ae619bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:15.950845 2ae448434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:15.950851 2ae448434700 0 lockdep start 2016-06-07 20:21:15.958473 2ae448434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:16.018194 2ae448434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:18.109353 2b2ee72e0700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:18.109358 2b2ee72e0700 0 lockdep start 2016-06-07 20:21:18.113496 2b2ee72e0700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:18.176924 2b2ee72e0700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:20.273141 2b1f88434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:20.273149 2b1f88434700 0 lockdep start 2016-06-07 20:21:20.281630 2b1f88434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:20.372798 2b1f88434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:22.462402 2aae2bb06700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:22.462410 2aae2bb06700 0 lockdep start 2016-06-07 20:21:22.472056 2aae2bb06700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:22.531955 2aae2bb06700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:24.630807 2b7107e91700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:24.630814 2b7107e91700 0 lockdep start 2016-06-07 20:21:24.634530 2b7107e91700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:25.119767 2b7107e91700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:27.210533 2af570b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:27.210538 2af570b6c700 0 lockdep start 2016-06-07 20:21:27.218096 2af570b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:27.280960 2af570b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:29.374725 2b58cf03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:29.374730 2b58cf03b700 0 lockdep start 2016-06-07 20:21:29.378733 2b58cf03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:29.435372 2b58cf03b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:31.539871 2ad95f943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:31.547238 2ad95f943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:31.618723 2ad95f943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:33.714483 2b39ed9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:33.714493 2b39ed9de700 0 lockdep start 2016-06-07 20:21:33.723101 2b39ed9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:33.786547 2b39ed9de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:35.880155 2b13fab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:35.880161 2b13fab5d700 0 lockdep start 2016-06-07 20:21:35.886345 2b13fab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:35.942393 2b13fab5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:38.042280 2b9b9c374700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:38.042286 2b9b9c374700 0 lockdep start 2016-06-07 20:21:38.046154 2b9b9c374700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:38.111807 2b9b9c374700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:40.202000 2b70c3d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:40.202006 2b70c3d5c700 0 lockdep start 2016-06-07 20:21:40.206179 2b70c3d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:40.270947 2b70c3d5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:42.349015 2b5f435f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:42.356365 2b5f435f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:42.412464 2b5f435f2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:44.486099 2b994a313700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:44.491988 2b994a313700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:45.907066 2b994a313700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:48.001555 2ae2f8f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:48.001566 2ae2f8f79700 0 lockdep start 2016-06-07 20:21:48.010590 2ae2f8f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:48.726306 2ae2f8f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:350: test_tiering: echo waiting for pg stats to flush waiting for pg stats to flush /home/yuriw/ceph/qa/workunits/cephtool/test.sh:351: test_tiering: sleep 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:349: test_tiering: grep ' 1 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:50.814439 2b9e0b423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:50.814447 2b9e0b423700 0 lockdep start 2016-06-07 20:21:50.824494 2b9e0b423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:51.094285 2b9e0b423700 0 lockdep stop cache2 4 5031 0 840G 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:353: test_tiering: expect_false ceph osd tier add slow cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier add slow cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:51.178875 2b3bcc1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:51.178879 2b3bcc1f3700 0 lockdep start 2016-06-07 20:21:51.185049 2b3bcc1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOTEMPTY: tier pool 'cache2' is not empty; --force-nonempty to force 2016-06-07 20:21:51.659270 2b3bcc1f3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:354: test_tiering: ceph osd tier add slow cache2 --force-nonempty *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:51.737688 2b5cc9e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:51.737694 2b5cc9e09700 0 lockdep start 2016-06-07 20:21:51.747287 2b5cc9e09700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache2' is now (or already was) a tier of 'slow' 2016-06-07 20:21:54.887388 2b5cc9e09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:355: test_tiering: ceph osd tier remove slow cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:54.977584 2ace4375d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:54.977589 2ace4375d700 0 lockdep start 2016-06-07 20:21:54.983238 2ace4375d700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache2' is now (or already was) not a tier of 'slow' 2016-06-07 20:21:57.705851 2ace4375d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:357: test_tiering: ceph osd pool ls /home/yuriw/ceph/qa/workunits/cephtool/test.sh:357: test_tiering: grep cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:57.778540 2b78999de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:57.791450 2b78999de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:57.861278 2b78999de700 0 lockdep stop cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:358: test_tiering: grep cache2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:358: test_tiering: ceph osd pool ls -f json-pretty *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:57.949851 2ae669355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:57.959796 2ae669355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:58.674287 2ae669355700 0 lockdep stop "cache2" /home/yuriw/ceph/qa/workunits/cephtool/test.sh:359: test_tiering: ceph osd pool ls detail /home/yuriw/ceph/qa/workunits/cephtool/test.sh:359: test_tiering: grep cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:58.749212 2b908ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:58.749217 2b908ab5d700 0 lockdep start 2016-06-07 20:21:58.755051 2b908ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:58.838658 2b908ab5d700 0 lockdep stop pool 4 'cache2' replicated size 3 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2 pgp_num 2 last_change 61 flags hashpspool stripe_width 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:360: test_tiering: ceph osd pool ls detail -f json-pretty /home/yuriw/ceph/qa/workunits/cephtool/test.sh:360: test_tiering: grep cache2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:58.901889 2aec3a33a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:58.907488 2aec3a33a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:58.961509 2aec3a33a700 0 lockdep stop "pool_name": "cache2", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:362: test_tiering: ceph osd pool delete cache cache --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:21:59.026891 2b8317e52700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:21:59.026896 2b8317e52700 0 lockdep start 2016-06-07 20:21:59.297700 2b8317e52700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache' removed 2016-06-07 20:22:01.272667 2b8317e52700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:363: test_tiering: ceph osd pool delete cache2 cache2 --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:01.367525 2affe2da1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:01.367530 2affe2da1700 0 lockdep start 2016-06-07 20:22:01.377527 2affe2da1700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache2' removed 2016-06-07 20:22:04.168432 2affe2da1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:366: test_tiering: ceph osd pool create snap_base 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:04.259224 2acc120ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:04.259230 2acc120ce700 0 lockdep start 2016-06-07 20:22:04.265271 2acc120ce700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'snap_base' created 2016-06-07 20:22:05.501840 2acc120ce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:367: test_tiering: ceph osd pool create snap_cache 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:05.585349 2b6880434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:05.585359 2b6880434700 0 lockdep start 2016-06-07 20:22:05.607387 2b6880434700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'snap_cache' created 2016-06-07 20:22:06.490958 2b6880434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:368: test_tiering: ceph osd pool mksnap snap_cache snapname *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:06.576130 2ae628f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:06.576139 2ae628f79700 0 lockdep start 2016-06-07 20:22:06.587479 2ae628f79700 -1 WARNING: the following dangerous and experimental features are enabled: * created pool snap_cache snap snapname 2016-06-07 20:22:09.835829 2ae628f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:369: test_tiering: expect_false ceph osd tier add snap_base snap_cache /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier add snap_base snap_cache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:09.915048 2afda1e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:09.915053 2afda1e09700 0 lockdep start 2016-06-07 20:22:09.921225 2afda1e09700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOTEMPTY: tier pool 'snap_cache' has snapshot state; it cannot be added as a tier without breaking the pool 2016-06-07 20:22:11.610915 2afda1e09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:370: test_tiering: ceph osd pool delete snap_base snap_base --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:11.696873 2b93220ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:11.707941 2b93220ce700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'snap_base' removed 2016-06-07 20:22:14.490840 2b93220ce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:371: test_tiering: ceph osd pool delete snap_cache snap_cache --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:14.582736 2b850781e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:14.582747 2b850781e700 0 lockdep start 2016-06-07 20:22:14.594549 2b850781e700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'snap_cache' removed 2016-06-07 20:22:16.550844 2b850781e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:374: test_tiering: ceph osd pool create eccache 2 2 erasure *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:16.648249 2b34efc5a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:16.648257 2b34efc5a700 0 lockdep start 2016-06-07 20:22:16.668444 2b34efc5a700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'eccache' created 2016-06-07 20:22:19.064756 2b34efc5a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:375: test_tiering: ceph osd pool create repbase 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:19.140985 2b3a2c026700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:19.140992 2b3a2c026700 0 lockdep start 2016-06-07 20:22:19.151756 2b3a2c026700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'repbase' created 2016-06-07 20:22:20.983387 2b3a2c026700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:376: test_tiering: expect_false ceph osd tier add repbase eccache /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd tier add repbase eccache *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:21.076600 2b8e3604d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:21.076607 2b8e3604d700 0 lockdep start 2016-06-07 20:22:21.086004 2b8e3604d700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOTSUP: tier pool 'eccache' is an ec pool, which cannot be a tier 2016-06-07 20:22:22.427956 2b8e3604d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:377: test_tiering: ceph osd pool delete repbase repbase --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:22.530271 2b061ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:22.530277 2b061ab5d700 0 lockdep start 2016-06-07 20:22:22.540367 2b061ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'repbase' removed 2016-06-07 20:22:24.098311 2b061ab5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:378: test_tiering: ceph osd pool delete eccache eccache --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:24.186034 2aca4dbf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:24.186042 2aca4dbf9700 0 lockdep start 2016-06-07 20:22:24.200213 2aca4dbf9700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'eccache' removed 2016-06-07 20:22:24.976297 2aca4dbf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:381: test_tiering: ceph osd pool create cache3 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:25.062427 2b4a87c07700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:25.083206 2b4a87c07700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache3' created 2016-06-07 20:22:26.627942 2b4a87c07700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:382: test_tiering: ceph osd tier add-cache slow cache3 1024000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:26.712949 2aae26f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:26.712955 2aae26f7a700 0 lockdep start 2016-06-07 20:22:26.732121 2aae26f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache3' is now (or already was) a cache tier of 'slow' 2016-06-07 20:22:27.291526 2aae26f7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:383: test_tiering: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:383: test_tiering: grep cache3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:383: test_tiering: grep 'target_bytes 1024000' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:383: test_tiering: grep bloom /home/yuriw/ceph/qa/workunits/cephtool/test.sh:383: test_tiering: grep 'false_positive_probability: 0.05' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:383: test_tiering: grep '1200s x4' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:27.389927 2b8efcf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:27.398470 2b8efcf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:27.468361 2b8efcf79700 0 lockdep stop pool 9 'cache3' replicated size 3 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2 pgp_num 2 last_change 80 flags hashpspool tier_of 1 cache_mode writeback target_bytes 1024000 hit_set bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 1200s x4 decay_rate 20 search_last_n 1 min_read_recency_for_promote 1 min_write_recency_for_promote 1 stripe_width 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:384: test_tiering: ceph osd tier remove slow cache3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:384: test_tiering: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:385: test_tiering: check_response 'EBUSY: tier pool '\''cache3'\'' is the overlay for '\''slow'\''; please remove-overlay first' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EBUSY: tier pool '\''cache3'\'' is the overlay for '\''slow'\''; please remove-overlay first' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EBUSY: tier pool '\''cache3'\'' is the overlay for '\''slow'\''; please remove-overlay first' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:386: test_tiering: ceph osd tier remove-overlay slow *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:28.003800 2b0264b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:28.003805 2b0264b6c700 0 lockdep start 2016-06-07 20:22:28.010827 2b0264b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * there is now (or already was) no overlay for 'slow' 2016-06-07 20:22:28.983434 2b0264b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:387: test_tiering: ceph osd tier remove slow cache3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:29.064310 2ae4597d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:29.064316 2ae4597d5700 0 lockdep start 2016-06-07 20:22:29.072077 2ae4597d5700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache3' is now (or already was) not a tier of 'slow' 2016-06-07 20:22:30.429586 2ae4597d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:388: test_tiering: grep cache3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:388: test_tiering: ceph osd pool ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:30.528586 2b5184d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:30.528592 2b5184d71700 0 lockdep start 2016-06-07 20:22:30.536068 2b5184d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:31.158126 2b5184d71700 0 lockdep stop cache3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:389: test_tiering: ceph osd pool delete cache3 cache3 --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:31.238714 2b0ce4233700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:31.238721 2b0ce4233700 0 lockdep start 2016-06-07 20:22:31.249107 2b0ce4233700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache3' removed 2016-06-07 20:22:31.985579 2b0ce4233700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:390: test_tiering: grep cache3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:390: test_tiering: ceph osd pool ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:32.083720 2b951b1f7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:32.083725 2b951b1f7700 0 lockdep start 2016-06-07 20:22:32.090122 2b951b1f7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:32.932372 2b951b1f7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:392: test_tiering: ceph osd pool delete slow2 slow2 --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:33.025795 2b16ff943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:33.025802 2b16ff943700 0 lockdep start 2016-06-07 20:22:33.036872 2b16ff943700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'slow2' removed 2016-06-07 20:22:33.461640 2b16ff943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:393: test_tiering: ceph osd pool delete slow slow --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:33.550588 2b5cf17d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:33.561923 2b5cf17d5700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'slow' removed 2016-06-07 20:22:34.649271 2b5cf17d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:396: test_tiering: ceph osd pool create datapool 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:34.736066 2ad53303b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:34.736071 2ad53303b700 0 lockdep start 2016-06-07 20:22:34.741298 2ad53303b700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'datapool' created 2016-06-07 20:22:36.285914 2ad53303b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:397: test_tiering: ceph osd pool create cachepool 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:36.369294 2b905d7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:36.388441 2b905d7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' created 2016-06-07 20:22:37.905008 2b905d7d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:398: test_tiering: ceph osd tier add-cache datapool cachepool 1024000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:37.996691 2b30e4434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:37.996700 2b30e4434700 0 lockdep start 2016-06-07 20:22:38.017824 2b30e4434700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' is now (or already was) a cache tier of 'datapool' 2016-06-07 20:22:39.399742 2b30e4434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:399: test_tiering: ceph osd tier cache-mode cachepool writeback *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:39.487413 2ba96bc34700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:39.487418 2ba96bc34700 0 lockdep start 2016-06-07 20:22:39.497307 2ba96bc34700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cachepool' to writeback 2016-06-07 20:22:41.241337 2ba96bc34700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:400: test_tiering: rados -p datapool put object /etc/passwd 2016-06-07 20:22:41.283570 2ab852f519c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:41.283637 2ab852f519c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:41.285124 2ab852f519c0 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:401: test_tiering: rados -p cachepool stat object 2016-06-07 20:22:41.866152 2b99aaf1b9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:41.866218 2b99aaf1b9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:41.867530 2b99aaf1b9c0 -1 WARNING: the following dangerous and experimental features are enabled: * cachepool/object mtime 2016-06-07 20:22:41.000000, size 5031 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:402: test_tiering: rados -p cachepool cache-flush object 2016-06-07 20:22:42.376424 2b1cb05169c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:42.376499 2b1cb05169c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:42.377980 2b1cb05169c0 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:403: test_tiering: rados -p datapool stat object 2016-06-07 20:22:42.420043 2b9b0775f9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:42.420115 2b9b0775f9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:42.421678 2b9b0775f9c0 -1 WARNING: the following dangerous and experimental features are enabled: * datapool/object mtime 2016-06-07 20:22:42.000000, size 5031 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:404: test_tiering: ceph osd tier remove-overlay datapool *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:42.524264 2b57b1eca700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:42.535343 2b57b1eca700 -1 WARNING: the following dangerous and experimental features are enabled: * there is now (or already was) no overlay for 'datapool' 2016-06-07 20:22:42.963379 2b57b1eca700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:405: test_tiering: ceph osd tier remove datapool cachepool *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:43.046985 2b2e4fdb5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:43.052831 2b2e4fdb5700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' is now (or already was) not a tier of 'datapool' 2016-06-07 20:22:43.779502 2b2e4fdb5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:406: test_tiering: ceph osd pool delete cachepool cachepool --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:43.862561 2b5c1ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:43.862567 2b5c1ad61700 0 lockdep start 2016-06-07 20:22:43.873947 2b5c1ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' removed 2016-06-07 20:22:45.307475 2b5c1ad61700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:407: test_tiering: ceph osd pool delete datapool datapool --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:45.389516 2b926bc7b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:45.389522 2b926bc7b700 0 lockdep start 2016-06-07 20:22:45.408206 2b926bc7b700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'datapool' removed 2016-06-07 20:22:47.453186 2b926bc7b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:410: test_tiering: ceph osd pool create datapool 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:47.542870 2b54b0b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:47.542875 2b54b0b6c700 0 lockdep start 2016-06-07 20:22:47.548750 2b54b0b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'datapool' created 2016-06-07 20:22:48.931556 2b54b0b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:411: test_tiering: ceph osd pool create cachepool 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:49.015601 2b2b120ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:49.015608 2b2b120ce700 0 lockdep start 2016-06-07 20:22:49.027662 2b2b120ce700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' created 2016-06-07 20:22:49.701020 2b2b120ce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:412: test_tiering: ceph osd tier add-cache datapool cachepool 1024000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:49.794691 2b5177d75700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:49.814911 2b5177d75700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' is now (or already was) a cache tier of 'datapool' 2016-06-07 20:22:52.203090 2b5177d75700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:413: test_tiering: ceph osd pool delete cachepool cachepool --yes-i-really-really-mean-it /home/yuriw/ceph/qa/workunits/cephtool/test.sh:413: test_tiering: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:414: test_tiering: check_response 'EBUSY: pool '\''cachepool'\'' is a tier of '\''datapool'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EBUSY: pool '\''cachepool'\'' is a tier of '\''datapool'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EBUSY: pool '\''cachepool'\'' is a tier of '\''datapool'\''' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:415: test_tiering: ceph osd pool delete datapool datapool --yes-i-really-really-mean-it /home/yuriw/ceph/qa/workunits/cephtool/test.sh:415: test_tiering: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:416: test_tiering: check_response 'EBUSY: pool '\''datapool'\'' has tiers cachepool' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EBUSY: pool '\''datapool'\'' has tiers cachepool' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EBUSY: pool '\''datapool'\'' has tiers cachepool' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:417: test_tiering: ceph osd tier remove-overlay datapool *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:53.403277 2b5635bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:53.403285 2b5635bf9700 0 lockdep start 2016-06-07 20:22:53.416211 2b5635bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * there is now (or already was) no overlay for 'datapool' 2016-06-07 20:22:54.151533 2b5635bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:418: test_tiering: ceph osd tier remove datapool cachepool *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:54.241411 2b871cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:54.252520 2b871cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' is now (or already was) not a tier of 'datapool' 2016-06-07 20:22:55.104478 2b871cf79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:419: test_tiering: ceph osd pool delete cachepool cachepool --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:55.192257 2b6d0b5b2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:55.192263 2b6d0b5b2700 0 lockdep start 2016-06-07 20:22:55.203340 2b6d0b5b2700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cachepool' removed 2016-06-07 20:22:57.250796 2b6d0b5b2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:420: test_tiering: ceph osd pool delete datapool datapool --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:57.334157 2af1a9cfa700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:57.345995 2af1a9cfa700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'datapool' removed 2016-06-07 20:22:58.324076 2af1a9cfa700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:423: test_tiering: ceph osd set notieragent *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:58.399219 2ba77cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:58.399224 2ba77cb6c700 0 lockdep start 2016-06-07 20:22:58.409288 2ba77cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * set notieragent 2016-06-07 20:22:59.790522 2ba77cb6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:424: test_tiering: ceph osd pool create datapool 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:22:59.883065 2ad91b1b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:22:59.883071 2ad91b1b7700 0 lockdep start 2016-06-07 20:22:59.893305 2ad91b1b7700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'datapool' created 2016-06-07 20:23:01.731905 2ad91b1b7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:425: test_tiering: ceph osd pool create cache4 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:01.820350 2b2106987700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:01.820356 2b2106987700 0 lockdep start 2016-06-07 20:23:01.829915 2b2106987700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache4' created 2016-06-07 20:23:02.654442 2b2106987700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:426: test_tiering: ceph osd tier add-cache datapool cache4 1024000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:02.751434 2b90619de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:02.751447 2b90619de700 0 lockdep start 2016-06-07 20:23:02.762894 2b90619de700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache4' is now (or already was) a cache tier of 'datapool' 2016-06-07 20:23:06.200620 2b90619de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:427: test_tiering: ceph osd tier cache-mode cache4 writeback *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:06.284068 2acae3c8a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:06.284073 2acae3c8a700 0 lockdep start 2016-06-07 20:23:06.290451 2acae3c8a700 -1 WARNING: the following dangerous and experimental features are enabled: * set cache-mode for pool 'cache4' to writeback 2016-06-07 20:23:06.349930 2acae3c8a700 0 lockdep stop //home/yuriw/ceph/qa/workunits/cephtool/test.sh:428: test_tiering: mktemp //home/yuriw/ceph/qa/workunits/cephtool/test.sh:428: test_tiering: grep tmp /home/yuriw/ceph/qa/workunits/cephtool/test.sh:428: test_tiering: tmpfile=/tmp/cephtool19083/tmp.RQVY3eSenL /home/yuriw/ceph/qa/workunits/cephtool/test.sh:429: test_tiering: dd if=/dev/zero of=/tmp/cephtool19083/tmp.RQVY3eSenL bs=4K count=1 1+0 records in 1+0 records out 4096 bytes (4.1 kB) copied, 0.000105524 s, 38.8 MB/s /home/yuriw/ceph/qa/workunits/cephtool/test.sh:430: test_tiering: ceph osd pool set cache4 target_max_objects 200 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:06.438841 2ad13c2b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:06.438846 2ad13c2b4700 0 lockdep start 2016-06-07 20:23:06.448553 2ad13c2b4700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 15 target_max_objects to 200 2016-06-07 20:23:08.063149 2ad13c2b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:431: test_tiering: ceph osd pool set cache4 target_max_bytes 1000000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:08.162721 2b5397ed1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:08.174232 2b5397ed1700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 15 target_max_bytes to 1000000 2016-06-07 20:23:09.251871 2b5397ed1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:432: test_tiering: rados -p cache4 put foo1 /tmp/cephtool19083/tmp.RQVY3eSenL 2016-06-07 20:23:09.299184 2b7a0ac7f9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:09.299400 2b7a0ac7f9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:09.300838 2b7a0ac7f9c0 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:433: test_tiering: rados -p cache4 put foo2 /tmp/cephtool19083/tmp.RQVY3eSenL 2016-06-07 20:23:10.264945 2b8baa99e9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:10.265136 2b8baa99e9c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:10.266528 2b8baa99e9c0 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:434: test_tiering: rm -f /tmp/cephtool19083/tmp.RQVY3eSenL /home/yuriw/ceph/qa/workunits/cephtool/test.sh:435: test_tiering: ceph tell 'osd.*' flush_pg_stats *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:10.837877 2ab0e0f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:10.837884 2ab0e0f79700 0 lockdep start 2016-06-07 20:23:10.849020 2ab0e0f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:11.390505 2ab0e0f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:436: test_tiering: grep datapool /home/yuriw/ceph/qa/workunits/cephtool/test.sh:436: test_tiering: ceph df /home/yuriw/ceph/qa/workunits/cephtool/test.sh:436: test_tiering: grep ' 2 ' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:11.478301 2b6eee00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:11.478307 2b6eee00d700 0 lockdep start 2016-06-07 20:23:11.482146 2b6eee00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:11.603553 2b6eee00d700 0 lockdep stop datapool 14 8192 0 841G 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:437: test_tiering: ceph osd tier remove-overlay datapool *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:11.695076 2b07d2b9d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:11.706860 2b07d2b9d700 -1 WARNING: the following dangerous and experimental features are enabled: * there is now (or already was) no overlay for 'datapool' 2016-06-07 20:23:13.050941 2b07d2b9d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:438: test_tiering: ceph osd tier remove datapool cache4 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:13.143655 2ae443278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:13.152648 2ae443278700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache4' is now (or already was) not a tier of 'datapool' 2016-06-07 20:23:13.984627 2ae443278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:439: test_tiering: ceph osd pool delete cache4 cache4 --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:14.075417 2affaaf7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:14.075433 2affaaf7a700 0 lockdep start 2016-06-07 20:23:14.085964 2affaaf7a700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache4' removed 2016-06-07 20:23:14.711658 2affaaf7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:440: test_tiering: ceph osd pool delete datapool datapool --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:14.799237 2aee90b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:14.799242 2aee90b6c700 0 lockdep start 2016-06-07 20:23:14.809848 2aee90b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'datapool' removed 2016-06-07 20:23:15.429752 2aee90b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:441: test_tiering: ceph osd unset notieragent *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:15.521945 2b2235395700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:15.521951 2b2235395700 0 lockdep start 2016-06-07 20:23:15.528128 2b2235395700 -1 WARNING: the following dangerous and experimental features are enabled: * unset notieragent 2016-06-07 20:23:16.768583 2b2235395700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:448: test_tiering: ceph osd pool create basepoolA 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:16.849636 2ba9657d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:16.862169 2ba9657d5700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'basepoolA' created 2016-06-07 20:23:18.261366 2ba9657d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:449: test_tiering: ceph osd pool create basepoolB 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:18.362027 2b0a7d576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:18.362033 2b0a7d576700 0 lockdep start 2016-06-07 20:23:18.369286 2b0a7d576700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'basepoolB' created 2016-06-07 20:23:19.932897 2b0a7d576700 0 lockdep stop //home/yuriw/ceph/qa/workunits/cephtool/test.sh:450: test_tiering: ceph osd dump //home/yuriw/ceph/qa/workunits/cephtool/test.sh:450: test_tiering: grep 'pool.*basepoolA' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:450: test_tiering: awk '{print $2;}' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:20.017298 2ab22761a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:20.017303 2ab22761a700 0 lockdep start 2016-06-07 20:23:20.021149 2ab22761a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:20.147164 2ab22761a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:450: test_tiering: poolA_id=16 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:451: test_tiering: grep 'pool.*basepoolB' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:451: test_tiering: awk '{print $2;}' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:451: test_tiering: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:20.241657 2b09fcd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:20.241667 2b09fcd71700 0 lockdep start 2016-06-07 20:23:20.250570 2b09fcd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:22.493828 2b09fcd71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:451: test_tiering: poolB_id=17 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:453: test_tiering: ceph osd pool create cache5 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:22.591052 2b0af3f09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:22.591063 2b0af3f09700 0 lockdep start 2016-06-07 20:23:22.601819 2b0af3f09700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache5' created 2016-06-07 20:23:24.094869 2b0af3f09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:454: test_tiering: ceph osd pool create cache6 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:24.194216 2b67833e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:24.204950 2b67833e3700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache6' created 2016-06-07 20:23:25.791502 2b67833e3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:455: test_tiering: ceph osd tier add basepoolA cache5 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:25.880032 2b616ec1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:25.890927 2b616ec1e700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache5' is now (or already was) a tier of 'basepoolA' 2016-06-07 20:23:26.478814 2b616ec1e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:456: test_tiering: ceph osd tier add basepoolB cache6 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:26.564172 2b11f6e22700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:26.564179 2b11f6e22700 0 lockdep start 2016-06-07 20:23:26.574474 2b11f6e22700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache6' is now (or already was) a tier of 'basepoolB' 2016-06-07 20:23:27.701045 2b11f6e22700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:457: test_tiering: ceph osd tier remove basepoolB cache5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:457: test_tiering: grep 'not a tier of' pool 'cache5' is now (or already was) not a tier of 'basepoolB' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:458: test_tiering: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:458: test_tiering: grep 'tier_of[ \t]\+16' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:458: test_tiering: grep 'pool.*'\''cache5'\''' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:27.978841 2ad4a7d0f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:27.987353 2ad4a7d0f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:28.210813 2ad4a7d0f700 0 lockdep stop pool 18 'cache5' replicated size 3 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2 pgp_num 2 last_change 125 flags hashpspool tier_of 16 stripe_width 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:459: test_tiering: ceph osd tier remove basepoolA cache6 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:459: test_tiering: grep 'not a tier of' pool 'cache6' is now (or already was) not a tier of 'basepoolA' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:460: test_tiering: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:460: test_tiering: grep 'tier_of[ \t]\+17' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:460: test_tiering: grep 'pool.*'\''cache6'\''' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:28.564600 2ba99cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:28.564606 2ba99cb6c700 0 lockdep start 2016-06-07 20:23:28.568782 2ba99cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:28.992277 2ba99cb6c700 0 lockdep stop pool 19 'cache6' replicated size 3 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2 pgp_num 2 last_change 127 flags hashpspool tier_of 17 stripe_width 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:462: test_tiering: grep 'not a tier of' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:462: test_tiering: ceph osd tier remove basepoolA cache5 pool 'cache5' is now (or already was) not a tier of 'basepoolA' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:463: test_tiering: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:463: test_tiering: grep 'pool.*'\''cache5'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:463: test_tiering: grep tier_of *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:29.644556 2acd25e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:29.648236 2acd25e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:30.995251 2acd25e09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:464: test_tiering: ceph osd tier remove basepoolB cache6 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:464: test_tiering: grep 'not a tier of' pool 'cache6' is now (or already was) not a tier of 'basepoolB' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:465: test_tiering: grep 'pool.*'\''cache6'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:465: test_tiering: grep tier_of /home/yuriw/ceph/qa/workunits/cephtool/test.sh:465: test_tiering: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:31.923531 2aca85355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:31.930960 2aca85355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:32.052586 2aca85355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:467: test_tiering: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:467: test_tiering: grep 'pool.*'\''basepoolA'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:467: test_tiering: grep tiers *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:32.150115 2ab0cd9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:32.150121 2ab0cd9de700 0 lockdep start 2016-06-07 20:23:32.155996 2ab0cd9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:32.276580 2ab0cd9de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:468: test_tiering: grep 'pool.*'\''basepoolB'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:468: test_tiering: grep tiers *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:32.353677 2abe83531700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:32.353683 2abe83531700 0 lockdep start 2016-06-07 20:23:32.360990 2abe83531700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:32.422393 2abe83531700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:470: test_tiering: ceph osd pool delete cache6 cache6 --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:32.510582 2ba970f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:32.521834 2ba970f79700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache6' removed 2016-06-07 20:23:32.946768 2ba970f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:471: test_tiering: ceph osd pool delete cache5 cache5 --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:33.033764 2b83fdc39700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:33.045254 2b83fdc39700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'cache5' removed 2016-06-07 20:23:33.729791 2b83fdc39700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:472: test_tiering: ceph osd pool delete basepoolB basepoolB --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:33.825308 2afb9e0ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:33.825314 2afb9e0ce700 0 lockdep start 2016-06-07 20:23:33.835116 2afb9e0ce700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'basepoolB' removed 2016-06-07 20:23:34.362495 2afb9e0ce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:473: test_tiering: ceph osd pool delete basepoolA basepoolA --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:34.449367 2b9f55395700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:34.449375 2b9f55395700 0 lockdep start 2016-06-07 20:23:34.461770 2b9f55395700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'basepoolA' removed 2016-06-07 20:23:36.256253 2b9f55395700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:36.351569 2ad82bd9f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:36.351575 2ad82bd9f700 0 lockdep start 2016-06-07 20:23:36.357349 2ad82bd9f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:36.779478 2ad82bd9f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_auth /home/yuriw/ceph/qa/workunits/cephtool/test.sh:478: test_auth: ceph auth add client.xx mon allow osd 'allow *' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:36.861608 2afb5f7de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:36.880637 2afb5f7de700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for client.xx 2016-06-07 20:23:37.762995 2afb5f7de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:479: test_auth: ceph auth export client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:37.858447 2b12ba859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:37.869741 2b12ba859700 -1 WARNING: the following dangerous and experimental features are enabled: * export auth(auid = 18446744073709551615 key=AQBJLVdXbr6zHRAAOBxu1/YP5ElzYPCkRuuuCQ== with 2 caps) 2016-06-07 20:23:38.244306 2b12ba859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:480: test_auth: ceph auth add client.xx -i client.xx.keyring *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:38.326758 2abeef278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:38.339052 2abeef278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:38.772577 2abeef278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:481: test_auth: rm -f client.xx.keyring /home/yuriw/ceph/qa/workunits/cephtool/test.sh:482: test_auth: grep client.xx /home/yuriw/ceph/qa/workunits/cephtool/test.sh:482: test_auth: ceph auth list *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:38.862751 2b649d7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:38.862756 2b649d7d5700 0 lockdep start 2016-06-07 20:23:38.868785 2b649d7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * installed auth entries: 2016-06-07 20:23:39.148036 2b649d7d5700 0 lockdep stop client.xx /home/yuriw/ceph/qa/workunits/cephtool/test.sh:483: test_auth: ceph auth get client.xx /home/yuriw/ceph/qa/workunits/cephtool/test.sh:483: test_auth: grep caps /home/yuriw/ceph/qa/workunits/cephtool/test.sh:483: test_auth: grep mon *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:39.231279 2aca66859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:39.231284 2aca66859700 0 lockdep start 2016-06-07 20:23:39.238650 2aca66859700 -1 WARNING: the following dangerous and experimental features are enabled: * exported keyring for client.xx 2016-06-07 20:23:39.556049 2aca66859700 0 lockdep stop caps mon = "allow" /home/yuriw/ceph/qa/workunits/cephtool/test.sh:484: test_auth: ceph auth get client.xx /home/yuriw/ceph/qa/workunits/cephtool/test.sh:484: test_auth: grep caps /home/yuriw/ceph/qa/workunits/cephtool/test.sh:484: test_auth: grep osd *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:39.645521 2b8cb8d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:39.658340 2b8cb8d71700 -1 WARNING: the following dangerous and experimental features are enabled: * exported keyring for client.xx 2016-06-07 20:23:40.653238 2b8cb8d71700 0 lockdep stop caps osd = "allow *" /home/yuriw/ceph/qa/workunits/cephtool/test.sh:485: test_auth: ceph auth get-key client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:40.743147 2b257eb5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:40.743152 2b257eb5d700 0 lockdep start 2016-06-07 20:23:40.749186 2b257eb5d700 -1 WARNING: the following dangerous and experimental features are enabled: * AQBJLVdXbr6zHRAAOBxu1/YP5ElzYPCkRuuuCQ==2016-06-07 20:23:40.951469 2b257eb5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:486: test_auth: ceph auth print-key client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:41.034369 2b6f65576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:41.034376 2b6f65576700 0 lockdep start 2016-06-07 20:23:41.040878 2b6f65576700 -1 WARNING: the following dangerous and experimental features are enabled: * AQBJLVdXbr6zHRAAOBxu1/YP5ElzYPCkRuuuCQ==2016-06-07 20:23:41.533057 2b6f65576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:487: test_auth: ceph auth print_key client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:41.633684 2b5377a04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:41.633690 2b5377a04700 0 lockdep start 2016-06-07 20:23:41.643759 2b5377a04700 -1 WARNING: the following dangerous and experimental features are enabled: * AQBJLVdXbr6zHRAAOBxu1/YP5ElzYPCkRuuuCQ==2016-06-07 20:23:41.707060 2b5377a04700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:488: test_auth: ceph auth caps client.xx osd 'allow rw' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:41.787101 2b8a5d355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:41.787106 2b8a5d355700 0 lockdep start 2016-06-07 20:23:41.792939 2b8a5d355700 -1 WARNING: the following dangerous and experimental features are enabled: * updated caps for client.xx 2016-06-07 20:23:42.930147 2b8a5d355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:489: test_auth: expect_false 'ceph auth get client.xx | grep caps | grep mon' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph auth get client.xx | grep caps | grep mon' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph auth get client.xx | grep caps | grep mon: command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:490: test_auth: ceph auth get client.xx /home/yuriw/ceph/qa/workunits/cephtool/test.sh:490: test_auth: grep osd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:490: test_auth: grep 'allow rw' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:43.027095 2b9cc83b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:43.027101 2b9cc83b4700 0 lockdep start 2016-06-07 20:23:43.035003 2b9cc83b4700 -1 WARNING: the following dangerous and experimental features are enabled: * exported keyring for client.xx 2016-06-07 20:23:43.509849 2b9cc83b4700 0 lockdep stop caps osd = "allow rw" /home/yuriw/ceph/qa/workunits/cephtool/test.sh:491: test_auth: ceph auth export /home/yuriw/ceph/qa/workunits/cephtool/test.sh:491: test_auth: grep client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:43.598163 2ac41a643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:43.609816 2ac41a643700 -1 WARNING: the following dangerous and experimental features are enabled: * exported master keyring 2016-06-07 20:23:44.202957 2ac41a643700 0 lockdep stop [client.xx] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:492: test_auth: ceph auth export -o authfile *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:44.283026 2b5fce643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:44.290092 2b5fce643700 -1 WARNING: the following dangerous and experimental features are enabled: * exported master keyring 2016-06-07 20:23:44.356116 2b5fce643700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:493: test_auth: ceph auth import -i authfile *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:44.438211 2b4c7a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:44.438216 2b4c7a91a700 0 lockdep start 2016-06-07 20:23:44.444259 2b4c7a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * imported keyring 2016-06-07 20:23:47.091420 2b4c7a91a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:494: test_auth: ceph auth export -o authfile2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:47.184100 2ab932f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:47.184105 2ab932f7a700 0 lockdep start 2016-06-07 20:23:47.187814 2ab932f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * exported master keyring 2016-06-07 20:23:47.741326 2ab932f7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:495: test_auth: diff authfile authfile2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:496: test_auth: rm authfile authfile2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:497: test_auth: ceph auth del client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:47.834873 2af460db1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:47.834878 2af460db1700 0 lockdep start 2016-06-07 20:23:47.842589 2af460db1700 -1 WARNING: the following dangerous and experimental features are enabled: * updated 2016-06-07 20:23:48.291103 2af460db1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:498: test_auth: expect_false ceph auth get client.xx /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph auth get client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:48.386266 2b72559de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:48.386272 2b72559de700 0 lockdep start 2016-06-07 20:23:48.392625 2b72559de700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOENT: failed to find client.xx in keyring 2016-06-07 20:23:48.462786 2b72559de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:501: test_auth: ceph /home/yuriw/ceph/qa/workunits/cephtool/test.sh:501: test_auth: echo -e 'auth add client.xx mon allow osd "allow *"\n' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:48.551716 2aca06212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:48.554831 2aca06212700 -1 WARNING: the following dangerous and experimental features are enabled: * Status: added key for client.xx 2016-06-07 20:23:49.247114 2aca06212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:502: test_auth: ceph auth get client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:49.344678 2b40bb4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:49.358013 2b40bb4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * exported keyring for client.xx [client.xx] key = AQBULVdX1wSNNBAAOyZfmyy+UFdO4dnMzB6Dpw== caps mon = "allow" caps osd = "allow *" 2016-06-07 20:23:49.682523 2b40bb4a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:504: test_auth: echo 'auth del client.xx' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:504: test_auth: ceph *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:49.777479 2ba229576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:49.783926 2ba229576700 -1 WARNING: the following dangerous and experimental features are enabled: * Status: updated 2016-06-07 20:23:50.106040 2ba229576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:505: test_auth: expect_false ceph auth get client.xx /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph auth get client.xx *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:50.196816 2aed8691a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:50.202989 2aed8691a700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOENT: failed to find client.xx in keyring 2016-06-07 20:23:51.179123 2aed8691a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:510: test_auth: local auid=444 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:511: test_auth: ceph-authtool --create-keyring --name client.TEST --gen-key --set-uid 444 TEST-keyring 2016-06-07 20:23:51.195552 2adea0b66e80 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:51.196167 2adea0b66e80 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:51.197547 2adea0b66e80 -1 WARNING: the following dangerous and experimental features are enabled: * creating TEST-keyring /home/yuriw/ceph/qa/workunits/cephtool/test.sh:512: test_auth: ceph auth import --in-file TEST-keyring *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:51.276627 2b5e743b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:51.276632 2b5e743b4700 0 lockdep start 2016-06-07 20:23:51.282509 2b5e743b4700 -1 WARNING: the following dangerous and experimental features are enabled: * imported keyring 2016-06-07 20:23:51.809422 2b5e743b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:513: test_auth: rm TEST-keyring /home/yuriw/ceph/qa/workunits/cephtool/test.sh:514: test_auth: ceph auth get client.TEST *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:51.903692 2b101f983700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:51.903698 2b101f983700 0 lockdep start 2016-06-07 20:23:51.918239 2b101f983700 -1 WARNING: the following dangerous and experimental features are enabled: * exported keyring for client.TEST 2016-06-07 20:23:52.641864 2b101f983700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:515: test_auth: check_response 'auid = 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='auid = 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'auid = 444' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:516: test_auth: ceph --format json-pretty auth get client.TEST *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:52.735501 2b793ff24700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:52.735508 2b793ff24700 0 lockdep start 2016-06-07 20:23:52.743825 2b793ff24700 -1 WARNING: the following dangerous and experimental features are enabled: * exported keyring for client.TEST 2016-06-07 20:23:53.023223 2b793ff24700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:517: test_auth: check_response '"auid": 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='"auid": 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- '"auid": 444' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:518: test_auth: ceph auth list *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:53.144072 2b6449e49700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:53.144077 2b6449e49700 0 lockdep start 2016-06-07 20:23:53.150297 2b6449e49700 -1 WARNING: the following dangerous and experimental features are enabled: * installed auth entries: 2016-06-07 20:23:53.422753 2b6449e49700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:519: test_auth: check_response 'auid: 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='auid: 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'auid: 444' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:520: test_auth: ceph --format json-pretty auth list *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:53.510874 2b1d0bee4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:53.510881 2b1d0bee4700 0 lockdep start 2016-06-07 20:23:53.518135 2b1d0bee4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:53.792902 2b1d0bee4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:521: test_auth: check_response '"auid": 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='"auid": 444' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- '"auid": 444' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:522: test_auth: ceph auth del client.TEST *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:53.892119 2aad2653f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:53.892125 2aad2653f700 0 lockdep start 2016-06-07 20:23:53.898468 2aad2653f700 -1 WARNING: the following dangerous and experimental features are enabled: * updated 2016-06-07 20:23:54.462229 2aad2653f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:54.543451 2b344bf3a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:54.543456 2b344bf3a700 0 lockdep start 2016-06-07 20:23:54.549232 2b344bf3a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:55.896040 2b344bf3a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_auth_profiles /home/yuriw/ceph/qa/workunits/cephtool/test.sh:527: test_auth_profiles: ceph auth add client.xx-profile-ro mon 'allow profile read-only' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:55.984794 2b39d6a08700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:55.984800 2b39d6a08700 0 lockdep start 2016-06-07 20:23:55.991057 2b39d6a08700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for client.xx-profile-ro 2016-06-07 20:23:56.676384 2b39d6a08700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:528: test_auth_profiles: ceph auth add client.xx-profile-rw mon 'allow profile read-write' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:56.764456 2acbd5e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:56.764462 2acbd5e09700 0 lockdep start 2016-06-07 20:23:56.770407 2acbd5e09700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for client.xx-profile-rw 2016-06-07 20:23:58.119435 2acbd5e09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:529: test_auth_profiles: ceph auth add client.xx-profile-rd mon 'allow profile role-definer' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:58.218627 2b4ca4434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:58.230085 2b4ca4434700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for client.xx-profile-rd 2016-06-07 20:23:58.665989 2b4ca4434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:531: test_auth_profiles: ceph auth export *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:58.760538 2b97a02b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:58.760546 2b97a02b4700 0 lockdep start 2016-06-07 20:23:58.766372 2b97a02b4700 -1 WARNING: the following dangerous and experimental features are enabled: * exported master keyring 2016-06-07 20:23:58.889498 2b97a02b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:534: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring status *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:58.976589 2ac65ffe6700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:58.983223 2ac65ffe6700 -1 WARNING: the following dangerous and experimental features are enabled: * cluster 8fedb721-e6d4-42fc-a3ae-49153fde812f health HEALTH_OK monmap e1: 3 mons at {a=127.0.0.1:7202/0,b=127.0.0.1:7203/0,c=127.0.0.1:7204/0} election epoch 6, quorum 0,1,2 a,b,c osdmap e133: 3 osds: 3 up, 3 in flags sortbitwise,require_jewel_osds pgmap v305: 8 pgs, 1 pools, 0 bytes data, 0 objects 88842 MB used, 2523 GB / 2750 GB avail 8 active+clean 2016-06-07 20:23:59.444395 2ac65ffe6700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:535: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:59.545159 2ad0c2744700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:59.545164 2ad0c2744700 0 lockdep start 2016-06-07 20:23:59.551587 2ad0c2744700 -1 WARNING: the following dangerous and experimental features are enabled: * epoch 133 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f created 2016-06-07 20:16:14.933213 modified 2016-06-07 20:23:35.584255 flags sortbitwise,require_jewel_osds pool 0 'rbd' replicated size 3 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 1 flags hashpspool stripe_width 0 max_osd 3 osd.0 up in weight 1 up_from 6 up_thru 122 down_at 0 last_clean_interval [0,0) 127.0.0.1:6832/28826 127.0.0.1:6833/28826 127.0.0.1:6834/28826 127.0.0.1:6835/28826 exists,up c8e248f1-e9df-405e-b941-418201377eef osd.1 up in weight 1 up_from 10 up_thru 124 down_at 0 last_clean_interval [0,0) 127.0.0.1:6856/31419 127.0.0.1:6857/31419 127.0.0.1:6858/31419 127.0.0.1:6859/31419 exists,up f2fddc8f-0f00-4b56-aa52-e248880c12f6 osd.2 up in weight 1 up_from 13 up_thru 124 down_at 0 last_clean_interval [0,0) 127.0.0.1:6881/18968 127.0.0.1:6882/18968 127.0.0.1:6883/18968 127.0.0.1:6884/18968 exists,up 455c7a54-0424-4e63-af14-378cff9c0ff1 2016-06-07 20:23:59.774422 2ad0c2744700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:536: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring pg dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:23:59.875428 2b624a252700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:23:59.875437 2b624a252700 0 lockdep start 2016-06-07 20:23:59.883635 2b624a252700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped all in format plain version 307 stamp 2016-06-07 20:24:00.206185 last_osdmap_epoch 133 last_pg_scan 133 full_ratio 0.99 nearfull_ratio 0.85 PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:06.735796 0'0 24:22 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:16:16.043762 0'0 2016-06-07 20:16:16.043762 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:04.786902 0'0 24:7 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:04.787444 0'0 24:22 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:16:16.043772 0'0 2016-06-07 20:16:16.043772 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:00.122375 0'0 22:8 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:00.030738 0'0 22:23 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:16:16.043775 0'0 2016-06-07 20:16:16.043775 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:01.344640 0'0 23:23 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:16:16.043777 0'0 2016-06-07 20:16:16.043777 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:02.704324 0'0 24:22 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:16:16.043778 0'0 2016-06-07 20:16:16.043778 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:05.804271 0'0 24:18 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 0 0 0 0 0 0 0 0 0 sum 0 0 0 0 0 0 0 0 OSD_STAT USED AVAIL TOTAL HB_IN PG_SUM 2 29615 kB 841 MB 916 MB [0,1] 8 1 29616 kB 841 MB 916 MB [0,2] 8 0 29614 kB 841 MB 916 MB [1,2] 8 sum 88846 kB 2523 MB 2750 MB 2016-06-07 20:24:01.022212 2b624a252700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:537: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring mon dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:01.100309 2b4b1291a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:01.372291 2b4b1291a700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped monmap epoch 1 epoch 1 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f last_changed 2016-06-07 20:15:54.015730 created 2016-06-07 20:15:54.015730 0: 127.0.0.1:7202/0 mon.a 1: 127.0.0.1:7203/0 mon.b 2: 127.0.0.1:7204/0 mon.c 2016-06-07 20:24:02.243596 2b4b1291a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:538: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring mds dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:02.320578 2b4e1e212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:02.330261 2b4e1e212700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped fsmap epoch 1 fs_name cephfs epoch 1 flags 0 created 0.000000 modified 0.000000 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed damaged stopped data_pools metadata_pool 0 inline_data disabled 2016-06-07 20:24:03.307861 2b4e1e212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:540: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring log foo /home/yuriw/ceph/qa/workunits/cephtool/test.sh:540: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:541: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:542: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring osd set noout /home/yuriw/ceph/qa/workunits/cephtool/test.sh:542: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:543: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:544: test_auth_profiles: ceph -n client.xx-profile-ro -k client.xx.keyring auth list /home/yuriw/ceph/qa/workunits/cephtool/test.sh:544: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:545: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:548: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring status *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:04.115781 2b271df0a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:04.122583 2b271df0a700 -1 WARNING: the following dangerous and experimental features are enabled: * cluster 8fedb721-e6d4-42fc-a3ae-49153fde812f health HEALTH_OK monmap e1: 3 mons at {a=127.0.0.1:7202/0,b=127.0.0.1:7203/0,c=127.0.0.1:7204/0} election epoch 6, quorum 0,1,2 a,b,c osdmap e133: 3 osds: 3 up, 3 in flags sortbitwise,require_jewel_osds pgmap v308: 8 pgs, 1 pools, 0 bytes data, 0 objects 88862 MB used, 2523 GB / 2750 GB avail 8 active+clean 2016-06-07 20:24:04.405857 2b271df0a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:549: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:04.545810 2afb43423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:04.545816 2afb43423700 0 lockdep start 2016-06-07 20:24:04.549660 2afb43423700 -1 WARNING: the following dangerous and experimental features are enabled: * epoch 133 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f created 2016-06-07 20:16:14.933213 modified 2016-06-07 20:23:35.584255 flags sortbitwise,require_jewel_osds pool 0 'rbd' replicated size 3 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 8 pgp_num 8 last_change 1 flags hashpspool stripe_width 0 max_osd 3 osd.0 up in weight 1 up_from 6 up_thru 122 down_at 0 last_clean_interval [0,0) 127.0.0.1:6832/28826 127.0.0.1:6833/28826 127.0.0.1:6834/28826 127.0.0.1:6835/28826 exists,up c8e248f1-e9df-405e-b941-418201377eef osd.1 up in weight 1 up_from 10 up_thru 124 down_at 0 last_clean_interval [0,0) 127.0.0.1:6856/31419 127.0.0.1:6857/31419 127.0.0.1:6858/31419 127.0.0.1:6859/31419 exists,up f2fddc8f-0f00-4b56-aa52-e248880c12f6 osd.2 up in weight 1 up_from 13 up_thru 124 down_at 0 last_clean_interval [0,0) 127.0.0.1:6881/18968 127.0.0.1:6882/18968 127.0.0.1:6883/18968 127.0.0.1:6884/18968 exists,up 455c7a54-0424-4e63-af14-378cff9c0ff1 2016-06-07 20:24:05.795186 2afb43423700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:550: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring pg dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:05.887386 2adb3c1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:05.899458 2adb3c1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped all in format plain version 310 stamp 2016-06-07 20:24:05.172437 last_osdmap_epoch 133 last_pg_scan 133 full_ratio 0.99 nearfull_ratio 0.85 PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:06.735796 0'0 24:22 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:16:16.043762 0'0 2016-06-07 20:16:16.043762 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:04.786902 0'0 24:7 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:04.787444 0'0 24:22 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:16:16.043772 0'0 2016-06-07 20:16:16.043772 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:00.122375 0'0 22:8 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:00.030738 0'0 22:23 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:16:16.043775 0'0 2016-06-07 20:16:16.043775 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:01.344640 0'0 23:23 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:16:16.043777 0'0 2016-06-07 20:16:16.043777 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:02.704324 0'0 24:22 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:16:16.043778 0'0 2016-06-07 20:16:16.043778 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:20:05.804271 0'0 24:18 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 0 0 0 0 0 0 0 0 0 sum 0 0 0 0 0 0 0 0 OSD_STAT USED AVAIL TOTAL HB_IN PG_SUM 2 29595 kB 841 MB 916 MB [0,1] 8 1 29595 kB 841 MB 916 MB [0,2] 8 0 29631 kB 841 MB 916 MB [1,2] 8 sum 88822 kB 2523 MB 2750 MB 2016-06-07 20:24:06.070323 2adb3c1f3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:551: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring mon dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:06.166010 2b865b943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:06.166021 2b865b943700 0 lockdep start 2016-06-07 20:24:06.178553 2b865b943700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped monmap epoch 1 epoch 1 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f last_changed 2016-06-07 20:15:54.015730 created 2016-06-07 20:15:54.015730 0: 127.0.0.1:7202/0 mon.a 1: 127.0.0.1:7203/0 mon.b 2: 127.0.0.1:7204/0 mon.c 2016-06-07 20:24:06.305642 2b865b943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:552: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring mds dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:06.398684 2ad90c3b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:06.398690 2ad90c3b4700 0 lockdep start 2016-06-07 20:24:06.404990 2ad90c3b4700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped fsmap epoch 1 fs_name cephfs epoch 1 flags 0 created 0.000000 modified 0.000000 tableserver 0 root 0 session_timeout 0 session_autoclose 0 max_file_size 0 last_failure 0 last_failure_osd_epoch 0 compat compat={},rocompat={},incompat={} max_mds 0 in up {} failed damaged stopped data_pools metadata_pool 0 inline_data disabled 2016-06-07 20:24:06.530440 2ad90c3b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:553: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring log foo *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:06.613536 2b8864f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:06.613546 2b8864f79700 0 lockdep start 2016-06-07 20:24:06.624618 2b8864f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:07.916627 2b8864f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:554: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring osd set noout *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:08.006988 2b8bb5a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:08.014008 2b8bb5a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * set noout 2016-06-07 20:24:08.746982 2b8bb5a1e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:555: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring osd unset noout *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:08.838902 2b2edbd5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:08.838912 2b2edbd5c700 0 lockdep start 2016-06-07 20:24:08.850401 2b2edbd5c700 -1 WARNING: the following dangerous and experimental features are enabled: * unset noout 2016-06-07 20:24:09.985525 2b2edbd5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:557: test_auth_profiles: ceph -n client.xx-profile-rw -k client.xx.keyring auth list /home/yuriw/ceph/qa/workunits/cephtool/test.sh:557: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:558: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:561: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring auth list *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:11.632428 2b421eb5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:11.632434 2b421eb5d700 0 lockdep start 2016-06-07 20:24:11.639408 2b421eb5d700 -1 WARNING: the following dangerous and experimental features are enabled: * installed auth entries: osd.0 key: AQCrK1dXqKj+ARAA2r4zftvTU/rV0AhbWbkh7Q== caps: [mon] allow profile osd caps: [osd] allow * osd.1 key: AQDFK1dXg0OhCxAA0PmTT3k2fkam+MukFmxH5g== caps: [mon] allow profile osd caps: [osd] allow * osd.2 key: AQBYLFdXqSg5HRAApkaAfA8kIFRNd2cqiBn9+Q== caps: [mon] allow profile osd caps: [osd] allow * client.admin key: AQB5K1dXvp1kOxAAtE4ARVtFaWSwYmHU8a6cRw== auid: 0 caps: [mds] allow * caps: [mon] allow * caps: [osd] allow * client.xx-profile-rd key: AQBeLVdX/UPeFxAAtXNtPDAYOCkCS/sLVvu1jg== caps: [mon] allow profile role-definer client.xx-profile-ro key: AQBcLVdX9U+QFRAAyecedrCYYXPtUXVjQ3Iq7A== caps: [mon] allow profile read-only client.xx-profile-rw key: AQBdLVdXoCEzMRAAtNITJi8tuZwHWvn+Nl7gSQ== caps: [mon] allow profile read-write 2016-06-07 20:24:11.871055 2b421eb5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:562: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring auth export *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:11.960559 2b211e212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:11.960568 2b211e212700 0 lockdep start 2016-06-07 20:24:11.968634 2b211e212700 -1 WARNING: the following dangerous and experimental features are enabled: * exported master keyring [osd.0] key = AQCrK1dXqKj+ARAA2r4zftvTU/rV0AhbWbkh7Q== caps mon = "allow profile osd" caps osd = "allow *" [osd.1] key = AQDFK1dXg0OhCxAA0PmTT3k2fkam+MukFmxH5g== caps mon = "allow profile osd" caps osd = "allow *" [osd.2] key = AQBYLFdXqSg5HRAApkaAfA8kIFRNd2cqiBn9+Q== caps mon = "allow profile osd" caps osd = "allow *" [client.admin] key = AQB5K1dXvp1kOxAAtE4ARVtFaWSwYmHU8a6cRw== auid = 0 caps mds = "allow *" caps mon = "allow *" caps osd = "allow *" [client.xx-profile-rd] key = AQBeLVdX/UPeFxAAtXNtPDAYOCkCS/sLVvu1jg== caps mon = "allow profile role-definer" [client.xx-profile-ro] key = AQBcLVdX9U+QFRAAyecedrCYYXPtUXVjQ3Iq7A== caps mon = "allow profile read-only" [client.xx-profile-rw] key = AQBdLVdXoCEzMRAAtNITJi8tuZwHWvn+Nl7gSQ== caps mon = "allow profile read-write" 2016-06-07 20:24:12.783148 2b211e212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:563: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring auth add client.xx-profile-foo *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:12.866472 2ade75576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:12.866478 2ade75576700 0 lockdep start 2016-06-07 20:24:12.872121 2ade75576700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for client.xx-profile-foo 2016-06-07 20:24:13.600111 2ade75576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:564: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring status *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:13.704737 2acd31bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:13.712372 2acd31bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * cluster 8fedb721-e6d4-42fc-a3ae-49153fde812f health HEALTH_OK monmap e1: 3 mons at {a=127.0.0.1:7202/0,b=127.0.0.1:7203/0,c=127.0.0.1:7204/0} election epoch 6, quorum 0,1,2 a,b,c osdmap e135: 3 osds: 3 up, 3 in flags sortbitwise,require_jewel_osds pgmap v315: 8 pgs, 1 pools, 0 bytes data, 0 objects 88792 MB used, 2523 GB / 2750 GB avail 8 active+clean 2016-06-07 20:24:14.239933 2acd31bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:565: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:565: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:566: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:567: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring pg dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:567: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:568: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:570: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring mon dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:14.810294 2ab09ff8b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:14.810301 2ab09ff8b700 0 lockdep start 2016-06-07 20:24:14.816184 2ab09ff8b700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped monmap epoch 1 epoch 1 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f last_changed 2016-06-07 20:15:54.015730 created 2016-06-07 20:15:54.015730 0: 127.0.0.1:7202/0 mon.a 1: 127.0.0.1:7203/0 mon.b 2: 127.0.0.1:7204/0 mon.c 2016-06-07 20:24:14.932019 2ab09ff8b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:572: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring mon add foo 1.1.1.1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:572: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:573: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:574: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring mds dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:574: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:575: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:576: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring log foo /home/yuriw/ceph/qa/workunits/cephtool/test.sh:576: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:577: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:578: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring osd set noout /home/yuriw/ceph/qa/workunits/cephtool/test.sh:578: test_auth_profiles: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:579: test_auth_profiles: check_response 'EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='EACCES: access denied' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'EACCES: access denied' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:581: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring auth del client.xx-profile-ro *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:17.001764 2afe5519a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:17.001770 2afe5519a700 0 lockdep start 2016-06-07 20:24:17.009404 2afe5519a700 -1 WARNING: the following dangerous and experimental features are enabled: * updated 2016-06-07 20:24:19.502932 2afe5519a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:582: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring auth del client.xx-profile-rw *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:19.717936 2b5f5b03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:19.717954 2b5f5b03b700 0 lockdep start 2016-06-07 20:24:19.729756 2b5f5b03b700 -1 WARNING: the following dangerous and experimental features are enabled: * updated 2016-06-07 20:24:20.670289 2b5f5b03b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:585: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring auth add client.xx-profile-rd2 mon 'allow profile role-definer' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:20.750811 2b251a2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:20.750816 2b251a2d3700 0 lockdep start 2016-06-07 20:24:20.760809 2b251a2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * added key for client.xx-profile-rd2 2016-06-07 20:24:21.399897 2b251a2d3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:587: test_auth_profiles: ceph -n client.xx-profile-rd -k client.xx.keyring auth export *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:21.468220 2ac2f6212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:21.468225 2ac2f6212700 0 lockdep start 2016-06-07 20:24:21.473288 2ac2f6212700 -1 WARNING: the following dangerous and experimental features are enabled: * exported master keyring 2016-06-07 20:24:21.997237 2ac2f6212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:590: test_auth_profiles: ceph -n client.xx-profile-rd2 -k client.xx.keyring.2 auth del client.xx-profile-rd *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:22.075549 2b4a22643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:22.083035 2b4a22643700 -1 WARNING: the following dangerous and experimental features are enabled: * updated 2016-06-07 20:24:24.538368 2b4a22643700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:593: test_auth_profiles: ceph auth del client.xx-profile-rd2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:24.597768 2b4317ed1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:24.603449 2b4317ed1700 -1 WARNING: the following dangerous and experimental features are enabled: * updated 2016-06-07 20:24:25.020009 2b4317ed1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:594: test_auth_profiles: rm -f client.xx.keyring client.xx.keyring.2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:25.094362 2b6a4e43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:25.094372 2b6a4e43e700 0 lockdep start 2016-06-07 20:24:25.102632 2b6a4e43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:25.162378 2b6a4e43e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_misc /home/yuriw/ceph/qa/workunits/cephtool/test.sh:621: test_mon_misc: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:621: test_mon_misc: grep '^epoch' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:25.224288 2b994703b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:25.224293 2b994703b700 0 lockdep start 2016-06-07 20:24:25.230172 2b994703b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:25.315368 2b994703b700 0 lockdep stop epoch 135 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:622: test_mon_misc: ceph --concise osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:622: test_mon_misc: grep '^epoch' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:25.378590 2b8e59eca700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:25.383865 2b8e59eca700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:25.545213 2b8e59eca700 0 lockdep stop epoch 135 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:625: test_mon_misc: ceph df *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:25.630051 2ac9bf4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:25.638174 2ac9bf4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:27.133041 2ac9bf4a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:626: test_mon_misc: grep GLOBAL /tmp/cephtool19083/test_invalid.19083 GLOBAL: /home/yuriw/ceph/qa/workunits/cephtool/test.sh:627: test_mon_misc: grep -v DIRTY /tmp/cephtool19083/test_invalid.19083 GLOBAL: SIZE AVAIL RAW USED %RAW USED 2750G 2523G 88838M 3.15 POOLS: NAME ID USED %USED MAX AVAIL OBJECTS rbd 0 0 0 841G 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:628: test_mon_misc: ceph df detail *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:27.311107 2b257243e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:27.316718 2b257243e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:27.370733 2b257243e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:629: test_mon_misc: grep CATEGORY /tmp/cephtool19083/test_invalid.19083 NAME ID CATEGORY QUOTA OBJECTS QUOTA BYTES USED %USED MAX AVAIL OBJECTS DIRTY READ WRITE RAW USED /home/yuriw/ceph/qa/workunits/cephtool/test.sh:630: test_mon_misc: grep DIRTY /tmp/cephtool19083/test_invalid.19083 NAME ID CATEGORY QUOTA OBJECTS QUOTA BYTES USED %USED MAX AVAIL OBJECTS DIRTY READ WRITE RAW USED /home/yuriw/ceph/qa/workunits/cephtool/test.sh:631: test_mon_misc: ceph df --format json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:27.458546 2b80b7ddf700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:27.464341 2b80b7ddf700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.040678 2b80b7ddf700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:632: test_mon_misc: grep total_bytes /tmp/cephtool19083/test_invalid.19083 {"stats":{"total_bytes":2953119227904,"total_used_bytes":92821909504,"total_avail_bytes":2710216527872},"pools":[{"name":"rbd","id":0,"stats":{"kb_used":0,"bytes_used":0,"max_avail":903310886615,"objects":0}}]} /home/yuriw/ceph/qa/workunits/cephtool/test.sh:633: test_mon_misc: grep -v dirty /tmp/cephtool19083/test_invalid.19083 {"stats":{"total_bytes":2953119227904,"total_used_bytes":92821909504,"total_avail_bytes":2710216527872},"pools":[{"name":"rbd","id":0,"stats":{"kb_used":0,"bytes_used":0,"max_avail":903310886615,"objects":0}}]} /home/yuriw/ceph/qa/workunits/cephtool/test.sh:634: test_mon_misc: ceph df detail --format json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:28.204470 2b6e8c374700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.204476 2b6e8c374700 0 lockdep start 2016-06-07 20:24:28.210218 2b6e8c374700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.264703 2b6e8c374700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:635: test_mon_misc: grep rd_bytes /tmp/cephtool19083/test_invalid.19083 {"stats":{"total_bytes":2953119227904,"total_used_bytes":92821909504,"total_avail_bytes":2710216527872,"total_objects":0},"pools":[{"name":"rbd","id":0,"stats":{"kb_used":0,"bytes_used":0,"max_avail":903310886615,"objects":0,"quota_objects":0,"quota_bytes":0,"dirty":0,"rd":0,"rd_bytes":0,"wr":0,"wr_bytes":0,"raw_bytes_used":0}}]} /home/yuriw/ceph/qa/workunits/cephtool/test.sh:636: test_mon_misc: grep dirty /tmp/cephtool19083/test_invalid.19083 {"stats":{"total_bytes":2953119227904,"total_used_bytes":92821909504,"total_avail_bytes":2710216527872,"total_objects":0},"pools":[{"name":"rbd","id":0,"stats":{"kb_used":0,"bytes_used":0,"max_avail":903310886615,"objects":0,"quota_objects":0,"quota_bytes":0,"dirty":0,"rd":0,"rd_bytes":0,"wr":0,"wr_bytes":0,"raw_bytes_used":0}}]} /home/yuriw/ceph/qa/workunits/cephtool/test.sh:637: test_mon_misc: ceph df --format xml /home/yuriw/ceph/qa/workunits/cephtool/test.sh:637: test_mon_misc: grep '' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:28.327936 2aac4ba04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.327940 2aac4ba04700 0 lockdep start 2016-06-07 20:24:28.331496 2aac4ba04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.385319 2aac4ba04700 0 lockdep stop 2953119227904928219095042710216527872rbd0009033108866150 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:638: test_mon_misc: ceph df detail --format xml /home/yuriw/ceph/qa/workunits/cephtool/test.sh:638: test_mon_misc: grep '' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:28.445689 2ae46ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.445693 2ae46ad61700 0 lockdep start 2016-06-07 20:24:28.451288 2ae46ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.504764 2ae46ad61700 0 lockdep stop 29531192279049282190950427102165278720rbd000903310886615000000000 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:640: test_mon_misc: ceph fsid *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:28.561479 2b22b43b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:28.565099 2b22b43b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 8fedb721-e6d4-42fc-a3ae-49153fde812f 2016-06-07 20:24:29.289341 2b22b43b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:641: test_mon_misc: ceph health *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:29.381180 2b63434e4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:29.381184 2b63434e4700 0 lockdep start 2016-06-07 20:24:29.387967 2b63434e4700 -1 WARNING: the following dangerous and experimental features are enabled: * HEALTH_OK 2016-06-07 20:24:29.957780 2b63434e4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:642: test_mon_misc: ceph health detail *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:30.034392 2b9354b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:30.043192 2b9354b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * HEALTH_OK 2016-06-07 20:24:30.100184 2b9354b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:643: test_mon_misc: ceph health --format json-pretty *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:30.171996 2b3d8a43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:30.172000 2b3d8a43e700 0 lockdep start 2016-06-07 20:24:30.175717 2b3d8a43e700 -1 WARNING: the following dangerous and experimental features are enabled: * { "health": { "health_services": [ { "mons": [ { "name": "a", "kb_total": 961301832, "kb_used": 30326148, "kb_avail": 882121260, "avail_percent": 91, "last_updated": "2016-06-07 20:23:57.208357", "store_stats": { "bytes_total": 14864242, "bytes_sst": 0, "bytes_log": 14863129, "bytes_misc": 1113, "last_updated": "0.000000" }, "health": "HEALTH_OK" }, { "name": "b", "kb_total": 961301832, "kb_used": 30326276, "kb_avail": 882121132, "avail_percent": 91, "last_updated": "2016-06-07 20:23:58.107584", "store_stats": { "bytes_total": 22356194, "bytes_sst": 0, "bytes_log": 22355081, "bytes_misc": 1113, "last_updated": "0.000000" }, "health": "HEALTH_OK" }, { "name": "c", "kb_total": 961301832, "kb_used": 30326532, "kb_avail": 882120876, "avail_percent": 91, "last_updated": "2016-06-07 20:23:58.562018", "store_stats": { "bytes_total": 22450497, "bytes_sst": 0, "bytes_log": 22449384, "bytes_misc": 1113, "last_updated": "0.000000" }, "health": "HEALTH_OK" } ] } ] }, "timechecks": { "epoch": 6, "round": 4, "round_status": "finished", "mons": [ { "name": "a", "skew": 0.000000, "latency": 0.000000, "health": "HEALTH_OK" }, { "name": "b", "skew": 0.000000, "latency": 0.253026, "health": "HEALTH_OK" }, { "name": "c", "skew": 0.000000, "latency": 0.252921, "health": "HEALTH_OK" } ] }, "summary": [], "overall_status": "HEALTH_OK", "detail": [] } 2016-06-07 20:24:31.550276 2b3d8a43e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:644: test_mon_misc: ceph health detail --format xml-pretty *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:31.625507 2abb03a04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:31.625512 2abb03a04700 0 lockdep start 2016-06-07 20:24:31.629113 2abb03a04700 -1 WARNING: the following dangerous and experimental features are enabled: * a 961301832 30326148 882121260 91 2016-06-07 20:23:57.208357 14864242 0 14863129 1113 0.000000 HEALTH_OK b 961301832 30326276 882121132 91 2016-06-07 20:23:58.107584 22356194 0 22355081 1113 0.000000 HEALTH_OK c 961301832 30326532 882120876 91 2016-06-07 20:23:58.562018 22450497 0 22449384 1113 0.000000 HEALTH_OK 6 4 finished a 0 0 HEALTH_OK b 0 0.253026 HEALTH_OK c 0 0.252921 HEALTH_OK HEALTH_OK 2016-06-07 20:24:31.845825 2abb03a04700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:646: test_mon_misc: ceph node ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:31.917548 2af777fe6700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:31.924161 2af777fe6700 -1 WARNING: the following dangerous and experimental features are enabled: * { "mon": { "smithi049": [ 0, 1, 2 ] }, "osd": { "smithi049": [ 0, 1, 2 ] } } 2016-06-07 20:24:32.914407 2af777fe6700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:647: test_mon_misc: for t in mon osd mds /home/yuriw/ceph/qa/workunits/cephtool/test.sh:648: test_mon_misc: ceph node ls mon *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:32.985264 2b7971576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:32.991024 2b7971576700 -1 WARNING: the following dangerous and experimental features are enabled: * { "smithi049": [ 0, 1, 2 ] } 2016-06-07 20:24:33.160748 2b7971576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:647: test_mon_misc: for t in mon osd mds /home/yuriw/ceph/qa/workunits/cephtool/test.sh:648: test_mon_misc: ceph node ls osd *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:33.232967 2b8cdf423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:33.238983 2b8cdf423700 -1 WARNING: the following dangerous and experimental features are enabled: * { "smithi049": [ 0, 1, 2 ] } 2016-06-07 20:24:34.110200 2b8cdf423700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:647: test_mon_misc: for t in mon osd mds /home/yuriw/ceph/qa/workunits/cephtool/test.sh:648: test_mon_misc: ceph node ls mds *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:34.193140 2b4b2ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:34.199064 2b4b2ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:34.497283 2b4b2ad61700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:651: test_mon_misc: ceph_watch_start /home/yuriw/ceph/qa/workunits/cephtool/test.sh:146: ceph_watch_start: local whatch_opt=--watch /home/yuriw/ceph/qa/workunits/cephtool/test.sh:148: ceph_watch_start: '[' -n '' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:152: ceph_watch_start: CEPH_WATCH_FILE=/tmp/cephtool19083/CEPH_WATCH_19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:154: ceph_watch_start: CEPH_WATCH_PID=27844 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:153: ceph_watch_start: ceph --watch //home/yuriw/ceph/qa/workunits/cephtool/test.sh:158: ceph_watch_start: seq 3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:158: ceph_watch_start: for i in '`seq 3`' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:159: ceph_watch_start: grep -q cluster /tmp/cephtool19083/CEPH_WATCH_19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:160: ceph_watch_start: sleep 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:34.594124 2b255c026700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:34.594129 2b255c026700 0 lockdep start 2016-06-07 20:24:34.597700 2b255c026700 -1 WARNING: the following dangerous and experimental features are enabled: * /home/yuriw/ceph/qa/workunits/cephtool/test.sh:158: ceph_watch_start: for i in '`seq 3`' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:159: ceph_watch_start: grep -q cluster /tmp/cephtool19083/CEPH_WATCH_19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:159: ceph_watch_start: break //home/yuriw/ceph/qa/workunits/cephtool/test.sh:652: test_mon_misc: date /home/yuriw/ceph/qa/workunits/cephtool/test.sh:652: test_mon_misc: mymsg='this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:653: test_mon_misc: ceph log 'this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:35.616020 2b5ee309f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:35.623016 2b5ee309f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:37.070284 2b5ee309f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:654: test_mon_misc: ceph_watch_wait 'this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:166: ceph_watch_wait: local 'regexp=this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:167: ceph_watch_wait: local timeout=30 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:169: ceph_watch_wait: '[' -n '' ']' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:173: ceph_watch_wait: seq 30 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:173: ceph_watch_wait: for i in '`seq ${timeout}`' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:174: ceph_watch_wait: grep -q 'this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016' /tmp/cephtool19083/CEPH_WATCH_19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:174: ceph_watch_wait: break /home/yuriw/ceph/qa/workunits/cephtool/test.sh:178: ceph_watch_wait: kill 27844 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:180: ceph_watch_wait: grep 'this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016' /tmp/cephtool19083/CEPH_WATCH_19083 2016-06-07 20:24:36.536932 mon.1 [INF] from='client.? 127.0.0.1:0/1073377606' entity='client.admin' cmd=[{"prefix": "log", "logtext": ["this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016"]}]: dispatch 2016-06-07 20:24:36.537062 client.4354 [INF] this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016 2016-06-07 20:24:36.537267 mon.0 [INF] from='client.4354 :/0' entity='client.admin' cmd=[{"prefix": "log", "logtext": ["this is a test log message 19083.Tue Jun 7 20:24:35 UTC 2016"]}]: dispatch /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 165: 27844 Terminated ceph $whatch_opt > $CEPH_WATCH_FILE /home/yuriw/ceph/qa/workunits/cephtool/test.sh:656: test_mon_misc: ceph mon metadata a *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:37.178400 2b5f85576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:37.178405 2b5f85576700 0 lockdep start 2016-06-07 20:24:37.184953 2b5f85576700 -1 WARNING: the following dangerous and experimental features are enabled: * { "arch": "x86_64", "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", "distro": "Ubuntu", "distro_codename": "trusty", "distro_description": "Ubuntu 14.04.4 LTS", "distro_version": "14.04", "hostname": "smithi049", "kernel_description": "#133-Ubuntu SMP Tue May 24 18:32:09 UTC 2016", "kernel_version": "3.13.0-87-generic", "mem_swap_kb": "0", "mem_total_kb": "32839832", "os": "Linux" } 2016-06-07 20:24:38.246904 2b5f85576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:657: test_mon_misc: ceph mon metadata *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:38.323018 2b1fe0434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:38.323024 2b1fe0434700 0 lockdep start 2016-06-07 20:24:38.332148 2b1fe0434700 -1 WARNING: the following dangerous and experimental features are enabled: * [ { "name": "a", "arch": "x86_64", "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", "distro": "Ubuntu", "distro_codename": "trusty", "distro_description": "Ubuntu 14.04.4 LTS", "distro_version": "14.04", "hostname": "smithi049", "kernel_description": "#133-Ubuntu SMP Tue May 24 18:32:09 UTC 2016", "kernel_version": "3.13.0-87-generic", "mem_swap_kb": "0", "mem_total_kb": "32839832", "os": "Linux" }, { "name": "b", "arch": "x86_64", "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", "distro": "Ubuntu", "distro_codename": "trusty", "distro_description": "Ubuntu 14.04.4 LTS", "distro_version": "14.04", "hostname": "smithi049", "kernel_description": "#133-Ubuntu SMP Tue May 24 18:32:09 UTC 2016", "kernel_version": "3.13.0-87-generic", "mem_swap_kb": "0", "mem_total_kb": "32839832", "os": "Linux" }, { "name": "c", "arch": "x86_64", "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", "distro": "Ubuntu", "distro_codename": "trusty", "distro_description": "Ubuntu 14.04.4 LTS", "distro_version": "14.04", "hostname": "smithi049", "kernel_description": "#133-Ubuntu SMP Tue May 24 18:32:09 UTC 2016", "kernel_version": "3.13.0-87-generic", "mem_swap_kb": "0", "mem_total_kb": "32839832", "os": "Linux" } ] 2016-06-07 20:24:38.474365 2b1fe0434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:658: test_mon_misc: ceph node ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:38.566965 2b6378f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:38.570516 2b6378f79700 -1 WARNING: the following dangerous and experimental features are enabled: * { "mon": { "smithi049": [ 0, 1, 2 ] }, "osd": { "smithi049": [ 0, 1, 2 ] } } 2016-06-07 20:24:38.790853 2b6378f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:38.865817 2ad7842f4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:38.872107 2ad7842f4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:38.942124 2ad7842f4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_mon /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1022: test_mon_mon: ceph --help mon *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** General usage: ============== usage: ceph [-h] [-c CEPHCONF] [-i INPUT_FILE] [-o OUTPUT_FILE] [--id CLIENT_ID] [--name CLIENT_NAME] [--cluster CLUSTER] [--admin-daemon ADMIN_SOCKET] [--admin-socket ADMIN_SOCKET_NOPE] [-s] [-w] [--watch-debug] [--watch-info] [--watch-sec] [--watch-warn] [--watch-error] [--version] [--verbose] [--concise] [-f {json,json-pretty,xml,xml-pretty,plain}] [--connect-timeout CLUSTER_TIMEOUT] Ceph administration tool optional arguments: -h, --help request mon help -c CEPHCONF, --conf CEPHCONF ceph configuration file -i INPUT_FILE, --in-file INPUT_FILE input file -o OUTPUT_FILE, --out-file OUTPUT_FILE output file --id CLIENT_ID, --user CLIENT_ID client id for authentication --name CLIENT_NAME, -n CLIENT_NAME client name for authentication --cluster CLUSTER cluster name --admin-daemon ADMIN_SOCKET submit admin-socket commands ("help" for help --admin-socket ADMIN_SOCKET_NOPE you probably mean --admin-daemon -s, --status show cluster status -w, --watch watch live cluster changes --watch-debug watch debug events --watch-info watch info events --watch-sec watch security events --watch-warn watch warn events --watch-error watch error events --version, -v display version --verbose make verbose --concise make less verbose -f {json,json-pretty,xml,xml-pretty,plain}, --format {json,json-pretty,xml,xml-pretty,plain} --connect-timeout CLUSTER_TIMEOUT set a timeout for connecting to the cluster 2016-06-07 20:24:39.054332 2af3883b4700 -1 WARNING: the following dangerous and experimental features are enabled: * Monitor commands: ================= [Contacting monitor, timeout after 5 seconds] 2016-06-07 20:24:39.057350 2af3883b4700 -1 WARNING: the following dangerous and experimental features are enabled: * mon add add new monitor named at mon compact cause compaction of monitor's leveldb storage mon dump {} dump formatted monmap (optionally from epoch) mon getmap {} get monmap mon metadata {} fetch metadata for mon mon remove remove monitor named mon rm remove monitor named mon scrub scrub the monitor stores mon stat summarize monitor status mon sync force {--yes-i-really-mean-it} force sync of and clear monitor store {--i-know-what-i-am-doing} mon_status report status of monitors 2016-06-07 20:24:39.810373 2af3883b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1024: test_mon_mon: ceph mon dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:39.884391 2b7becd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:39.890324 2b7becd71700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped monmap epoch 1 epoch 1 fsid 8fedb721-e6d4-42fc-a3ae-49153fde812f last_changed 2016-06-07 20:15:54.015730 created 2016-06-07 20:15:54.015730 0: 127.0.0.1:7202/0 mon.a 1: 127.0.0.1:7203/0 mon.b 2: 127.0.0.1:7204/0 mon.c 2016-06-07 20:24:41.338270 2b7becd71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1025: test_mon_mon: ceph mon getmap -o /tmp/cephtool19083/monmap.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:41.422670 2b150f278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:41.431300 2b150f278700 -1 WARNING: the following dangerous and experimental features are enabled: * got monmap epoch 1 2016-06-07 20:24:41.503001 2b150f278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1026: test_mon_mon: '[' -s /tmp/cephtool19083/monmap.19083 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1028: test_mon_mon: ceph mon_status *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:41.570781 2b2fbf278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:41.784228 2b2fbf278700 -1 WARNING: the following dangerous and experimental features are enabled: * {"name":"c","rank":2,"state":"peon","election_epoch":6,"quorum":[0,1,2],"outside_quorum":[],"extra_probe_peers":[],"sync_provider":[],"monmap":{"epoch":1,"fsid":"8fedb721-e6d4-42fc-a3ae-49153fde812f","modified":"2016-06-07 20:15:54.015730","created":"2016-06-07 20:15:54.015730","mons":[{"rank":0,"name":"a","addr":"127.0.0.1:7202\/0"},{"rank":1,"name":"b","addr":"127.0.0.1:7203\/0"},{"rank":2,"name":"c","addr":"127.0.0.1:7204\/0"}]}} 2016-06-07 20:24:43.268769 2b2fbf278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:43.337488 2ac9baf7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:43.337493 2ac9baf7a700 0 lockdep start 2016-06-07 20:24:43.343039 2ac9baf7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:43.402876 2ac9baf7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_osd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1036: test_mon_osd: bl=192.168.0.1:0/1000 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1038: test_mon_osd: bl_json='192.168.0.1:0\\/1000' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1039: test_mon_osd: ceph osd blacklist add 192.168.0.1:0/1000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:43.475923 2b187643e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:43.475927 2b187643e700 0 lockdep start 2016-06-07 20:24:43.481538 2b187643e700 -1 WARNING: the following dangerous and experimental features are enabled: * blacklisting 192.168.0.1:0/1000 until 2016-06-07 21:24:43.890178 (3600 sec) 2016-06-07 20:24:44.506038 2b187643e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1040: test_mon_osd: ceph osd blacklist ls /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1040: test_mon_osd: grep 192.168.0.1:0/1000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:44.578408 2b7b0cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:44.578414 2b7b0cd71700 0 lockdep start 2016-06-07 20:24:44.583639 2b7b0cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * listed 1 entries 2016-06-07 20:24:45.101811 2b7b0cd71700 0 lockdep stop 192.168.0.1:0/1000 2016-06-07 21:24:43.890178 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1041: test_mon_osd: ceph osd blacklist ls --format=json-pretty /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1041: test_mon_osd: grep '192.168.0.1:0\\/1000' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:45.167843 2ac1ad355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:45.176702 2ac1ad355700 -1 WARNING: the following dangerous and experimental features are enabled: * listed 1 entries 2016-06-07 20:24:45.235759 2ac1ad355700 0 lockdep stop "addr": "192.168.0.1:0\/1000", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1042: test_mon_osd: ceph osd dump --format=json-pretty /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1042: test_mon_osd: grep 192.168.0.1:0/1000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:45.304126 2b0256859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:45.304132 2b0256859700 0 lockdep start 2016-06-07 20:24:45.312700 2b0256859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:45.954731 2b0256859700 0 lockdep stop "192.168.0.1:0/1000": "2016-06-07 21:24:43.890178" /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1043: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1043: test_mon_osd: grep '^blacklist 192.168.0.1:0/1000' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:46.027259 2b9b41355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:46.036573 2b9b41355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:46.304510 2b9b41355700 0 lockdep stop blacklist 192.168.0.1:0/1000 expires 2016-06-07 21:24:43.890178 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1044: test_mon_osd: ceph osd blacklist rm 192.168.0.1:0/1000 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:46.377341 2b77a3a44700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:46.377348 2b77a3a44700 0 lockdep start 2016-06-07 20:24:46.387559 2b77a3a44700 -1 WARNING: the following dangerous and experimental features are enabled: * un-blacklisting 192.168.0.1:0/1000 2016-06-07 20:24:47.610325 2b77a3a44700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1045: test_mon_osd: expect_false 'ceph osd blacklist ls | grep 192.168.0.1:0/1000' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd blacklist ls | grep 192.168.0.1:0/1000' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd blacklist ls | grep 192.168.0.1:0/1000: No such file or directory /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1047: test_mon_osd: bl=192.168.0.1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1049: test_mon_osd: ceph osd blacklist add 192.168.0.1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:47.688375 2b237cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:47.688380 2b237cb6c700 0 lockdep start 2016-06-07 20:24:47.694303 2b237cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * blacklisting 192.168.0.1:0/0 until 2016-06-07 21:24:49.183240 (3600 sec) 2016-06-07 20:24:50.131737 2b237cb6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1050: test_mon_osd: ceph osd blacklist ls /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1050: test_mon_osd: grep 192.168.0.1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:50.211175 2b09e3fe6700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:50.211181 2b09e3fe6700 0 lockdep start 2016-06-07 20:24:50.217004 2b09e3fe6700 -1 WARNING: the following dangerous and experimental features are enabled: * listed 1 entries 2016-06-07 20:24:51.636229 2b09e3fe6700 0 lockdep stop 192.168.0.1:0/0 2016-06-07 21:24:49.183240 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1051: test_mon_osd: ceph osd blacklist rm 192.168.0.1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:51.712679 2acc2a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:51.725558 2acc2a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * un-blacklisting 192.168.0.1:0/0 2016-06-07 20:24:53.871932 2acc2a91a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1052: test_mon_osd: expect_false 'ceph osd blacklist ls | grep 192.168.0.1' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd blacklist ls | grep 192.168.0.1' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd blacklist ls | grep 192.168.0.1: command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1053: test_mon_osd: expect_false 'ceph osd blacklist 192.168.0.1/-1' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd blacklist 192.168.0.1/-1' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd blacklist 192.168.0.1/-1: No such file or directory /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1054: test_mon_osd: expect_false 'ceph osd blacklist 192.168.0.1/foo' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd blacklist 192.168.0.1/foo' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd blacklist 192.168.0.1/foo: No such file or directory /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1057: test_mon_osd: expect_false 'ceph osd blacklist 1234.56.78.90/100' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd blacklist 1234.56.78.90/100' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd blacklist 1234.56.78.90/100: No such file or directory /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1060: test_mon_osd: ceph osd blacklist add 192.168.0.1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:53.963209 2b5f13943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:53.963214 2b5f13943700 0 lockdep start 2016-06-07 20:24:53.969140 2b5f13943700 -1 WARNING: the following dangerous and experimental features are enabled: * blacklisting 192.168.0.1:0/0 until 2016-06-07 21:24:54.946067 (3600 sec) 2016-06-07 20:24:56.235870 2b5f13943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1061: test_mon_osd: ceph osd blacklist ls /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1061: test_mon_osd: grep 192.168.0.1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:56.350500 2b40ff943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:56.350508 2b40ff943700 0 lockdep start 2016-06-07 20:24:56.358451 2b40ff943700 -1 WARNING: the following dangerous and experimental features are enabled: * listed 1 entries 2016-06-07 20:24:56.422254 2b40ff943700 0 lockdep stop 192.168.0.1:0/0 2016-06-07 21:24:54.946067 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1062: test_mon_osd: ceph osd blacklist clear *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:56.501593 2b2c2e53f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:56.501604 2b2c2e53f700 0 lockdep start 2016-06-07 20:24:56.508388 2b2c2e53f700 -1 WARNING: the following dangerous and experimental features are enabled: * removed all blacklist entries 2016-06-07 20:24:59.815014 2b2c2e53f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1063: test_mon_osd: expect_false 'ceph osd blacklist ls | grep 192.168.0.1' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd blacklist ls | grep 192.168.0.1' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd blacklist ls | grep 192.168.0.1: command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1068: test_mon_osd: ceph osd crush reweight-all *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:24:59.893722 2b31affe6700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:24:59.904388 2b31affe6700 -1 WARNING: the following dangerous and experimental features are enabled: * reweighted crush hierarchy 2016-06-07 20:25:00.681854 2b31affe6700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1069: test_mon_osd: ceph osd crush tunables legacy *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:00.765560 2b7d30f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:00.765566 2b7d30f79700 0 lockdep start 2016-06-07 20:25:00.776345 2b7d30f79700 -1 WARNING: the following dangerous and experimental features are enabled: * adjusted tunables profile to legacy 2016-06-07 20:25:03.414452 2b7d30f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1070: test_mon_osd: ceph osd crush show-tunables /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1070: test_mon_osd: grep argonaut *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:03.495305 2b337bf8e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:03.495309 2b337bf8e700 0 lockdep start 2016-06-07 20:25:03.501286 2b337bf8e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:03.580982 2b337bf8e700 0 lockdep stop "profile": "argonaut", "minimum_required_version": "argonaut", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1071: test_mon_osd: ceph osd crush tunables bobtail *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:03.662726 2b88e30b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:03.672132 2b88e30b4700 -1 WARNING: the following dangerous and experimental features are enabled: * adjusted tunables profile to bobtail 2016-06-07 20:25:05.009199 2b88e30b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1072: test_mon_osd: ceph osd crush show-tunables /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1072: test_mon_osd: grep bobtail *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:05.081207 2ba59381e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:05.081216 2ba59381e700 0 lockdep start 2016-06-07 20:25:05.090292 2ba59381e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:06.970162 2ba59381e700 0 lockdep stop "profile": "bobtail", "minimum_required_version": "bobtail", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1073: test_mon_osd: ceph osd crush tunables firefly *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:07.051352 2aded3531700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:07.057100 2aded3531700 -1 WARNING: the following dangerous and experimental features are enabled: * adjusted tunables profile to firefly 2016-06-07 20:25:08.781581 2aded3531700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1074: test_mon_osd: ceph osd crush show-tunables /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1074: test_mon_osd: grep firefly *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:08.862815 2b500c374700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:08.862820 2b500c374700 0 lockdep start 2016-06-07 20:25:08.868677 2b500c374700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:08.932156 2b500c374700 0 lockdep stop "profile": "firefly", "minimum_required_version": "firefly", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1076: test_mon_osd: ceph osd crush set-tunable straw_calc_version 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:09.006948 2b2401a34700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:09.006953 2b2401a34700 0 lockdep start 2016-06-07 20:25:09.012676 2b2401a34700 -1 WARNING: the following dangerous and experimental features are enabled: * adjusted tunable straw_calc_version to 0 2016-06-07 20:25:10.451769 2b2401a34700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1077: test_mon_osd: ceph osd crush get-tunable straw_calc_version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1077: test_mon_osd: grep 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:10.527344 2b194b75d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:10.527351 2b194b75d700 0 lockdep start 2016-06-07 20:25:10.536182 2b194b75d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:10.616644 2b194b75d700 0 lockdep stop 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1078: test_mon_osd: ceph osd crush set-tunable straw_calc_version 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:10.686504 2b873fa04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:10.686510 2b873fa04700 0 lockdep start 2016-06-07 20:25:10.692248 2b873fa04700 -1 WARNING: the following dangerous and experimental features are enabled: * adjusted tunable straw_calc_version to 1 2016-06-07 20:25:13.544147 2b873fa04700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1079: test_mon_osd: ceph osd crush get-tunable straw_calc_version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1079: test_mon_osd: grep 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:13.610258 2b5de1cfa700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:13.621151 2b5de1cfa700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:14.965230 2b5de1cfa700 0 lockdep stop 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1085: test_mon_osd: ceph osd scrub 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:15.055796 2b6b66e62700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:15.063916 2b6b66e62700 -1 WARNING: the following dangerous and experimental features are enabled: * osd.0 instructed to scrub 2016-06-07 20:25:15.129321 2b6b66e62700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1086: test_mon_osd: ceph osd deep-scrub 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:15.207330 2b3166744700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:15.207334 2b3166744700 0 lockdep start 2016-06-07 20:25:15.213016 2b3166744700 -1 WARNING: the following dangerous and experimental features are enabled: * osd.0 instructed to deep-scrub 2016-06-07 20:25:15.291025 2b3166744700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1087: test_mon_osd: ceph osd repair 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:15.376831 2ba4697d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:15.376837 2ba4697d5700 0 lockdep start 2016-06-07 20:25:15.383242 2ba4697d5700 -1 WARNING: the following dangerous and experimental features are enabled: * osd.0 instructed to repair 2016-06-07 20:25:16.530982 2ba4697d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set noup *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:16.608859 2b7bc119a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:16.615867 2b7bc119a700 -1 WARNING: the following dangerous and experimental features are enabled: * set noup 2016-06-07 20:25:17.385507 2b7bc119a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset noup *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:17.461773 2b94f3e1d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:17.473630 2b94f3e1d700 -1 WARNING: the following dangerous and experimental features are enabled: * unset noup 2016-06-07 20:25:20.050067 2b94f3e1d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set nodown *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:20.136796 2ae9eef7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:20.136803 2ae9eef7a700 0 lockdep start 2016-06-07 20:25:20.145296 2ae9eef7a700 -1 WARNING: the following dangerous and experimental features are enabled: * set nodown 2016-06-07 20:25:22.737602 2ae9eef7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset nodown *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:22.835318 2ad91e00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:22.835325 2ad91e00d700 0 lockdep start 2016-06-07 20:25:22.844966 2ad91e00d700 -1 WARNING: the following dangerous and experimental features are enabled: * unset nodown 2016-06-07 20:25:24.832430 2ad91e00d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set noin *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:24.917465 2b6322f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:24.917472 2b6322f7a700 0 lockdep start 2016-06-07 20:25:24.931776 2b6322f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * set noin 2016-06-07 20:25:28.421508 2b6322f7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset noin *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:28.503330 2afac3ece700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:28.503337 2afac3ece700 0 lockdep start 2016-06-07 20:25:28.512379 2afac3ece700 -1 WARNING: the following dangerous and experimental features are enabled: * unset noin 2016-06-07 20:25:30.850679 2afac3ece700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set noout *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:30.927668 2b0baf4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:30.939210 2b0baf4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * set noout 2016-06-07 20:25:31.743121 2b0baf4a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset noout *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:31.822885 2ab792859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:31.834634 2ab792859700 -1 WARNING: the following dangerous and experimental features are enabled: * unset noout 2016-06-07 20:25:34.862807 2ab792859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set noscrub *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:34.949156 2b66541f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:34.955135 2b66541f3700 -1 WARNING: the following dangerous and experimental features are enabled: * set noscrub 2016-06-07 20:25:37.752113 2b66541f3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset noscrub *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:37.835756 2b8c16252700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:37.835762 2b8c16252700 0 lockdep start 2016-06-07 20:25:37.841286 2b8c16252700 -1 WARNING: the following dangerous and experimental features are enabled: * unset noscrub 2016-06-07 20:25:39.950819 2b8c16252700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set nodeep-scrub *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:40.032529 2b1cc1cba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:40.041142 2b1cc1cba700 -1 WARNING: the following dangerous and experimental features are enabled: * set nodeep-scrub 2016-06-07 20:25:42.644348 2b1cc1cba700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset nodeep-scrub *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:42.745806 2b3d63b86700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:42.753072 2b3d63b86700 -1 WARNING: the following dangerous and experimental features are enabled: * unset nodeep-scrub 2016-06-07 20:25:44.140820 2b3d63b86700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set nobackfill *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:44.229456 2b5330434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:44.229460 2b5330434700 0 lockdep start 2016-06-07 20:25:44.235554 2b5330434700 -1 WARNING: the following dangerous and experimental features are enabled: * set nobackfill 2016-06-07 20:25:46.824605 2b5330434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset nobackfill *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:46.901010 2b21afe2f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:46.910027 2b21afe2f700 -1 WARNING: the following dangerous and experimental features are enabled: * unset nobackfill 2016-06-07 20:25:51.060321 2b21afe2f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set norebalance *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:51.136099 2b8cca2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:51.136106 2b8cca2d3700 0 lockdep start 2016-06-07 20:25:51.143549 2b8cca2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * set norebalance 2016-06-07 20:25:51.922889 2b8cca2d3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset norebalance *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:52.004854 2ac3f34a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:52.004859 2ac3f34a4700 0 lockdep start 2016-06-07 20:25:52.011186 2ac3f34a4700 -1 WARNING: the following dangerous and experimental features are enabled: * unset norebalance 2016-06-07 20:25:54.901959 2ac3f34a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set norecover *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:54.978797 2b01f7c19700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:54.978802 2b01f7c19700 0 lockdep start 2016-06-07 20:25:54.984737 2b01f7c19700 -1 WARNING: the following dangerous and experimental features are enabled: * set norecover 2016-06-07 20:25:58.728884 2b01f7c19700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset norecover *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:25:58.816979 2ac2edeca700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:25:58.831432 2ac2edeca700 -1 WARNING: the following dangerous and experimental features are enabled: * unset norecover 2016-06-07 20:26:01.399146 2ac2edeca700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set notieragent *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:01.481578 2aca0bd5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:01.481582 2aca0bd5c700 0 lockdep start 2016-06-07 20:26:01.491673 2aca0bd5c700 -1 WARNING: the following dangerous and experimental features are enabled: * set notieragent 2016-06-07 20:26:03.273618 2aca0bd5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset notieragent *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:03.351524 2b89c5355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:03.364911 2b89c5355700 -1 WARNING: the following dangerous and experimental features are enabled: * unset notieragent 2016-06-07 20:26:04.085722 2b89c5355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set full *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:04.154564 2acb52a59700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:04.161591 2acb52a59700 -1 WARNING: the following dangerous and experimental features are enabled: * set full 2016-06-07 20:26:05.148693 2acb52a59700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset full *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:05.230429 2aeab1cba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:05.230435 2aeab1cba700 0 lockdep start 2016-06-07 20:26:05.240716 2aeab1cba700 -1 WARNING: the following dangerous and experimental features are enabled: * unset full 2016-06-07 20:26:08.734017 2aeab1cba700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1089: test_mon_osd: for f in noup nodown noin noout noscrub nodeep-scrub nobackfill norebalance norecover notieragent full sortbitwise /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1091: test_mon_osd: ceph osd set sortbitwise *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:08.830253 2b4917c79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:08.843848 2b4917c79700 -1 WARNING: the following dangerous and experimental features are enabled: * set sortbitwise 2016-06-07 20:26:09.823485 2b4917c79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1092: test_mon_osd: ceph osd unset sortbitwise *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:09.900776 2b35d3ace700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:09.900781 2b35d3ace700 0 lockdep start 2016-06-07 20:26:09.907514 2b35d3ace700 -1 WARNING: the following dangerous and experimental features are enabled: * unset sortbitwise 2016-06-07 20:26:10.472581 2b35d3ace700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1094: test_mon_osd: ceph osd set sortbitwise *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:10.561280 2af21f943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:10.567015 2af21f943700 -1 WARNING: the following dangerous and experimental features are enabled: * set sortbitwise 2016-06-07 20:26:13.241648 2af21f943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1095: test_mon_osd: expect_false ceph osd set bogus /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd set bogus *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:13.328409 2b9e374a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:13.328416 2b9e374a4700 0 lockdep start 2016-06-07 20:26:13.333898 2b9e374a4700 -1 WARNING: the following dangerous and experimental features are enabled: * Invalid command: bogus not in full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|sortbitwise|require_jewel_osds osd set full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|sortbitwise|require_jewel_osds : set Error EINVAL: invalid command 2016-06-07 20:26:13.388333 2b9e374a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1096: test_mon_osd: expect_false ceph osd unset bogus /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd unset bogus *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:13.469184 2b988e899700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:13.481374 2b988e899700 -1 WARNING: the following dangerous and experimental features are enabled: * Invalid command: bogus not in full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|sortbitwise osd unset full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|sortbitwise : unset Error EINVAL: invalid command 2016-06-07 20:26:14.554111 2b988e899700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1097: test_mon_osd: ceph osd set require_jewel_osds *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:14.639430 2b5d0b09f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:14.639440 2b5d0b09f700 0 lockdep start 2016-06-07 20:26:14.651551 2b5d0b09f700 -1 WARNING: the following dangerous and experimental features are enabled: * set require_jewel_osds 2016-06-07 20:26:16.885228 2b5d0b09f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1098: test_mon_osd: expect_false ceph osd unset require_jewel_osds /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd unset require_jewel_osds *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:16.970142 2afef6f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:16.970147 2afef6f7a700 0 lockdep start 2016-06-07 20:26:16.980530 2afef6f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * Invalid command: require_jewel_osds not in full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|sortbitwise osd unset full|pause|noup|nodown|noout|noin|nobackfill|norebalance|norecover|noscrub|nodeep-scrub|notieragent|sortbitwise : unset Error EINVAL: invalid command 2016-06-07 20:26:17.806592 2afef6f7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1100: test_mon_osd: ceph osd set noup *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:17.882785 2af019cba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:17.882790 2af019cba700 0 lockdep start 2016-06-07 20:26:17.888756 2af019cba700 -1 WARNING: the following dangerous and experimental features are enabled: * set noup 2016-06-07 20:26:18.367766 2af019cba700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1101: test_mon_osd: ceph osd down 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:18.453541 2b9cab75d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:18.466060 2b9cab75d700 -1 WARNING: the following dangerous and experimental features are enabled: * marked down osd.0. 2016-06-07 20:26:19.486741 2b9cab75d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1102: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1102: test_mon_osd: grep 'osd.0 down' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:19.578486 2b4d5f1b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:19.585396 2b4d5f1b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:20.631278 2b4d5f1b7700 0 lockdep stop osd.0 down in weight 1 up_from 6 up_thru 175 down_at 179 last_clean_interval [0,0) 127.0.0.1:6832/28826 127.0.0.1:6833/28826 127.0.0.1:6834/28826 127.0.0.1:6835/28826 exists c8e248f1-e9df-405e-b941-418201377eef /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1103: test_mon_osd: ceph osd unset noup *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:20.715016 2ace3b71d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:20.715021 2ace3b71d700 0 lockdep start 2016-06-07 20:26:20.724196 2ace3b71d700 -1 WARNING: the following dangerous and experimental features are enabled: * unset noup 2016-06-07 20:26:22.614284 2ace3b71d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1104: test_mon_osd: max_run=1000 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i=0 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:22.692298 2b44359de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:22.695727 2b44359de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:23.249821 2b44359de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (0/1000)' waiting for osd.0 to come back up (0/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:24.325907 2b7737fe6700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:24.332650 2b7737fe6700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:24.390919 2b7737fe6700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (1/1000)' waiting for osd.0 to come back up (1/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:25.468858 2b0ba0d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:25.474734 2b0ba0d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:25.556486 2b0ba0d71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (2/1000)' waiting for osd.0 to come back up (2/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:26.645616 2b359e43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:26.649455 2b359e43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:26.712133 2b359e43e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (3/1000)' waiting for osd.0 to come back up (3/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:27.804262 2b37d119a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:27.811688 2b37d119a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:27.871769 2b37d119a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (4/1000)' waiting for osd.0 to come back up (4/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:28.963880 2b2055815700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:28.963885 2b2055815700 0 lockdep start 2016-06-07 20:26:29.030656 2b2055815700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:29.094972 2b2055815700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (5/1000)' waiting for osd.0 to come back up (5/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:30.172144 2b90dea48700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:30.172149 2b90dea48700 0 lockdep start 2016-06-07 20:26:30.175670 2b90dea48700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:30.235476 2b90dea48700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (6/1000)' waiting for osd.0 to come back up (6/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:31.313449 2ba096d61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:31.322010 2ba096d61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:31.786636 2ba096d61700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (7/1000)' waiting for osd.0 to come back up (7/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:32.857365 2ac39f983700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:32.864331 2ac39f983700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:32.922947 2ac39f983700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (8/1000)' waiting for osd.0 to come back up (8/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:34.012139 2aac6fccb700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:34.019624 2aac6fccb700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:34.076700 2aac6fccb700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (9/1000)' waiting for osd.0 to come back up (9/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:35.172546 2b36d60ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:35.172554 2b36d60ce700 0 lockdep start 2016-06-07 20:26:35.180854 2b36d60ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:35.236697 2b36d60ce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (10/1000)' waiting for osd.0 to come back up (10/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:36.330819 2b08ea2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:36.330824 2b08ea2d3700 0 lockdep start 2016-06-07 20:26:36.338693 2b08ea2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:36.406391 2b08ea2d3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (11/1000)' waiting for osd.0 to come back up (11/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:37.490005 2ada76859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:37.490010 2ada76859700 0 lockdep start 2016-06-07 20:26:37.493724 2ada76859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:37.559541 2ada76859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (12/1000)' waiting for osd.0 to come back up (12/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:38.637451 2b860a744700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:38.637456 2b860a744700 0 lockdep start 2016-06-07 20:26:38.643420 2b860a744700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:38.698885 2b860a744700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (13/1000)' waiting for osd.0 to come back up (13/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:39.770592 2ad375cba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:39.770597 2ad375cba700 0 lockdep start 2016-06-07 20:26:39.776372 2ad375cba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:39.831313 2ad375cba700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (14/1000)' waiting for osd.0 to come back up (14/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:40.912430 2b8f6e899700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:40.916110 2b8f6e899700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:42.910874 2b8f6e899700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (15/1000)' waiting for osd.0 to come back up (15/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:43.988401 2b276d9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:43.988406 2b276d9de700 0 lockdep start 2016-06-07 20:26:43.994162 2b276d9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:44.054560 2b276d9de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (16/1000)' waiting for osd.0 to come back up (16/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:45.139369 2b822f5b2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:45.139373 2b822f5b2700 0 lockdep start 2016-06-07 20:26:45.143147 2b822f5b2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:46.386340 2b822f5b2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (17/1000)' waiting for osd.0 to come back up (17/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:47.472241 2ab5e5576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:47.663296 2ab5e5576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:47.730574 2ab5e5576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (18/1000)' waiting for osd.0 to come back up (18/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:48.823806 2b045e252700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:48.834747 2b045e252700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:48.896453 2b045e252700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (19/1000)' waiting for osd.0 to come back up (19/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:49.970579 2b2f63ed1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:49.991248 2b2f63ed1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:50.548874 2b2f63ed1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (20/1000)' waiting for osd.0 to come back up (20/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:51.689708 2abeb4fb9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:51.696835 2abeb4fb9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:51.747579 2abeb4fb9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (21/1000)' waiting for osd.0 to come back up (21/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:52.815280 2b666a859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:52.815284 2b666a859700 0 lockdep start 2016-06-07 20:26:52.821164 2b666a859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:52.879259 2b666a859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (22/1000)' waiting for osd.0 to come back up (22/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:53.945258 2b83d95f4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:53.980102 2b83d95f4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:54.033890 2b83d95f4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (23/1000)' waiting for osd.0 to come back up (23/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:55.615074 2b35f2f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:55.615080 2b35f2f7a700 0 lockdep start 2016-06-07 20:26:55.620776 2b35f2f7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:55.677193 2b35f2f7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1107: test_mon_osd: echo 'waiting for osd.0 to come back up (24/1000)' waiting for osd.0 to come back up (24/1000) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1108: test_mon_osd: sleep 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i++ )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1105: test_mon_osd: (( i < 1000 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: grep 'osd.0 up' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1106: test_mon_osd: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:26:56.779367 2b66cfa04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:26:56.779375 2b66cfa04700 0 lockdep start 2016-06-07 20:26:56.786411 2b66cfa04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:00.479640 2b66cfa04700 0 lockdep stop osd.0 up in weight 1 up_from 183 up_thru 175 down_at 179 last_clean_interval [6,182) 127.0.0.1:6832/28826 127.0.0.1:6826/1028826 127.0.0.1:6827/1028826 127.0.0.1:6828/1028826 exists,up c8e248f1-e9df-405e-b941-418201377eef /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1110: test_mon_osd: break /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1113: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1113: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:00.569343 2ab7b19de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:00.575297 2ab7b19de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:00.632495 2ab7b19de700 0 lockdep stop osd.0 up in weight 1 up_from 183 up_thru 175 down_at 179 last_clean_interval [6,182) 127.0.0.1:6832/28826 127.0.0.1:6826/1028826 127.0.0.1:6827/1028826 127.0.0.1:6828/1028826 exists,up c8e248f1-e9df-405e-b941-418201377eef /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1115: test_mon_osd: ceph osd thrash 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:00.706749 2b4193d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:00.721322 2b4193d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * will thrash map for 0 epochs 2016-06-07 20:27:01.195941 2b4193d5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1117: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1117: test_mon_osd: grep 'osd.0 up' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:01.274009 2acf89576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:01.279816 2acf89576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:02.921034 2acf89576700 0 lockdep stop osd.0 up in weight 1 up_from 183 up_thru 185 down_at 179 last_clean_interval [6,182) 127.0.0.1:6832/28826 127.0.0.1:6826/1028826 127.0.0.1:6827/1028826 127.0.0.1:6828/1028826 exists,up c8e248f1-e9df-405e-b941-418201377eef /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1119: test_mon_osd: ceph osd find 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:02.984233 2aad3c0a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:02.993094 2aad3c0a7700 -1 WARNING: the following dangerous and experimental features are enabled: * { "osd": 1, "ip": "127.0.0.1:6856\/31419", "crush_location": { "host": "smithi049", "root": "default" } } 2016-06-07 20:27:03.551392 2aad3c0a7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1120: test_mon_osd: ceph osd find osd.1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:03.636223 2af54f531700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:03.645782 2af54f531700 -1 WARNING: the following dangerous and experimental features are enabled: * { "osd": 1, "ip": "127.0.0.1:6856\/31419", "crush_location": { "host": "smithi049", "root": "default" } } 2016-06-07 20:27:03.961861 2af54f531700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1121: test_mon_osd: expect_false osd find osd.xyz /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: osd find osd.xyz /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: osd: command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1122: test_mon_osd: expect_false osd find xyz /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: osd find xyz /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: osd: command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1123: test_mon_osd: expect_false osd find 0.1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: osd find 0.1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: osd: command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1124: test_mon_osd: ceph --format plain osd find 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:04.051181 2b4f1f3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:04.051185 2b4f1f3e3700 0 lockdep start 2016-06-07 20:27:04.056863 2b4f1f3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * { "osd": 1, "ip": "127.0.0.1:6856\/31419", "crush_location": { "host": "smithi049", "root": "default" } } 2016-06-07 20:27:04.110005 2b4f1f3e3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1125: test_mon_osd: ceph osd metadata 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1125: test_mon_osd: grep distro *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:04.187430 2acbe4f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:04.187436 2acbe4f79700 0 lockdep start 2016-06-07 20:27:04.197453 2acbe4f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:04.720256 2acbe4f79700 0 lockdep stop "distro": "Ubuntu", "distro_codename": "trusty", "distro_description": "Ubuntu 14.04.4 LTS", "distro_version": "14.04", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1126: test_mon_osd: ceph --format plain osd metadata 1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1126: test_mon_osd: grep distro *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:04.805032 2b14773e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:04.805037 2b14773e3700 0 lockdep start 2016-06-07 20:27:04.816102 2b14773e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:04.875776 2b14773e3700 0 lockdep stop "distro": "Ubuntu", "distro_codename": "trusty", "distro_description": "Ubuntu 14.04.4 LTS", "distro_version": "14.04", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1127: test_mon_osd: ceph osd out 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:04.960529 2b4a1d355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:04.972905 2b4a1d355700 -1 WARNING: the following dangerous and experimental features are enabled: * marked out osd.0. 2016-06-07 20:27:05.808053 2b4a1d355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1128: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1128: test_mon_osd: grep 'osd.0.*out' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:05.889330 2b2bb4f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:05.889339 2b2bb4f79700 0 lockdep start 2016-06-07 20:27:05.900467 2b2bb4f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:07.588194 2b2bb4f79700 0 lockdep stop osd.0 up out weight 0 up_from 183 up_thru 187 down_at 179 last_clean_interval [6,182) 127.0.0.1:6832/28826 127.0.0.1:6826/1028826 127.0.0.1:6827/1028826 127.0.0.1:6828/1028826 exists,up c8e248f1-e9df-405e-b941-418201377eef /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1129: test_mon_osd: ceph osd in 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:07.684337 2b0818d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:07.684346 2b0818d71700 0 lockdep start 2016-06-07 20:27:07.696341 2b0818d71700 -1 WARNING: the following dangerous and experimental features are enabled: * marked in osd.0. 2016-06-07 20:27:09.835373 2b0818d71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1130: test_mon_osd: ceph osd dump /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1130: test_mon_osd: grep 'osd.0.*in' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:09.914500 2ac5dad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:09.921648 2ac5dad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:09.980640 2ac5dad61700 0 lockdep stop osd.0 up in weight 1 up_from 183 up_thru 187 down_at 179 last_clean_interval [6,182) 127.0.0.1:6832/28826 127.0.0.1:6826/1028826 127.0.0.1:6827/1028826 127.0.0.1:6828/1028826 exists,up c8e248f1-e9df-405e-b941-418201377eef /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1131: test_mon_osd: ceph osd find 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:10.059943 2b0373fa2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:10.068409 2b0373fa2700 -1 WARNING: the following dangerous and experimental features are enabled: * { "osd": 0, "ip": "127.0.0.1:6832\/28826", "crush_location": { "host": "smithi049", "root": "default" } } 2016-06-07 20:27:10.288943 2b0373fa2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1133: test_mon_osd: f=/tmp/cephtool19083/map.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1134: test_mon_osd: ceph osd getcrushmap -o /tmp/cephtool19083/map.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:10.366081 2af660db1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:10.374502 2af660db1700 -1 WARNING: the following dangerous and experimental features are enabled: * got crush map from osdmap epoch 193 2016-06-07 20:27:12.563963 2af660db1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1135: test_mon_osd: '[' -s /tmp/cephtool19083/map.19083 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1136: test_mon_osd: ceph osd setcrushmap -i /tmp/cephtool19083/map.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:12.641124 2ba5a7d9c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:12.641129 2ba5a7d9c700 0 lockdep start 2016-06-07 20:27:12.646863 2ba5a7d9c700 -1 WARNING: the following dangerous and experimental features are enabled: * set crush map 2016-06-07 20:27:14.641093 2ba5a7d9c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1137: test_mon_osd: rm /tmp/cephtool19083/map.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1138: test_mon_osd: ceph osd getmap -o /tmp/cephtool19083/map.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:14.739664 2afadab9d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:14.739669 2afadab9d700 0 lockdep start 2016-06-07 20:27:14.746757 2afadab9d700 -1 WARNING: the following dangerous and experimental features are enabled: * got osdmap epoch 195 2016-06-07 20:27:15.941121 2afadab9d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1139: test_mon_osd: '[' -s /tmp/cephtool19083/map.19083 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1140: test_mon_osd: rm /tmp/cephtool19083/map.19083 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1141: test_mon_osd: ceph osd getmaxosd //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1141: test_mon_osd: sed -e 's/max_osd = //' -e 's/ in epoch.*//' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:16.035350 2b5a67a04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:16.035356 2b5a67a04700 0 lockdep start 2016-06-07 20:27:16.042601 2b5a67a04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:16.519525 2b5a67a04700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1141: test_mon_osd: save=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1142: test_mon_osd: '[' 3 -gt 0 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1143: test_mon_osd: ceph osd setmaxosd 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1143: test_mon_osd: grep EBUSY Error EBUSY: cannot shrink max_osd to 2 because osd.2 (and possibly others) still in use /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1144: test_mon_osd: ceph osd setmaxosd 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:17.032644 2ba5a17d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:17.032651 2ba5a17d5700 0 lockdep start 2016-06-07 20:27:17.044484 2ba5a17d5700 -1 WARNING: the following dangerous and experimental features are enabled: * set new max_osd = 10 2016-06-07 20:27:17.827391 2ba5a17d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1145: test_mon_osd: ceph osd getmaxosd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1145: test_mon_osd: grep 'max_osd = 10' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:17.913996 2b9977e04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:17.914004 2b9977e04700 0 lockdep start 2016-06-07 20:27:17.923235 2b9977e04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:17.985559 2b9977e04700 0 lockdep stop max_osd = 10 in epoch 196 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1146: test_mon_osd: ceph osd setmaxosd 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:18.062016 2b7ff7e16700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:18.067875 2b7ff7e16700 -1 WARNING: the following dangerous and experimental features are enabled: * set new max_osd = 3 2016-06-07 20:27:18.699722 2b7ff7e16700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1147: test_mon_osd: ceph osd getmaxosd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1147: test_mon_osd: grep 'max_osd = 3' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:18.775112 2ab1ce212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:18.775117 2ab1ce212700 0 lockdep start 2016-06-07 20:27:18.778790 2ab1ce212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:19.454528 2ab1ce212700 0 lockdep stop max_osd = 3 in epoch 197 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1149: test_mon_osd: ceph osd ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:19.542730 2b1bd40a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:19.542735 2b1bd40a7700 0 lockdep start 2016-06-07 20:27:19.546716 2b1bd40a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:20.356210 2b1bd40a7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1149: test_mon_osd: for id in '`ceph osd ls`' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1150: test_mon_osd: retry_eagain 5 map_enxio_to_eagain ceph tell osd.0 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:49: retry_eagain: local max=5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:50: retry_eagain: shift /home/yuriw/ceph/qa/workunits/cephtool/test.sh:51: retry_eagain: local status /home/yuriw/ceph/qa/workunits/cephtool/test.sh:52: retry_eagain: local tmpfile=/tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:53: retry_eagain: local count //home/yuriw/ceph/qa/workunits/cephtool/test.sh:54: retry_eagain: seq 1 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:54: retry_eagain: for count in '$(seq 1 $max)' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:55: retry_eagain: status=0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:56: retry_eagain: map_enxio_to_eagain ceph tell osd.0 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:57: retry_eagain: test 0 = 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:59: retry_eagain: break /home/yuriw/ceph/qa/workunits/cephtool/test.sh:63: retry_eagain: test 1 = 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:66: retry_eagain: cat /tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:79: map_enxio_to_eagain: local status=0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:80: map_enxio_to_eagain: local tmpfile=/tmp/cephtool19083/map_enxio_to_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:82: map_enxio_to_eagain: ceph tell osd.0 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:83: map_enxio_to_eagain: test 0 '!=' 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:87: map_enxio_to_eagain: cat /tmp/cephtool19083/map_enxio_to_eagain.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:20.445328 2b38f9a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:20.455515 2b38f9a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * { "version": "ceph version 10.2.0-2079-g47591d6 (47591d620ef48c668472ed467eb5bbf0eda89a70)" } 2016-06-07 20:27:20.468780 2b38f9a1e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:88: map_enxio_to_eagain: rm /tmp/cephtool19083/map_enxio_to_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:89: map_enxio_to_eagain: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:67: retry_eagain: rm /tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:68: retry_eagain: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1149: test_mon_osd: for id in '`ceph osd ls`' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1150: test_mon_osd: retry_eagain 5 map_enxio_to_eagain ceph tell osd.1 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:49: retry_eagain: local max=5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:50: retry_eagain: shift /home/yuriw/ceph/qa/workunits/cephtool/test.sh:51: retry_eagain: local status /home/yuriw/ceph/qa/workunits/cephtool/test.sh:52: retry_eagain: local tmpfile=/tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:53: retry_eagain: local count //home/yuriw/ceph/qa/workunits/cephtool/test.sh:54: retry_eagain: seq 1 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:54: retry_eagain: for count in '$(seq 1 $max)' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:55: retry_eagain: status=0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:56: retry_eagain: map_enxio_to_eagain ceph tell osd.1 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:57: retry_eagain: test 0 = 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:59: retry_eagain: break /home/yuriw/ceph/qa/workunits/cephtool/test.sh:63: retry_eagain: test 1 = 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:66: retry_eagain: cat /tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:79: map_enxio_to_eagain: local status=0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:80: map_enxio_to_eagain: local tmpfile=/tmp/cephtool19083/map_enxio_to_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:82: map_enxio_to_eagain: ceph tell osd.1 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:83: map_enxio_to_eagain: test 0 '!=' 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:87: map_enxio_to_eagain: cat /tmp/cephtool19083/map_enxio_to_eagain.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:20.557629 2ad597983700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:20.557634 2ad597983700 0 lockdep start 2016-06-07 20:27:20.563488 2ad597983700 -1 WARNING: the following dangerous and experimental features are enabled: * { "version": "ceph version 10.2.0-2079-g47591d6 (47591d620ef48c668472ed467eb5bbf0eda89a70)" } 2016-06-07 20:27:21.138790 2ad597983700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:88: map_enxio_to_eagain: rm /tmp/cephtool19083/map_enxio_to_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:89: map_enxio_to_eagain: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:67: retry_eagain: rm /tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:68: retry_eagain: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1149: test_mon_osd: for id in '`ceph osd ls`' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1150: test_mon_osd: retry_eagain 5 map_enxio_to_eagain ceph tell osd.2 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:49: retry_eagain: local max=5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:50: retry_eagain: shift /home/yuriw/ceph/qa/workunits/cephtool/test.sh:51: retry_eagain: local status /home/yuriw/ceph/qa/workunits/cephtool/test.sh:52: retry_eagain: local tmpfile=/tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:53: retry_eagain: local count //home/yuriw/ceph/qa/workunits/cephtool/test.sh:54: retry_eagain: seq 1 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:54: retry_eagain: for count in '$(seq 1 $max)' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:55: retry_eagain: status=0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:56: retry_eagain: map_enxio_to_eagain ceph tell osd.2 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:57: retry_eagain: test 0 = 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:59: retry_eagain: break /home/yuriw/ceph/qa/workunits/cephtool/test.sh:63: retry_eagain: test 1 = 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:66: retry_eagain: cat /tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:79: map_enxio_to_eagain: local status=0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:80: map_enxio_to_eagain: local tmpfile=/tmp/cephtool19083/map_enxio_to_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:82: map_enxio_to_eagain: ceph tell osd.2 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:83: map_enxio_to_eagain: test 0 '!=' 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:87: map_enxio_to_eagain: cat /tmp/cephtool19083/map_enxio_to_eagain.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:21.241089 2b5a580a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:21.252185 2b5a580a7700 -1 WARNING: the following dangerous and experimental features are enabled: * { "version": "ceph version 10.2.0-2079-g47591d6 (47591d620ef48c668472ed467eb5bbf0eda89a70)" } 2016-06-07 20:27:21.270336 2b5a580a7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:88: map_enxio_to_eagain: rm /tmp/cephtool19083/map_enxio_to_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:89: map_enxio_to_eagain: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:67: retry_eagain: rm /tmp/cephtool19083/retry_eagain.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:68: retry_eagain: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1153: test_mon_osd: grep EBUSY /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1153: test_mon_osd: ceph osd rm 0 Error EBUSY: osd.0 is still up; must be down before removal. ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:1155: test_mon_osd: ceph osd ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:21.506231 2b7247f7b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:21.510102 2b7247f7b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:21.631839 2b7247f7b700 0 lockdep stop //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1155: test_mon_osd: echo 0 1 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1155: test_mon_osd: local 'old_osds=0 1 2' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1156: test_mon_osd: ceph osd create *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:21.727907 2b8ac7fa5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:21.727918 2b8ac7fa5700 0 lockdep start 2016-06-07 20:27:21.739942 2b8ac7fa5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:22.299624 2b8ac7fa5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1156: test_mon_osd: id=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1157: test_mon_osd: ceph osd find 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:22.392753 2b98b7b42700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:22.392760 2b98b7b42700 0 lockdep start 2016-06-07 20:27:22.404230 2b98b7b42700 -1 WARNING: the following dangerous and experimental features are enabled: * { "osd": 3, "ip": ":\/0", "crush_location": {} } 2016-06-07 20:27:23.073394 2b98b7b42700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1158: test_mon_osd: ceph osd lost 3 --yes-i-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:23.152821 2b5f41355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:23.152827 2b5f41355700 0 lockdep start 2016-06-07 20:27:23.158744 2b5f41355700 -1 WARNING: the following dangerous and experimental features are enabled: * marked osd lost in epoch 0 2016-06-07 20:27:24.187472 2b5f41355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1159: test_mon_osd: expect_false ceph osd setmaxosd 3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd setmaxosd 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:24.268367 2afac9bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:24.268372 2afac9bf9700 0 lockdep start 2016-06-07 20:27:24.274389 2afac9bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EBUSY: cannot shrink max_osd to 3 because osd.3 (and possibly others) still in use 2016-06-07 20:27:24.335832 2afac9bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:1160: test_mon_osd: ceph osd ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:24.419146 2b8fba859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:24.419150 2b8fba859700 0 lockdep start 2016-06-07 20:27:24.425294 2b8fba859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:24.484385 2b8fba859700 0 lockdep stop //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1160: test_mon_osd: echo 0 1 2 3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1160: test_mon_osd: local 'new_osds=0 1 2 3' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1161: test_mon_osd: echo 0 1 2 3 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1161: test_mon_osd: sed -e 's/0 1 2//' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1161: test_mon_osd: for id in '$(echo $new_osds | sed -e "s/$old_osds//")' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1162: test_mon_osd: ceph osd rm 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:24.573975 2ae476a08700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:24.573979 2ae476a08700 0 lockdep start 2016-06-07 20:27:24.579747 2ae476a08700 -1 WARNING: the following dangerous and experimental features are enabled: * removed osd.3 2016-06-07 20:27:26.235872 2ae476a08700 0 lockdep stop //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1165: test_mon_osd: uuidgen /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1165: test_mon_osd: uuid=70c96642-2cf1-4438-bd85-e1797665e035 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1166: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:26.319921 2ab89519a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:26.319927 2ab89519a700 0 lockdep start 2016-06-07 20:27:26.325816 2ab89519a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:27.051355 2ab89519a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1166: test_mon_osd: id=3 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1167: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:27.143299 2abe6a643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:27.155872 2abe6a643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:27.218817 2abe6a643700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1167: test_mon_osd: id2=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1168: test_mon_osd: '[' 3 = 3 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1169: test_mon_osd: ceph osd rm 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:27.300795 2b348fed1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:27.300800 2b348fed1700 0 lockdep start 2016-06-07 20:27:27.311004 2b348fed1700 -1 WARNING: the following dangerous and experimental features are enabled: * removed osd.3 2016-06-07 20:27:28.401985 2b348fed1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1171: test_mon_osd: ceph --help osd *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** General usage: ============== usage: ceph [-h] [-c CEPHCONF] [-i INPUT_FILE] [-o OUTPUT_FILE] [--id CLIENT_ID] [--name CLIENT_NAME] [--cluster CLUSTER] [--admin-daemon ADMIN_SOCKET] [--admin-socket ADMIN_SOCKET_NOPE] [-s] [-w] [--watch-debug] [--watch-info] [--watch-sec] [--watch-warn] [--watch-error] [--version] [--verbose] [--concise] [-f {json,json-pretty,xml,xml-pretty,plain}] [--connect-timeout CLUSTER_TIMEOUT] Ceph administration tool optional arguments: -h, --help request mon help -c CEPHCONF, --conf CEPHCONF ceph configuration file -i INPUT_FILE, --in-file INPUT_FILE input file -o OUTPUT_FILE, --out-file OUTPUT_FILE output file --id CLIENT_ID, --user CLIENT_ID client id for authentication --name CLIENT_NAME, -n CLIENT_NAME client name for authentication --cluster CLUSTER cluster name --admin-daemon ADMIN_SOCKET submit admin-socket commands ("help" for help --admin-socket ADMIN_SOCKET_NOPE you probably mean --admin-daemon -s, --status show cluster status -w, --watch watch live cluster changes --watch-debug watch debug events --watch-info watch info events --watch-sec watch security events --watch-warn watch warn events --watch-error watch error events --version, -v display version --verbose make verbose --concise make less verbose -f {json,json-pretty,xml,xml-pretty,plain}, --format {json,json-pretty,xml,xml-pretty,plain} --connect-timeout CLUSTER_TIMEOUT set a timeout for connecting to the cluster 2016-06-07 20:27:28.498468 2b0ecde09700 -1 WARNING: the following dangerous and experimental features are enabled: * Monitor commands: ================= [Contacting monitor, timeout after 5 seconds] 2016-06-07 20:27:28.506946 2b0ecde09700 -1 WARNING: the following dangerous and experimental features are enabled: * osd blacklist add|rm add (optionally until seconds {} from now) or remove from blacklist osd blacklist clear clear all blacklisted clients osd blacklist ls show blacklisted clients osd blocked-by print histogram of which OSDs are blocking their peers osd create {} {} ID) osd crush add add or update crushmap position and [...] weight for with and location osd crush add-bucket add no-parent (probably root) crush bucket of type osd crush create-or-move [.. for at/to location .] osd crush dump dump crush map osd crush get-tunable straw_calc_version get crush tunable osd crush link [...] link existing entry for under location osd crush move [...] move existing entry for to location osd crush remove {} remove from crush map ( everywhere, or just at ) osd crush rename-bucket rename bucket to osd crush reweight change 's weight to in crush map osd crush reweight-all recalculate the weights for the tree to ensure they sum correctly osd crush reweight-subtree change all leaf items beneath to in crush map osd crush rm {} remove from crush map ( everywhere, or just at ) osd crush rule create-erasure create crush rule for erasure {} coded pool created with ( default default) osd crush rule create-simple create crush rule to start from {firstn|indep} , replicate across buckets of type , using a choose mode of (default firstn; indep best for erasure pools) osd crush rule dump {} dump crush rule (default all) osd crush rule list list crush rules osd crush rule ls list crush rules osd crush rule rm remove crush rule osd crush set set crush map from input file osd crush set update crushmap position and weight [...] for to with location osd crush set-tunable straw_calc_ set crush tunable to version osd crush show-tunables show current crush tunables osd crush tree dump crush buckets and items in a tree view osd crush tunables legacy|argonaut| set crush tunables values to bobtail|firefly|hammer|jewel|optimal| default osd crush unlink {} unlink from crush map ( everywhere, or just at ) osd deep-scrub initiate deep scrub on osd osd df {plain|tree} show OSD utilization osd down [...] set osd(s) [...] down osd dump {} print summary of OSD map osd erasure-code-profile get get erasure code profile osd erasure-code-profile ls list all erasure code profiles osd erasure-code-profile rm remove erasure code profile osd erasure-code-profile set create erasure code profile { [...]} with [ ...] pairs. Add a --force at the end to override an existing profile (VERY DANGEROUS) osd find find osd in the CRUSH map and show its location osd getcrushmap {} get CRUSH map osd getmap {} get OSD map osd getmaxosd show largest OSD id osd in [...] set osd(s) [...] in osd lost {--yes-i- mark osd as permanently lost. THIS really-mean-it} DESTROYS DATA IF NO MORE REPLICAS EXIST, BE CAREFUL osd ls {} show all OSD ids osd lspools {} list pools osd map find pg for in with {} [namespace] osd metadata {} fetch metadata for osd {id} (default all) osd out [...] set osd(s) [...] out osd pause pause osd osd perf print dump of OSD perf summary stats osd pg-temp { [... > [...]} ]] (developers only) osd pool create create pool {} {replicated|erasure} {} {} {} osd pool delete {} delete pool {--yes-i-really-really-mean-it} osd pool get size|min_size| get pool parameter crash_replay_interval|pg_num|pgp_num| crush_ruleset|hashpspool|nodelete| nopgchange|nosizechange|write_fadvise_ dontneed|noscrub|nodeep-scrub|hit_set_ type|hit_set_period|hit_set_count|hit_ set_fpp|auid|target_max_objects|target_ max_bytes|cache_target_dirty_ratio| cache_target_dirty_high_ratio|cache_ target_full_ratio|cache_min_flush_age| cache_min_evict_age|erasure_code_ profile|min_read_recency_for_promote| all|min_write_recency_for_promote|fast_ read|hit_set_grade_decay_rate|hit_set_ search_last_n|scrub_min_interval|scrub_ max_interval|deep_scrub_interval| recovery_priority|recovery_op_priority| scrub_priority osd pool get-quota obtain object or byte limits for pool osd pool ls {detail} list pools osd pool mksnap make snapshot in osd pool rename rename to osd pool rm {} {-- remove pool yes-i-really-really-mean-it} osd pool rmsnap remove snapshot from osd pool set size|min_size| set pool parameter to crash_replay_interval|pg_num|pgp_num| crush_ruleset|hashpspool|nodelete| nopgchange|nosizechange|write_fadvise_ dontneed|noscrub|nodeep-scrub|hit_set_ type|hit_set_period|hit_set_count|hit_ set_fpp|use_gmt_hitset|debug_fake_ec_ pool|target_max_bytes|target_max_ objects|cache_target_dirty_ratio|cache_ target_dirty_high_ratio|cache_target_ full_ratio|cache_min_flush_age|cache_ min_evict_age|auid|min_read_recency_ for_promote|min_write_recency_for_ promote|fast_read|hit_set_grade_decay_ rate|hit_set_search_last_n|scrub_min_ interval|scrub_max_interval|deep_scrub_ interval|recovery_priority|recovery_op_ priority|scrub_priority {--yes-i- really-mean-it} osd pool set-quota max_ set object or byte limit on pool objects|max_bytes osd pool stats {} obtain stats from all pools, or from specified pool osd primary-affinity <= 1.0 osd primary-temp |-1 ( osd.id)> developers only) osd repair initiate repair on osd osd reweight reweight osd to 0.0 < < 1.0 osd reweight-by-pg {} {} reweight OSDs by PG distribution {} { [...]} [overload-percentage-for- consideration, default 120] osd reweight-by-utilization {} reweight OSDs by utilization [overload- {} {} {--no-increasing} percentage-for-consideration, default 120] osd rm [...] remove osd(s) [...] in osd scrub initiate scrub on osd osd set full|pause|noup|nodown|noout| set noin|nobackfill|norebalance|norecover| noscrub|nodeep-scrub|notieragent| sortbitwise|require_jewel_osds osd setcrushmap set crush map from input file osd setmaxosd set new maximum osd value osd stat print summary of OSD map osd test-reweight-by-pg {} dry run of reweight OSDs by PG {} {} { distribution [overload-percentage-for- [...]} consideration, default 120] osd test-reweight-by-utilization dry run of reweight OSDs by {} {} {} {--no- utilization [overload-percentage-for- increasing} consideration, default 120] osd thrash thrash OSDs for osd tier add {-- add the tier (the second force-nonempty} one) to base pool (the first one) osd tier add-cache add a cache (the second one) of size to existing pool (the first one) osd tier cache-mode none| specify the caching mode for cache writeback|forward|readonly|readforward| tier proxy|readproxy {--yes-i-really-mean- it} osd tier remove remove the tier (the second one) from base pool (the first one) osd tier remove-overlay remove the overlay pool for base pool osd tier rm remove the tier (the second one) from base pool (the first one) osd tier rm-overlay remove the overlay pool for base pool osd tier set-overlay set the overlay pool for base pool to be osd tree {} print OSD tree osd unpause unpause osd osd unset full|pause|noup|nodown|noout| unset noin|nobackfill|norebalance|norecover| noscrub|nodeep-scrub|notieragent| sortbitwise osd utilization get basic pg distribution stats 2016-06-07 20:27:29.102291 2b0ecde09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1174: test_mon_osd: ceph osd setmaxosd 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:29.186881 2b8ded7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:29.193125 2b8ded7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * set new max_osd = 3 2016-06-07 20:27:29.625502 2b8ded7d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1175: test_mon_osd: ceph osd getmaxosd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1175: test_mon_osd: grep 'max_osd = 3' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:29.717156 2b52f35f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:29.717163 2b52f35f2700 0 lockdep start 2016-06-07 20:27:29.723218 2b52f35f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:30.768255 2b52f35f2700 0 lockdep stop max_osd = 3 in epoch 203 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1176: test_mon_osd: local max_osd=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1178: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1178: test_mon_osd: grep EINVAL Error EINVAL: id 0 already in use and does not match uuid 70c96642-2cf1-4438-bd85-e1797665e035 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1179: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1179: test_mon_osd: grep EINVAL Error EINVAL: id 2 already in use and does not match uuid 70c96642-2cf1-4438-bd85-e1797665e035 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1181: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:31.378276 2b8601e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:31.378282 2b8601e09700 0 lockdep start 2016-06-07 20:27:31.384492 2b8601e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:32.672684 2b8601e09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1181: test_mon_osd: id=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1182: test_mon_osd: '[' 3 = 3 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1183: test_mon_osd: ceph osd find 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:32.749982 2b4254026700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:32.756016 2b4254026700 -1 WARNING: the following dangerous and experimental features are enabled: * { "osd": 3, "ip": ":\/0", "crush_location": {} } 2016-06-07 20:27:32.815089 2b4254026700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1184: test_mon_osd: max_osd=4 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1185: test_mon_osd: ceph osd getmaxosd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1185: test_mon_osd: grep 'max_osd = 4' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:32.890253 2b006119a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:32.890257 2b006119a700 0 lockdep start 2016-06-07 20:27:32.894091 2b006119a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:33.010639 2b006119a700 0 lockdep stop max_osd = 4 in epoch 204 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1187: test_mon_osd: grep EINVAL /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1187: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 2 Error EINVAL: uuid 70c96642-2cf1-4438-bd85-e1797665e035 already in use for different id 3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1188: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 4 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1188: test_mon_osd: grep EINVAL Error EINVAL: uuid 70c96642-2cf1-4438-bd85-e1797665e035 already in use for different id 3 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1189: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:33.926599 2b66f610e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:33.926616 2b66f610e700 0 lockdep start 2016-06-07 20:27:33.938155 2b66f610e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:34.213666 2b66f610e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1189: test_mon_osd: id2=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1190: test_mon_osd: '[' 3 = 3 ']' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1191: test_mon_osd: ceph osd create 70c96642-2cf1-4438-bd85-e1797665e035 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:34.300310 2b937e859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:34.300317 2b937e859700 0 lockdep start 2016-06-07 20:27:34.312797 2b937e859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:34.401923 2b937e859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1191: test_mon_osd: id2=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1192: test_mon_osd: '[' 3 = 3 ']' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1194: test_mon_osd: uuidgen /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1194: test_mon_osd: uuid=51baf79d-5663-4150-9d45-b541ac676262 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1195: test_mon_osd: local gap_start=4 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1196: test_mon_osd: ceph osd create 51baf79d-5663-4150-9d45-b541ac676262 104 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:34.472987 2af98a00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:34.486245 2af98a00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:36.198246 2af98a00d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1196: test_mon_osd: id=104 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1197: test_mon_osd: '[' 104 = 104 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1198: test_mon_osd: max_osd=105 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1199: test_mon_osd: ceph osd getmaxosd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1199: test_mon_osd: grep 'max_osd = 105' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:36.288657 2b0117d9c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:36.288663 2b0117d9c700 0 lockdep start 2016-06-07 20:27:36.294681 2b0117d9c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:37.112750 2b0117d9c700 0 lockdep stop max_osd = 105 in epoch 205 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1201: test_mon_osd: ceph osd create 51baf79d-5663-4150-9d45-b541ac676262 4 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1201: test_mon_osd: grep EINVAL Error EINVAL: uuid 51baf79d-5663-4150-9d45-b541ac676262 already in use for different id 104 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1207: test_mon_osd: local next_osd /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1208: test_mon_osd: test '' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1211: test_mon_osd: next_osd=4 //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1213: test_mon_osd: ceph osd create *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:37.639050 2acd8519a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:37.639055 2acd8519a700 0 lockdep start 2016-06-07 20:27:37.644759 2acd8519a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:39.463520 2acd8519a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1213: test_mon_osd: id=4 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1214: test_mon_osd: '[' 4 = 4 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1216: test_mon_osd: next_osd=5 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:1217: test_mon_osd: uuidgen //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1217: test_mon_osd: ceph osd create 131ee146-5e81-451d-93df-0734604829fa *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:39.557159 2ab43ed61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:39.557164 2ab43ed61700 0 lockdep start 2016-06-07 20:27:39.568961 2ab43ed61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:41.231204 2ab43ed61700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1217: test_mon_osd: id=5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1218: test_mon_osd: '[' 5 = 5 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1220: test_mon_osd: next_osd=6 ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:1221: test_mon_osd: uuidgen //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1221: test_mon_osd: ceph osd create e98f86f4-6c60-4ea9-945a-35b007c4505b 6 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:41.302687 2b83232b8700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:41.308646 2b83232b8700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:42.743496 2b83232b8700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1221: test_mon_osd: id=6 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1222: test_mon_osd: '[' 6 = 6 ']' ///home/yuriw/ceph/qa/workunits/cephtool/test.sh:1224: test_mon_osd: ceph osd ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:42.814103 2b1cf4f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:42.821083 2b1cf4f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:42.883742 2b1cf4f79700 0 lockdep stop //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1224: test_mon_osd: echo 0 1 2 3 4 5 6 104 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1224: test_mon_osd: local 'new_osds=0 1 2 3 4 5 6 104' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1225: test_mon_osd: sed -e 's/0 1 2//' //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1225: test_mon_osd: echo 0 1 2 3 4 5 6 104 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1225: test_mon_osd: for id in '$(echo $new_osds | sed -e "s/$old_osds//")' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1226: test_mon_osd: '[' 3 -ge 3 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1227: test_mon_osd: ceph osd rm 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:42.975918 2aaab3f2f700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:42.975925 2aaab3f2f700 0 lockdep start 2016-06-07 20:27:42.989224 2aaab3f2f700 -1 WARNING: the following dangerous and experimental features are enabled: * removed osd.3 2016-06-07 20:27:45.228217 2aaab3f2f700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1225: test_mon_osd: for id in '$(echo $new_osds | sed -e "s/$old_osds//")' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1226: test_mon_osd: '[' 4 -ge 3 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1227: test_mon_osd: ceph osd rm 4 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:45.304435 2b40c99de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:45.304440 2b40c99de700 0 lockdep start 2016-06-07 20:27:45.338253 2b40c99de700 -1 WARNING: the following dangerous and experimental features are enabled: * removed osd.4 2016-06-07 20:27:46.266450 2b40c99de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1225: test_mon_osd: for id in '$(echo $new_osds | sed -e "s/$old_osds//")' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1226: test_mon_osd: '[' 5 -ge 3 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1227: test_mon_osd: ceph osd rm 5 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:46.363183 2b9ef4333700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:46.363191 2b9ef4333700 0 lockdep start 2016-06-07 20:27:46.370973 2b9ef4333700 -1 WARNING: the following dangerous and experimental features are enabled: * removed osd.5 2016-06-07 20:27:47.433848 2b9ef4333700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1225: test_mon_osd: for id in '$(echo $new_osds | sed -e "s/$old_osds//")' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1226: test_mon_osd: '[' 6 -ge 3 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1227: test_mon_osd: ceph osd rm 6 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:47.532727 2b0a7c1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:47.532732 2b0a7c1f3700 0 lockdep start 2016-06-07 20:27:47.540174 2b0a7c1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * removed osd.6 2016-06-07 20:27:48.558959 2b0a7c1f3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1225: test_mon_osd: for id in '$(echo $new_osds | sed -e "s/$old_osds//")' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1226: test_mon_osd: '[' 104 -ge 3 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1227: test_mon_osd: ceph osd rm 104 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:48.640484 2ac56e2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:48.645737 2ac56e2d3700 -1 WARNING: the following dangerous and experimental features are enabled: * removed osd.104 2016-06-07 20:27:49.239381 2ac56e2d3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1229: test_mon_osd: ceph osd setmaxosd 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:49.313995 2ad5019de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:49.320709 2ad5019de700 -1 WARNING: the following dangerous and experimental features are enabled: * set new max_osd = 3 2016-06-07 20:27:52.111556 2ad5019de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1231: test_mon_osd: ceph osd ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:52.203847 2ba781f0a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:52.203853 2ba781f0a700 0 lockdep start 2016-06-07 20:27:52.207776 2ba781f0a700 -1 WARNING: the following dangerous and experimental features are enabled: * 0 1 2 2016-06-07 20:27:52.271077 2ba781f0a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1232: test_mon_osd: ceph osd pool create data 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:52.346884 2b9459576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:52.357786 2b9459576700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'data' created 2016-06-07 20:27:54.122970 2b9459576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1233: test_mon_osd: ceph osd lspools /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1233: test_mon_osd: grep data *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:54.212716 2b8d06744700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:54.219967 2b8d06744700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:54.741314 2b8d06744700 0 lockdep stop 0 rbd,20 data, /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1234: test_mon_osd: grep 'pool.*data.*object.*foo.*pg.*up.*acting' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1234: test_mon_osd: ceph osd map data foo *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:54.819477 2afaffe5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:54.819483 2afaffe5d700 0 lockdep start 2016-06-07 20:27:54.933335 2afaffe5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:56.477950 2afaffe5d700 0 lockdep stop osdmap e217 pool 'data' (20) object 'foo' -> pg 20.7fc1f406 (20.6) -> up ([1,2,0], p1) acting ([1,2,0], p1) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1235: test_mon_osd: ceph osd map data foo namespace /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1235: test_mon_osd: grep 'pool.*data.*object.*namespace/foo.*pg.*up.*acting' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:56.566202 2b0a501f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:56.566207 2b0a501f3700 0 lockdep start 2016-06-07 20:27:56.572291 2b0a501f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:57.062243 2b0a501f3700 0 lockdep stop osdmap e217 pool 'data' (20) object 'namespace/foo' -> pg 20.9091039 (20.9) -> up ([2,0,1], p2) acting ([2,0,1], p2) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1236: test_mon_osd: ceph osd pool delete data data --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:57.148128 2ae2a2212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:57.148133 2ae2a2212700 0 lockdep start 2016-06-07 20:27:57.153832 2ae2a2212700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'data' removed 2016-06-07 20:27:57.636708 2ae2a2212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1238: test_mon_osd: ceph osd pause *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:57.720707 2b096f1f7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:57.729530 2b096f1f7700 -1 WARNING: the following dangerous and experimental features are enabled: * set pauserd,pausewr 2016-06-07 20:27:59.052858 2b096f1f7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1239: test_mon_osd: grep 'flags pauserd,pausewr' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1239: test_mon_osd: ceph osd dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:59.144202 2b50f9355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:59.152274 2b50f9355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:59.328149 2b50f9355700 0 lockdep stop flags pauserd,pausewr,sortbitwise,require_jewel_osds /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1240: test_mon_osd: ceph osd unpause *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:27:59.420625 2b2a8ea48700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:27:59.428191 2b2a8ea48700 -1 WARNING: the following dangerous and experimental features are enabled: * unset pauserd,pausewr 2016-06-07 20:28:00.161288 2b2a8ea48700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1242: test_mon_osd: ceph osd tree *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:00.231519 2af6634a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:00.238350 2af6634a4700 -1 WARNING: the following dangerous and experimental features are enabled: * ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -1 3.00000 root default -2 3.00000 host smithi049 0 1.00000 osd.0 up 1.00000 1.00000 1 1.00000 osd.1 up 1.00000 1.00000 2 1.00000 osd.2 up 1.00000 1.00000 2016-06-07 20:28:01.012776 2af6634a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1243: test_mon_osd: ceph osd perf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:01.104377 2ad877b7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:01.104385 2ad877b7a700 0 lockdep start 2016-06-07 20:28:01.113702 2ad877b7a700 -1 WARNING: the following dangerous and experimental features are enabled: * osd fs_commit_latency(ms) fs_apply_latency(ms) 2 179 1978 1 206 2329 0 202 1106 2016-06-07 20:28:01.751925 2ad877b7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1244: test_mon_osd: ceph osd blocked-by *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:01.825988 2aebe9bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:01.825993 2aebe9bf9700 0 lockdep start 2016-06-07 20:28:01.833464 2aebe9bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * osd num_blocked 2016-06-07 20:28:02.707459 2aebe9bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1246: test_mon_osd: ceph osd stat /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1246: test_mon_osd: grep up, *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:02.780846 2b5d3bea3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:02.780851 2b5d3bea3700 0 lockdep start 2016-06-07 20:28:02.786628 2b5d3bea3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:04.162586 2b5d3bea3700 0 lockdep stop osdmap e220: 3 osds: 3 up, 3 in /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:04.253221 2b3c46859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:04.253227 2b3c46859700 0 lockdep start 2016-06-07 20:28:04.257275 2b3c46859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:04.840153 2b3c46859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_osd_pool /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1254: test_mon_osd_pool: ceph osd pool create data 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:04.930934 2b6d431b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:04.941817 2b6d431b7700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'data' created 2016-06-07 20:28:06.264814 2b6d431b7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1255: test_mon_osd_pool: ceph osd pool mksnap data datasnap *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:06.362927 2b7f0d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:06.362938 2b7f0d19a700 0 lockdep start 2016-06-07 20:28:06.375048 2b7f0d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * created pool data snap datasnap 2016-06-07 20:28:08.470207 2b7f0d19a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1256: test_mon_osd_pool: rados -p data lssnap /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1256: test_mon_osd_pool: grep datasnap 2016-06-07 20:28:08.513688 2adc975889c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:08.513868 2adc975889c0 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:08.515359 2adc975889c0 -1 WARNING: the following dangerous and experimental features are enabled: * 1 datasnap 2016.06.07 20:28:07 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1257: test_mon_osd_pool: ceph osd pool rmsnap data datasnap *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:09.065176 2b8c56d61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:09.065187 2b8c56d61700 0 lockdep start 2016-06-07 20:28:09.085846 2b8c56d61700 -1 WARNING: the following dangerous and experimental features are enabled: * removed pool data snap datasnap 2016-06-07 20:28:10.223109 2b8c56d61700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1258: test_mon_osd_pool: expect_false ceph osd pool rmsnap pool_fake snapshot /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool rmsnap pool_fake snapshot *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:10.308328 2ac959eca700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:10.308335 2ac959eca700 0 lockdep start 2016-06-07 20:28:10.314097 2ac959eca700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOENT: unrecognized pool 'pool_fake' 2016-06-07 20:28:11.405533 2ac959eca700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1259: test_mon_osd_pool: ceph osd pool delete data data --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:11.495274 2b7bad576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:11.514646 2b7bad576700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'data' removed 2016-06-07 20:28:12.056800 2b7bad576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1261: test_mon_osd_pool: ceph osd pool create data2 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:12.158920 2b667a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:12.158926 2b667a91a700 0 lockdep start 2016-06-07 20:28:12.168209 2b667a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'data2' created 2016-06-07 20:28:13.613232 2b667a91a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1262: test_mon_osd_pool: ceph osd pool rename data2 data3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:13.701290 2b22d3e91700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:13.701301 2b22d3e91700 0 lockdep start 2016-06-07 20:28:13.712408 2b22d3e91700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'data2' renamed to 'data3' 2016-06-07 20:28:15.348962 2b22d3e91700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1263: test_mon_osd_pool: ceph osd lspools /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1263: test_mon_osd_pool: grep data3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:15.430188 2af012b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:15.430197 2af012b5d700 0 lockdep start 2016-06-07 20:28:15.439250 2af012b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:16.964300 2af012b5d700 0 lockdep stop 0 rbd,22 data3, /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1264: test_mon_osd_pool: ceph osd pool delete data3 data3 --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:17.067500 2ad2071b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:17.067506 2ad2071b7700 0 lockdep start 2016-06-07 20:28:17.078842 2ad2071b7700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'data3' removed 2016-06-07 20:28:18.578914 2ad2071b7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1266: test_mon_osd_pool: ceph osd pool create replicated 12 12 replicated *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:18.674588 2b84de0ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:18.674595 2b84de0ce700 0 lockdep start 2016-06-07 20:28:18.687292 2b84de0ce700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'replicated' created 2016-06-07 20:28:19.619694 2b84de0ce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1267: test_mon_osd_pool: ceph osd pool create replicated 12 12 replicated *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:19.710828 2b2173b46700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:19.722218 2b2173b46700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'replicated' already exists 2016-06-07 20:28:19.847233 2b2173b46700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1268: test_mon_osd_pool: ceph osd pool create replicated 12 12 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:19.932560 2aacabeb8700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:19.932565 2aacabeb8700 0 lockdep start 2016-06-07 20:28:19.941275 2aacabeb8700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'replicated' already exists 2016-06-07 20:28:20.116241 2aacabeb8700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1269: test_mon_osd_pool: ceph osd pool create replicated 12 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:20.192617 2b805fba6700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:20.192622 2b805fba6700 0 lockdep start 2016-06-07 20:28:20.202307 2b805fba6700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'replicated' already exists 2016-06-07 20:28:20.586306 2b805fba6700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1271: test_mon_osd_pool: expect_false ceph osd pool create replicated 12 12 erasure /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool create replicated 12 12 erasure *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:20.678862 2b67c8d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:20.678868 2b67c8d71700 0 lockdep start 2016-06-07 20:28:20.688995 2b67c8d71700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: pool 'replicated' cannot change to type erasure 2016-06-07 20:28:20.754932 2b67c8d71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1272: test_mon_osd_pool: grep replicated /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1272: test_mon_osd_pool: ceph osd lspools *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:20.828169 2b5091a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:20.828173 2b5091a1e700 0 lockdep start 2016-06-07 20:28:20.834121 2b5091a1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:21.607154 2b5091a1e700 0 lockdep stop 0 rbd,23 replicated, /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1273: test_mon_osd_pool: ceph osd pool delete replicated replicated --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:21.684903 2ad8071b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:21.695425 2ad8071b7700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'replicated' removed 2016-06-07 20:28:22.754094 2ad8071b7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:22.841837 2af16a0ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:22.841845 2af16a0ce700 0 lockdep start 2016-06-07 20:28:22.847389 2af16a0ce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:23.675142 2af16a0ce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_osd_pool_quota /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1283: test_mon_osd_pool_quota: ceph osd pool create tmp-quota-pool 36 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:23.760887 2af80ae62700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:23.772156 2af80ae62700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'tmp-quota-pool' created 2016-06-07 20:28:24.902163 2af80ae62700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1287: test_mon_osd_pool_quota: expect_false ceph osd pool set-quota tmp-quota-pool max_fooness 10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set-quota tmp-quota-pool max_fooness 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:24.987431 2b297119a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:24.987438 2b297119a700 0 lockdep start 2016-06-07 20:28:24.996309 2b297119a700 -1 WARNING: the following dangerous and experimental features are enabled: * Invalid command: max_fooness not in max_objects|max_bytes osd pool set-quota max_objects|max_bytes : set object or byte limit on pool Error EINVAL: invalid command 2016-06-07 20:28:25.061097 2b297119a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1288: test_mon_osd_pool_quota: expect_false ceph osd pool set-quota tmp-quota-pool max_bytes -1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set-quota tmp-quota-pool max_bytes -1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:25.143131 2b8dd00a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:25.143135 2b8dd00a7700 0 lockdep start 2016-06-07 20:28:25.154313 2b8dd00a7700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: error parsing value '-22': expected numerical value, got: -1 2016-06-07 20:28:25.240432 2b8dd00a7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1289: test_mon_osd_pool_quota: expect_false ceph osd pool set-quota tmp-quota-pool max_objects aaa /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set-quota tmp-quota-pool max_objects aaa *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:25.336320 2af1715f4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:25.336325 2af1715f4700 0 lockdep start 2016-06-07 20:28:25.342451 2af1715f4700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: error parsing value '-22': unrecognized modifier 'a' 2016-06-07 20:28:25.461010 2af1715f4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1293: test_mon_osd_pool_quota: ceph osd pool set-quota tmp-quota-pool max_bytes 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:25.551709 2b66a2704700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:25.551714 2b66a2704700 0 lockdep start 2016-06-07 20:28:25.562968 2b66a2704700 -1 WARNING: the following dangerous and experimental features are enabled: * set-quota max_bytes = 10 for pool tmp-quota-pool 2016-06-07 20:28:27.658895 2b66a2704700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1294: test_mon_osd_pool_quota: ceph osd pool set-quota tmp-quota-pool max_objects 10M *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:27.745450 2aaad9e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:27.745455 2aaad9e09700 0 lockdep start 2016-06-07 20:28:27.756884 2aaad9e09700 -1 WARNING: the following dangerous and experimental features are enabled: * set-quota max_objects = 10485760 for pool tmp-quota-pool 2016-06-07 20:28:28.431931 2aaad9e09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1298: test_mon_osd_pool_quota: grep 'max bytes.*10B' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:28.522067 2b030c1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:28.522078 2b030c1f3700 0 lockdep start 2016-06-07 20:28:28.533488 2b030c1f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:28.596557 2b030c1f3700 0 lockdep stop max bytes : 10B /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1299: test_mon_osd_pool_quota: ceph osd pool get-quota tmp-quota-pool /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1299: test_mon_osd_pool_quota: grep 'max objects.*10240k objects' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:28.679888 2ac452b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:28.679898 2ac452b5d700 0 lockdep start 2016-06-07 20:28:28.690998 2ac452b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:28.810755 2ac452b5d700 0 lockdep stop max objects: 10240k objects /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1303: test_mon_osd_pool_quota: ceph osd pool get-quota tmp-quota-pool --format=json-pretty /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1304: test_mon_osd_pool_quota: grep '"quota_max_objects":.*10485760' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:28.891598 2b4aaf03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:28.891603 2b4aaf03b700 0 lockdep start 2016-06-07 20:28:28.897931 2b4aaf03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:29.072809 2b4aaf03b700 0 lockdep stop "quota_max_objects": 10485760, /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1306: test_mon_osd_pool_quota: grep '"quota_max_bytes":.*10' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1305: test_mon_osd_pool_quota: ceph osd pool get-quota tmp-quota-pool --format=json-pretty *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:29.168228 2afeee643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:29.178973 2afeee643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:29.245515 2afeee643700 0 lockdep stop "quota_max_bytes": 10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1310: test_mon_osd_pool_quota: ceph osd pool set-quota tmp-quota-pool max_bytes 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:29.316890 2b6165cba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:29.335631 2b6165cba700 -1 WARNING: the following dangerous and experimental features are enabled: * set-quota max_bytes = 0 for pool tmp-quota-pool 2016-06-07 20:28:30.319786 2b6165cba700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1311: test_mon_osd_pool_quota: ceph osd pool set-quota tmp-quota-pool max_objects 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:30.402336 2b732ffae700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:30.421593 2b732ffae700 -1 WARNING: the following dangerous and experimental features are enabled: * set-quota max_objects = 0 for pool tmp-quota-pool 2016-06-07 20:28:32.366218 2b732ffae700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1315: test_mon_osd_pool_quota: ceph osd pool get-quota tmp-quota-pool /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1315: test_mon_osd_pool_quota: grep 'max bytes.*N/A' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:32.458077 2b1aa6643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:32.463945 2b1aa6643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:33.355730 2b1aa6643700 0 lockdep stop max bytes : N/A /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1316: test_mon_osd_pool_quota: ceph osd pool get-quota tmp-quota-pool /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1316: test_mon_osd_pool_quota: grep 'max objects.*N/A' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:33.449828 2aae5ac5e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:33.449834 2aae5ac5e700 0 lockdep start 2016-06-07 20:28:33.461677 2aae5ac5e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:33.838769 2aae5ac5e700 0 lockdep stop max objects: N/A /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1319: test_mon_osd_pool_quota: ceph osd pool delete tmp-quota-pool tmp-quota-pool --yes-i-really-really-mean-it *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:33.933576 2b61c59de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:33.944787 2b61c59de700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'tmp-quota-pool' removed 2016-06-07 20:28:34.538561 2b61c59de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:34.630126 2ad9eaa08700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:34.630131 2ad9eaa08700 0 lockdep start 2016-06-07 20:28:34.637635 2ad9eaa08700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:36.030058 2ad9eaa08700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_pg /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1324: test_mon_pg: ceph pg debug unfound_objects_exist *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:36.126456 2ac2bcb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:36.133682 2ac2bcb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * FALSE 2016-06-07 20:28:36.658076 2ac2bcb6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1325: test_mon_pg: ceph pg debug degraded_pgs_exist *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:36.745606 2b0897d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:36.745611 2b0897d5c700 0 lockdep start 2016-06-07 20:28:36.752727 2b0897d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * FALSE 2016-06-07 20:28:36.877152 2b0897d5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1326: test_mon_pg: ceph pg deep-scrub 0.0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:36.971623 2b1f0efba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:36.984046 2b1f0efba700 -1 WARNING: the following dangerous and experimental features are enabled: * instructing pg 0.0 on osd.0 to deep-scrub 2016-06-07 20:28:37.355377 2b1f0efba700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1327: test_mon_pg: ceph pg dump *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:37.440358 2b9b79576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:37.440363 2b9b79576700 0 lockdep start 2016-06-07 20:28:37.446458 2b9b79576700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped all in format plain version 548 stamp 2016-06-07 20:28:35.247227 last_osdmap_epoch 241 last_pg_scan 241 full_ratio 0.99 nearfull_ratio 0.85 PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.0 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777538 0'0 194:56 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0 0 0 0 0 0 0 0 0 sum 0 0 0 0 0 0 0 0 OSD_STAT USED AVAIL TOTAL HB_IN PG_SUM 2 29767 kB 841 MB 916 MB [0,1] 8 1 29767 kB 841 MB 916 MB [0,2] 8 0 29766 kB 841 MB 916 MB [1,2] 8 sum 89301 kB 2523 MB 2750 MB 2016-06-07 20:28:37.786453 2b9b79576700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1328: test_mon_pg: ceph pg dump pgs_brief --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:37.875797 2b8e88b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:37.886753 2b8e88b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped pgs_brief in format json [{"pgid":"0.7","state":"active+clean","up":[1,0,2],"acting":[1,0,2],"up_primary":1,"acting_primary":1},{"pgid":"0.6","state":"active+clean","up":[0,1,2],"acting":[0,1,2],"up_primary":0,"acting_primary":0},{"pgid":"0.5","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0},{"pgid":"0.4","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0},{"pgid":"0.3","state":"active+clean","up":[2,0,1],"acting":[2,0,1],"up_primary":2,"acting_primary":2},{"pgid":"0.2","state":"active+clean","up":[0,1,2],"acting":[0,1,2],"up_primary":0,"acting_primary":0},{"pgid":"0.1","state":"active+clean","up":[2,0,1],"acting":[2,0,1],"up_primary":2,"acting_primary":2},{"pgid":"0.0","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0}] 2016-06-07 20:28:38.207415 2b8e88b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1329: test_mon_pg: ceph pg dump pgs --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:38.302958 2b571e643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:38.302965 2b571e643700 0 lockdep start 2016-06-07 20:28:38.312224 2b571e643700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped pgs in format json [{"pgid":"0.7","version":"0'0","reported_seq":"66","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777230","last_change":"2016-06-07 20:27:13.777230","last_active":"2016-06-07 20:27:13.777230","last_peered":"2016-06-07 20:27:13.777230","last_clean":"2016-06-07 20:27:13.777230","last_became_active":"2016-06-07 20:27:13.776922","last_became_peered":"2016-06-07 20:27:13.776922","last_unstale":"2016-06-07 20:27:13.777230","last_undegraded":"2016-06-07 20:27:13.777230","last_fullsized":"2016-06-07 20:27:13.777230","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043782","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043782","last_clean_scrub_stamp":"2016-06-07 20:16:16.043782","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[1,0,2],"acting":[1,0,2],"blocked_by":[],"up_primary":1,"acting_primary":1},{"pgid":"0.6","version":"0'0","reported_seq":"54","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.779436","last_change":"2016-06-07 20:27:13.779436","last_active":"2016-06-07 20:27:13.779436","last_peered":"2016-06-07 20:27:13.779436","last_clean":"2016-06-07 20:27:13.779436","last_became_active":"2016-06-07 20:27:13.779202","last_became_peered":"2016-06-07 20:27:13.779202","last_unstale":"2016-06-07 20:27:13.779436","last_undegraded":"2016-06-07 20:27:13.779436","last_fullsized":"2016-06-07 20:27:13.779436","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:22.631572","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:22.631572","last_clean_scrub_stamp":"2016-06-07 20:25:22.631572","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,1,2],"acting":[0,1,2],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.5","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778365","last_change":"2016-06-07 20:27:13.778365","last_active":"2016-06-07 20:27:13.778365","last_peered":"2016-06-07 20:27:13.778365","last_clean":"2016-06-07 20:27:13.778365","last_became_active":"2016-06-07 20:27:13.777780","last_became_peered":"2016-06-07 20:27:13.777780","last_unstale":"2016-06-07 20:27:13.778365","last_undegraded":"2016-06-07 20:27:13.778365","last_fullsized":"2016-06-07 20:27:13.778365","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:19.662334","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:19.662334","last_clean_scrub_stamp":"2016-06-07 20:25:19.662334","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.4","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778247","last_change":"2016-06-07 20:27:13.778247","last_active":"2016-06-07 20:27:13.778247","last_peered":"2016-06-07 20:27:13.778247","last_clean":"2016-06-07 20:27:13.778247","last_became_active":"2016-06-07 20:27:13.777573","last_became_peered":"2016-06-07 20:27:13.777573","last_unstale":"2016-06-07 20:27:13.778247","last_undegraded":"2016-06-07 20:27:13.778247","last_fullsized":"2016-06-07 20:27:13.778247","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:18.662129","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:18.662129","last_clean_scrub_stamp":"2016-06-07 20:25:18.662129","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.3","version":"0'0","reported_seq":"56","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777640","last_change":"2016-06-07 20:27:13.777640","last_active":"2016-06-07 20:27:13.777640","last_peered":"2016-06-07 20:27:13.777640","last_clean":"2016-06-07 20:27:13.777640","last_became_active":"2016-06-07 20:27:13.777081","last_became_peered":"2016-06-07 20:27:13.777081","last_unstale":"2016-06-07 20:27:13.777640","last_undegraded":"2016-06-07 20:27:13.777640","last_fullsized":"2016-06-07 20:27:13.777640","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043774","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043774","last_clean_scrub_stamp":"2016-06-07 20:16:16.043774","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[2,0,1],"acting":[2,0,1],"blocked_by":[],"up_primary":2,"acting_primary":2},{"pgid":"0.2","version":"0'0","reported_seq":"54","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778078","last_change":"2016-06-07 20:27:13.778078","last_active":"2016-06-07 20:27:13.778078","last_peered":"2016-06-07 20:27:13.778078","last_clean":"2016-06-07 20:27:13.778078","last_became_active":"2016-06-07 20:27:13.777445","last_became_peered":"2016-06-07 20:27:13.777445","last_unstale":"2016-06-07 20:27:13.778078","last_undegraded":"2016-06-07 20:27:13.778078","last_fullsized":"2016-06-07 20:27:13.778078","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:17.630976","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:17.630976","last_clean_scrub_stamp":"2016-06-07 20:25:17.630976","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,1,2],"acting":[0,1,2],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.1","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778973","last_change":"2016-06-07 20:27:13.778973","last_active":"2016-06-07 20:27:13.778973","last_peered":"2016-06-07 20:27:13.778973","last_clean":"2016-06-07 20:27:13.778973","last_became_active":"2016-06-07 20:27:13.778758","last_became_peered":"2016-06-07 20:27:13.778758","last_unstale":"2016-06-07 20:27:13.778973","last_undegraded":"2016-06-07 20:27:13.778973","last_fullsized":"2016-06-07 20:27:13.778973","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043770","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043770","last_clean_scrub_stamp":"2016-06-07 20:16:16.043770","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[2,0,1],"acting":[2,0,1],"blocked_by":[],"up_primary":2,"acting_primary":2},{"pgid":"0.0","version":"0'0","reported_seq":"56","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777538","last_change":"2016-06-07 20:27:13.777538","last_active":"2016-06-07 20:27:13.777538","last_peered":"2016-06-07 20:27:13.777538","last_clean":"2016-06-07 20:27:13.777538","last_became_active":"2016-06-07 20:27:13.777240","last_became_peered":"2016-06-07 20:27:13.777240","last_unstale":"2016-06-07 20:27:13.777538","last_undegraded":"2016-06-07 20:27:13.777538","last_fullsized":"2016-06-07 20:27:13.777538","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:16.630469","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:16.630469","last_clean_scrub_stamp":"2016-06-07 20:25:16.630469","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0}] 2016-06-07 20:28:38.835931 2b571e643700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1330: test_mon_pg: ceph pg dump pools --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:38.936120 2ae6f519a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:38.942672 2ae6f519a700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped pools in format json [{"poolid":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24}] 2016-06-07 20:28:39.007200 2ae6f519a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1331: test_mon_pg: ceph pg dump osds --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:39.091153 2b79ecf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:39.091158 2b79ecf79700 0 lockdep start 2016-06-07 20:28:39.098464 2b79ecf79700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped osds in format json [{"osd":2,"kb":961301832,"kb_used":30481812,"kb_avail":881965596,"hb_in":[0,1],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,1],"upper_bound":1024},"fs_perf_stat":{"commit_latency_ms":192,"apply_latency_ms":6513}},{"osd":1,"kb":961301832,"kb_used":30482108,"kb_avail":881965300,"hb_in":[0,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,6],"upper_bound":16384},"fs_perf_stat":{"commit_latency_ms":201,"apply_latency_ms":9991}},{"osd":0,"kb":961301832,"kb_used":30480928,"kb_avail":881966480,"hb_in":[1,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,12],"upper_bound":8192},"fs_perf_stat":{"commit_latency_ms":140,"apply_latency_ms":8001}}] 2016-06-07 20:28:39.155582 2b79ecf79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1332: test_mon_pg: ceph pg dump sum --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:39.235118 2afcfda1e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:39.235122 2afcfda1e700 0 lockdep start 2016-06-07 20:28:39.241128 2afcfda1e700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped sum in format json {"version":550,"stamp":"2016-06-07 20:28:40.127871","last_osdmap_epoch":241,"last_pg_scan":241,"min_last_epoch_clean":0,"full_ratio":0.990000,"near_full_ratio":0.850000,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24},"osd_stats_sum":{"kb":2883905496,"kb_used":91451236,"kb_avail":2645890988,"hb_in":[],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,0,0,6],"upper_bound":16384},"fs_perf_stat":{"commit_latency_ms":575,"apply_latency_ms":25193}},"osd_epochs":{},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0}} 2016-06-07 20:28:40.981838 2afcfda1e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1333: test_mon_pg: ceph pg dump all --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:41.074235 2b4803e80700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:41.074244 2b4803e80700 0 lockdep start 2016-06-07 20:28:41.087930 2b4803e80700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped all in format json {"version":551,"stamp":"2016-06-07 20:28:41.229501","last_osdmap_epoch":241,"last_pg_scan":241,"min_last_epoch_clean":0,"full_ratio":0.990000,"near_full_ratio":0.850000,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24},"osd_stats_sum":{"kb":2883905496,"kb_used":91453996,"kb_avail":2645888228,"hb_in":[],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"fs_perf_stat":{"commit_latency_ms":482,"apply_latency_ms":26687}},"osd_epochs":{},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0},"pg_stats":[{"pgid":"0.7","version":"0'0","reported_seq":"66","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777230","last_change":"2016-06-07 20:27:13.777230","last_active":"2016-06-07 20:27:13.777230","last_peered":"2016-06-07 20:27:13.777230","last_clean":"2016-06-07 20:27:13.777230","last_became_active":"2016-06-07 20:27:13.776922","last_became_peered":"2016-06-07 20:27:13.776922","last_unstale":"2016-06-07 20:27:13.777230","last_undegraded":"2016-06-07 20:27:13.777230","last_fullsized":"2016-06-07 20:27:13.777230","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043782","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043782","last_clean_scrub_stamp":"2016-06-07 20:16:16.043782","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[1,0,2],"acting":[1,0,2],"blocked_by":[],"up_primary":1,"acting_primary":1},{"pgid":"0.6","version":"0'0","reported_seq":"54","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.779436","last_change":"2016-06-07 20:27:13.779436","last_active":"2016-06-07 20:27:13.779436","last_peered":"2016-06-07 20:27:13.779436","last_clean":"2016-06-07 20:27:13.779436","last_became_active":"2016-06-07 20:27:13.779202","last_became_peered":"2016-06-07 20:27:13.779202","last_unstale":"2016-06-07 20:27:13.779436","last_undegraded":"2016-06-07 20:27:13.779436","last_fullsized":"2016-06-07 20:27:13.779436","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:22.631572","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:22.631572","last_clean_scrub_stamp":"2016-06-07 20:25:22.631572","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,1,2],"acting":[0,1,2],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.5","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778365","last_change":"2016-06-07 20:27:13.778365","last_active":"2016-06-07 20:27:13.778365","last_peered":"2016-06-07 20:27:13.778365","last_clean":"2016-06-07 20:27:13.778365","last_became_active":"2016-06-07 20:27:13.777780","last_became_peered":"2016-06-07 20:27:13.777780","last_unstale":"2016-06-07 20:27:13.778365","last_undegraded":"2016-06-07 20:27:13.778365","last_fullsized":"2016-06-07 20:27:13.778365","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:19.662334","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:19.662334","last_clean_scrub_stamp":"2016-06-07 20:25:19.662334","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.4","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778247","last_change":"2016-06-07 20:27:13.778247","last_active":"2016-06-07 20:27:13.778247","last_peered":"2016-06-07 20:27:13.778247","last_clean":"2016-06-07 20:27:13.778247","last_became_active":"2016-06-07 20:27:13.777573","last_became_peered":"2016-06-07 20:27:13.777573","last_unstale":"2016-06-07 20:27:13.778247","last_undegraded":"2016-06-07 20:27:13.778247","last_fullsized":"2016-06-07 20:27:13.778247","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:18.662129","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:18.662129","last_clean_scrub_stamp":"2016-06-07 20:25:18.662129","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.3","version":"0'0","reported_seq":"56","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777640","last_change":"2016-06-07 20:27:13.777640","last_active":"2016-06-07 20:27:13.777640","last_peered":"2016-06-07 20:27:13.777640","last_clean":"2016-06-07 20:27:13.777640","last_became_active":"2016-06-07 20:27:13.777081","last_became_peered":"2016-06-07 20:27:13.777081","last_unstale":"2016-06-07 20:27:13.777640","last_undegraded":"2016-06-07 20:27:13.777640","last_fullsized":"2016-06-07 20:27:13.777640","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043774","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043774","last_clean_scrub_stamp":"2016-06-07 20:16:16.043774","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[2,0,1],"acting":[2,0,1],"blocked_by":[],"up_primary":2,"acting_primary":2},{"pgid":"0.2","version":"0'0","reported_seq":"54","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778078","last_change":"2016-06-07 20:27:13.778078","last_active":"2016-06-07 20:27:13.778078","last_peered":"2016-06-07 20:27:13.778078","last_clean":"2016-06-07 20:27:13.778078","last_became_active":"2016-06-07 20:27:13.777445","last_became_peered":"2016-06-07 20:27:13.777445","last_unstale":"2016-06-07 20:27:13.778078","last_undegraded":"2016-06-07 20:27:13.778078","last_fullsized":"2016-06-07 20:27:13.778078","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:17.630976","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:17.630976","last_clean_scrub_stamp":"2016-06-07 20:25:17.630976","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,1,2],"acting":[0,1,2],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.1","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778973","last_change":"2016-06-07 20:27:13.778973","last_active":"2016-06-07 20:27:13.778973","last_peered":"2016-06-07 20:27:13.778973","last_clean":"2016-06-07 20:27:13.778973","last_became_active":"2016-06-07 20:27:13.778758","last_became_peered":"2016-06-07 20:27:13.778758","last_unstale":"2016-06-07 20:27:13.778973","last_undegraded":"2016-06-07 20:27:13.778973","last_fullsized":"2016-06-07 20:27:13.778973","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043770","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043770","last_clean_scrub_stamp":"2016-06-07 20:16:16.043770","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[2,0,1],"acting":[2,0,1],"blocked_by":[],"up_primary":2,"acting_primary":2},{"pgid":"0.0","version":"0'0","reported_seq":"56","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777538","last_change":"2016-06-07 20:27:13.777538","last_active":"2016-06-07 20:27:13.777538","last_peered":"2016-06-07 20:27:13.777538","last_clean":"2016-06-07 20:27:13.777538","last_became_active":"2016-06-07 20:27:13.777240","last_became_peered":"2016-06-07 20:27:13.777240","last_unstale":"2016-06-07 20:27:13.777538","last_undegraded":"2016-06-07 20:27:13.777538","last_fullsized":"2016-06-07 20:27:13.777538","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:16.630469","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:16.630469","last_clean_scrub_stamp":"2016-06-07 20:25:16.630469","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0}],"pool_stats":[{"poolid":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24}],"osd_stats":[{"osd":2,"kb":961301832,"kb_used":30484572,"kb_avail":881962836,"hb_in":[0,1],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":176,"apply_latency_ms":6786}},{"osd":1,"kb":961301832,"kb_used":30484868,"kb_avail":881962540,"hb_in":[0,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"fs_perf_stat":{"commit_latency_ms":108,"apply_latency_ms":11485}},{"osd":0,"kb":961301832,"kb_used":30484556,"kb_avail":881962852,"hb_in":[1,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":198,"apply_latency_ms":8416}}]} 2016-06-07 20:28:41.962562 2b4803e80700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1334: test_mon_pg: ceph pg dump pgs_brief osds --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:42.045870 2b4869bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:42.045876 2b4869bf9700 0 lockdep start 2016-06-07 20:28:42.051929 2b4869bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped osds,pgs_brief in format json [{"osd":2,"kb":961301832,"kb_used":30484572,"kb_avail":881962836,"hb_in":[0,1],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":176,"apply_latency_ms":6786}},{"osd":1,"kb":961301832,"kb_used":30484868,"kb_avail":881962540,"hb_in":[0,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"fs_perf_stat":{"commit_latency_ms":108,"apply_latency_ms":11485}},{"osd":0,"kb":961301832,"kb_used":30484556,"kb_avail":881962852,"hb_in":[1,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":198,"apply_latency_ms":8416}}][{"pgid":"0.7","state":"active+clean","up":[1,0,2],"acting":[1,0,2],"up_primary":1,"acting_primary":1},{"pgid":"0.6","state":"active+clean","up":[0,1,2],"acting":[0,1,2],"up_primary":0,"acting_primary":0},{"pgid":"0.5","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0},{"pgid":"0.4","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0},{"pgid":"0.3","state":"active+clean","up":[2,0,1],"acting":[2,0,1],"up_primary":2,"acting_primary":2},{"pgid":"0.2","state":"active+clean","up":[0,1,2],"acting":[0,1,2],"up_primary":0,"acting_primary":0},{"pgid":"0.1","state":"active+clean","up":[2,0,1],"acting":[2,0,1],"up_primary":2,"acting_primary":2},{"pgid":"0.0","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0}] 2016-06-07 20:28:42.144527 2b4869bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1335: test_mon_pg: ceph pg dump pools osds pgs_brief --format=json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:42.233225 2b7de99de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:42.233231 2b7de99de700 0 lockdep start 2016-06-07 20:28:42.244375 2b7de99de700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped osds,pgs_brief,pools in format json [{"poolid":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24}][{"osd":2,"kb":961301832,"kb_used":30484572,"kb_avail":881962836,"hb_in":[0,1],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":176,"apply_latency_ms":6786}},{"osd":1,"kb":961301832,"kb_used":30484868,"kb_avail":881962540,"hb_in":[0,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"fs_perf_stat":{"commit_latency_ms":108,"apply_latency_ms":11485}},{"osd":0,"kb":961301832,"kb_used":30484556,"kb_avail":881962852,"hb_in":[1,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":198,"apply_latency_ms":8416}}][{"pgid":"0.7","state":"active+clean","up":[1,0,2],"acting":[1,0,2],"up_primary":1,"acting_primary":1},{"pgid":"0.6","state":"active+clean","up":[0,1,2],"acting":[0,1,2],"up_primary":0,"acting_primary":0},{"pgid":"0.5","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0},{"pgid":"0.4","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0},{"pgid":"0.3","state":"active+clean","up":[2,0,1],"acting":[2,0,1],"up_primary":2,"acting_primary":2},{"pgid":"0.2","state":"active+clean","up":[0,1,2],"acting":[0,1,2],"up_primary":0,"acting_primary":0},{"pgid":"0.1","state":"active+clean","up":[2,0,1],"acting":[2,0,1],"up_primary":2,"acting_primary":2},{"pgid":"0.0","state":"active+clean","up":[0,2,1],"acting":[0,2,1],"up_primary":0,"acting_primary":0}] 2016-06-07 20:28:42.968555 2b7de99de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1336: test_mon_pg: ceph pg dump_json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:43.047899 2acb519de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:43.047904 2acb519de700 0 lockdep start 2016-06-07 20:28:43.054072 2acb519de700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped all in format json {"version":551,"stamp":"2016-06-07 20:28:41.229501","last_osdmap_epoch":241,"last_pg_scan":241,"min_last_epoch_clean":194,"full_ratio":0.990000,"near_full_ratio":0.850000,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24},"osd_stats_sum":{"kb":2883905496,"kb_used":91453996,"kb_avail":2645888228,"hb_in":[],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"fs_perf_stat":{"commit_latency_ms":482,"apply_latency_ms":26687}},"osd_epochs":{},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0},"pg_stats":[{"pgid":"0.7","version":"0'0","reported_seq":"66","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777230","last_change":"2016-06-07 20:27:13.777230","last_active":"2016-06-07 20:27:13.777230","last_peered":"2016-06-07 20:27:13.777230","last_clean":"2016-06-07 20:27:13.777230","last_became_active":"2016-06-07 20:27:13.776922","last_became_peered":"2016-06-07 20:27:13.776922","last_unstale":"2016-06-07 20:27:13.777230","last_undegraded":"2016-06-07 20:27:13.777230","last_fullsized":"2016-06-07 20:27:13.777230","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043782","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043782","last_clean_scrub_stamp":"2016-06-07 20:16:16.043782","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[1,0,2],"acting":[1,0,2],"blocked_by":[],"up_primary":1,"acting_primary":1},{"pgid":"0.6","version":"0'0","reported_seq":"54","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.779436","last_change":"2016-06-07 20:27:13.779436","last_active":"2016-06-07 20:27:13.779436","last_peered":"2016-06-07 20:27:13.779436","last_clean":"2016-06-07 20:27:13.779436","last_became_active":"2016-06-07 20:27:13.779202","last_became_peered":"2016-06-07 20:27:13.779202","last_unstale":"2016-06-07 20:27:13.779436","last_undegraded":"2016-06-07 20:27:13.779436","last_fullsized":"2016-06-07 20:27:13.779436","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:22.631572","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:22.631572","last_clean_scrub_stamp":"2016-06-07 20:25:22.631572","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,1,2],"acting":[0,1,2],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.5","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778365","last_change":"2016-06-07 20:27:13.778365","last_active":"2016-06-07 20:27:13.778365","last_peered":"2016-06-07 20:27:13.778365","last_clean":"2016-06-07 20:27:13.778365","last_became_active":"2016-06-07 20:27:13.777780","last_became_peered":"2016-06-07 20:27:13.777780","last_unstale":"2016-06-07 20:27:13.778365","last_undegraded":"2016-06-07 20:27:13.778365","last_fullsized":"2016-06-07 20:27:13.778365","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:19.662334","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:19.662334","last_clean_scrub_stamp":"2016-06-07 20:25:19.662334","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.4","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778247","last_change":"2016-06-07 20:27:13.778247","last_active":"2016-06-07 20:27:13.778247","last_peered":"2016-06-07 20:27:13.778247","last_clean":"2016-06-07 20:27:13.778247","last_became_active":"2016-06-07 20:27:13.777573","last_became_peered":"2016-06-07 20:27:13.777573","last_unstale":"2016-06-07 20:27:13.778247","last_undegraded":"2016-06-07 20:27:13.778247","last_fullsized":"2016-06-07 20:27:13.778247","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:18.662129","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:18.662129","last_clean_scrub_stamp":"2016-06-07 20:25:18.662129","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.3","version":"0'0","reported_seq":"56","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777640","last_change":"2016-06-07 20:27:13.777640","last_active":"2016-06-07 20:27:13.777640","last_peered":"2016-06-07 20:27:13.777640","last_clean":"2016-06-07 20:27:13.777640","last_became_active":"2016-06-07 20:27:13.777081","last_became_peered":"2016-06-07 20:27:13.777081","last_unstale":"2016-06-07 20:27:13.777640","last_undegraded":"2016-06-07 20:27:13.777640","last_fullsized":"2016-06-07 20:27:13.777640","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043774","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043774","last_clean_scrub_stamp":"2016-06-07 20:16:16.043774","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[2,0,1],"acting":[2,0,1],"blocked_by":[],"up_primary":2,"acting_primary":2},{"pgid":"0.2","version":"0'0","reported_seq":"54","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778078","last_change":"2016-06-07 20:27:13.778078","last_active":"2016-06-07 20:27:13.778078","last_peered":"2016-06-07 20:27:13.778078","last_clean":"2016-06-07 20:27:13.778078","last_became_active":"2016-06-07 20:27:13.777445","last_became_peered":"2016-06-07 20:27:13.777445","last_unstale":"2016-06-07 20:27:13.778078","last_undegraded":"2016-06-07 20:27:13.778078","last_fullsized":"2016-06-07 20:27:13.778078","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:17.630976","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:17.630976","last_clean_scrub_stamp":"2016-06-07 20:25:17.630976","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,1,2],"acting":[0,1,2],"blocked_by":[],"up_primary":0,"acting_primary":0},{"pgid":"0.1","version":"0'0","reported_seq":"55","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.778973","last_change":"2016-06-07 20:27:13.778973","last_active":"2016-06-07 20:27:13.778973","last_peered":"2016-06-07 20:27:13.778973","last_clean":"2016-06-07 20:27:13.778973","last_became_active":"2016-06-07 20:27:13.778758","last_became_peered":"2016-06-07 20:27:13.778758","last_unstale":"2016-06-07 20:27:13.778973","last_undegraded":"2016-06-07 20:27:13.778973","last_fullsized":"2016-06-07 20:27:13.778973","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:16:16.043770","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:16:16.043770","last_clean_scrub_stamp":"2016-06-07 20:16:16.043770","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[2,0,1],"acting":[2,0,1],"blocked_by":[],"up_primary":2,"acting_primary":2},{"pgid":"0.0","version":"0'0","reported_seq":"56","reported_epoch":"194","state":"active+clean","last_fresh":"2016-06-07 20:27:13.777538","last_change":"2016-06-07 20:27:13.777538","last_active":"2016-06-07 20:27:13.777538","last_peered":"2016-06-07 20:27:13.777538","last_clean":"2016-06-07 20:27:13.777538","last_became_active":"2016-06-07 20:27:13.777240","last_became_peered":"2016-06-07 20:27:13.777240","last_unstale":"2016-06-07 20:27:13.777538","last_undegraded":"2016-06-07 20:27:13.777538","last_fullsized":"2016-06-07 20:27:13.777538","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":1,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2016-06-07 20:25:16.630469","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2016-06-07 20:25:16.630469","last_clean_scrub_stamp":"2016-06-07 20:25:16.630469","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"up":[0,2,1],"acting":[0,2,1],"blocked_by":[],"up_primary":0,"acting_primary":0}],"pool_stats":[{"poolid":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24}],"osd_stats":[{"osd":2,"kb":961301832,"kb_used":30484572,"kb_avail":881962836,"hb_in":[0,1],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":176,"apply_latency_ms":6786}},{"osd":1,"kb":961301832,"kb_used":30484868,"kb_avail":881962540,"hb_in":[0,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,1],"upper_bound":4096},"fs_perf_stat":{"commit_latency_ms":108,"apply_latency_ms":11485}},{"osd":0,"kb":961301832,"kb_used":30484556,"kb_avail":881962852,"hb_in":[1,2],"hb_out":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"fs_perf_stat":{"commit_latency_ms":198,"apply_latency_ms":8416}}]} 2016-06-07 20:28:43.174697 2acb519de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1337: test_mon_pg: ceph pg dump_pools_json *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:43.250978 2aebd4b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:43.250983 2aebd4b6c700 0 lockdep start 2016-06-07 20:28:43.258485 2aebd4b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped pools in format json [{"poolid":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24}] 2016-06-07 20:28:43.434408 2aebd4b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1338: test_mon_pg: ceph pg dump_stuck inactive *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:43.512914 2ac52e33a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:43.512919 2ac52e33a700 0 lockdep start 2016-06-07 20:28:43.519232 2ac52e33a700 -1 WARNING: the following dangerous and experimental features are enabled: * ok 2016-06-07 20:28:44.048298 2ac52e33a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1339: test_mon_pg: ceph pg dump_stuck unclean *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:44.137198 2b6633278700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:44.137203 2b6633278700 0 lockdep start 2016-06-07 20:28:44.148896 2b6633278700 -1 WARNING: the following dangerous and experimental features are enabled: * ok 2016-06-07 20:28:44.526167 2b6633278700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1340: test_mon_pg: ceph pg dump_stuck stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:44.627828 2b6ebb71d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:44.627841 2b6ebb71d700 0 lockdep start 2016-06-07 20:28:44.639479 2b6ebb71d700 -1 WARNING: the following dangerous and experimental features are enabled: * ok 2016-06-07 20:28:46.016147 2b6ebb71d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1341: test_mon_pg: ceph pg dump_stuck undersized *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:46.104342 2b2e2e643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:46.104352 2b2e2e643700 0 lockdep start 2016-06-07 20:28:46.116854 2b2e2e643700 -1 WARNING: the following dangerous and experimental features are enabled: * ok 2016-06-07 20:28:46.655530 2b2e2e643700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1342: test_mon_pg: ceph pg dump_stuck degraded *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:46.739989 2b7d7cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:46.739997 2b7d7cd71700 0 lockdep start 2016-06-07 20:28:46.752039 2b7d7cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * ok 2016-06-07 20:28:47.269638 2b7d7cd71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1343: test_mon_pg: ceph pg ls *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:47.368324 2accaba44700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:47.376527 2accaba44700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:47.457958 2accaba44700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1344: test_mon_pg: ceph pg ls 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:47.549714 2b19e11da700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:47.562435 2b19e11da700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:47.988044 2b19e11da700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1345: test_mon_pg: ceph pg ls stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:48.078234 2b192fcef700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:48.091155 2b192fcef700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:48.359621 2b192fcef700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1346: test_mon_pg: expect_false ceph pg ls scrubq /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph pg ls scrubq *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:48.437241 2b326a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:48.437245 2b326a91a700 0 lockdep start 2016-06-07 20:28:48.444699 2b326a91a700 -1 WARNING: the following dangerous and experimental features are enabled: * scrubq not valid: scrubq not in active|clean|down|replay|splitting|scrubbing|degraded|inconsistent|peering|repair|recovering|backfill_wait|incomplete|stale|remapped|deep_scrub|backfill|backfill_toofull|recovery_wait|undersized|activating|peered Invalid command: unused arguments: [u'scrubq'] pg ls {} {active|clean|down|replay|splitting|scrubbing|degraded|inconsistent|peering|repair|recovering|backfill_wait|incomplete|stale|remapped|deep_scrub|backfill|backfill_toofull|recovery_wait|undersized|activating|peered [active|clean|down|replay|splitting|scrubbing|degraded|inconsistent|peering|repair|recovering|backfill_wait|incomplete|stale|remapped|deep_scrub|backfill|backfill_toofull|recovery_wait|undersized|activating|peered...]} : list pg with specific pool, osd, state Error EINVAL: invalid command 2016-06-07 20:28:48.505254 2b326a91a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1347: test_mon_pg: ceph pg ls active stale repair recovering *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:48.593238 2b6b1f75d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:48.593247 2b6b1f75d700 0 lockdep start 2016-06-07 20:28:48.606925 2b6b1f75d700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:48.827466 2b6b1f75d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1348: test_mon_pg: ceph pg ls 0 active *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:48.909289 2b79aaf7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:48.909300 2b79aaf7a700 0 lockdep start 2016-06-07 20:28:48.921872 2b79aaf7a700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:49.398946 2b79aaf7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1349: test_mon_pg: ceph pg ls 0 active stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:49.483457 2b1f275f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:49.483467 2b1f275f2700 0 lockdep start 2016-06-07 20:28:49.502138 2b1f275f2700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:49.565709 2b1f275f2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1350: test_mon_pg: ceph pg ls-by-primary osd.0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:49.653432 2afb89cba700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:49.653443 2afb89cba700 0 lockdep start 2016-06-07 20:28:49.667982 2afb89cba700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 2016-06-07 20:28:49.862052 2afb89cba700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1351: test_mon_pg: ceph pg ls-by-primary osd.0 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:49.962208 2b41df7de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:49.962213 2b41df7de700 0 lockdep start 2016-06-07 20:28:49.968177 2b41df7de700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 2016-06-07 20:28:51.941506 2b41df7de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1352: test_mon_pg: ceph pg ls-by-primary osd.0 active *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:52.058030 2ae8d97d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:52.063854 2ae8d97d5700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 2016-06-07 20:28:53.038245 2ae8d97d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1353: test_mon_pg: ceph pg ls-by-primary osd.0 active stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:53.124131 2b088e704700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:53.135429 2b088e704700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 2016-06-07 20:28:53.405606 2b088e704700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1354: test_mon_pg: ceph pg ls-by-primary osd.0 0 active stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:53.486174 2b335d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:53.496977 2b335d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 2016-06-07 20:28:53.554930 2b335d19a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1355: test_mon_pg: ceph pg ls-by-osd osd.0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:53.652079 2b5dd59de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:53.652086 2b5dd59de700 0 lockdep start 2016-06-07 20:28:53.657578 2b5dd59de700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:53.781404 2b5dd59de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1356: test_mon_pg: ceph pg ls-by-osd osd.0 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:53.866909 2b7b96b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:53.866914 2b7b96b5d700 0 lockdep start 2016-06-07 20:28:53.872159 2b7b96b5d700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:54.197796 2b7b96b5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1357: test_mon_pg: ceph pg ls-by-osd osd.0 active *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:54.291136 2b9d77e2b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:54.302846 2b9d77e2b700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean+scrubbing+deep 2016-06-07 20:28:44.446022 0'0 241:57 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:16.630469 0'0 2016-06-07 20:25:16.630469 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:54.572399 2b9d77e2b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1358: test_mon_pg: ceph pg ls-by-osd osd.0 active stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:54.663150 2baa86643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:54.663156 2baa86643700 0 lockdep start 2016-06-07 20:28:54.683081 2baa86643700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:28:49.940266 0'0 241:58 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:28:49.940223 0'0 2016-06-07 20:28:49.940223 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:55.475983 2baa86643700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1359: test_mon_pg: ceph pg ls-by-osd osd.0 0 active stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:55.567251 2b96fb07b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:55.567262 2b96fb07b700 0 lockdep start 2016-06-07 20:28:55.587833 2b96fb07b700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:28:49.940266 0'0 241:58 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:28:49.940223 0'0 2016-06-07 20:28:49.940223 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:55.960173 2b96fb07b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1360: test_mon_pg: ceph pg ls-by-pool rbd *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:56.057462 2b16ab4e4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:56.069222 2b16ab4e4700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:28:49.940266 0'0 241:58 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:28:49.940223 0'0 2016-06-07 20:28:49.940223 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:56.140282 2b16ab4e4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1361: test_mon_pg: ceph pg ls-by-pool rbd active stale *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:56.228225 2b700d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:56.228232 2b700d19a700 0 lockdep start 2016-06-07 20:28:56.237797 2b700d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES LOG DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP 0.0 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:28:49.940266 0'0 241:58 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:28:49.940223 0'0 2016-06-07 20:28:49.940223 0.1 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778973 0'0 194:55 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043770 0'0 2016-06-07 20:16:16.043770 0.2 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778078 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:17.630976 0'0 2016-06-07 20:25:17.630976 0.3 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777640 0'0 194:56 [2,0,1] 2 [2,0,1] 2 0'0 2016-06-07 20:16:16.043774 0'0 2016-06-07 20:16:16.043774 0.4 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778247 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:18.662129 0'0 2016-06-07 20:25:18.662129 0.5 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.778365 0'0 194:55 [0,2,1] 0 [0,2,1] 0 0'0 2016-06-07 20:25:19.662334 0'0 2016-06-07 20:25:19.662334 0.6 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.779436 0'0 194:54 [0,1,2] 0 [0,1,2] 0 0'0 2016-06-07 20:25:22.631572 0'0 2016-06-07 20:25:22.631572 0.7 0 0 0 0 0 0 0 0 active+clean 2016-06-07 20:27:13.777230 0'0 194:66 [1,0,2] 1 [1,0,2] 1 0'0 2016-06-07 20:16:16.043782 0'0 2016-06-07 20:16:16.043782 2016-06-07 20:28:56.303767 2b700d19a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1364: test_mon_pg: ceph pg getmap -o /tmp/cephtool19083/map.19083 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:56.388703 2afb8ee62700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:56.396286 2afb8ee62700 -1 WARNING: the following dangerous and experimental features are enabled: * got pgmap version 560 2016-06-07 20:28:56.716666 2afb8ee62700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1365: test_mon_pg: '[' -s /tmp/cephtool19083/map.19083 ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1366: test_mon_pg: ceph pg map 0.0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1366: test_mon_pg: grep acting *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:56.800201 2b5b59eca700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:56.800209 2b5b59eca700 0 lockdep start 2016-06-07 20:28:56.810765 2b5b59eca700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:56.901091 2b5b59eca700 0 lockdep stop osdmap e241 pg 0.0 (0.0) -> up [0,2,1] acting [0,2,1] /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1367: test_mon_pg: ceph pg repair 0.0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:56.978290 2ac6882b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:57.044506 2ac6882b4700 -1 WARNING: the following dangerous and experimental features are enabled: * instructing pg 0.0 on osd.0 to repair 2016-06-07 20:28:57.602870 2ac6882b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1368: test_mon_pg: ceph pg scrub 0.0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:57.678514 2ba42fbce700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:57.678519 2ba42fbce700 0 lockdep start 2016-06-07 20:28:57.684791 2ba42fbce700 -1 WARNING: the following dangerous and experimental features are enabled: * instructing pg 0.0 on osd.0 to scrub 2016-06-07 20:28:57.774838 2ba42fbce700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1370: test_mon_pg: ceph pg set_full_ratio 0.90 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:57.860655 2b8996212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:57.871524 2b8996212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:58.709193 2b8996212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1371: test_mon_pg: ceph pg dump --format=plain /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1371: test_mon_pg: grep '^full_ratio 0.9' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:58.795995 2ab913f00700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:58.802891 2ab913f00700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped all in format plain 2016-06-07 20:28:59.121682 2ab913f00700 0 lockdep stop full_ratio 0.9 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1372: test_mon_pg: ceph pg set_full_ratio 0.95 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:28:59.206125 2b7e5ba04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:28:59.217352 2b7e5ba04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:00.207323 2b7e5ba04700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1373: test_mon_pg: ceph pg set_nearfull_ratio 0.90 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:00.301573 2ae86fed1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:00.308565 2ae86fed1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:00.985064 2ae86fed1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1374: test_mon_pg: ceph pg dump --format=plain /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1374: test_mon_pg: grep '^nearfull_ratio 0.9' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:01.074781 2b8b7dbf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:01.239789 2b8b7dbf9700 -1 WARNING: the following dangerous and experimental features are enabled: * dumped all in format plain 2016-06-07 20:29:02.385589 2b8b7dbf9700 0 lockdep stop nearfull_ratio 0.9 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1375: test_mon_pg: ceph pg set_nearfull_ratio 0.85 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:02.473033 2b9768d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:02.473038 2b9768d71700 0 lockdep start 2016-06-07 20:29:02.484006 2b9768d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:03.176487 2b9768d71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1376: test_mon_pg: grep pgs: /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1376: test_mon_pg: ceph pg stat *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:03.254699 2abcbe43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:03.260210 2abcbe43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:03.683692 2abcbe43e700 0 lockdep stop v567: 8 pgs: 8 active+clean; 0 bytes data, 89470 MB used, 2523 GB / 2750 GB avail /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1377: test_mon_pg: ceph pg 0.0 query *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:03.774534 2b37b57d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:03.774544 2b37b57d5700 0 lockdep start 2016-06-07 20:29:03.785815 2b37b57d5700 -1 WARNING: the following dangerous and experimental features are enabled: * { "state": "active+clean", "snap_trimq": "[]", "epoch": 241, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "actingbackfill": [ "0", "1", "2" ], "info": { "pgid": "0.0", "last_update": "0'0", "last_complete": "0'0", "log_tail": "0'0", "last_user_version": 0, "last_backfill": "MAX", "last_backfill_bitwise": 0, "purged_snaps": "[]", "history": { "epoch_created": 1, "last_epoch_started": 194, "last_epoch_clean": 194, "last_epoch_split": 0, "last_epoch_marked_full": 168, "same_up_since": 191, "same_interval_since": 193, "same_primary_since": 193, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871" }, "stats": { "version": "0'0", "reported_seq": "60", "reported_epoch": "241", "state": "active+clean", "last_fresh": "2016-06-07 20:29:00.655941", "last_change": "2016-06-07 20:29:00.655941", "last_active": "2016-06-07 20:29:00.655941", "last_peered": "2016-06-07 20:29:00.655941", "last_clean": "2016-06-07 20:29:00.655941", "last_became_active": "2016-06-07 20:27:13.777240", "last_became_peered": "2016-06-07 20:27:13.777240", "last_unstale": "2016-06-07 20:29:00.655941", "last_undegraded": "2016-06-07 20:29:00.655941", "last_fullsized": "2016-06-07 20:29:00.655941", "mapping_epoch": 193, "log_start": "0'0", "ondisk_log_start": "0'0", "created": 1, "last_epoch_clean": 194, "parent": "0.0", "parent_split_bits": 0, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871", "log_size": 0, "ondisk_log_size": 0, "stats_invalid": false, "dirty_stats_invalid": false, "omap_stats_invalid": false, "hitset_stats_invalid": false, "hitset_bytes_stats_invalid": false, "pin_stats_invalid": false, "stat_sum": { "num_bytes": 0, "num_objects": 0, "num_object_clones": 0, "num_object_copies": 0, "num_objects_missing_on_primary": 0, "num_objects_missing": 0, "num_objects_degraded": 0, "num_objects_misplaced": 0, "num_objects_unfound": 0, "num_objects_dirty": 0, "num_whiteouts": 0, "num_read": 0, "num_read_kb": 0, "num_write": 0, "num_write_kb": 0, "num_scrub_errors": 0, "num_shallow_scrub_errors": 0, "num_deep_scrub_errors": 0, "num_objects_recovered": 0, "num_bytes_recovered": 0, "num_keys_recovered": 0, "num_objects_omap": 0, "num_objects_hit_set_archive": 0, "num_bytes_hit_set_archive": 0, "num_flush": 0, "num_flush_kb": 0, "num_evict": 0, "num_evict_kb": 0, "num_promote": 0, "num_flush_mode_high": 0, "num_flush_mode_low": 0, "num_evict_mode_some": 0, "num_evict_mode_full": 0, "num_objects_pinned": 0 }, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "blocked_by": [], "up_primary": 0, "acting_primary": 0 }, "empty": 1, "dne": 0, "incomplete": 0, "last_epoch_started": 194, "hit_set_history": { "current_last_update": "0'0", "history": [] } }, "peer_info": [ { "peer": "1", "pgid": "0.0", "last_update": "0'0", "last_complete": "0'0", "log_tail": "0'0", "last_user_version": 0, "last_backfill": "MAX", "last_backfill_bitwise": 0, "purged_snaps": "[]", "history": { "epoch_created": 1, "last_epoch_started": 194, "last_epoch_clean": 194, "last_epoch_split": 0, "last_epoch_marked_full": 168, "same_up_since": 191, "same_interval_since": 193, "same_primary_since": 193, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871" }, "stats": { "version": "0'0", "reported_seq": "48", "reported_epoch": "189", "state": "remapped+peering", "last_fresh": "2016-06-07 20:27:08.125638", "last_change": "2016-06-07 20:27:08.121961", "last_active": "2016-06-07 20:26:12.503454", "last_peered": "2016-06-07 20:26:12.503454", "last_clean": "2016-06-07 20:26:12.503454", "last_became_active": "2016-06-07 20:26:12.503302", "last_became_peered": "2016-06-07 20:26:12.503302", "last_unstale": "2016-06-07 20:27:08.125638", "last_undegraded": "2016-06-07 20:27:08.125638", "last_fullsized": "2016-06-07 20:27:08.125638", "mapping_epoch": 193, "log_start": "0'0", "ondisk_log_start": "0'0", "created": 1, "last_epoch_clean": 186, "parent": "0.0", "parent_split_bits": 0, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:25:16.630469", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:25:16.630469", "last_clean_scrub_stamp": "2016-06-07 20:25:16.630469", "log_size": 0, "ondisk_log_size": 0, "stats_invalid": false, "dirty_stats_invalid": false, "omap_stats_invalid": false, "hitset_stats_invalid": false, "hitset_bytes_stats_invalid": false, "pin_stats_invalid": false, "stat_sum": { "num_bytes": 0, "num_objects": 0, "num_object_clones": 0, "num_object_copies": 0, "num_objects_missing_on_primary": 0, "num_objects_missing": 0, "num_objects_degraded": 0, "num_objects_misplaced": 0, "num_objects_unfound": 0, "num_objects_dirty": 0, "num_whiteouts": 0, "num_read": 0, "num_read_kb": 0, "num_write": 0, "num_write_kb": 0, "num_scrub_errors": 0, "num_shallow_scrub_errors": 0, "num_deep_scrub_errors": 0, "num_objects_recovered": 0, "num_bytes_recovered": 0, "num_keys_recovered": 0, "num_objects_omap": 0, "num_objects_hit_set_archive": 0, "num_bytes_hit_set_archive": 0, "num_flush": 0, "num_flush_kb": 0, "num_evict": 0, "num_evict_kb": 0, "num_promote": 0, "num_flush_mode_high": 0, "num_flush_mode_low": 0, "num_evict_mode_some": 0, "num_evict_mode_full": 0, "num_objects_pinned": 0 }, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "blocked_by": [], "up_primary": 0, "acting_primary": 0 }, "empty": 1, "dne": 0, "incomplete": 0, "last_epoch_started": 194, "hit_set_history": { "current_last_update": "0'0", "history": [] } }, { "peer": "2", "pgid": "0.0", "last_update": "0'0", "last_complete": "0'0", "log_tail": "0'0", "last_user_version": 0, "last_backfill": "MAX", "last_backfill_bitwise": 0, "purged_snaps": "[]", "history": { "epoch_created": 1, "last_epoch_started": 194, "last_epoch_clean": 194, "last_epoch_split": 0, "last_epoch_marked_full": 168, "same_up_since": 191, "same_interval_since": 193, "same_primary_since": 193, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871" }, "stats": { "version": "0'0", "reported_seq": "55", "reported_epoch": "192", "state": "remapped", "last_fresh": "2016-06-07 20:27:11.700906", "last_change": "2016-06-07 20:27:11.700906", "last_active": "2016-06-07 20:27:09.432576", "last_peered": "2016-06-07 20:27:09.432576", "last_clean": "2016-06-07 20:26:12.503454", "last_became_active": "2016-06-07 20:27:09.431734", "last_became_peered": "2016-06-07 20:27:09.431734", "last_unstale": "2016-06-07 20:27:11.700906", "last_undegraded": "2016-06-07 20:27:11.700906", "last_fullsized": "2016-06-07 20:27:11.700906", "mapping_epoch": 193, "log_start": "0'0", "ondisk_log_start": "0'0", "created": 1, "last_epoch_clean": 190, "parent": "0.0", "parent_split_bits": 0, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:25:16.630469", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:25:16.630469", "last_clean_scrub_stamp": "2016-06-07 20:25:16.630469", "log_size": 0, "ondisk_log_size": 0, "stats_invalid": false, "dirty_stats_invalid": false, "omap_stats_invalid": false, "hitset_stats_invalid": false, "hitset_bytes_stats_invalid": false, "pin_stats_invalid": false, "stat_sum": { "num_bytes": 0, "num_objects": 0, "num_object_clones": 0, "num_object_copies": 0, "num_objects_missing_on_primary": 0, "num_objects_missing": 0, "num_objects_degraded": 0, "num_objects_misplaced": 0, "num_objects_unfound": 0, "num_objects_dirty": 0, "num_whiteouts": 0, "num_read": 0, "num_read_kb": 0, "num_write": 0, "num_write_kb": 0, "num_scrub_errors": 0, "num_shallow_scrub_errors": 0, "num_deep_scrub_errors": 0, "num_objects_recovered": 0, "num_bytes_recovered": 0, "num_keys_recovered": 0, "num_objects_omap": 0, "num_objects_hit_set_archive": 0, "num_bytes_hit_set_archive": 0, "num_flush": 0, "num_flush_kb": 0, "num_evict": 0, "num_evict_kb": 0, "num_promote": 0, "num_flush_mode_high": 0, "num_flush_mode_low": 0, "num_evict_mode_some": 0, "num_evict_mode_full": 0, "num_objects_pinned": 0 }, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "blocked_by": [], "up_primary": 0, "acting_primary": 0 }, "empty": 1, "dne": 0, "incomplete": 0, "last_epoch_started": 194, "hit_set_history": { "current_last_update": "0'0", "history": [] } } ], "recovery_state": [ { "name": "Started\/Primary\/Active", "enter_time": "2016-06-07 20:27:13.573399", "might_have_unfound": [], "recovery_progress": { "backfill_targets": [], "waiting_on_backfill": [], "last_backfill_started": "MIN", "backfill_info": { "begin": "MIN", "end": "MIN", "objects": [] }, "peer_backfill_info": [], "backfills_in_flight": [], "recovering": [], "pg_backend": { "pull_from_peer": [], "pushing": [] } }, "scrub": { "scrubber.epoch_start": "193", "scrubber.active": false, "scrubber.state": "INACTIVE", "scrubber.start": "MIN", "scrubber.end": "MIN", "scrubber.subset_last_update": "0'0", "scrubber.deep": false, "scrubber.seed": 0, "scrubber.waiting_on": 0, "scrubber.waiting_on_whom": [] } }, { "name": "Started", "enter_time": "2016-06-07 20:27:12.632984" } ], "agent_state": {} } 2016-06-07 20:29:03.800627 2b37b57d5700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1378: test_mon_pg: ceph tell 0.0 query *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:03.893121 2b60e11da700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:03.893128 2b60e11da700 0 lockdep start 2016-06-07 20:29:03.904664 2b60e11da700 -1 WARNING: the following dangerous and experimental features are enabled: * { "state": "active+clean", "snap_trimq": "[]", "epoch": 241, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "actingbackfill": [ "0", "1", "2" ], "info": { "pgid": "0.0", "last_update": "0'0", "last_complete": "0'0", "log_tail": "0'0", "last_user_version": 0, "last_backfill": "MAX", "last_backfill_bitwise": 0, "purged_snaps": "[]", "history": { "epoch_created": 1, "last_epoch_started": 194, "last_epoch_clean": 194, "last_epoch_split": 0, "last_epoch_marked_full": 168, "same_up_since": 191, "same_interval_since": 193, "same_primary_since": 193, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871" }, "stats": { "version": "0'0", "reported_seq": "60", "reported_epoch": "241", "state": "active+clean", "last_fresh": "2016-06-07 20:29:00.655941", "last_change": "2016-06-07 20:29:00.655941", "last_active": "2016-06-07 20:29:00.655941", "last_peered": "2016-06-07 20:29:00.655941", "last_clean": "2016-06-07 20:29:00.655941", "last_became_active": "2016-06-07 20:27:13.777240", "last_became_peered": "2016-06-07 20:27:13.777240", "last_unstale": "2016-06-07 20:29:00.655941", "last_undegraded": "2016-06-07 20:29:00.655941", "last_fullsized": "2016-06-07 20:29:00.655941", "mapping_epoch": 193, "log_start": "0'0", "ondisk_log_start": "0'0", "created": 1, "last_epoch_clean": 194, "parent": "0.0", "parent_split_bits": 0, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871", "log_size": 0, "ondisk_log_size": 0, "stats_invalid": false, "dirty_stats_invalid": false, "omap_stats_invalid": false, "hitset_stats_invalid": false, "hitset_bytes_stats_invalid": false, "pin_stats_invalid": false, "stat_sum": { "num_bytes": 0, "num_objects": 0, "num_object_clones": 0, "num_object_copies": 0, "num_objects_missing_on_primary": 0, "num_objects_missing": 0, "num_objects_degraded": 0, "num_objects_misplaced": 0, "num_objects_unfound": 0, "num_objects_dirty": 0, "num_whiteouts": 0, "num_read": 0, "num_read_kb": 0, "num_write": 0, "num_write_kb": 0, "num_scrub_errors": 0, "num_shallow_scrub_errors": 0, "num_deep_scrub_errors": 0, "num_objects_recovered": 0, "num_bytes_recovered": 0, "num_keys_recovered": 0, "num_objects_omap": 0, "num_objects_hit_set_archive": 0, "num_bytes_hit_set_archive": 0, "num_flush": 0, "num_flush_kb": 0, "num_evict": 0, "num_evict_kb": 0, "num_promote": 0, "num_flush_mode_high": 0, "num_flush_mode_low": 0, "num_evict_mode_some": 0, "num_evict_mode_full": 0, "num_objects_pinned": 0 }, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "blocked_by": [], "up_primary": 0, "acting_primary": 0 }, "empty": 1, "dne": 0, "incomplete": 0, "last_epoch_started": 194, "hit_set_history": { "current_last_update": "0'0", "history": [] } }, "peer_info": [ { "peer": "1", "pgid": "0.0", "last_update": "0'0", "last_complete": "0'0", "log_tail": "0'0", "last_user_version": 0, "last_backfill": "MAX", "last_backfill_bitwise": 0, "purged_snaps": "[]", "history": { "epoch_created": 1, "last_epoch_started": 194, "last_epoch_clean": 194, "last_epoch_split": 0, "last_epoch_marked_full": 168, "same_up_since": 191, "same_interval_since": 193, "same_primary_since": 193, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871" }, "stats": { "version": "0'0", "reported_seq": "48", "reported_epoch": "189", "state": "remapped+peering", "last_fresh": "2016-06-07 20:27:08.125638", "last_change": "2016-06-07 20:27:08.121961", "last_active": "2016-06-07 20:26:12.503454", "last_peered": "2016-06-07 20:26:12.503454", "last_clean": "2016-06-07 20:26:12.503454", "last_became_active": "2016-06-07 20:26:12.503302", "last_became_peered": "2016-06-07 20:26:12.503302", "last_unstale": "2016-06-07 20:27:08.125638", "last_undegraded": "2016-06-07 20:27:08.125638", "last_fullsized": "2016-06-07 20:27:08.125638", "mapping_epoch": 193, "log_start": "0'0", "ondisk_log_start": "0'0", "created": 1, "last_epoch_clean": 186, "parent": "0.0", "parent_split_bits": 0, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:25:16.630469", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:25:16.630469", "last_clean_scrub_stamp": "2016-06-07 20:25:16.630469", "log_size": 0, "ondisk_log_size": 0, "stats_invalid": false, "dirty_stats_invalid": false, "omap_stats_invalid": false, "hitset_stats_invalid": false, "hitset_bytes_stats_invalid": false, "pin_stats_invalid": false, "stat_sum": { "num_bytes": 0, "num_objects": 0, "num_object_clones": 0, "num_object_copies": 0, "num_objects_missing_on_primary": 0, "num_objects_missing": 0, "num_objects_degraded": 0, "num_objects_misplaced": 0, "num_objects_unfound": 0, "num_objects_dirty": 0, "num_whiteouts": 0, "num_read": 0, "num_read_kb": 0, "num_write": 0, "num_write_kb": 0, "num_scrub_errors": 0, "num_shallow_scrub_errors": 0, "num_deep_scrub_errors": 0, "num_objects_recovered": 0, "num_bytes_recovered": 0, "num_keys_recovered": 0, "num_objects_omap": 0, "num_objects_hit_set_archive": 0, "num_bytes_hit_set_archive": 0, "num_flush": 0, "num_flush_kb": 0, "num_evict": 0, "num_evict_kb": 0, "num_promote": 0, "num_flush_mode_high": 0, "num_flush_mode_low": 0, "num_evict_mode_some": 0, "num_evict_mode_full": 0, "num_objects_pinned": 0 }, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "blocked_by": [], "up_primary": 0, "acting_primary": 0 }, "empty": 1, "dne": 0, "incomplete": 0, "last_epoch_started": 194, "hit_set_history": { "current_last_update": "0'0", "history": [] } }, { "peer": "2", "pgid": "0.0", "last_update": "0'0", "last_complete": "0'0", "log_tail": "0'0", "last_user_version": 0, "last_backfill": "MAX", "last_backfill_bitwise": 0, "purged_snaps": "[]", "history": { "epoch_created": 1, "last_epoch_started": 194, "last_epoch_clean": 194, "last_epoch_split": 0, "last_epoch_marked_full": 168, "same_up_since": 191, "same_interval_since": 193, "same_primary_since": 193, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:29:00.655871", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:28:49.940223", "last_clean_scrub_stamp": "2016-06-07 20:29:00.655871" }, "stats": { "version": "0'0", "reported_seq": "55", "reported_epoch": "192", "state": "remapped", "last_fresh": "2016-06-07 20:27:11.700906", "last_change": "2016-06-07 20:27:11.700906", "last_active": "2016-06-07 20:27:09.432576", "last_peered": "2016-06-07 20:27:09.432576", "last_clean": "2016-06-07 20:26:12.503454", "last_became_active": "2016-06-07 20:27:09.431734", "last_became_peered": "2016-06-07 20:27:09.431734", "last_unstale": "2016-06-07 20:27:11.700906", "last_undegraded": "2016-06-07 20:27:11.700906", "last_fullsized": "2016-06-07 20:27:11.700906", "mapping_epoch": 193, "log_start": "0'0", "ondisk_log_start": "0'0", "created": 1, "last_epoch_clean": 190, "parent": "0.0", "parent_split_bits": 0, "last_scrub": "0'0", "last_scrub_stamp": "2016-06-07 20:25:16.630469", "last_deep_scrub": "0'0", "last_deep_scrub_stamp": "2016-06-07 20:25:16.630469", "last_clean_scrub_stamp": "2016-06-07 20:25:16.630469", "log_size": 0, "ondisk_log_size": 0, "stats_invalid": false, "dirty_stats_invalid": false, "omap_stats_invalid": false, "hitset_stats_invalid": false, "hitset_bytes_stats_invalid": false, "pin_stats_invalid": false, "stat_sum": { "num_bytes": 0, "num_objects": 0, "num_object_clones": 0, "num_object_copies": 0, "num_objects_missing_on_primary": 0, "num_objects_missing": 0, "num_objects_degraded": 0, "num_objects_misplaced": 0, "num_objects_unfound": 0, "num_objects_dirty": 0, "num_whiteouts": 0, "num_read": 0, "num_read_kb": 0, "num_write": 0, "num_write_kb": 0, "num_scrub_errors": 0, "num_shallow_scrub_errors": 0, "num_deep_scrub_errors": 0, "num_objects_recovered": 0, "num_bytes_recovered": 0, "num_keys_recovered": 0, "num_objects_omap": 0, "num_objects_hit_set_archive": 0, "num_bytes_hit_set_archive": 0, "num_flush": 0, "num_flush_kb": 0, "num_evict": 0, "num_evict_kb": 0, "num_promote": 0, "num_flush_mode_high": 0, "num_flush_mode_low": 0, "num_evict_mode_some": 0, "num_evict_mode_full": 0, "num_objects_pinned": 0 }, "up": [ 0, 2, 1 ], "acting": [ 0, 2, 1 ], "blocked_by": [], "up_primary": 0, "acting_primary": 0 }, "empty": 1, "dne": 0, "incomplete": 0, "last_epoch_started": 194, "hit_set_history": { "current_last_update": "0'0", "history": [] } } ], "recovery_state": [ { "name": "Started\/Primary\/Active", "enter_time": "2016-06-07 20:27:13.573399", "might_have_unfound": [], "recovery_progress": { "backfill_targets": [], "waiting_on_backfill": [], "last_backfill_started": "MIN", "backfill_info": { "begin": "MIN", "end": "MIN", "objects": [] }, "peer_backfill_info": [], "backfills_in_flight": [], "recovering": [], "pg_backend": { "pull_from_peer": [], "pushing": [] } }, "scrub": { "scrubber.epoch_start": "193", "scrubber.active": false, "scrubber.state": "INACTIVE", "scrubber.start": "MIN", "scrubber.end": "MIN", "scrubber.subset_last_update": "0'0", "scrubber.deep": false, "scrubber.seed": 0, "scrubber.waiting_on": 0, "scrubber.waiting_on_whom": [] } }, { "name": "Started", "enter_time": "2016-06-07 20:27:12.632984" } ], "agent_state": {} } 2016-06-07 20:29:04.028849 2b60e11da700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1379: test_mon_pg: ceph quorum enter *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:04.122340 2b6e30026700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:04.129890 2b6e30026700 -1 WARNING: the following dangerous and experimental features are enabled: * started responding to quorum, initiated new election 2016-06-07 20:29:04.302146 2b6e30026700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1380: test_mon_pg: ceph quorum_status *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:04.391651 2b56ee859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:04.399009 2b56ee859700 -1 WARNING: the following dangerous and experimental features are enabled: * {"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","monmap":{"epoch":1,"fsid":"8fedb721-e6d4-42fc-a3ae-49153fde812f","modified":"2016-06-07 20:15:54.015730","created":"2016-06-07 20:15:54.015730","mons":[{"rank":0,"name":"a","addr":"127.0.0.1:7202\/0"},{"rank":1,"name":"b","addr":"127.0.0.1:7203\/0"},{"rank":2,"name":"c","addr":"127.0.0.1:7204\/0"}]}} 2016-06-07 20:29:04.958559 2b56ee859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1381: test_mon_pg: ceph report /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1381: test_mon_pg: grep osd_stats *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:05.037516 2ba0c3eda700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:05.046214 2ba0c3eda700 -1 WARNING: the following dangerous and experimental features are enabled: * report 1097914002 2016-06-07 20:29:06.933231 2ba0c3eda700 0 lockdep stop "osd_stats_sum": { "osd_stats": [ /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1382: test_mon_pg: ceph status *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:07.022822 2accae4ff700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:07.022827 2accae4ff700 0 lockdep start 2016-06-07 20:29:07.026546 2accae4ff700 -1 WARNING: the following dangerous and experimental features are enabled: * cluster 8fedb721-e6d4-42fc-a3ae-49153fde812f health HEALTH_OK monmap e1: 3 mons at {a=127.0.0.1:7202/0,b=127.0.0.1:7203/0,c=127.0.0.1:7204/0} election epoch 8, quorum 0,1,2 a,b,c osdmap e241: 3 osds: 3 up, 3 in flags sortbitwise,require_jewel_osds pgmap v569: 8 pgs, 1 pools, 0 bytes data, 0 objects 89500 MB used, 2523 GB / 2750 GB avail 8 active+clean 2016-06-07 20:29:07.111492 2accae4ff700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1383: test_mon_pg: ceph -s *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:07.188376 2b731b4e4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:07.192275 2b731b4e4700 -1 WARNING: the following dangerous and experimental features are enabled: * cluster 8fedb721-e6d4-42fc-a3ae-49153fde812f health HEALTH_OK monmap e1: 3 mons at {a=127.0.0.1:7202/0,b=127.0.0.1:7203/0,c=127.0.0.1:7204/0} election epoch 8, quorum 0,1,2 a,b,c osdmap e241: 3 osds: 3 up, 3 in flags sortbitwise,require_jewel_osds pgmap v569: 8 pgs, 1 pools, 0 bytes data, 0 objects 89500 MB used, 2523 GB / 2750 GB avail 8 active+clean 2016-06-07 20:29:07.855485 2b731b4e4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1388: test_mon_pg: ceph tell osd.0 version *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:07.936643 2b3701c39700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:07.950743 2b3701c39700 -1 WARNING: the following dangerous and experimental features are enabled: * { "version": "ceph version 10.2.0-2079-g47591d6 (47591d620ef48c668472ed467eb5bbf0eda89a70)" } 2016-06-07 20:29:07.967107 2b3701c39700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1389: test_mon_pg: expect_false ceph tell osd.9999 version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph tell osd.9999 version *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:08.049849 2b4a0ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:08.049855 2b4a0ab5d700 0 lockdep start 2016-06-07 20:29:08.056141 2b4a0ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOENT: problem getting command descriptions from osd.9999 2016-06-07 20:29:08.321746 2b4a0ab5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1390: test_mon_pg: expect_false ceph tell osd.foo version /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph tell osd.foo version *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:08.415435 2ae8be43e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:08.426708 2ae8be43e700 -1 WARNING: the following dangerous and experimental features are enabled: * error handling command target: osd id foo not integer 2016-06-07 20:29:08.431963 2ae8be43e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1394: test_mon_pg: ceph tell osd.0 dump_pg_recovery_stats /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1394: test_mon_pg: grep Started *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:08.502221 2ad85d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:08.502225 2ad85d19a700 0 lockdep start 2016-06-07 20:29:08.512275 2ad85d19a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:08.530807 2ad85d19a700 0 lockdep stop "Started", "Started", "Started", "Started", "Started", "Started", "Started", "Started", "Started" "Started", "Started", "Started", "Started", "Started", "Started", /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1396: test_mon_pg: ceph osd reweight 0 0.9 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:08.609097 2ba7cc233700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:08.609103 2ba7cc233700 0 lockdep start 2016-06-07 20:29:08.620457 2ba7cc233700 -1 WARNING: the following dangerous and experimental features are enabled: * reweighted osd.0 to 0.9 (e666) 2016-06-07 20:29:09.184985 2ba7cc233700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1397: test_mon_pg: expect_false ceph osd reweight 0 -1 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd reweight 0 -1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:09.259713 2b2e56e22700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:09.372780 2b2e56e22700 -1 WARNING: the following dangerous and experimental features are enabled: * Invalid command: -1.0 not in range [0.0, 1.0] osd reweight : reweight osd to 0.0 < < 1.0 Error EINVAL: invalid command 2016-06-07 20:29:10.397120 2b2e56e22700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1398: test_mon_pg: ceph osd reweight osd.0 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:10.467564 2afe9375d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:10.475562 2afe9375d700 -1 WARNING: the following dangerous and experimental features are enabled: * reweighted osd.0 to 1 (10000) 2016-06-07 20:29:11.035334 2afe9375d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1400: test_mon_pg: ceph osd primary-affinity osd.0 .9 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:11.136674 2ae1342b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:11.136679 2ae1342b4700 0 lockdep start 2016-06-07 20:29:11.142493 2ae1342b4700 -1 WARNING: the following dangerous and experimental features are enabled: * set osd.0 primary-affinity to 0.9 (8589822) 2016-06-07 20:29:11.738494 2ae1342b4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1401: test_mon_pg: expect_false ceph osd primary-affinity osd.0 -2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd primary-affinity osd.0 -2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:11.831491 2ba51b3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:11.831496 2ba51b3e3700 0 lockdep start 2016-06-07 20:29:11.837719 2ba51b3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * Invalid command: -2.0 not in range [0.0, 1.0] osd primary-affinity : adjust osd primary-affinity from 0.0 <= <= 1.0 Error EINVAL: invalid command 2016-06-07 20:29:12.364390 2ba51b3e3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1402: test_mon_pg: expect_false ceph osd primary-affinity osd.9999 .5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd primary-affinity osd.9999 .5 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:12.442427 2b3c419de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:12.442432 2b3c419de700 0 lockdep start 2016-06-07 20:29:12.448177 2b3c419de700 -1 WARNING: the following dangerous and experimental features are enabled: * Error ENOENT: osd.9999 does not exist 2016-06-07 20:29:12.508003 2b3c419de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1403: test_mon_pg: ceph osd primary-affinity osd.0 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:12.591212 2b88c73e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:12.602739 2b88c73e3700 -1 WARNING: the following dangerous and experimental features are enabled: * set osd.0 primary-affinity to 1 (8655362) 2016-06-07 20:29:13.274402 2b88c73e3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1405: test_mon_pg: ceph osd pg-temp 0.0 0 1 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:13.366940 2acc0af7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:13.366945 2acc0af7a700 0 lockdep start 2016-06-07 20:29:13.377355 2acc0af7a700 -1 WARNING: the following dangerous and experimental features are enabled: * set 0.0 pg_temp mapping to [0,1,2] 2016-06-07 20:29:16.876652 2acc0af7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1406: test_mon_pg: ceph osd pg-temp 0.0 osd.1 osd.0 osd.2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:16.958280 2afaa7d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:16.970395 2afaa7d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * set 0.0 pg_temp mapping to [1,0,2] 2016-06-07 20:29:21.096318 2afaa7d5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1407: test_mon_pg: expect_false ceph osd pg-temp asdf qwer /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pg-temp asdf qwer *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:21.158349 2b71e691a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:21.158354 2b71e691a700 0 lockdep start 2016-06-07 20:29:21.165135 2b71e691a700 -1 WARNING: the following dangerous and experimental features are enabled: * Invalid command: pgid has no . osd pg-temp { [...]} : set pg_temp mapping pgid:[ [...]] (developers only) Error EINVAL: invalid command 2016-06-07 20:29:21.223822 2b71e691a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1408: test_mon_pg: expect_false ceph osd pg-temp 0.0 asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pg-temp 0.0 asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:21.299057 2afed33e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:21.310178 2afed33e3700 -1 WARNING: the following dangerous and experimental features are enabled: * asdf not valid: osd id asdf not integer Invalid command: unused arguments: [u'asdf'] osd pg-temp { [...]} : set pg_temp mapping pgid:[ [...]] (developers only) Error EINVAL: invalid command 2016-06-07 20:29:21.365813 2afed33e3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1409: test_mon_pg: expect_false ceph osd pg-temp 0.0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pg-temp 0.0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:21.448269 2b0786859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:21.454070 2b0786859700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: unable to parse 'id' value(s) '' 2016-06-07 20:29:24.091194 2b0786859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1992: main: set +x *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:24.178430 2aea8cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:24.178437 2aea8cb6c700 0 lockdep start 2016-06-07 20:29:24.184118 2aea8cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:25.211825 2aea8cb6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1991: main: test_mon_osd_pool_set /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1416: test_mon_osd_pool_set: TEST_POOL_GETSET=pool_getset /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1417: test_mon_osd_pool_set: ceph osd pool create pool_getset 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:25.286536 2b7041bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:25.286541 2b7041bf9700 0 lockdep start 2016-06-07 20:29:25.292326 2b7041bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'pool_getset' created 2016-06-07 20:29:28.834958 2b7041bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1418: test_mon_osd_pool_set: wait_for_clean /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1083: wait_for_clean: local status=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1084: wait_for_clean: local num_active_clean=-1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1085: wait_for_clean: local cur_active_clean /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1086: wait_for_clean: local -i timer=0 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1087: wait_for_clean: get_num_pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1001: get_num_pgs: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1002: get_num_pgs: xmlstarlet sel -t -m //pgmap/num_pgs -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1087: wait_for_clean: local num_pgs=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1088: wait_for_clean: test 9 '!=' 0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' -1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1097: wait_for_clean: timer=0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1098: wait_for_clean: num_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 0 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' 8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 1 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=2 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' 8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 2 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=3 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' 8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 3 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=4 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' 8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 4 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=5 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: break /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1108: wait_for_clean: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1419: test_mon_osd_pool_set: ceph osd pool get pool_getset all *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:37.432858 2ac5c1815700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:37.446380 2ac5c1815700 -1 WARNING: the following dangerous and experimental features are enabled: * size: 3 min_size: 1 crash_replay_interval: 0 pg_num: 1 pgp_num: 1 crush_ruleset: 0 hashpspool: true nodelete: false nopgchange: false nosizechange: false write_fadvise_dontneed: false noscrub: false nodeep-scrub: false use_gmt_hitset: 1 auid: 0 min_write_recency_for_promote: 0 fast_read: 0 2016-06-07 20:29:37.911041 2ac5c1815700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1421: test_mon_osd_pool_set: for s in pg_num pgp_num size min_size crash_replay_interval crush_ruleset /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1422: test_mon_osd_pool_set: ceph osd pool get pool_getset pg_num *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:37.994727 2b27b9e09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:37.994732 2b27b9e09700 0 lockdep start 2016-06-07 20:29:38.005943 2b27b9e09700 -1 WARNING: the following dangerous and experimental features are enabled: * pg_num: 1 2016-06-07 20:29:38.326450 2b27b9e09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1421: test_mon_osd_pool_set: for s in pg_num pgp_num size min_size crash_replay_interval crush_ruleset /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1422: test_mon_osd_pool_set: ceph osd pool get pool_getset pgp_num *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:38.404519 2ae62e10e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:38.404525 2ae62e10e700 0 lockdep start 2016-06-07 20:29:38.414531 2ae62e10e700 -1 WARNING: the following dangerous and experimental features are enabled: * pgp_num: 1 2016-06-07 20:29:38.700981 2ae62e10e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1421: test_mon_osd_pool_set: for s in pg_num pgp_num size min_size crash_replay_interval crush_ruleset /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1422: test_mon_osd_pool_set: ceph osd pool get pool_getset size *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:38.783822 2af110334700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:38.783826 2af110334700 0 lockdep start 2016-06-07 20:29:38.789701 2af110334700 -1 WARNING: the following dangerous and experimental features are enabled: * size: 3 2016-06-07 20:29:39.162709 2af110334700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1421: test_mon_osd_pool_set: for s in pg_num pgp_num size min_size crash_replay_interval crush_ruleset /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1422: test_mon_osd_pool_set: ceph osd pool get pool_getset min_size *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:39.250070 2b68f0233700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:39.260222 2b68f0233700 -1 WARNING: the following dangerous and experimental features are enabled: * min_size: 1 2016-06-07 20:29:39.584753 2b68f0233700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1421: test_mon_osd_pool_set: for s in pg_num pgp_num size min_size crash_replay_interval crush_ruleset /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1422: test_mon_osd_pool_set: ceph osd pool get pool_getset crash_replay_interval *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:39.661098 2b91d1bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:39.661102 2b91d1bf9700 0 lockdep start 2016-06-07 20:29:39.667244 2b91d1bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * crash_replay_interval: 0 2016-06-07 20:29:40.963128 2b91d1bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1421: test_mon_osd_pool_set: for s in pg_num pgp_num size min_size crash_replay_interval crush_ruleset /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1422: test_mon_osd_pool_set: ceph osd pool get pool_getset crush_ruleset *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:41.044185 2b9e3600d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:41.044190 2b9e3600d700 0 lockdep start 2016-06-07 20:29:41.054307 2b9e3600d700 -1 WARNING: the following dangerous and experimental features are enabled: * crush_ruleset: 0 2016-06-07 20:29:41.117505 2b9e3600d700 0 lockdep stop //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1425: test_mon_osd_pool_set: ceph osd pool get pool_getset size //home/yuriw/ceph/qa/workunits/cephtool/test.sh:1425: test_mon_osd_pool_set: sed -e 's/size: //' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:41.202010 2aab8ef7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:41.202015 2aab8ef7a700 0 lockdep start 2016-06-07 20:29:41.208041 2aab8ef7a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:42.629461 2aab8ef7a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1425: test_mon_osd_pool_set: old_size=3 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1426: test_mon_osd_pool_set: (( new_size = old_size + 1 )) /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1427: test_mon_osd_pool_set: ceph osd pool set pool_getset size 4 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:42.697723 2b4c02859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:42.707300 2b4c02859700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 size to 4 2016-06-07 20:29:43.526150 2b4c02859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1428: test_mon_osd_pool_set: ceph osd pool get pool_getset size /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1428: test_mon_osd_pool_set: grep 'size: 4' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:43.623239 2b8d935b2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:43.623243 2b8d935b2700 0 lockdep start 2016-06-07 20:29:43.628913 2b8d935b2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:44.774295 2b8d935b2700 0 lockdep stop size: 4 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1429: test_mon_osd_pool_set: ceph osd pool set pool_getset size 3 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:44.847589 2ace9bd32700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:44.858911 2ace9bd32700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 size to 3 2016-06-07 20:29:45.527479 2ace9bd32700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1431: test_mon_osd_pool_set: ceph osd pool create pool_erasure 1 1 erasure *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:45.605471 2ad52b4e4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:45.617368 2ad52b4e4700 -1 WARNING: the following dangerous and experimental features are enabled: * pool 'pool_erasure' created 2016-06-07 20:29:46.336389 2ad52b4e4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1432: test_mon_osd_pool_set: wait_for_clean /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1083: wait_for_clean: local status=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1084: wait_for_clean: local num_active_clean=-1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1085: wait_for_clean: local cur_active_clean /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1086: wait_for_clean: local -i timer=0 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1087: wait_for_clean: get_num_pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1001: get_num_pgs: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1002: get_num_pgs: xmlstarlet sel -t -m //pgmap/num_pgs -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1087: wait_for_clean: local num_pgs=10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1088: wait_for_clean: test 10 '!=' 0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' -1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1097: wait_for_clean: timer=0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1098: wait_for_clean: num_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 0 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' 8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 1 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=2 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 8 = 10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 8 '!=' 8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 2 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=3 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 10 = 10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: break /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1108: wait_for_clean: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1433: test_mon_osd_pool_set: set +e /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1434: test_mon_osd_pool_set: ceph osd pool set pool_erasure size 4444 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1435: test_mon_osd_pool_set: check_response 'not change the size' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:94: check_response: expected_string='not change the size' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:95: check_response: retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:96: check_response: expected_retcode= /home/yuriw/ceph/qa/workunits/cephtool/test.sh:97: check_response: '[' '' -a '!=' ']' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:102: check_response: grep --quiet -- 'not change the size' /tmp/cephtool19083/test_invalid.19083 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1436: test_mon_osd_pool_set: set -e /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1437: test_mon_osd_pool_set: ceph osd pool get pool_erasure erasure_code_profile *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:52.310514 2ac0834a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:52.310519 2ac0834a4700 0 lockdep start 2016-06-07 20:29:52.316624 2ac0834a4700 -1 WARNING: the following dangerous and experimental features are enabled: * erasure_code_profile: default 2016-06-07 20:29:52.588700 2ac0834a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1439: test_mon_osd_pool_set: auid=5555 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1440: test_mon_osd_pool_set: ceph osd pool set pool_getset auid 5555 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:52.670665 2b78dc374700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:52.670673 2b78dc374700 0 lockdep start 2016-06-07 20:29:52.681267 2b78dc374700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 auid to 5555 2016-06-07 20:29:53.913435 2b78dc374700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1441: test_mon_osd_pool_set: grep 5555 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:54.001432 2b1757423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:54.001438 2b1757423700 0 lockdep start 2016-06-07 20:29:54.013142 2b1757423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:55.358336 2b1757423700 0 lockdep stop auid: 5555 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1442: test_mon_osd_pool_set: ceph --format=xml osd pool get pool_getset auid /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1442: test_mon_osd_pool_set: grep 5555 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:55.448723 2af8c31f7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:55.448728 2af8c31f7700 0 lockdep start 2016-06-07 20:29:55.454567 2af8c31f7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:56.226835 2af8c31f7700 0 lockdep stop pool_getset255555 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1443: test_mon_osd_pool_set: ceph osd pool set pool_getset auid 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:56.302777 2b6617c56700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:56.302789 2b6617c56700 0 lockdep start 2016-06-07 20:29:56.313850 2b6617c56700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 auid to 0 2016-06-07 20:29:57.599185 2b6617c56700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1445: test_mon_osd_pool_set: for flag in hashpspool nodelete nopgchange nosizechange write_fadvise_dontneed noscrub nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1446: test_mon_osd_pool_set: ceph osd pool set pool_getset hashpspool false *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:57.692184 2b5023943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:57.711319 2b5023943700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 hashpspool to false 2016-06-07 20:29:58.682681 2b5023943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: grep 'hashpspool: false' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: ceph osd pool get pool_getset hashpspool *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:58.779235 2aea16987700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:58.779242 2aea16987700 0 lockdep start 2016-06-07 20:29:58.790801 2aea16987700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:59.411501 2aea16987700 0 lockdep stop hashpspool: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1448: test_mon_osd_pool_set: ceph osd pool set pool_getset hashpspool true *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:29:59.493783 2ac00f03b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:29:59.493792 2ac00f03b700 0 lockdep start 2016-06-07 20:29:59.506227 2ac00f03b700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 hashpspool to true 2016-06-07 20:30:00.482350 2ac00f03b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: grep 'hashpspool: true' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: ceph osd pool get pool_getset hashpspool *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:00.566520 2adb2ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:00.566526 2adb2ad61700 0 lockdep start 2016-06-07 20:30:00.577153 2adb2ad61700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:01.137272 2adb2ad61700 0 lockdep stop hashpspool: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1450: test_mon_osd_pool_set: ceph osd pool set pool_getset hashpspool 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:01.219201 2b71c3943700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:01.219212 2b71c3943700 0 lockdep start 2016-06-07 20:30:01.230300 2b71c3943700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 hashpspool to 1 2016-06-07 20:30:03.296904 2b71c3943700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: ceph osd pool get pool_getset hashpspool /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: grep 'hashpspool: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:03.386437 2ad770434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:03.386442 2ad770434700 0 lockdep start 2016-06-07 20:30:03.392557 2ad770434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:03.570548 2ad770434700 0 lockdep stop hashpspool: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1452: test_mon_osd_pool_set: ceph osd pool set pool_getset hashpspool 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:03.657315 2aacb295a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:03.657320 2aacb295a700 0 lockdep start 2016-06-07 20:30:03.667652 2aacb295a700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 hashpspool to 0 2016-06-07 20:30:04.302030 2aacb295a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: ceph osd pool get pool_getset hashpspool /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: grep 'hashpspool: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:04.385111 2ac0ad576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:04.385121 2ac0ad576700 0 lockdep start 2016-06-07 20:30:04.397458 2ac0ad576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:05.759445 2ac0ad576700 0 lockdep stop hashpspool: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1454: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset hashpspool asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset hashpspool asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:05.858100 2b6a3d9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:05.858106 2b6a3d9de700 0 lockdep start 2016-06-07 20:30:05.868171 2b6a3d9de700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:05.995442 2b6a3d9de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1455: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset hashpspool 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset hashpspool 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:06.088716 2b502b531700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:06.108142 2b502b531700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:06.792993 2b502b531700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1445: test_mon_osd_pool_set: for flag in hashpspool nodelete nopgchange nosizechange write_fadvise_dontneed noscrub nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1446: test_mon_osd_pool_set: ceph osd pool set pool_getset nodelete false *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:06.883365 2b527cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:06.883372 2b527cf79700 0 lockdep start 2016-06-07 20:30:06.894868 2b527cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodelete to false 2016-06-07 20:30:07.583536 2b527cf79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: ceph osd pool get pool_getset nodelete /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: grep 'nodelete: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:07.674091 2af035576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:07.674097 2af035576700 0 lockdep start 2016-06-07 20:30:07.681239 2af035576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:08.206235 2af035576700 0 lockdep stop nodelete: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1448: test_mon_osd_pool_set: ceph osd pool set pool_getset nodelete true *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:08.296693 2b91301f3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:08.296699 2b91301f3700 0 lockdep start 2016-06-07 20:30:08.308175 2b91301f3700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodelete to true 2016-06-07 20:30:09.045218 2b91301f3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: grep 'nodelete: true' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: ceph osd pool get pool_getset nodelete *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:09.139923 2b5981bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:09.152093 2b5981bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:09.671082 2b5981bf9700 0 lockdep stop nodelete: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1450: test_mon_osd_pool_set: ceph osd pool set pool_getset nodelete 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:09.746561 2b80ce00d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:09.756432 2b80ce00d700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodelete to 1 2016-06-07 20:30:10.282120 2b80ce00d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: ceph osd pool get pool_getset nodelete /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: grep 'nodelete: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:10.371865 2b902de09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:10.371873 2b902de09700 0 lockdep start 2016-06-07 20:30:10.384014 2b902de09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:12.329082 2b902de09700 0 lockdep stop nodelete: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1452: test_mon_osd_pool_set: ceph osd pool set pool_getset nodelete 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:12.409835 2adc6643e700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:12.409841 2adc6643e700 0 lockdep start 2016-06-07 20:30:12.420155 2adc6643e700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodelete to 0 2016-06-07 20:30:13.143495 2adc6643e700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: ceph osd pool get pool_getset nodelete /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: grep 'nodelete: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:13.232912 2b0d0a643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:13.232918 2b0d0a643700 0 lockdep start 2016-06-07 20:30:13.242436 2b0d0a643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:13.715923 2b0d0a643700 0 lockdep stop nodelete: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1454: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nodelete asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nodelete asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:13.802766 2acd46859700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:13.814210 2acd46859700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:14.096720 2acd46859700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1455: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nodelete 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nodelete 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:14.178424 2ac45ae62700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:14.178430 2ac45ae62700 0 lockdep start 2016-06-07 20:30:14.188726 2ac45ae62700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:14.414469 2ac45ae62700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1445: test_mon_osd_pool_set: for flag in hashpspool nodelete nopgchange nosizechange write_fadvise_dontneed noscrub nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1446: test_mon_osd_pool_set: ceph osd pool set pool_getset nopgchange false *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:14.513263 2ba7dab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:14.533990 2ba7dab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nopgchange to false 2016-06-07 20:30:15.974583 2ba7dab5d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: ceph osd pool get pool_getset nopgchange /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: grep 'nopgchange: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:16.075364 2b5201bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:16.075369 2b5201bf9700 0 lockdep start 2016-06-07 20:30:16.139055 2b5201bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:17.082716 2b5201bf9700 0 lockdep stop nopgchange: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1448: test_mon_osd_pool_set: ceph osd pool set pool_getset nopgchange true *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:17.171497 2ada8cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:17.192225 2ada8cb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nopgchange to true 2016-06-07 20:30:17.866589 2ada8cb6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: ceph osd pool get pool_getset nopgchange /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: grep 'nopgchange: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:17.952483 2b9281bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:17.952488 2b9281bf9700 0 lockdep start 2016-06-07 20:30:17.958169 2b9281bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:18.633558 2b9281bf9700 0 lockdep stop nopgchange: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1450: test_mon_osd_pool_set: ceph osd pool set pool_getset nopgchange 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:18.721981 2aef52da1700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:18.721987 2aef52da1700 0 lockdep start 2016-06-07 20:30:18.740866 2aef52da1700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nopgchange to 1 2016-06-07 20:30:19.628739 2aef52da1700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: ceph osd pool get pool_getset nopgchange /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: grep 'nopgchange: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:19.716685 2b6b559de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:19.728184 2b6b559de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:20.817126 2b6b559de700 0 lockdep stop nopgchange: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1452: test_mon_osd_pool_set: ceph osd pool set pool_getset nopgchange 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:20.906211 2b3afcb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:20.906216 2b3afcb6c700 0 lockdep start 2016-06-07 20:30:20.918185 2b3afcb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nopgchange to 0 2016-06-07 20:30:22.357955 2b3afcb6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: ceph osd pool get pool_getset nopgchange /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: grep 'nopgchange: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:22.439747 2b8b6ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:22.452479 2b8b6ab5d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:22.978738 2b8b6ab5d700 0 lockdep stop nopgchange: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1454: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nopgchange asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nopgchange asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:23.066023 2abda22d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:23.072075 2abda22d3700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:23.291626 2abda22d3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1455: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nopgchange 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nopgchange 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:23.391278 2aef7cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:23.391284 2aef7cf79700 0 lockdep start 2016-06-07 20:30:23.401451 2aef7cf79700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:23.721796 2aef7cf79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1445: test_mon_osd_pool_set: for flag in hashpspool nodelete nopgchange nosizechange write_fadvise_dontneed noscrub nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1446: test_mon_osd_pool_set: ceph osd pool set pool_getset nosizechange false *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:23.801452 2b58aee22700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:23.801457 2b58aee22700 0 lockdep start 2016-06-07 20:30:23.823770 2b58aee22700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nosizechange to false 2016-06-07 20:30:25.642554 2b58aee22700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: ceph osd pool get pool_getset nosizechange /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: grep 'nosizechange: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:25.726066 2b45e1576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:25.738290 2b45e1576700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:31.730443 2b45e1576700 0 lockdep stop nosizechange: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1448: test_mon_osd_pool_set: ceph osd pool set pool_getset nosizechange true *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:31.812053 2b116fd9c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:31.812058 2b116fd9c700 0 lockdep start 2016-06-07 20:30:31.823254 2b116fd9c700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nosizechange to true 2016-06-07 20:30:32.644238 2b116fd9c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: ceph osd pool get pool_getset nosizechange /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: grep 'nosizechange: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:32.717148 2ade09395700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:32.717153 2ade09395700 0 lockdep start 2016-06-07 20:30:32.722828 2ade09395700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:33.247521 2ade09395700 0 lockdep stop nosizechange: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1450: test_mon_osd_pool_set: ceph osd pool set pool_getset nosizechange 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:33.331292 2b7b0f605700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:33.331297 2b7b0f605700 0 lockdep start 2016-06-07 20:30:33.341109 2b7b0f605700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nosizechange to 1 2016-06-07 20:30:34.018271 2b7b0f605700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: grep 'nosizechange: true' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: ceph osd pool get pool_getset nosizechange *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:34.096016 2b02942b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:34.107695 2b02942b4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:34.669108 2b02942b4700 0 lockdep stop nosizechange: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1452: test_mon_osd_pool_set: ceph osd pool set pool_getset nosizechange 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:34.755911 2ad5b75b2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:34.755917 2ad5b75b2700 0 lockdep start 2016-06-07 20:30:34.766940 2ad5b75b2700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nosizechange to 0 2016-06-07 20:30:37.911636 2ad5b75b2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: ceph osd pool get pool_getset nosizechange /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: grep 'nosizechange: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:38.008649 2b8fd7a04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:38.008661 2b8fd7a04700 0 lockdep start 2016-06-07 20:30:38.028261 2b8fd7a04700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:38.795019 2b8fd7a04700 0 lockdep stop nosizechange: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1454: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nosizechange asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nosizechange asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:38.890499 2afea919a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:38.890505 2afea919a700 0 lockdep start 2016-06-07 20:30:38.902069 2afea919a700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:39.159789 2afea919a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1455: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nosizechange 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nosizechange 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:39.271377 2b68f4d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:39.282190 2b68f4d71700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:39.346598 2b68f4d71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1445: test_mon_osd_pool_set: for flag in hashpspool nodelete nopgchange nosizechange write_fadvise_dontneed noscrub nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1446: test_mon_osd_pool_set: ceph osd pool set pool_getset write_fadvise_dontneed false *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:39.432883 2ab59f983700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:39.444221 2ab59f983700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 write_fadvise_dontneed to false 2016-06-07 20:30:40.582739 2ab59f983700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: ceph osd pool get pool_getset write_fadvise_dontneed /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: grep 'write_fadvise_dontneed: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:40.676693 2b531d7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:40.683406 2b531d7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:40.813872 2b531d7d5700 0 lockdep stop write_fadvise_dontneed: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1448: test_mon_osd_pool_set: ceph osd pool set pool_getset write_fadvise_dontneed true *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:40.886108 2b049e683700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:40.886114 2b049e683700 0 lockdep start 2016-06-07 20:30:40.896142 2b049e683700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 write_fadvise_dontneed to true 2016-06-07 20:30:41.321330 2b049e683700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: ceph osd pool get pool_getset write_fadvise_dontneed /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: grep 'write_fadvise_dontneed: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:41.408321 2b7d264ff700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:41.408330 2b7d264ff700 0 lockdep start 2016-06-07 20:30:41.420205 2b7d264ff700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:42.265111 2b7d264ff700 0 lockdep stop write_fadvise_dontneed: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1450: test_mon_osd_pool_set: ceph osd pool set pool_getset write_fadvise_dontneed 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:42.350956 2b4bc5355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:42.350961 2b4bc5355700 0 lockdep start 2016-06-07 20:30:42.360760 2b4bc5355700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 write_fadvise_dontneed to 1 2016-06-07 20:30:43.642851 2b4bc5355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: ceph osd pool get pool_getset write_fadvise_dontneed /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: grep 'write_fadvise_dontneed: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:43.727436 2b5db40a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:43.738733 2b5db40a7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:44.218174 2b5db40a7700 0 lockdep stop write_fadvise_dontneed: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1452: test_mon_osd_pool_set: ceph osd pool set pool_getset write_fadvise_dontneed 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:44.301392 2b794b4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:44.301401 2b794b4a4700 0 lockdep start 2016-06-07 20:30:44.313673 2b794b4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 write_fadvise_dontneed to 0 2016-06-07 20:30:45.033819 2b794b4a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: ceph osd pool get pool_getset write_fadvise_dontneed /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: grep 'write_fadvise_dontneed: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:45.135228 2ba9edc39700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:45.135234 2ba9edc39700 0 lockdep start 2016-06-07 20:30:45.141938 2ba9edc39700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:45.883593 2ba9edc39700 0 lockdep stop write_fadvise_dontneed: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1454: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset write_fadvise_dontneed asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset write_fadvise_dontneed asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:45.974704 2b95e600d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:45.974716 2b95e600d700 0 lockdep start 2016-06-07 20:30:45.987017 2b95e600d700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:47.183599 2b95e600d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1455: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset write_fadvise_dontneed 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset write_fadvise_dontneed 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:47.284703 2b7ce6212700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:47.284714 2b7ce6212700 0 lockdep start 2016-06-07 20:30:47.303883 2b7ce6212700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:47.735745 2b7ce6212700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1445: test_mon_osd_pool_set: for flag in hashpspool nodelete nopgchange nosizechange write_fadvise_dontneed noscrub nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1446: test_mon_osd_pool_set: ceph osd pool set pool_getset noscrub false *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:47.832859 2b9914d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:47.832864 2b9914d71700 0 lockdep start 2016-06-07 20:30:47.844009 2b9914d71700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 noscrub to false 2016-06-07 20:30:49.279060 2b9914d71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: ceph osd pool get pool_getset noscrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: grep 'noscrub: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:49.371366 2ae2bd7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:49.371372 2ae2bd7d5700 0 lockdep start 2016-06-07 20:30:49.382812 2ae2bd7d5700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:49.442414 2ae2bd7d5700 0 lockdep stop noscrub: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1448: test_mon_osd_pool_set: ceph osd pool set pool_getset noscrub true *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:49.521914 2b23f59de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:49.534435 2b23f59de700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 noscrub to true 2016-06-07 20:30:52.182155 2b23f59de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: grep 'noscrub: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:52.260573 2b62303f4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:52.260578 2b62303f4700 0 lockdep start 2016-06-07 20:30:52.266191 2b62303f4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:53.239465 2b62303f4700 0 lockdep stop noscrub: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1450: test_mon_osd_pool_set: ceph osd pool set pool_getset noscrub 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:53.329824 2aab580e7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:53.329829 2aab580e7700 0 lockdep start 2016-06-07 20:30:53.336921 2aab580e7700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 noscrub to 1 2016-06-07 20:30:54.025387 2aab580e7700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: ceph osd pool get pool_getset noscrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: grep 'noscrub: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:54.109884 2b8033423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:54.109889 2b8033423700 0 lockdep start 2016-06-07 20:30:54.121692 2b8033423700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:54.491898 2b8033423700 0 lockdep stop noscrub: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1452: test_mon_osd_pool_set: ceph osd pool set pool_getset noscrub 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:54.572537 2b46a0f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:54.572543 2b46a0f79700 0 lockdep start 2016-06-07 20:30:54.582248 2b46a0f79700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 noscrub to 0 2016-06-07 20:30:55.208389 2b46a0f79700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: ceph osd pool get pool_getset noscrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: grep 'noscrub: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:55.299092 2b66322d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:55.299098 2b66322d3700 0 lockdep start 2016-06-07 20:30:55.306220 2b66322d3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:56.449829 2b66322d3700 0 lockdep stop noscrub: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1454: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset noscrub asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset noscrub asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:56.535215 2b975b4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:56.535219 2b975b4a4700 0 lockdep start 2016-06-07 20:30:56.541006 2b975b4a4700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:57.362201 2b975b4a4700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1455: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset noscrub 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset noscrub 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:57.446525 2b3aa1355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:57.446530 2b3aa1355700 0 lockdep start 2016-06-07 20:30:57.466593 2b3aa1355700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:30:57.742516 2b3aa1355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1445: test_mon_osd_pool_set: for flag in hashpspool nodelete nopgchange nosizechange write_fadvise_dontneed noscrub nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1446: test_mon_osd_pool_set: ceph osd pool set pool_getset nodeep-scrub false *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:57.842835 2b52cb3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:57.842840 2b52cb3e3700 0 lockdep start 2016-06-07 20:30:57.848852 2b52cb3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodeep-scrub to false 2016-06-07 20:30:58.333280 2b52cb3e3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1447: test_mon_osd_pool_set: grep 'nodeep-scrub: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:58.417966 2b58f1bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:58.417972 2b58f1bf9700 0 lockdep start 2016-06-07 20:30:58.427665 2b58f1bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:58.486488 2b58f1bf9700 0 lockdep stop nodeep-scrub: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1448: test_mon_osd_pool_set: ceph osd pool set pool_getset nodeep-scrub true *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:58.560147 2b6376899700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:58.565923 2b6376899700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodeep-scrub to true 2016-06-07 20:30:59.343816 2b6376899700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: ceph osd pool get pool_getset nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1449: test_mon_osd_pool_set: grep 'nodeep-scrub: true' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:59.435552 2ab5acd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:59.435563 2ab5acd71700 0 lockdep start 2016-06-07 20:30:59.446948 2ab5acd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:59.519162 2ab5acd71700 0 lockdep stop nodeep-scrub: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1450: test_mon_osd_pool_set: ceph osd pool set pool_getset nodeep-scrub 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:30:59.593734 2abe8f82b700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:30:59.604008 2abe8f82b700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodeep-scrub to 1 2016-06-07 20:31:00.385575 2abe8f82b700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: grep 'nodeep-scrub: true' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1451: test_mon_osd_pool_set: ceph osd pool get pool_getset nodeep-scrub *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:00.460088 2af233e05700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:00.471300 2af233e05700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:00.531692 2af233e05700 0 lockdep stop nodeep-scrub: true /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1452: test_mon_osd_pool_set: ceph osd pool set pool_getset nodeep-scrub 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:00.613792 2b17735f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:00.613797 2b17735f2700 0 lockdep start 2016-06-07 20:31:00.623704 2b17735f2700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nodeep-scrub to 0 2016-06-07 20:31:03.152468 2b17735f2700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: ceph osd pool get pool_getset nodeep-scrub /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1453: test_mon_osd_pool_set: grep 'nodeep-scrub: false' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:03.244702 2b66d4d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:03.256414 2b66d4d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:03.728248 2b66d4d71700 0 lockdep stop nodeep-scrub: false /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1454: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nodeep-scrub asdf /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nodeep-scrub asdf *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:03.816389 2b11f9355700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:03.816395 2b11f9355700 0 lockdep start 2016-06-07 20:31:03.986958 2b11f9355700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:31:04.215443 2b11f9355700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1455: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset nodeep-scrub 2 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset nodeep-scrub 2 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:04.293234 2b529f75d700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:04.304371 2b529f75d700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EINVAL: expecting value 'true', 'false', '0', or '1' 2016-06-07 20:31:04.417279 2b529f75d700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1458: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset scrub_min_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset scrub_min_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset scrub_min_interval | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1459: test_mon_osd_pool_set: ceph osd pool set pool_getset scrub_min_interval 123456 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:04.508633 2ab5de33a700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:04.508638 2ab5de33a700 0 lockdep start 2016-06-07 20:31:04.519263 2ab5de33a700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 scrub_min_interval to 123456 2016-06-07 20:31:05.249875 2ab5de33a700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1460: test_mon_osd_pool_set: ceph osd pool get pool_getset scrub_min_interval /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1460: test_mon_osd_pool_set: grep 'scrub_min_interval: 123456' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:05.332994 2b5118f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:05.332999 2b5118f79700 0 lockdep start 2016-06-07 20:31:05.338875 2b5118f79700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:05.893207 2b5118f79700 0 lockdep stop scrub_min_interval: 123456 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1461: test_mon_osd_pool_set: ceph osd pool set pool_getset scrub_min_interval 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:05.978686 2ac857d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:05.999602 2ac857d5c700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 scrub_min_interval to 0 2016-06-07 20:31:07.034845 2ac857d5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1462: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset scrub_min_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset scrub_min_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset scrub_min_interval | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1464: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset scrub_max_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset scrub_max_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset scrub_max_interval | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1465: test_mon_osd_pool_set: ceph osd pool set pool_getset scrub_max_interval 123456 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:07.127843 2b2cc3531700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:07.139614 2b2cc3531700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 scrub_max_interval to 123456 2016-06-07 20:31:08.273529 2b2cc3531700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1466: test_mon_osd_pool_set: ceph osd pool get pool_getset scrub_max_interval /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1466: test_mon_osd_pool_set: grep 'scrub_max_interval: 123456' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:08.375971 2b0608b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:08.387385 2b0608b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:08.453647 2b0608b6c700 0 lockdep stop scrub_max_interval: 123456 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1467: test_mon_osd_pool_set: ceph osd pool set pool_getset scrub_max_interval 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:08.525483 2b5903c07700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:08.525489 2b5903c07700 0 lockdep start 2016-06-07 20:31:08.535410 2b5903c07700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 scrub_max_interval to 0 2016-06-07 20:31:09.272552 2b5903c07700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1468: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset scrub_max_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset scrub_max_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset scrub_max_interval | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1470: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset deep_scrub_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset deep_scrub_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset deep_scrub_interval | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1471: test_mon_osd_pool_set: ceph osd pool set pool_getset deep_scrub_interval 123456 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:09.367011 2ad710434700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:09.386210 2ad710434700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 deep_scrub_interval to 123456 2016-06-07 20:31:09.948097 2ad710434700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1472: test_mon_osd_pool_set: grep 'deep_scrub_interval: 123456' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1472: test_mon_osd_pool_set: ceph osd pool get pool_getset deep_scrub_interval *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:10.030604 2ac3bfb46700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:10.041840 2ac3bfb46700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:11.425843 2ac3bfb46700 0 lockdep stop deep_scrub_interval: 123456 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1473: test_mon_osd_pool_set: ceph osd pool set pool_getset deep_scrub_interval 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:11.523569 2adb2fd5c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:11.523579 2adb2fd5c700 0 lockdep start 2016-06-07 20:31:11.535160 2adb2fd5c700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 deep_scrub_interval to 0 2016-06-07 20:31:12.693869 2adb2fd5c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1474: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset deep_scrub_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset deep_scrub_interval | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset deep_scrub_interval | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1476: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset recovery_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset recovery_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset recovery_priority | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1477: test_mon_osd_pool_set: ceph osd pool set pool_getset recovery_priority 5 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:12.784607 2b8928d71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:12.792781 2b8928d71700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 recovery_priority to 5 2016-06-07 20:31:13.632825 2b8928d71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1478: test_mon_osd_pool_set: ceph osd pool get pool_getset recovery_priority /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1478: test_mon_osd_pool_set: grep 'recovery_priority: 5' *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:13.714172 2b204b1b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:13.714183 2b204b1b7700 0 lockdep start 2016-06-07 20:31:13.725401 2b204b1b7700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:14.146961 2b204b1b7700 0 lockdep stop recovery_priority: 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1479: test_mon_osd_pool_set: ceph osd pool set pool_getset recovery_priority 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:14.245210 2b8be2a48700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:14.245216 2b8be2a48700 0 lockdep start 2016-06-07 20:31:14.255439 2b8be2a48700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 recovery_priority to 0 2016-06-07 20:31:15.494254 2b8be2a48700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1480: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset recovery_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset recovery_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset recovery_priority | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1482: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset recovery_op_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset recovery_op_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset recovery_op_priority | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1483: test_mon_osd_pool_set: ceph osd pool set pool_getset recovery_op_priority 5 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:15.572861 2b260a643700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:15.582387 2b260a643700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 recovery_op_priority to 5 2016-06-07 20:31:17.326684 2b260a643700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1484: test_mon_osd_pool_set: grep 'recovery_op_priority: 5' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1484: test_mon_osd_pool_set: ceph osd pool get pool_getset recovery_op_priority *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:17.422175 2b815f5f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:17.422182 2b815f5f2700 0 lockdep start 2016-06-07 20:31:17.429180 2b815f5f2700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:17.548233 2b815f5f2700 0 lockdep stop recovery_op_priority: 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1485: test_mon_osd_pool_set: ceph osd pool set pool_getset recovery_op_priority 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:17.631140 2b8229bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:17.631145 2b8229bf9700 0 lockdep start 2016-06-07 20:31:17.641297 2b8229bf9700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 recovery_op_priority to 0 2016-06-07 20:31:18.277952 2b8229bf9700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1486: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset recovery_op_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset recovery_op_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset recovery_op_priority | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1488: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset scrub_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset scrub_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset scrub_priority | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1489: test_mon_osd_pool_set: ceph osd pool set pool_getset scrub_priority 5 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:18.379304 2b900de09700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:18.398225 2b900de09700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 scrub_priority to 5 2016-06-07 20:31:19.822533 2b900de09700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1490: test_mon_osd_pool_set: grep 'scrub_priority: 5' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1490: test_mon_osd_pool_set: ceph osd pool get pool_getset scrub_priority *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:19.909169 2b4954b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:19.909175 2b4954b6c700 0 lockdep start 2016-06-07 20:31:19.926864 2b4954b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:20.099247 2b4954b6c700 0 lockdep stop scrub_priority: 5 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1491: test_mon_osd_pool_set: ceph osd pool set pool_getset scrub_priority 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:20.187224 2b64dcb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:20.187234 2b64dcb6c700 0 lockdep start 2016-06-07 20:31:20.198053 2b64dcb6c700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 scrub_priority to 0 2016-06-07 20:31:22.144662 2b64dcb6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1492: test_mon_osd_pool_set: expect_false 'ceph osd pool get pool_getset scrub_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: 'ceph osd pool get pool_getset scrub_priority | grep '\''.'\''' /home/yuriw/ceph/qa/workunits/cephtool/test.sh: line 32: ceph osd pool get pool_getset scrub_priority | grep '.': command not found /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1494: test_mon_osd_pool_set: ceph osd pool set pool_getset nopgchange 1 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:22.218399 2ba58d9de700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:22.218406 2ba58d9de700 0 lockdep start 2016-06-07 20:31:22.228026 2ba58d9de700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nopgchange to 1 2016-06-07 20:31:23.367004 2ba58d9de700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1495: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset pg_num 10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset pg_num 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:23.444981 2b9042a48700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:23.444986 2b9042a48700 0 lockdep start 2016-06-07 20:31:23.455240 2b9042a48700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EPERM: pool pg_num change is disabled; you must unset nopgchange flag for the pool first 2016-06-07 20:31:23.925666 2b9042a48700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1496: test_mon_osd_pool_set: expect_false ceph osd pool set pool_getset pgp_num 10 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:31: expect_false: set -x /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: ceph osd pool set pool_getset pgp_num 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:24.015253 2b8b1cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:24.015257 2b8b1cd71700 0 lockdep start 2016-06-07 20:31:24.026448 2b8b1cd71700 -1 WARNING: the following dangerous and experimental features are enabled: * Error EPERM: pool pgp_num change is disabled; you must unset nopgchange flag for the pool first 2016-06-07 20:31:24.611387 2b8b1cd71700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:32: expect_false: return 0 /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1497: test_mon_osd_pool_set: ceph osd pool set pool_getset nopgchange 0 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:24.699886 2b616b3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:24.699892 2b616b3e3700 0 lockdep start 2016-06-07 20:31:24.710035 2b616b3e3700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 nopgchange to 0 2016-06-07 20:31:26.267272 2b616b3e3700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1498: test_mon_osd_pool_set: ceph osd pool set pool_getset pg_num 10 *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** 2016-06-07 20:31:26.351603 2b1e60b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * 2016-06-07 20:31:26.351609 2b1e60b6c700 0 lockdep start 2016-06-07 20:31:26.362002 2b1e60b6c700 -1 WARNING: the following dangerous and experimental features are enabled: * set pool 25 pg_num to 10 2016-06-07 20:31:27.817626 2b1e60b6c700 0 lockdep stop /home/yuriw/ceph/qa/workunits/cephtool/test.sh:1499: test_mon_osd_pool_set: wait_for_clean /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1083: wait_for_clean: local status=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1084: wait_for_clean: local num_active_clean=-1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1085: wait_for_clean: local cur_active_clean /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1086: wait_for_clean: local -i timer=0 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1087: wait_for_clean: get_num_pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1001: get_num_pgs: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1002: get_num_pgs: xmlstarlet sel -t -m //pgmap/num_pgs -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1087: wait_for_clean: local num_pgs=19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1088: wait_for_clean: test 19 '!=' 0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 10 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 10 '!=' -1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1097: wait_for_clean: timer=0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1098: wait_for_clean: num_active_clean=10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 0 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 10 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 10 '!=' 10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 1 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=2 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 10 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 10 '!=' 10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 2 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=3 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1097: wait_for_clean: timer=0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1098: wait_for_clean: num_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 0 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 1 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=2 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 2 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=3 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 3 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=4 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 4 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=5 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 5 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=6 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 6 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 7 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=8 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 8 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 9 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=10 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 10 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=11 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 11 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=12 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 12 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=13 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 13 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=14 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 14 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=15 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 15 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=16 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 16 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=17 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 17 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=18 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 18 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 19 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=20 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 20 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=21 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 21 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=22 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 22 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=23 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 23 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=24 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 24 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=25 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 25 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=26 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 26 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=27 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 27 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=28 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 28 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=29 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 29 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=30 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 30 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=31 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 31 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=32 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 32 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=33 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 33 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=34 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 34 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=35 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 35 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=36 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 36 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=37 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 37 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=38 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 38 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=39 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 39 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=40 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 40 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=41 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 41 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=42 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 42 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=43 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 43 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=44 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 44 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=45 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 45 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=46 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 46 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=47 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 47 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=48 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 48 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=49 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 49 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=50 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 50 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=51 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 51 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=52 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 52 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=53 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 53 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=54 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 54 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=55 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 55 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=56 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 56 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=57 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 57 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=58 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 58 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=59 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 59 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=60 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 60 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=61 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 61 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=62 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 62 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=63 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 63 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=64 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 64 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=65 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 65 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=66 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 66 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=67 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 67 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=68 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 68 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=69 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 69 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=70 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 70 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=71 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 71 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=72 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 72 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=73 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 9 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 9 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 73 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=74 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 7 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 7 '!=' 9 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1097: wait_for_clean: timer=0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1098: wait_for_clean: num_active_clean=7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 0 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 7 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 7 '!=' 7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 1 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=2 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 7 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 7 '!=' 7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 2 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=3 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 7 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 7 '!=' 7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 3 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=4 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1095: wait_for_clean: test 7 = 19 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1096: wait_for_clean: test 7 '!=' 7 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1099: wait_for_clean: get_is_making_recovery_progress //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: xmlstarlet sel -t -m //pgmap/recovering_keys_per_sec -v . -o ' ' -t -m //pgmap/recovering_bytes_per_sec -v . -o ' ' -t -m //pgmap/recovering_objects_per_sec -v . //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: ceph --format xml status /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:942: get_is_making_recovery_progress: local progress= /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:943: get_is_making_recovery_progress: test -n '' /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1101: wait_for_clean: (( timer >= 3000 )) /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1105: wait_for_clean: sleep .1 //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: expr 4 + 1 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1106: wait_for_clean: timer=5 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1090: wait_for_clean: true //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: get_num_active_clean //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:966: get_num_active_clean: local 'expression=(' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:967: get_num_active_clean: expression+='contains(.,'\''active'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:968: get_num_active_clean: expression+='contains(.,'\''clean'\'') and ' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:969: get_num_active_clean: expression+='not(contains(.,'\''stale'\''))' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:970: get_num_active_clean: expression+=')' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:974: get_num_active_clean: ceph --format xml pg dump pgs //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:976: get_num_active_clean: grep -cv '^$' //home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:975: get_num_active_clean: xmlstarlet sel -t -m '//pg_stat/state[(contains(.,'\''active'\'') and contains(.,'\''clean'\'') and not(contains(.,'\''stale'\'')))]' -v . -n /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1094: wait_for_clean: cur_active_clean=0 /home/yuriw/ceph/qa/workunits/cephtool/../ceph-helpers.sh:1: wait_for_clean: rm -fr /tmp/cephtool19083