Ceph » rgw

The 'bilog autotrim' command in this test sometimes fails to trim all entries, causing a failed assert(len(active_bilog) == 0).

From http://qa-proxy.ceph.com/teuthology/cbodley-2018-09-17_14:26:55-rgw:multisite-wip-multisite-testing-distro-basic-smithi/3033357/teuthology.log:

2018-09-17T15:05:58.143 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0']
2018-09-17T15:05:58.143 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'autotrim', '--rgw-sync-log-trim-max-buckets', '1', '--rgw-sync-log-trim-min-cold-buckets', '0', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0']
2018-09-17T15:05:58.143 INFO:teuthology.orchestra.run.smithi037:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog autotrim --rgw-sync-log-trim-max-buckets 1 --rgw-sync-log-trim-min-cold-buckets 0 --cluster c1 --debug-rgw 0 --debug-ms 0'
2018-09-17T15:05:59.808 INFO:tasks.util.rgw:rgwadmin: c1.client.0 : ['bilog', 'list', '--bucket', 'soltxr-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false']
2018-09-17T15:05:59.809 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'c1', 'bilog', 'list', '--bucket', 'soltxr-46', '--cluster', 'c1', '--debug-rgw', '0', '--debug-ms', '0', '--rgw-cache-enabled', 'false']
2018-09-17T15:05:59.809 INFO:teuthology.orchestra.run.smithi037:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster c1 bilog list --bucket soltxr-46 --cluster c1 --debug-rgw 0 --debug-ms 0 --rgw-cache-enabled false'
2018-09-17T15:06:00.624 INFO:teuthology.orchestra.run.smithi037.stdout:[{"op_id":"00000000001.31.4","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1351","op":"write","object":"a","instance":"","state":"pending","index_ver":1,"timestamp":"0.000000","ver":{"pool":-1,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]},{"op_id":"00000000002.32.5","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1351","op":"write","object":"a","instance":"","state":"complete","index_ver":2,"timestamp":"2018-09-17 15:05:47.328407316Z","ver":{"pool":8,"epoch":3},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]},{"op_id":"00000000003.33.4","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1353","op":"write","object":"b","instance":"","state":"pending","index_ver":3,"timestamp":"0.000000","ver":{"pool":-1,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]},{"op_id":"00000000004.34.5","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1353","op":"write","object":"b","instance":"","state":"complete","index_ver":4,"timestamp":"2018-09-17 15:05:47.502896761Z","ver":{"pool":8,"epoch":1},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]},{"op_id":"00000000005.35.4","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1355","op":"write","object":"c","instance":"","state":"pending","index_ver":5,"timestamp":"0.000000","ver":{"pool":-1,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]},{"op_id":"00000000006.36.5","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1355","op":"write","object":"c","instance":"","state":"complete","index_ver":6,"timestamp":"2018-09-17 15:05:47.657797139Z","ver":{"pool":8,"epoch":3},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]},{"op_id":"00000000007.37.4","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1358","op":"write","object":"d","instance":"","state":"pending","index_ver":7,"timestamp":"0.000000","ver":{"pool":-1,"epoch":0},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]},{"op_id":"00000000008.38.5","op_tag":"6b494045-35f4-4a6d-9a86-dea6396200e2.4559.1358","op":"write","object":"d","instance":"","state":"complete","index_ver":8,"timestamp":"2018-09-17 15:05:47.848360747Z","ver":{"pool":8,"epoch":4},"bilog_flags":0,"versioned":false,"owner":"","owner_display_name":"","zones_trace":["6b494045-35f4-4a6d-9a86-dea6396200e2"]}]
2018-09-17T15:06:00.635 INFO:tasks.rgw_multisite_tests:tasks.rgw_multi.tests.test_bucket_index_log_trim ... FAIL
2018-09-17T15:06:00.635 INFO:tasks.rgw_multisite_tests:
2018-09-17T15:06:00.635 INFO:tasks.rgw_multisite_tests:======================================================================
2018-09-17T15:06:00.635 INFO:tasks.rgw_multisite_tests:FAIL: tasks.rgw_multi.tests.test_bucket_index_log_trim
2018-09-17T15:06:00.635 INFO:tasks.rgw_multisite_tests:----------------------------------------------------------------------
2018-09-17T15:06:00.635 INFO:tasks.rgw_multisite_tests:Traceback (most recent call last):
2018-09-17T15:06:00.635 INFO:tasks.rgw_multisite_tests:  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:    self.test(*self.arg)
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-multisite-testing/qa/tasks/rgw_multi/tests.py", line 1120, in test_bucket_index_log_trim
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:    assert(len(active_bilog) == 0)
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:AssertionError
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:----------------------------------------------------------------------
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:Ran 21 tests in 787.673s
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:
2018-09-17T15:06:00.636 INFO:tasks.rgw_multisite_tests:FAILED (failures=1)