Bug #17526
Updated by Loïc Dachary over 7 years ago
In the same way obj_bencher.cc verifies the metadata object exists before trying to read from the pool when required to do sequential read bench, it must verify that the hostname used to build object names is the same as the one used when writing the objects. Otherwise it will fail later with a core dump because there are no safeguard against that kind of error. Steps to reproduce: * rados bench -p rbd 2 write --no-cleanup * sudo hostname somethingelse * rados bench -p rbd 20 seq On hammer it fails with: <pre> (gdb) bt bt #0 __memcmp_sse4_1 () at ../sysdeps/x86_64/multiarch/memcmp-sse4.S:69 #1 0x0000000000872829 in ObjBencher::seq_read_bench (this=0x7fffffffe730, seconds_to_run=20, num_objects=36, concurrentios=16, pid=1587, no_verify=false) at common/obj_bencher.cc:592 #2 0x00000000008703d5 in ObjBencher::aio_bench (this=0x7fffffffe730, operation=2, secondsToRun=20, maxObjectsToCreate=0, concurrentios=16, op_size=4194304, cleanup=true, run_name=0x0, no_verify=false) at common/obj_bencher.cc:208 Python Exception <class 'IndexError'> list index out of range: #3 0x000000000084b340 in rados_tool_common (opts=std::map with 1 elements, nargs=std::vector of length 3, capacity 8 = {...}) at tools/rados/rados.cc:2271 #4 0x0000000000850e61 in main (argc=6, argv=0x7fffffffecd8) at tools/rados/rados.cc:2732 (gdb) frame 1 frame 1 #1 0x0000000000872829 in ObjBencher::seq_read_bench (this=0x7fffffffe730, seconds_to_run=20, num_objects=36, concurrentios=16, pid=1587, no_verify=false) at common/obj_bencher.cc:592 592 if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) { (gdb) list list 587 // invalidate internal crc cache 588 cur_contents->invalidate_crc(); 589 590 if (!no_verify) { 591 snprintf(data.object_contents, data.object_size, "I'm the %16dth object!", current_index); 592 if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) { 593 cerr << name[slot] << " is not correct!" << std::endl; 594 ++errors; 595 } 596 } (gdb) print cur_contents->c_str() print cur_contents->c_str() $3 = 0x0 </pre> on master ( 003dbc83e5cdb2c1984578f19d55383924db9ce1 ) it fails with <pre> $ bin/rados bench -p rbd 2 seq sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 0 0 0 0 0 0 - 0 benchmark_data_abc_707_object0 is not correct! read got -2 error during benchmark: -2 error 2: (2) No such file or directory </pre>