Project

General

Profile

Bug #17526

rados bench seq must verify the hostname

Added by Loïc Dachary about 6 years ago. Updated almost 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

In the same way obj_bencher.cc verifies the metadata object exists before trying to read from the pool when required to do sequential read bench, it must verify that the hostname used to build object names is the same as the one used when writing the objects. Otherwise it will fail later with a core dump because there are no safeguard against that kind of error.

Steps to reproduce:

  • rados bench -p rbd 2 write --no-cleanup
  • sudo hostname somethingelse
  • rados bench -p rbd 20 seq

On hammer it fails with:

(gdb) bt
bt
#0  __memcmp_sse4_1 () at ../sysdeps/x86_64/multiarch/memcmp-sse4.S:69
#1  0x0000000000872829 in ObjBencher::seq_read_bench (this=0x7fffffffe730,
    seconds_to_run=20, num_objects=36, concurrentios=16, pid=1587,
    no_verify=false) at common/obj_bencher.cc:592
#2  0x00000000008703d5 in ObjBencher::aio_bench (this=0x7fffffffe730,
    operation=2, secondsToRun=20, maxObjectsToCreate=0, concurrentios=16,
    op_size=4194304, cleanup=true, run_name=0x0, no_verify=false)
    at common/obj_bencher.cc:208
Python Exception <class 'IndexError'> list index out of range:
#3  0x000000000084b340 in rados_tool_common (opts=std::map with 1 elements,
    nargs=std::vector of length 3, capacity 8 = {...})
    at tools/rados/rados.cc:2271
#4  0x0000000000850e61 in main (argc=6, argv=0x7fffffffecd8)
    at tools/rados/rados.cc:2732
(gdb) frame 1
frame 1
#1  0x0000000000872829 in ObjBencher::seq_read_bench (this=0x7fffffffe730,
    seconds_to_run=20, num_objects=36, concurrentios=16, pid=1587,
    no_verify=false) at common/obj_bencher.cc:592
592           if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) {
(gdb) list
list
587         // invalidate internal crc cache
588         cur_contents->invalidate_crc();
589
590         if (!no_verify) {
591           snprintf(data.object_contents, data.object_size, "I'm the %16dth object!", current_index);
592           if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) {
593             cerr << name[slot] << " is not correct!" << std::endl;
594             ++errors;
595           }
596         }
(gdb) print cur_contents->c_str()
print cur_contents->c_str()
$3 = 0x0

on master ( 003dbc83e5cdb2c1984578f19d55383924db9ce1 ) it fails with

$ bin/rados bench -p rbd 2 seq
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
benchmark_data_abc_707_object0 is not correct!
read got -2
error during benchmark: -2
error 2: (2) No such file or directory

Related issues

Copied to Ceph - Backport #18433: jewel: rados bench seq must verify the hostname Resolved

History

#1 Updated by Loïc Dachary about 6 years ago

  • Status changed from New to 12

#2 Updated by Loïc Dachary about 6 years ago

  • Backport set to hammer,jewel

#3 Updated by Loïc Dachary about 6 years ago

  • Description updated (diff)

#4 Updated by Loïc Dachary about 6 years ago

  • Status changed from 12 to Fix Under Review
  • Assignee set to Loïc Dachary

#5 Updated by Loïc Dachary almost 6 years ago

  • Status changed from Fix Under Review to Resolved
  • Backport changed from hammer,jewel to jewel

#6 Updated by Loïc Dachary almost 6 years ago

  • Status changed from Resolved to Pending Backport

#7 Updated by Nathan Cutler almost 6 years ago

  • Copied to Backport #18433: jewel: rados bench seq must verify the hostname added

#8 Updated by Nathan Cutler almost 6 years ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF