Project

General

Profile

Bug #17526

rados bench seq must verify the hostname

Added by Loic Dachary about 1 year ago. Updated 9 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
10/06/2016
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Release:
Needs Doc:
No

Description

In the same way obj_bencher.cc verifies the metadata object exists before trying to read from the pool when required to do sequential read bench, it must verify that the hostname used to build object names is the same as the one used when writing the objects. Otherwise it will fail later with a core dump because there are no safeguard against that kind of error.

Steps to reproduce:

  • rados bench -p rbd 2 write --no-cleanup
  • sudo hostname somethingelse
  • rados bench -p rbd 20 seq

On hammer it fails with:

(gdb) bt
bt
#0  __memcmp_sse4_1 () at ../sysdeps/x86_64/multiarch/memcmp-sse4.S:69
#1  0x0000000000872829 in ObjBencher::seq_read_bench (this=0x7fffffffe730,
    seconds_to_run=20, num_objects=36, concurrentios=16, pid=1587,
    no_verify=false) at common/obj_bencher.cc:592
#2  0x00000000008703d5 in ObjBencher::aio_bench (this=0x7fffffffe730,
    operation=2, secondsToRun=20, maxObjectsToCreate=0, concurrentios=16,
    op_size=4194304, cleanup=true, run_name=0x0, no_verify=false)
    at common/obj_bencher.cc:208
Python Exception <class 'IndexError'> list index out of range:
#3  0x000000000084b340 in rados_tool_common (opts=std::map with 1 elements,
    nargs=std::vector of length 3, capacity 8 = {...})
    at tools/rados/rados.cc:2271
#4  0x0000000000850e61 in main (argc=6, argv=0x7fffffffecd8)
    at tools/rados/rados.cc:2732
(gdb) frame 1
frame 1
#1  0x0000000000872829 in ObjBencher::seq_read_bench (this=0x7fffffffe730,
    seconds_to_run=20, num_objects=36, concurrentios=16, pid=1587,
    no_verify=false) at common/obj_bencher.cc:592
592           if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) {
(gdb) list
list
587         // invalidate internal crc cache
588         cur_contents->invalidate_crc();
589
590         if (!no_verify) {
591           snprintf(data.object_contents, data.object_size, "I'm the %16dth object!", current_index);
592           if (memcmp(data.object_contents, cur_contents->c_str(), data.object_size) != 0) {
593             cerr << name[slot] << " is not correct!" << std::endl;
594             ++errors;
595           }
596         }
(gdb) print cur_contents->c_str()
print cur_contents->c_str()
$3 = 0x0

on master ( 003dbc83e5cdb2c1984578f19d55383924db9ce1 ) it fails with

$ bin/rados bench -p rbd 2 seq
  sec Cur ops   started  finished  avg MB/s  cur MB/s last lat(s)  avg lat(s)
    0       0         0         0         0         0           -           0
benchmark_data_abc_707_object0 is not correct!
read got -2
error during benchmark: -2
error 2: (2) No such file or directory

Related issues

Copied to Ceph - Backport #18433: jewel: rados bench seq must verify the hostname Resolved

History

#1 Updated by Loic Dachary about 1 year ago

  • Status changed from New to Verified

#2 Updated by Loic Dachary about 1 year ago

  • Backport set to hammer,jewel

#3 Updated by Loic Dachary about 1 year ago

  • Description updated (diff)

#4 Updated by Loic Dachary about 1 year ago

  • Status changed from Verified to Need Review
  • Assignee set to Loic Dachary

#5 Updated by Loic Dachary 11 months ago

  • Status changed from Need Review to Resolved
  • Backport changed from hammer,jewel to jewel

#6 Updated by Loic Dachary 10 months ago

  • Status changed from Resolved to Pending Backport

#7 Updated by Nathan Cutler 10 months ago

  • Copied to Backport #18433: jewel: rados bench seq must verify the hostname added

#8 Updated by Nathan Cutler 9 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF