Bug #11948: tests: test-erasure-code.sh osd out is racy - Ceph - Ceph

Actions

Copy link

Bug #11948

closed

tests: test-erasure-code.sh osd out is racy

Added by Loïc Dachary almost 9 years ago. Updated almost 3 years ago.

Status:

Can't reproduce

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

other

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

    ./ceph osd out ${initial_osds[$last]} || return 1
    ! get_osds $poolname $objname | grep '\<'${initial_osds[$last]}'\>' || return 1

found in test-erasure-code.sh, it is possible that get_osds gets results before osd out had time to complete.

http://jenkins.ceph.dachary.org/job/ceph/LABELS=ubuntu-14.04&&x86_64/5776/console

rados_put_get: 95: ./ceph osd out 4
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2015-06-10 00:41:51.456073 2b123d8c8700 -1 WARNING: the following dangerous and experimental features are enabled: shec
2015-06-10 00:41:51.457803 2b123d8c8700 -1 WARNING: the following dangerous and experimental features are enabled: shec
2015-06-10 00:41:51.460017 2b123d8c8700 -1 WARNING: the following dangerous and experimental features are enabled: shec
marked out osd.4. 
rados_put_get: 96: get_osds ecpool SOMETHING
get_osds: 585: local poolname=ecpool
get_osds: 586: local objectname=SOMETHING
rados_put_get: 96: grep '\<4\>'
get_osds: 588: ceph --format xml osd map ecpool SOMETHING
get_osds: 589: xmlstarlet sel -t -m //acting/osd -v . -o ' '
2147483647 0 4 
rados_put_get: 96: return 1
TEST_rados_put_get_jerasure: 163: return 1
run: 47: return 1
main: 1170: code=1

Actions

Copy link

Updated by Loïc Dachary over 8 years ago

Assignee deleted (~~Loïc Dachary~~)

Actions

Copy link

Updated by Kefu Chai about 8 years ago

./test/erasure-code/test-erasure-code.sh:92: rados_put_get:  local -a initial_osds
./test/erasure-code/test-erasure-code.sh:93: rados_put_get:  local last=2
./test/erasure-code/test-erasure-code.sh:94: rados_put_get:  ./ceph osd out 4
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
marked out osd.4. 
./test/erasure-code/test-erasure-code.sh:95: rados_put_get:  get_osds ecpool SOMETHING
./test/erasure-code/test-erasure-code.sh:95: rados_put_get:  grep '\<4\>'
../qa/workunits/ceph-helpers.sh:641: get_osds:  local objectname=SOMETHING
.../qa/workunits/ceph-helpers.sh:644: get_osds:  ceph --format xml osd map ecpool SOMETHING
.../qa/workunits/ceph-helpers.sh:644: get_osds:  xmlstarlet sel -t -m //acting/osd -v . -o ' '
../qa/workunits/ceph-helpers.sh:644: get_osds:  local 'osds=6 0 4 '
../qa/workunits/ceph-helpers.sh:646: get_osds:  echo 6 0 4
6 0 4
./test/erasure-code/test-erasure-code.sh:95: rados_put_get:  return 1
./test/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure:  return 1
./test/erasure-code/test-erasure-code.sh:46: run:  return 1

spotted again at https://jenkins.ceph.com/job/ceph-pull-requests/3286/consoleFull

Actions

Copy link