Project

General

Profile

Bug #17919

tests: test/mon/osd-crush.sh is racy

Added by Loïc Dachary about 6 years ago. Updated about 6 years ago.

Status:
Can't reproduce
Priority:
Urgent
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

https://jenkins.ceph.com/job/ceph-pull-requests/14124/console

//home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:762: get_config:  xmlstarlet sel -t -m //mon_initial_members -v . -n
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/workunits/ceph-helpers.sh:379: run_mon:  test -z a,b,c
//home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/osd-crush.sh:287: TEST_crush_repair_faulty_crushmap:  ceph osd crush dump --format=xml
//home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/osd-crush.sh:287: TEST_crush_repair_faulty_crushmap:  xmlstarlet sel -t -m //buckets/bucket -v .
======================= testdir/osd-crush/mon.a.log
2016-11-15 20:10:22.863050 7f6f011b6780  0 ceph version 11.0.2-1574-g4027970 (4027970a4465c70c9a8e362de07a5fe590b88ed1), process ceph-mon, pid 31369
2016-11-15 20:10:22.877288 7f6f011b6780  0 load: jerasure load: lrc load: isa 
2016-11-15 20:10:22.877550 7f6f011b6780  1 leveldb: Recovering log #3
2016-11-15 20:10:22.877591 7f6f011b6780  1 leveldb: Level-0 table #5: started
2016-11-15 20:10:22.881339 7f6f011b6780  1 leveldb: Level-0 table #5: 765 bytes OK
2016-11-15 20:10:22.886951 7f6f011b6780  1 leveldb: Delete type=3 #2

2016-11-15 20:10:22.886999 7f6f011b6780  1 leveldb: Delete type=0 #3

2016-11-15 20:10:22.887185 7f6f011b6780 10 obtain_monmap
2016-11-15 20:10:22.887241 7f6f011b6780 10 obtain_monmap found mkfs monmap
2016-11-15 20:10:22.894238 7f6ef97a2700  2 Event(0x7f6f09fcd680 nevent=5000 time_id=1).set_owner idx=2 owner=140114608662272
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/osd-crush.sh:287: TEST_crush_repair_faulty_crushmap:  test
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/osd-crush.sh:287: TEST_crush_repair_faulty_crushmap:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/mon/osd-crush.sh:33: run:  return 1

The test:

    #now it can be successful to set the empty crush map
    ceph osd setcrushmap -i $empty_map.map || return 1

    # should be an empty crush map without any buckets
    ! test $(ceph osd crush dump --format=xml | \
           $XMLSTARLET sel -t -m "//buckets/bucket" -v .) || return 1

must retry until it becomes true because ceph osd setcrushmap returns before the action is complete and the ceph osd crush dump races against it.


Related issues

Related to Ceph - Bug #18022: tests: osd-crush.sh sometime fails Can't reproduce 11/24/2016

History

#1 Updated by Kefu Chai about 6 years ago

  • Status changed from 12 to Resolved
  • Assignee set to Loïc Dachary

#2 Updated by Loïc Dachary about 6 years ago

  • File consoleText.3.gz added
  • Status changed from Resolved to 12

#3 Updated by Loïc Dachary about 6 years ago

  • Duplicated by Bug #18022: tests: osd-crush.sh sometime fails added

#5 Updated by Loïc Dachary about 6 years ago

  • File deleted (consoleText.3.gz)

#6 Updated by Loïc Dachary about 6 years ago

  • Duplicated by deleted (Bug #18022: tests: osd-crush.sh sometime fails)

#7 Updated by Loïc Dachary about 6 years ago

  • Related to Bug #18022: tests: osd-crush.sh sometime fails added

#8 Updated by Loïc Dachary about 6 years ago

  • Status changed from 12 to Can't reproduce

Also available in: Atom PDF