Project

General

Profile

Bug #20770

test_pidfile.sh test is failing 2 places

Added by David Zafman 5 months ago. Updated 4 months ago.

Status:
Verified
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
07/25/2017
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Release:
Needs Doc:
No

Description

I've seen both of these on Jenkins make check runs.

test_pidfile.sh line 55


 run_osd $dir 0 2>&1 | grep "failed to lock pidfile" || return 1

/home/dzafman/ceph/src/test/test_pidfile.sh:55: TEST_pidfile:  run_osd td/pidfile 0
/home/dzafman/ceph/src/test/test_pidfile.sh:55: TEST_pidfile:  grep 'failed to lock pidfile'
/home/dzafman/ceph/src/test/test_pidfile.sh:55: TEST_pidfile:  return 1
/home/dzafman/ceph/src/test/test_pidfile.sh:22: run:  return 1

test_pidfile.sh line 36


    expect_failure $dir "ignore empty --pid-file" ceph-mon \
        -f \
        --log-to-stderr \
        --pid-file= \
        --id $id \
        --mon-data=$data \
        --run-dir=$dir || return 1

/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_pidfile.sh:29: TEST_without_pidfile:  local data=td/pidfile/osd1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_pidfile.sh:30: TEST_without_pidfile:  local id=1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_pidfile.sh:32: TEST_without_pidfile:  ceph-mon --id 1 --mkfs --mon-data=td/pidfile/osd1 --run-dir=td/pidfile
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_pidfile.sh:36: TEST_without_pidfile:  expect_failure td/pidfile 'ignore empty --pid-file' ceph-mon -f --log-to-stderr --pid-file= --id 1 --mon-data=td/pidfile/osd1 --run-dir=td/pidfile
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1532: expect_failure:  local dir=td/pidfile
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1533: expect_failure:  shift
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1534: expect_failure:  local 'expected=ignore empty --pid-file'
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1535: expect_failure:  shift
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1536: expect_failure:  local success
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1538: expect_failure:  ceph-mon -f --log-to-stderr --pid-file= --id 1 --mon-data=td/pidfile/osd1 --run-dir=td/pidfile
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1541: expect_failure:  success=false
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1544: expect_failure:  false
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1544: expect_failure:  grep --quiet 'ignore empty --pid-file' td/pidfile/out
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1545: expect_failure:  cat td/pidfile/out
/home/jenkins-build/build/workspace/ceph-pull-requests/qa/standalone/ceph-helpers.sh:1546: expect_failure:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_pidfile.sh:42: TEST_without_pidfile:  return 1
/home/jenkins-build/build/workspace/ceph-pull-requests/src/test/test_pidfile.sh:22: run:  return 1


Related issues

Related to RADOS - Bug #20975: test_pidfile.sh is flaky Resolved 08/10/2017
Copied to Ceph - Backport #21050: luminous: test_pidfile.sh test is failing 2 places Rejected

History

#1 Updated by David Zafman 5 months ago

  • Description updated (diff)

#2 Updated by David Zafman 5 months ago

  • Assignee changed from Kefu Chai to David Zafman

#4 Updated by David Zafman 4 months ago

This is still failing sometimes in TEST_without_pidfile() even after adding a sleep 1.

#5 Updated by David Zafman 4 months ago

  • Status changed from New to Verified

#6 Updated by David Zafman 4 months ago

  • Status changed from Verified to In Progress

#7 Updated by David Zafman 4 months ago

  • Status changed from In Progress to Testing
  • Backport set to luminous

#8 Updated by David Zafman 4 months ago

  • Status changed from Testing to Need Review

#9 Updated by David Zafman 4 months ago

  • Status changed from Need Review to Pending Backport

#10 Updated by David Zafman 4 months ago

To backport all the test-pidfile.sh cherry-pick 4 pull requests using the sha1s in this order:

https://github.com/ceph/ceph/pull/16587 sha1 20fee153cada2ea7659199a009679f978cd8fce1
https://github.com/ceph/ceph/pull/16635 sha1 c76f306ff56e46ee5818689a951987f7cefecb31
https://github.com/ceph/ceph/pull/16977 sha1 ff3de2304497544033837bb8d0c809a9e54a3e6e
https://github.com/ceph/ceph/pull/16987 sha1 16ea9b15717e1f2d49d871e5ec60c37c766a29dc

#11 Updated by Nathan Cutler 4 months ago

  • Copied to Backport #21050: luminous: test_pidfile.sh test is failing 2 places added

#12 Updated by Nathan Cutler 4 months ago

David Zafman wrote:

To backport all the test-pidfile.sh cherry-pick 4 pull requests using the sha1s in this order:

https://github.com/ceph/ceph/pull/16587 sha1 20fee153cada2ea7659199a009679f978cd8fce1
https://github.com/ceph/ceph/pull/16635 sha1 c76f306ff56e46ee5818689a951987f7cefecb31

These two are already in luminous.

https://github.com/ceph/ceph/pull/16977 sha1 ff3de2304497544033837bb8d0c809a9e54a3e6e
https://github.com/ceph/ceph/pull/16987 sha1 16ea9b15717e1f2d49d871e5ec60c37c766a29dc

I cherry-picked these; see https://github.com/ceph/ceph/pull/17161

#13 Updated by Nathan Cutler 4 months ago

  • Duplicated by Bug #20975: test_pidfile.sh is flaky added

#14 Updated by David Zafman 4 months ago

Another change is needed too. I've requested that in the pull request.

https://github.com/ceph/ceph/pull/17052 sha1 4d56de3608629571d07b48e9e3219420218c0b99

#15 Updated by Nathan Cutler 4 months ago

  • Duplicated by deleted (Bug #20975: test_pidfile.sh is flaky)

#16 Updated by Nathan Cutler 4 months ago

  • Related to Bug #20975: test_pidfile.sh is flaky added

#17 Updated by Nathan Cutler 4 months ago

  • Status changed from Pending Backport to Resolved
  • Backport deleted (luminous)

luminous backport rejected because the test continued to fail

#18 Updated by David Zafman 4 months ago

  • Status changed from Resolved to Verified

This problem still hasn't been solved. The is disabled, so moving back to verified.

Also available in: Atom PDF