Bug #44790: qa: adjust-ulimits invoked after being removed - CephFS - Ceph

Actions

Copy link

Bug #44790

closed

qa: adjust-ulimits invoked after being removed

Added by Greg Farnum about 4 years ago. Updated about 4 years ago.

Status:

Duplicate

Priority:

High

Assignee:

Category:

Testing

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(FS):

Labels (FS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

http://pulpito.ceph.com/gregf-2020-03-26_16:03:14-fs-wip-greg-testing-326-distro-basic-smithi/4891336/

2020-03-26T16:57:22.660 DEBUG:teuthology.run_tasks:Unwinding manager install
2020-03-26T16:57:22.677 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...
2020-03-26T16:57:22.677 INFO:teuthology.orchestra.run.smithi029:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits
2020-03-26T16:57:22.681 INFO:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits
...
2020-03-26T16:59:29.422 DEBUG:teuthology.run_tasks:Unwinding manager ceph
2020-03-26T16:59:29.447 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges...
2020-03-26T16:59:29.447 INFO:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --log-early osd dump --format=json
2020-03-26T16:59:29.473 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: adjust-ulimits: command not found

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Greg Farnum about 4 years ago

This popped up a couple times many years ago; I'm not sure if it's a very rare race in shutdown or what. No apparent changes in the FS qa suites that could have caused it.

Actions

Copy link

Updated by Greg Farnum about 4 years ago

Happened again in /a/gregf-2020-04-01_18:45:43-fs-wip-greg-testing-41-distro-basic-smithi/4914097

I did happen to notice an error in the log right before it started unwinding, but it looks like the test was already in shutdown mode and I'm not sure if it's relevant:

2020-04-01T20:45:17.443 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.b
2020-04-01T20:45:17.443 DEBUG:teuthology.run_tasks:Unwinding manager install
2020-04-01T20:45:17.461 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits...

Actions

Copy link

Updated by Greg Farnum about 4 years ago

Priority changed from Normal to High

And again: http://pulpito.ceph.com/gregf-2020-04-03_05:44:43-fs-wip-greg-testing-42-1-distro-basic-smithi/4921286/

Actions

Copy link

Updated by Greg Farnum about 4 years ago

http://pulpito.ceph.com/yuriw-2020-04-07_17:47:23-fs-wip-octopus-rgw-msg-fixes-distro-basic-smithi/4931869/
http://pulpito.ceph.com/yuriw-2020-04-07_17:47:23-fs-wip-octopus-rgw-msg-fixes-distro-basic-smithi/4931949/

These are both fsx runs and it wasn't just adjust-ulimits, hit it with fusermount as well. But still seems to be we are somehow doing teardown out of order.

Actions

Copy link