Bug #54460: snaptest-multiple-capsnaps.sh test failure - CephFS - Ceph

Actions

Copy link

Bug #54460

closed

snaptest-multiple-capsnaps.sh test failure

Added by Jeff Layton about 2 years ago. Updated 10 months ago.

Status:

Resolved

Priority:

Normal

Assignee:

Milind Changire

Category:

Correctness/Safety

Target version:

Ceph - v18.0.0

% Done:

100%

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(FS):

MDS

Labels (FS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

Test failure on quincy run:

https://pulpito.ceph.com/yuriw-2022-03-01_20:21:46-fs-wip-yuri-testing-2022-02-28-0823-quincy-distro-default-smithi/6715199

2022-03-01T22:21:46.296 INFO:tasks.workunit.client.0.smithi152.stderr:+ grep qwer .snap/4/a
2022-03-01T22:21:46.303 INFO:tasks.workunit.client.0.smithi152.stdout:qwer
2022-03-01T22:21:46.304 INFO:tasks.workunit.client.0.smithi152.stderr:+ stat .snap/4/a
2022-03-01T22:21:46.305 INFO:tasks.workunit.client.0.smithi152.stderr:+ grep 'Size: 5'
2022-03-01T22:21:46.307 INFO:tasks.workunit.client.0.smithi152.stdout:  Size: 5                 Blocks: 1          IO Block: 4194304 regular file
2022-03-01T22:21:46.308 INFO:tasks.workunit.client.0.smithi152.stderr:+ stat .snap/4/a
2022-03-01T22:21:46.309 INFO:tasks.workunit.client.0.smithi152.stderr:+ grep -- -rw-rw-rw-
2022-03-01T22:21:46.312 DEBUG:teuthology.orchestra.run:got remote process result: 1
2022-03-01T22:21:46.313 INFO:tasks.workunit:Stopping ['fs/snaps'] on client.0...

Looks like the snap/4/a grep failed, so the permissions were wrong?

Files

test-snaptest-multiple-capsnaps.sh (709 Bytes) test-snaptest-multiple-capsnaps.sh

test harness

Milind Changire, 05/13/2022 07:25 AM

Subtasks 3 (0 open — 3 closed)

Actions

Copy link

Updated by Venky Shankar about 2 years ago

Status changed from New to Triaged
Assignee set to Milind Changire
Target version set to v18.0.0
Backport set to quincy, pacific

Actions

Copy link

Updated by Venky Shankar about 2 years ago

Another instance - http://pulpito.front.sepia.ceph.com/yuriw-2022-03-29_20:09:22-fs-wip-yuri-testing-2022-03-29-0741-quincy-distro-default-smithi/6767087/

Milind, please take a look.

Actions

Copy link

Updated by Milind Changire almost 2 years ago

File test-snaptest-multiple-capsnaps.sh test-snaptest-multiple-capsnaps.sh added

client-type: fuse

Iteratively running shell scripts under qa/workunits/fs/snaps/ on a fuse mount with a 10 seconds pause between execution of two consecutive scripts can easily reproduce the error within 2 iterations of the entire script set under the mentioned directory.
If sleep time between two shell scripts is reduced to 1 second in the above test then it is hard to reproduce the error.
The error is easily reproducible on a T480s Lenovo ThinkPad.
Exclusively running qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh in iterations does not reproduce the error quicker.

Attached shell script is used to reproduce the error.

Actions

Copy link

Updated by Venky Shankar over 1 year ago

Milind Changire wrote:

client-type: fuse

Iteratively running shell scripts under qa/workunits/fs/snaps/ on a fuse mount with a 10 seconds pause between execution of two consecutive scripts can easily reproduce the error within 2 iterations of the entire script set under the mentioned directory.

If sleep time between two shell scripts is reduced to 1 second in the above test then it is hard to reproduce the error.

The error is easily reproducible on a T480s Lenovo ThinkPad.

Exclusively running qa/workunits/fs/snaps/snaptest-multiple-capsnaps.sh in iterations does not reproduce the error quicker.

Attached shell script is used to reproduce the error.

Were you able to RCA the problem Milind?

Actions

Copy link