Project

General

Profile

Bug #50219

qa/standalone/erasure-code/test-erasure-eio.sh fails since pg is not in recovery_unfound

Added by Neha Ojha almost 3 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2021-04-07T23:19:35.754 INFO:tasks.workunit.client.0.smithi039.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound:  ceph pg dump pgs
2021-04-07T23:19:36.094 INFO:tasks.workunit.client.0.smithi039.stdout:PG_STAT  OBJECTS  MISSING_ON_PRIMARY  DEGRADED  MISPLACED  UNFOUND  BYTES    OMAP_BYTES*  OMAP_KEYS*  LOG  DISK_LOG  STATE                       STATE_STAMP                      VERSION  REPORTED  UP              UP_PRIMARY  ACTING          ACTING_PRIMARY  LAST_SCRUB  SCRUB_STAMP                      LAST_DEEP_SCRUB  DEEP_SCRUB_STAMP                 SNAPTRIMQ_LEN
2021-04-07T23:19:36.095 INFO:tasks.workunit.client.0.smithi039.stdout:2.0          301                   0       301          0        0  1232896            0           0  101       101  active+undersized+degraded  2021-04-07T23:19:32.317506+0000   51'301    62:526  [3,1,4,0,NONE]           3  [3,1,4,0,NONE]               3         0'0  2021-04-07T23:15:27.626514+0000              0'0  2021-04-07T23:15:27.626514+0000              0
2021-04-07T23:19:36.095 INFO:tasks.workunit.client.0.smithi039.stdout:1.3            0                   0         0          0        0        0            0           0    0         0                active+clean  2021-04-07T23:17:02.911857+0000      0'0     58:52         [4,1,2]           4         [4,1,2]               4         0'0  2021-04-07T23:14:28.323017+0000              0'0  2021-04-07T23:14:28.323017+0000              0
2021-04-07T23:19:36.095 INFO:tasks.workunit.client.0.smithi039.stdout:1.2            0                   0         0          0        0        0            0           0    0         0                active+clean  2021-04-07T23:15:26.929019+0000      0'0     58:90         [0,4,1]           0         [0,4,1]               0         0'0  2021-04-07T23:14:28.323017+0000              0'0  2021-04-07T23:14:28.323017+0000              0
2021-04-07T23:19:36.095 INFO:tasks.workunit.client.0.smithi039.stdout:1.1            0                   0         0          0        0        0            0           0    0         0                active+clean  2021-04-07T23:15:27.284994+0000      0'0     58:28         [4,3,0]           4         [4,3,0]               4         0'0  2021-04-07T23:14:28.323017+0000              0'0  2021-04-07T23:14:28.323017+0000              0
2021-04-07T23:19:36.096 INFO:tasks.workunit.client.0.smithi039.stdout:1.0            0                   0         0          0        0        0            0           0    0         0       activating+undersized  2021-04-07T23:19:31.758115+0000      0'0     61:82           [1,0]           1           [1,0]               1         0'0  2021-04-07T23:14:28.323017+0000              0'0  2021-04-07T23:14:28.323017+0000              0
2021-04-07T23:19:36.096 INFO:tasks.workunit.client.0.smithi039.stdout:
2021-04-07T23:19:36.096 INFO:tasks.workunit.client.0.smithi039.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details.
2021-04-07T23:19:36.097 INFO:tasks.workunit.client.0.smithi039.stderr:dumped pgs
2021-04-07T23:19:36.105 INFO:tasks.workunit.client.0.smithi039.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:566: TEST_ec_backfill_unfound:  ceph pg 2.0 list_unfound
2021-04-07T23:19:36.230 INFO:tasks.workunit.client.0.smithi039.stdout:{
2021-04-07T23:19:36.231 INFO:tasks.workunit.client.0.smithi039.stdout:    "num_missing": 0,
2021-04-07T23:19:36.231 INFO:tasks.workunit.client.0.smithi039.stdout:    "num_unfound": 0,
2021-04-07T23:19:36.231 INFO:tasks.workunit.client.0.smithi039.stdout:    "objects": [],
2021-04-07T23:19:36.231 INFO:tasks.workunit.client.0.smithi039.stdout:    "state": "Active",
2021-04-07T23:19:36.231 INFO:tasks.workunit.client.0.smithi039.stdout:    "available_might_have_unfound": true,
2021-04-07T23:19:36.232 INFO:tasks.workunit.client.0.smithi039.stdout:    "might_have_unfound": [],
2021-04-07T23:19:36.232 INFO:tasks.workunit.client.0.smithi039.stdout:    "more": false
2021-04-07T23:19:36.232 INFO:tasks.workunit.client.0.smithi039.stdout:}
...
2021-04-07T23:19:36.578 INFO:tasks.workunit.client.0.smithi039.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:569: TEST_ec_backfill_unfound:  ceph pg 2.0 list_unfound
2021-04-07T23:19:36.578 INFO:tasks.workunit.client.0.smithi039.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:569: TEST_ec_backfill_unfound:  grep -q obj250
2021-04-07T23:19:36.579 INFO:tasks.workunit.client.0.smithi039.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:569: TEST_ec_backfill_unfound:  return 1

/a/nojha-2021-04-07_22:43:15-rados-wip-default-mclock-distro-basic-smithi/6027586

History

#1 Updated by Neha Ojha almost 3 years ago

  • Status changed from New to Closed

https://pulpito.ceph.com/nojha-2021-04-07_23:49:43-rados:standalone-master-distro-basic-smithi/ - the same test passes in master so this is an issue with my wip

#2 Updated by Laura Flores over 1 year ago

  • Status changed from Closed to New

The failure actually reproduced here:

/a/lflores-2022-10-17_18:19:55-rados:standalone-main-distro-default-smithi/7071287

2022-10-17T22:30:07.442 INFO:tasks.workunit.client.0.smithi090.stdout:    "scrubber": {
2022-10-17T22:30:07.442 INFO:tasks.workunit.client.0.smithi090.stdout:        "active": false,
2022-10-17T22:30:07.443 INFO:tasks.workunit.client.0.smithi090.stdout:        "must_scrub": false,
2022-10-17T22:30:07.443 INFO:tasks.workunit.client.0.smithi090.stdout:        "must_deep_scrub": false,
2022-10-17T22:30:07.443 INFO:tasks.workunit.client.0.smithi090.stdout:        "must_repair": false,
2022-10-17T22:30:07.443 INFO:tasks.workunit.client.0.smithi090.stdout:        "need_auto": false,
2022-10-17T22:30:07.444 INFO:tasks.workunit.client.0.smithi090.stdout:        "scrub_reg_stamp": "2022-10-19T06:57:02.186482+0000",
2022-10-17T22:30:07.444 INFO:tasks.workunit.client.0.smithi090.stdout:        "schedule": "scrub scheduled @ 2022-10-19T06:57:02.186" 
2022-10-17T22:30:07.444 INFO:tasks.workunit.client.0.smithi090.stdout:    },
2022-10-17T22:30:07.444 INFO:tasks.workunit.client.0.smithi090.stdout:    "agent_state": {}
2022-10-17T22:30:07.445 INFO:tasks.workunit.client.0.smithi090.stdout:}
2022-10-17T22:30:07.447 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound:  ceph pg 2.0 list_unfound
2022-10-17T22:30:07.447 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound:  grep -q obj250
2022-10-17T22:30:07.448 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound:  return 1
2022-10-17T22:30:07.448 INFO:tasks.workunit.client.0.smithi090.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run:  return 1

#3 Updated by Laura Flores over 1 year ago

  • Tags set to test-failure

#4 Updated by Radoslaw Zarzynski over 1 year ago

  • Status changed from New to Closed

The original issue was caused by a commit in a wip branch being tested, so it's highly unprobable it's a reoccurence. Closing. Dissected the new into: https://tracker.ceph.com/issues/57989.

Also available in: Atom PDF