Bug #51462: rgw: resolve empty ordered bucket listing results w/ CLS filtering - rgw - Ceph

Actions

Copy link

Bug #51462

closed

rgw: resolve empty ordered bucket listing results w/ CLS filtering

Added by J. Eric Ivancich almost 3 years ago. Updated over 1 year ago.

Status:

Resolved

Priority:

High

Assignee:

J. Eric Ivancich

Target version:

Ceph - v17.0.0

% Done:

100%

Source:

Tags:

Backport:

octopus, pacific

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

42125

Crash signature (v1):

Crash signature (v2):

Description

Bucket listing filtering was moved from the RGW layer to the CLS layer to improve efficiency. However currently if there are enough entries that are filtered out, the CLS call may return zero entries back to RGW. Since we did not mark how far we got, calling it again will yield the same result, causing the process to be stuck and fail.

Related issues 2 (0 open — 2 closed)

Actions

Copy link

Updated by J. Eric Ivancich almost 3 years ago

Subject changed from rgw: resolve broken listing to rgw: resolve empty ordered bucket listing results w/ CLS filtering

Actions

Copy link

Updated by J. Eric Ivancich almost 3 years ago

Description updated (diff)

Actions

Copy link

Updated by Casey Bodley almost 3 years ago

Status changed from In Progress to Fix Under Review

Actions

Copy link

Updated by Casey Bodley over 2 years ago

Status changed from Fix Under Review to Pending Backport

Actions

Copy link

Updated by Backport Bot over 2 years ago

Copied to Backport #52075: pacific: rgw: resolve empty ordered bucket listing results w/ CLS filtering added

Actions

Copy link

Updated by Backport Bot over 2 years ago

Copied to Backport #52076: octopus: rgw: resolve empty ordered bucket listing results w/ CLS filtering added

Actions

Copy link

Updated by Dan van der Ster about 2 years ago

Hi, I found this via https://bugzilla.redhat.com/show_bug.cgi?id=1975939 -- after upgrading from 14.2.21 to 15.2.15 we have one bucket which the user cannot list. On the rgws we have lots of ERROR marker failed to make forward progress:

2022-02-18T01:09:54.996+0100 7f847946c700  0 RGWRados::Bucket::List::list_objects_ordered: ERROR marker failed to make forward progress; attempt=2, prev_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7/_manifests/tags/20201119/current/link, cur_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7/_manifests/tags/20201119/current/link
2022-02-18T01:09:58.594+0100 7f8498d2c700  0 RGWRados::Bucket::List::list_objects_ordered: ERROR marker failed to make forward progress; attempt=3, prev_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7/_manifests/tags/20201129/current/link[wS0hQgJWFFNPJ-Ve.tSv5fNYepi-XkD], cur_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7/_manifests/tags/20201129/current/link[wS0hQgJWFFNPJ-Ve.tSv5fNYepi-XkD]
2022-02-18T01:10:13.969+0100 7f8504006700  0 RGWRados::Bucket::List::list_objects_ordered: ERROR marker failed to make forward progress; attempt=3, prev_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7_jdk8/_manifests/tags/20201129/current/link[euGwo3OfQOute7NaT1XMVjW19VKkYBI], cur_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7_jdk8/_manifests/tags/20201129/current/link[euGwo3OfQOute7NaT1XMVjW19VKkYBI]
2022-02-18T01:10:16.474+0100 7f84a5545700  0 RGWRados::Bucket::List::list_objects_ordered: ERROR marker failed to make forward progress; attempt=3, prev_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7_jdkmulti/_manifests/tags/20201119/current/link[HO9TWgG9HxdwIncLUaDhzxmgqx0z4vZ], cur_marker=docker/registry/v2/repositories/acc-co/docker/acc-base-images/acc_cc7_jdkmulti/_manifests/tags/20201119/current/link[HO9TWgG9HxdwIncLUaDhzxmgqx0z4vZ]
...

The octopus backport looks non-trivial, but we'll try it. (OTOH it applies cleanly to pacific).
Do you think this PR#42125 is likely to fix?

Actions

Copy link