Project

General

Profile

Activity

From 01/25/2022 to 02/23/2022

02/23/2022

07:05 PM rgw-testing Backport #54388 (Resolved): quincy: rgw/tests: Add lifecycle test-suite to teuthology run
Backport Bot
07:01 PM Bug #54112 (Fix Under Review): segmentation fault in UserAsyncRefreshHandler::init_fetch
Casey Bodley
06:57 PM rgw-testing Bug #54387 (Pending Backport): rgw/tests: Add lifecycle test-suite to teuthology run
Execute lifecycle s3-tests in the teuthology test-suite by configuring
required storage classes and 'rgw lc debug in...
Soumya Koduri
05:01 PM Backport #52010 (In Progress): octopus: radosgw-admin bi list fails with Input/Output error
New PR: https://github.com/ceph/ceph/pull/45088 Dan van der Ster
05:01 PM Backport #52009 (In Progress): pacific: radosgw-admin bi list fails with Input/Output error
New PR: https://github.com/ceph/ceph/pull/45087 Dan van der Ster
02:12 PM Backport #53007 (In Progress): pacific: segfault on FIPS enabled server as result of EVP_md5 disa...
Casey Bodley
02:12 PM Backport #53008 (In Progress): octopus: segfault on FIPS enabled server as result of EVP_md5 disa...
Casey Bodley
09:09 AM Bug #51429: radosgw-admin bi list fails with Input/Output error
Dan van der Ster wrote:
> After updating to 15.2.15 we now have at least one bucket which we cannot list (the user i...
Dan van der Ster
08:57 AM Bug #51462: rgw: resolve empty ordered bucket listing results w/ CLS filtering
Good news, the collected backports in https://github.com/ceph/ceph/pull/45088 fixed the bucket listing issue for us.
...
Dan van der Ster

02/22/2022

07:36 PM Backport #54162: quincy: OpsLogRados::log segfaults in rgw/multisite suite
Casey Bodley wrote:
> https://github.com/ceph/ceph/pull/45074
merged
Yuri Weinstein
07:34 PM Bug #54193: rgw/dbstore: Add dbstore tests to 'make check'
https://github.com/ceph/ceph/pull/45061 merged Yuri Weinstein
08:32 AM Bug #54363 (Resolved): segfault when Resharding occurs during LC
on master a segfault occurs when dynamic resharding is triggered during lc processing:
repro:
- write 10,000,00...
Mark Kogan
02:43 AM Bug #54361 (Pending Backport): common/compressor: crash when using qat compressor concurrently

@export QAT_SECTION_NAME=SSL@
@MON=1 OSD=1 MDS=0 MGR=1 RGW=1 ../src/vstart.sh -n -d --bluestore -X \@
@...
Hualong Feng

02/21/2022

08:38 PM Bug #54124 (Resolved): smaller object workloads wane and terminate after a few hours
Vikhyat Umrao
08:37 PM Bug #54124: smaller object workloads wane and terminate after a few hours
Tim Wilkinson wrote:
> Again using rgw_max_concurrent_requests=2048 and rgw_thread_pool_size=1024, the repeatability...
Vikhyat Umrao
08:35 PM Bug #54124: smaller object workloads wane and terminate after a few hours
Again using rgw_max_concurrent_requests=2048 and rgw_thread_pool_size=1024, the repeatability testing using s3 auth a... Tim Wilkinson
07:27 PM Bug #54359 (New): dbstore: heavy use of macros and goto
the macros in sqliteDB.cc:
* obscure error handling and control flow (@goto out@)
* rely on 'external' state like @...
Casey Bodley
05:49 PM Bug #54355 (New): dbstore: confusing use of virtual inheritance for DBOps
virtual inheritance is usually a sign that class hierarchies are too complicated. here it's used because each SQL op ... Casey Bodley
05:32 PM Bug #54354 (New): dbstore: DB::objectmap leaks memory and is not thread-safe between DB instances
what exactly is objectmap for? it only seems to be used for SQLiteDB::ListAllObjects() as a way to know which buckets... Casey Bodley
03:06 PM Bug #54346 (New): dbstore: bind text/blob columns with SQLITE_STATIC instead of SQLITE_TRANSIENT
use of SQLITE_TRANSIENT indicates that the given memory isn't stable over the lifetime of the operation, so requests ... Casey Bodley
12:50 PM Bug #52339: rgw upgrade 14.2.22 slow requests
Just wanted to drop a comment that we upgraded our ceph cluster to pacific and are not seeing this issue anymore.
We...
Maximilian Stinsky

02/20/2022

07:16 AM Bug #54325 (New): lua: elasticsearch example script does not check for null object/bucket
Request.Object and Request.Bucket are optional fields, and needs to be checked before being used
script: https://git...
Yuval Lifshitz

02/18/2022

10:22 PM Bug #51429: radosgw-admin bi list fails with Input/Output error
After updating to 15.2.15 we now have at least one bucket which we cannot list (the user is trying to rclone sync the... Dan van der Ster
10:15 PM Backport #54074 (In Progress): pacific: rgw: fix bucket index list minor calculation bug
Backport for this is included in https://github.com/ceph/ceph/pull/45087 Dan van der Ster
10:01 PM Backport #54072 (In Progress): octopus: rgw: fix bucket index list minor calculation bug
Dan van der Ster
10:00 PM Backport #54072: octopus: rgw: fix bucket index list minor calculation bug
Backport for this included in https://github.com/ceph/ceph/pull/45088 Dan van der Ster
09:32 PM Backport #52076 (In Progress): octopus: rgw: resolve empty ordered bucket listing results w/ CLS ...
Dan van der Ster
09:10 PM Backport #52075 (In Progress): pacific: rgw: resolve empty ordered bucket listing results w/ CLS ...
Dan van der Ster
06:53 PM Bug #51462: rgw: resolve empty ordered bucket listing results w/ CLS filtering
Hi, I found this via https://bugzilla.redhat.com/show_bug.cgi?id=1975939 -- after upgrading from 14.2.21 to 15.2.15 w... Dan van der Ster
07:32 AM Bug #53737: Ceph_Multi-Site : Data added in primary RGW while secondary RGW service is down does ...
I think this issue is fixed by this PR: https://github.com/ceph/ceph/pull/41316
But this PR is not backport to oct...
Mule Te

02/17/2022

04:52 PM Bug #53956: pacific radosgw-admin binary reports incorrect stats on quincy cluster
hmm, i suspected a backward-compatibility issue with the RGWBucketInfo's encode/decode, but pacific and quincy are id... Casey Bodley
03:08 PM Bug #53956: pacific radosgw-admin binary reports incorrect stats on quincy cluster
the log for pacific only shows the original 11 shards:
"max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#",
w...
Casey Bodley
04:32 PM Bug #54265: s3select: c++20 compilation error in boost/msm/back/state_machine.hpp
Gal Salomon wrote:
> does s3select the only boost_state_machine user?
i believe so - at least s3select was the on...
Casey Bodley
04:13 PM Bug #54265: s3select: c++20 compilation error in boost/msm/back/state_machine.hpp
Yes, please use something else.
Matt
Matt Benjamin
04:06 PM Bug #54265: s3select: c++20 compilation error in boost/msm/back/state_machine.hpp
indeed,
we found that (Girjeh and myself) there is a faster(3x) alternative for boost_state_machine.
the thing i...
Gal Salomon
03:13 PM Bug #54265: s3select: c++20 compilation error in boost/msm/back/state_machine.hpp
I think I heard that we're eliminating the dependency on boost::state_machine--its apparently a large source of CPU o... Matt Benjamin
03:32 PM Bug #51574 (Fix Under Review): Segfault when uploading file
Daniel Gryniewicz
03:24 PM Bug #54266 (Fix Under Review): rgw: cmake configure error on fedora-37/rawhide
Casey Bodley
02:48 PM Backport #54162 (In Progress): quincy: OpsLogRados::log segfaults in rgw/multisite suite
https://github.com/ceph/ceph/pull/45074 Casey Bodley

02/16/2022

11:35 PM Bug #54307 (New): test_cls_rgw.sh: 'index_list_delimited' test times out
/a/yuriw-2022-02-16_15:53:49-rados-wip-yuri11-testing-2022-02-15-1643-distro-default-smithi/6688784
Last test run ...
Laura Flores
12:17 AM Feature #8956 (Resolved): rgw: support bucket notification
Matt Benjamin
12:17 AM Feature #8956: rgw: support bucket notification
Seems hard to argue with. Matt Benjamin

02/15/2022

11:45 PM Feature #8956: rgw: support bucket notification
can we close this out as this is already supported? (since nautilus) Joseph Mundackal
05:45 PM Backport #54291 (New): quincy: rgw/dbstore: Add dbstore tests to 'make check'
Backport Bot
05:42 PM Bug #54193 (Pending Backport): rgw/dbstore: Add dbstore tests to 'make check'
Soumya Koduri
06:26 AM Bug #54173: User delete with purge data didn’t delete the data (multisite)
Casey Bodley wrote:
> does sync status show that sync has caught up with the deletes?
Seems like not, I don't und...
Ist Gab

02/14/2022

07:29 PM Bug #51574: Segfault when uploading file
I confirmed that the test script in https://github.com/jgraichen/ceph-rgw-51574 also results in segfault in 16.2.7:
...
Jan Graichen
05:56 PM Backport #54278 (Resolved): pacific: rgw: remove rgw_rados_pool_pg_num_min and its use on pool cr...
https://github.com/ceph/ceph/pull/46235 Backport Bot
05:56 PM Backport #54277 (Resolved): quincy: rgw: remove rgw_rados_pool_pg_num_min and its use on pool cre...
https://github.com/ceph/ceph/pull/46234 Backport Bot
05:51 PM Bug #52673 (Pending Backport): rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation
Neha Ojha
05:03 PM Bug #52673: rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation
https://github.com/ceph/ceph/pull/44983 merged Yuri Weinstein
03:26 PM Bug #54054: dencoder: missing lots of rgw and cls_rgw types in src/tools/ceph-dencoder/rgw_types.h
example commit that added cls_rgw_lc_get_entry_ret in https://github.com/ceph/ceph/pull/44743/commits/c8291d9662a8dca... Casey Bodley
12:57 PM Bug #54266: rgw: cmake configure error on fedora-37/rawhide
https://github.com/ceph/ceph/pull/45022 Kaleb KEITHLEY
03:43 AM Bug #54214: nautilus: unable to delete the tag of the current version object in a bucket with ver...
Casey Bodley wrote:
> do you know if this happens on supported ceph versions?
This was found on nautilus.
Jiahui Zeng

02/12/2022

04:51 PM Bug #54266 (Pending Backport): rgw: cmake configure error on fedora-37/rawhide
see https://kojipkgs.fedoraproject.org//work/tasks/6624/82706624/build.log, excerpted below
...
CMake Error: Th...
Kaleb KEITHLEY
12:17 AM Bug #54265 (Resolved): s3select: c++20 compilation error in boost/msm/back/state_machine.hpp
set CMAKE_CXX_STANDARD=20:... Casey Bodley

02/11/2022

08:37 PM Bug #54124: smaller object workloads wane and terminate after a few hours
The retesting with the tweaks described above allowed the hybrid-48hr runs to execute for the duration. For the sake ... Tim Wilkinson
04:02 PM Bug #54260 (New): unify RGWRados::guard_reshard() and RGWRados::Bucket::UpdateIndex::guard_reshard()
we have two functions that essentially do the same thing, we shouldn't duplicate them Casey Bodley
07:53 AM Backport #54073 (Resolved): quincy: rgw: fix bucket index list minor calculation bug
Konstantin Shalygin
02:35 AM Bug #54254 (Resolved): when the remove-all parameter of rgw admin operation trim usage interface ...
when the remove-all parameter of rgw admin operation trim usage interface is set false, the usage is trimmed. i use t... peng liang

02/10/2022

11:45 PM Bug #52673 (In Progress): rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation
Vikhyat Umrao
06:43 PM Bug #52673: rgw: remove rgw_rados_pool_pg_num_min and its use on pool creation
opened https://github.com/ceph/ceph/pull/44983 which removes rgw_rados_pool_pg_num_min Casey Bodley
11:37 PM Backport #54143: quincy: bilog trim: segfault in RGWRadosBILogTrimCR::send_request if bucket shar...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/44906
merged
Yuri Weinstein
11:36 PM Backport #54073: quincy: rgw: fix bucket index list minor calculation bug
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/44852
merged
Yuri Weinstein
05:48 PM Bug #54124: smaller object workloads wane and terminate after a few hours
quick update of the current status (pasting from mail):... Mark Kogan
03:06 PM Bug #54124 (Triaged): smaller object workloads wane and terminate after a few hours
Casey Bodley
03:53 PM Bug #54249 (New): beast: limit the number of accepted connections
the beast frontend only limits the number of concurrent requests (rgw_max_concurrent_requests) in-flight, but does no... Casey Bodley
03:27 PM Bug #54173 (Need More Info): User delete with purge data didn’t delete the data (multisite)
Casey Bodley
03:27 PM Bug #54173: User delete with purge data didn’t delete the data (multisite)
does sync status show that sync has caught up with the deletes? Casey Bodley
03:25 PM Bug #54181 (Won't Fix): RGW does not support iso8601 date format in Date Header
see discussion in PR Casey Bodley
03:23 PM Bug #54214: nautilus: unable to delete the tag of the current version object in a bucket with ver...
do you know if this happens on supported ceph versions? Casey Bodley
02:59 PM Bug #54247 (Resolved): rgw/crypt/barbican: Cannot create secret
this started failing recently. http://qa-proxy.ceph.com/teuthology/teuthology-2022-02-10_03:05:03-rgw-master-distro-d... Casey Bodley

02/09/2022

01:40 PM Backport #54229 (Rejected): pacific: The storage class attr is not correct when the object is rew...
Backport Bot
01:40 PM Backport #54228 (Rejected): octopus: The storage class attr is not correct when the object is rew...
Backport Bot
01:40 PM Backport #54227 (In Progress): quincy: The storage class attr is not correct when the object is r...
https://github.com/ceph/ceph/pull/50520 Backport Bot
01:39 PM Bug #53790 (Pending Backport): The storage class attr is not correct when the object is rewrited ...
Casey Bodley
06:55 AM Backport #52242: octopus: RadosGW requests to Open Policy Agent add 40ms latency
please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/44950
ceph-backport.sh versi...
Benoît Knecht
06:54 AM Backport #52241: pacific: RadosGW requests to Open Policy Agent add 40ms latency
please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/44949
ceph-backport.sh versi...
Benoît Knecht
04:05 AM Bug #54214 (New): nautilus: unable to delete the tag of the current version object in a bucket wi...
Set the tag to the object that exists in a bucket which already enables versioning for the objects in the bucket. The... Jiahui Zeng

02/08/2022

06:57 PM Bug #54208 (Fix Under Review): Data race in RGWDataChangesLog::ChangeStatus
Adam Emerson
06:36 PM Bug #54208 (Resolved): Data race in RGWDataChangesLog::ChangeStatus
Write to ChangeStatus while holding the wrong lock. Adam Emerson
03:30 PM Backport #54202 (Rejected): pacific: rgw:copy-object:x-amz-copy-source-if-* headers not work
Backport Bot
03:30 PM Backport #54201 (In Progress): quincy: rgw:copy-object:x-amz-copy-source-if-* headers not work
https://github.com/ceph/ceph/pull/50519 Backport Bot
03:30 PM Backport #54200 (Rejected): octopus: rgw:copy-object:x-amz-copy-source-if-* headers not work
Backport Bot
03:27 PM Bug #53945 (Pending Backport): rgw:copy-object:x-amz-copy-source-if-* headers not work
Casey Bodley
12:39 PM Bug #54193 (Pending Backport): rgw/dbstore: Add dbstore tests to 'make check'
Include & run dbstore unit tests as part of 'make check' Soumya Koduri
12:35 PM Backport #54192 (New): quincy: Early return in cloud tiering
Backport Bot
12:30 PM Bug #54183 (Pending Backport): Early return in cloud tiering
Daniel Gryniewicz
10:27 AM Bug #54124: smaller object workloads wane and terminate after a few hours
bringing attention to the balancer (as noted in the email thread)
After discussion with Tim -
The cluster has more ...
Mark Kogan
10:15 AM Bug #54124: smaller object workloads wane and terminate after a few hours
Casey Bodley wrote:
> thanks Mark! with rgw_max_concurrent_requests=1024, it seems like we should still get consiste...
Mark Kogan
02:34 AM Bug #53668: Why not add a xxx.retry obJ to metadata synchronization at multisite for exception re...
Casey Bodley wrote:
> in general, object uploads tend to be way more frequent than metadata changes like bucket/user...
Jinghua Zeng

02/07/2022

05:37 PM Bug #54124: smaller object workloads wane and terminate after a few hours
> I at least would strongly suspect it. I think you'd need to verify by inspection that an RGW cluster in the waning... Matt Benjamin
05:31 PM Bug #54124: smaller object workloads wane and terminate after a few hours
thanks Mark! with rgw_max_concurrent_requests=1024, it seems like we should still get consistent performance without ... Casey Bodley
12:17 PM Bug #54124: smaller object workloads wane and terminate after a few hours
thank you for the detailed information for analysis,
from the logs in https://tracker.ceph.com/issues/54124#note...
Mark Kogan
05:19 PM Fix #54174 (Fix Under Review): rgw dbstore test env init wrong
Casey Bodley
06:41 AM Fix #54174: rgw dbstore test env init wrong
Could you please provide more details about the issue? Soumya Koduri
06:39 AM Fix #54174 (Resolved): rgw dbstore test env init wrong
joke lee
05:05 PM Bug #54183 (Pending Backport): Early return in cloud tiering
In transition_obj_to_cloud(), there's an early return in an error case, where the target is not removed from the list. Daniel Gryniewicz
04:31 PM Bug #54181 (Won't Fix): RGW does not support iso8601 date format in Date Header
RGW checks only rfc2616 date format in header "Date".
When client uses iso8601 format, HTTP code=403...
Dmitry Kvashnin
09:12 AM Bug #51429: radosgw-admin bi list fails with Input/Output error
J. Eric Ivancich wrote:
> Pawel Stefanski wrote:
> > Tried with Pacific backport found here https://github.com/ceph...
Pawel Stefanski
03:50 AM Bug #54173 (Closed): User delete with purge data didn’t delete the data (multisite)
Hi,
In my multisite setup I’ve deleted a user with radosgw-admin user rm —purge-data but the objects still in the ...
Ist Gab

02/06/2022

03:40 PM Bug #54167 (Resolved): segfault in radosgw-admin bucket rewrite --bucket
using debian package `16.2.7-1~bpo11+1` on bullseye
The following always crashes. It happens on multiple buckets.
...
Matthew Darwin

02/05/2022

02:00 PM Backport #54162 (Resolved): quincy: OpsLogRados::log segfaults in rgw/multisite suite
Backport Bot
01:55 PM Bug #54130 (Pending Backport): OpsLogRados::log segfaults in rgw/multisite suite
Casey Bodley
08:46 AM Backport #54145 (In Progress): octopus: bilog trim: segfault in RGWRadosBILogTrimCR::send_request...
Mykola Golub
08:39 AM Backport #54144 (In Progress): pacific: bilog trim: segfault in RGWRadosBILogTrimCR::send_request...
Mykola Golub
08:38 AM Backport #54143 (In Progress): quincy: bilog trim: segfault in RGWRadosBILogTrimCR::send_request ...
Mykola Golub

02/04/2022

08:02 PM Backport #54157 (Resolved): quincy: rgw: "reshard cancel" errors with "invalid argument"
https://github.com/ceph/ceph/pull/49090 Backport Bot
08:02 PM Backport #54156 (Rejected): octopus: rgw: "reshard cancel" errors with "invalid argument"
Backport Bot
08:01 PM Backport #54155 (Resolved): pacific: rgw: "reshard cancel" errors with "invalid argument"
https://github.com/ceph/ceph/pull/49091 Backport Bot
08:01 PM Backport #54154 (Resolved): quincy: rgw: in bucket reshard list, clarify new num shards is tentative
https://github.com/ceph/ceph/pull/45504 Backport Bot
08:01 PM Backport #54153 (Rejected): octopus: rgw: in bucket reshard list, clarify new num shards is tenta...
Backport Bot
08:01 PM Backport #54152 (Resolved): pacific: rgw: in bucket reshard list, clarify new num shards is tenta...
https://github.com/ceph/ceph/pull/45509 Backport Bot
08:01 PM Backport #54151 (New): quincy: rgwlc: version of cls_rgw_lc_get_entry_ret is not updated
Backport Bot
08:01 PM Backport #54150 (Resolved): quincy: PostObj ignores error, may lose data
https://github.com/ceph/ceph/pull/45501 Backport Bot
08:00 PM Backport #54149 (Resolved): octopus: PostObj ignores error, may lose data
https://github.com/ceph/ceph/pull/45503 Backport Bot
08:00 PM Backport #54148 (Resolved): pacific: PostObj ignores error, may lose data
https://github.com/ceph/ceph/pull/45502 Backport Bot
08:00 PM Backport #54147 (Resolved): pacific: admin: datalog list always return max-entries
https://github.com/ceph/ceph/pull/45500 Backport Bot
08:00 PM Backport #54146 (Resolved): quincy: admin: datalog list always return max-entries
https://github.com/ceph/ceph/pull/45499 Backport Bot
08:00 PM Backport #54145 (Rejected): octopus: bilog trim: segfault in RGWRadosBILogTrimCR::send_request if...
https://github.com/ceph/ceph/pull/44908 Backport Bot
08:00 PM Backport #54144 (Resolved): pacific: bilog trim: segfault in RGWRadosBILogTrimCR::send_request if...
https://github.com/ceph/ceph/pull/44907 Backport Bot
08:00 PM Backport #54143 (Resolved): quincy: bilog trim: segfault in RGWRadosBILogTrimCR::send_request if ...
https://github.com/ceph/ceph/pull/44906 Backport Bot
07:57 PM Bug #54116 (Pending Backport): admin: datalog list always return max-entries
Casey Bodley
07:56 PM Bug #53705 (Pending Backport): rgw: in bucket reshard list, clarify new num shards is tentative
Casey Bodley
07:56 PM Bug #54114 (Pending Backport): PostObj ignores error, may lose data
Casey Bodley
07:56 PM Bug #53927 (Pending Backport): rgwlc: version of cls_rgw_lc_get_entry_ret is not updated
Casey Bodley
07:56 PM Bug #53429 (Pending Backport): rgw: "reshard cancel" errors with "invalid argument"
Casey Bodley
07:55 PM Bug #54119 (Pending Backport): bilog trim: segfault in RGWRadosBILogTrimCR::send_request if bucke...
Casey Bodley
07:19 PM Bug #51429: radosgw-admin bi list fails with Input/Output error
We attempted to apply the backport as well (although on Octopus, 15.2.14). It appeared the issue was still present as... Justin Goetz
06:26 PM Bug #51429: radosgw-admin bi list fails with Input/Output error
Pawel Stefanski wrote:
> Tried with Pacific backport found here https://github.com/ceph/ceph/pull/44815
> but still...
J. Eric Ivancich
09:07 AM Bug #51429: radosgw-admin bi list fails with Input/Output error
Tried with Pacific backport found here https://github.com/ceph/ceph/pull/44815
but still has issues with "ERROR: bi_...
Pawel Stefanski
06:50 PM Bug #54124: smaller object workloads wane and terminate after a few hours
Let's try that again with some newlines ...... Tim Wilkinson
06:44 PM Bug #54124: smaller object workloads wane and terminate after a few hours
rgw and rados bench workloads were executed without apparent issue. Using s3cmd to create buckets, populate them, lis... Tim Wilkinson
08:25 AM Bug #53913: rgw: s3website crashes after upgrade from octopus to pacific
Is that enough? previous lines are just osd spams
-33> 2022-02-04T08:36:56.009+0100 7f5968b77700 10 monclient: ...
Hubert Niedlich

02/03/2022

11:04 PM Bug #51327: Format of date/time header "x-amz-object-lock-retain-until-date" is incorrect (does n...
We were able to build and test the backport here: https://github.com/ceph/ceph/pull/43656 on our Octopus v15.2.14 clu... Justin Goetz
09:34 PM Bug #54130 (Fix Under Review): OpsLogRados::log segfaults in rgw/multisite suite
Casey Bodley
06:43 PM Bug #54130: OpsLogRados::log segfaults in rgw/multisite suite
thanks for taking a look! another option to consider is letting RGWRados own OpsLogRados and handle its init/shutdown... Casey Bodley
06:33 PM Bug #54130: OpsLogRados::log segfaults in rgw/multisite suite
Seems to be caused by the Store being de-allocated when a realm is reloaded. OpsLogRados is not reinitialized to use ... Cory Snyder
03:45 PM Bug #54130 (Resolved): OpsLogRados::log segfaults in rgw/multisite suite
... Casey Bodley
07:43 PM Bug #54119 (Fix Under Review): bilog trim: segfault in RGWRadosBILogTrimCR::send_request if bucke...
Mykola Golub
06:18 PM Bug #53668: Why not add a xxx.retry obJ to metadata synchronization at multisite for exception re...
Casey Bodley wrote:
> metadata sync *could* have an error repo, but we rarely see issues with metadata sync catching...
Christian Rohmann
04:41 PM Bug #53668 (Need More Info): Why not add a xxx.retry obJ to metadata synchronization at multisite...
in general, object uploads tend to be way more frequent than metadata changes like bucket/user creation. the datalog ... Casey Bodley
04:37 PM Bug #54124: smaller object workloads wane and terminate after a few hours
Vikhyat Umrao wrote:
> > The generic sized workload has executed without issues throughout the full COSBench test cy...
Vikhyat Umrao
04:25 PM Bug #54124: smaller object workloads wane and terminate after a few hours
> The generic sized workload has executed without issues throughout the full COSBench test cycle ..
Tim - can you ...
Vikhyat Umrao
02:16 PM Bug #54124: smaller object workloads wane and terminate after a few hours
... Tim Wilkinson
02:12 PM Bug #54124: smaller object workloads wane and terminate after a few hours
Correction, the above .xml is the generic sized workload that executes without issue.
Below is the workload in qu...
Tim Wilkinson
02:10 PM Bug #54124: smaller object workloads wane and terminate after a few hours
An example of the workload, this the 48hr hybrid test ...... Tim Wilkinson
02:04 PM Bug #54124 (Resolved): smaller object workloads wane and terminate after a few hours
In our initial baseline tests with quincy, we use two sets of object sizes.
The generic sized workload looks like ...
Tim Wilkinson
03:47 PM Bug #53708: ceph multisite sync deleted unversioned object failed
versioned object sync has never been especially stable. we have some test cases around suspended versions that fail o... Casey Bodley
03:30 PM Bug #53739: the log_usage func may cause all usage logs to be written to the same user shard
It also appears to be fixed in newer nautilus. Daniel Gryniewicz
03:29 PM Bug #53739 (Won't Fix - EOL): the log_usage func may cause all usage logs to be written to the sa...
note that nautilus is end-of-life, and the referenced code in log_usage() no longer exists in the later nautilus poin... Casey Bodley
03:26 PM Bug #53745 (Need More Info): crash on null coroutine under RGWDataSyncShardCR::stop_spawned_services
does this happen on more recent ceph releases? note that nautilus is end of life, so this won't be fixed there Casey Bodley
03:21 PM Bug #53913 (Need More Info): rgw: s3website crashes after upgrade from octopus to pacific
in the bug description, you share 8 lines of the radosgw log leading up to the crash. are you able to capture a log o... Casey Bodley
03:12 PM Bug #53956 (Triaged): pacific radosgw-admin binary reports incorrect stats on quincy cluster
Casey Bodley
03:12 PM Bug #53956 (New): pacific radosgw-admin binary reports incorrect stats on quincy cluster
Casey Bodley
03:09 PM Bug #54054: dencoder: missing lots of rgw and cls_rgw types in src/tools/ceph-dencoder/rgw_types.h
we'll consider this as an intern project ;) Matt Benjamin
03:02 PM Bug #53918 (Need More Info): librados segfault during AMQP notification tests
Casey Bodley

02/02/2022

06:06 PM Bug #54119 (Resolved): bilog trim: segfault in RGWRadosBILogTrimCR::send_request if bucket shard ...
... Mykola Golub
02:56 PM Bug #54116 (Fix Under Review): admin: datalog list always return max-entries
Yuval Lifshitz
02:05 PM Bug #54116 (Resolved): admin: datalog list always return max-entries
when running: radosgw-admin datalog list --max-entries <num>
we will always get "num" amount of entries, regardless ...
Yuval Lifshitz
01:56 PM Bug #54114 (Resolved): PostObj ignores error, may lose data
Casey Bodley
09:42 AM Bug #54112 (Resolved): segmentation fault in UserAsyncRefreshHandler::init_fetch
We observed a segfault in UserAsyncRefreshHandler::init_fetch on 4 separate nodes. The cluster is running Pacific v16... Cory Snyder

02/01/2022

09:52 PM Backport #52010: octopus: radosgw-admin bi list fails with Input/Output error
PR: https://github.com/ceph/ceph/pull/44858 J. Eric Ivancich
03:45 PM rgw-testing Bug #54103 (Fix Under Review): rgw/verify cannot install s3cmd on ubuntu
Casey Bodley
03:30 PM rgw-testing Bug #54103 (Resolved): rgw/verify cannot install s3cmd on ubuntu
the rgw/verify task adds s3cmd via extra_packages to install as a dependency of test_rgw_datacache.py
on ubuntu, t...
Casey Bodley
03:37 PM rgw-testing Bug #54104 (Pending Backport): test_rgw_datacache.py: s3cmd fails with '403 (SignatureDoesNotMatc...
note that the rgw/verify suite is currently pinned to centos, so we don't see these ubuntu failures in master
exam...
Casey Bodley
03:22 PM rgw-testing Bug #54102 (Resolved): rgw/verify suite should not pin centos
valgrind used to crash on ubuntu (https://tracker.ceph.com/issues/18126), so we had pinned it to run only on centos. ... Casey Bodley
10:34 AM Bug #46563: Metadata synchronization failed,"metadata is behind on 1 shards" appear
I now observed a crash of a RADOSGW daemon:... Christian Rohmann
08:44 AM Bug #49666: RGW crash due to PerfCounters::inc assert_condition during multisite syncing
Christian Rohmann wrote:
> The issue appeared again around the time the machine was rebooed
>
> [...]
Most lik...
Christian Rohmann
08:43 AM Bug #49666: RGW crash due to PerfCounters::inc assert_condition during multisite syncing
The issue appeared again around the time the machine was rebooed... Christian Rohmann
08:11 AM Backport #54073 (In Progress): quincy: rgw: fix bucket index list minor calculation bug
Konstantin Shalygin

01/31/2022

10:50 PM Backport #54093 (Resolved): quincy: rgwlc: warn at level 0 if lifecycle processing for a valid bu...
https://github.com/ceph/ceph/pull/45498 Backport Bot
10:50 PM Backport #54092 (Resolved): pacific: rgwlc: warn at level 0 if lifecycle processing for a valid b...
https://github.com/ceph/ceph/pull/45497 Backport Bot
10:49 PM Bug #53728 (Pending Backport): rgwlc: warn at level 0 if lifecycle processing for a valid bucket ...
Casey Bodley
08:31 PM Backport #54085 (Resolved): octopus: librgw: make rgw file handle versioned
https://github.com/ceph/ceph/pull/45496 Backport Bot
08:31 PM Backport #54084 (Resolved): quincy: librgw: make rgw file handle versioned
https://github.com/ceph/ceph/pull/45494 Backport Bot
08:31 PM Backport #54083 (Resolved): pacific: librgw: make rgw file handle versioned
https://github.com/ceph/ceph/pull/45495 Backport Bot
08:26 PM Bug #50194 (Pending Backport): librgw: make rgw file handle versioned
Casey Bodley
06:10 PM Backport #54080 (Rejected): octopus: The decode_json of default placement in zonegroup does not m...
Backport Bot
06:10 PM Backport #54079 (Rejected): pacific: The decode_json of default placement in zonegroup does not m...
Backport Bot
06:10 PM Backport #54078 (Resolved): quincy: The decode_json of default placement in zonegroup does not ma...
https://github.com/ceph/ceph/pull/45493 Backport Bot
06:05 PM Backport #54077 (Resolved): pacific: Memory leak in radosgw-admin bucket chown command
https://github.com/ceph/ceph/pull/45491 Backport Bot
06:05 PM Backport #54076 (Resolved): quincy: Memory leak in radosgw-admin bucket chown command
https://github.com/ceph/ceph/pull/45490 Backport Bot
06:05 PM Bug #54016 (Pending Backport): The decode_json of default placement in zonegroup does not match d...
Casey Bodley
06:05 PM Backport #54075 (Resolved): octopus: Memory leak in radosgw-admin bucket chown command
https://github.com/ceph/ceph/pull/45492 Backport Bot
06:05 PM Backport #54074 (Resolved): pacific: rgw: fix bucket index list minor calculation bug
Backport Bot
06:05 PM Backport #54073 (Resolved): quincy: rgw: fix bucket index list minor calculation bug
https://github.com/ceph/ceph/pull/44852 Backport Bot
06:05 PM Backport #54072 (Resolved): octopus: rgw: fix bucket index list minor calculation bug
Backport Bot
06:04 PM Bug #53856 (Pending Backport): rgw: fix bucket index list minor calculation bug
Casey Bodley
06:02 PM Feature #47856 (Resolved): Add infile arg to role-policy put
Casey Bodley
06:02 PM Bug #53599 (Pending Backport): Memory leak in radosgw-admin bucket chown command
Casey Bodley
06:01 PM Bug #53725 (Resolved): radosgw-admin rm s3 key which exist in userinfo return error "unable to fi...
Casey Bodley
06:01 PM Bug #53727 (Resolved): radosgw-admin user info --access-key return -13(Permission denie) with a n...
Casey Bodley
12:36 PM Bug #53248 (Resolved): rgw: fix bi put not using right bucket index shard
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are ... Loïc Dachary
12:17 PM Backport #53850 (Resolved): octopus: rgw:lc:remove lifecycle entry on bucket deletion
This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/44730
m...
Loïc Dachary
12:17 PM Backport #53290 (Resolved): octopus: rgw: fix bi put not using right bucket index shard
This update was made using the script "backport-resolve-issue".
backport PR https://github.com/ceph/ceph/pull/44167
m...
Loïc Dachary

01/28/2022

08:55 PM Bug #53956: pacific radosgw-admin binary reports incorrect stats on quincy cluster
Sure, let me know if this isn't what you need. ... Tim Wilkinson
03:57 PM Bug #53956 (Need More Info): pacific radosgw-admin binary reports incorrect stats on quincy cluster
thanks Tim. are you able to provide debug logs (--debug-rgw=20 --debug-ms=1) from these radosgw-admin commands? prefe... Casey Bodley
02:50 PM Bug #54054 (Resolved): dencoder: missing lots of rgw and cls_rgw types in src/tools/ceph-dencoder...
the ceph-dencoder and ceph-object-corpus give us regression test coverage of the encodings listed in rgw_types.h, so ... Casey Bodley

01/27/2022

09:50 PM Bug #53941: Trim thread crashes when cluster misconfigured to have empty endpoints
https://github.com/ceph/ceph/pull/44733 merged Yuri Weinstein
03:15 PM Bug #53941 (Pending Backport): Trim thread crashes when cluster misconfigured to have empty endpo...
Casey Bodley
08:59 PM Backport #52009: pacific: radosgw-admin bi list fails with Input/Output error
PR: https://github.com/ceph/ceph/pull/44815 J. Eric Ivancich
08:54 PM Backport #52009: pacific: radosgw-admin bi list fails with Input/Output error
Casey Bodley wrote:
> https://github.com/ceph/ceph/pull/42404 merged prematurely and is causing ceph_test_cls_rgw fa...
J. Eric Ivancich
03:31 PM Bug #53599 (Fix Under Review): Memory leak in radosgw-admin bucket chown command
Casey Bodley
03:28 PM Bug #53664 (Fix Under Review): radosgw-admin subuser create can modify AK-SK pair of other subuser
Casey Bodley
03:25 PM Backport #54041 (Resolved): pacific: remove bucket API returns NoSuchKey than NoSuchBucket
https://github.com/ceph/ceph/pull/45489 Backport Bot
03:25 PM Backport #54040 (Rejected): quincy: remove bucket API returns NoSuchKey than NoSuchBucket
Backport Bot
03:25 PM Backport #54039 (Rejected): octopus: remove bucket API returns NoSuchKey than NoSuchBucket
Backport Bot
03:22 PM Bug #53731 (Pending Backport): remove bucket API returns NoSuchKey than NoSuchBucket
Casey Bodley
03:20 PM Backport #54038 (Rejected): pacific: Trim thread crashes when cluster misconfigured to have empty...
Backport Bot
03:20 PM Backport #54037 (Rejected): octopus: Trim thread crashes when cluster misconfigured to have empty...
Backport Bot
03:18 PM Bug #53878 (Fix Under Review): The storage class attr of the noncurrent object is not correct whe...
Casey Bodley
03:18 PM Bug #53902 (Fix Under Review): Failed to download objects when the append operation uses differen...
Casey Bodley
03:11 PM Backport #54036 (Rejected): octopus: crypt: can't load client cert from /home/ubuntu/cephtest/ca/...
https://github.com/ceph/ceph/pull/45416 Backport Bot
03:10 PM Bug #53945 (Fix Under Review): rgw:copy-object:x-amz-copy-source-if-* headers not work
Casey Bodley
03:10 PM Backport #54035 (Resolved): pacific: crypt: can't load client cert from /home/ubuntu/cephtest/ca/...
https://github.com/ceph/ceph/pull/45413 Backport Bot
03:07 PM Bug #54016 (Fix Under Review): The decode_json of default placement in zonegroup does not match d...
Casey Bodley
03:05 PM Bug #52085 (Pending Backport): crypt: can't load client cert from /home/ubuntu/cephtest/ca/kmip-c...
Casey Bodley
01:02 PM Bug #52900: segfault on FIPS enabled server as result of EVP_md5 disabled in openssl
backport PRs have been created and noted in above tracker issues Mark Kogan
12:56 PM Backport #53008: octopus: segfault on FIPS enabled server as result of EVP_md5 disabled in openssl
backport PR: https://github.com/ceph/ceph/pull/44806 Mark Kogan
12:20 PM Feature #51193 (Resolved): (rfe) add version support to lua scripting
Matan Breizman

01/26/2022

07:24 PM Backport #53850: octopus: rgw:lc:remove lifecycle entry on bucket deletion
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/44730
merged
Yuri Weinstein
04:42 PM Backport #53007: pacific: segfault on FIPS enabled server as result of EVP_md5 disabled in openssl
backport PR: https://github.com/ceph/ceph/pull/44778 Mark Kogan
04:33 PM Backport #53290: octopus: rgw: fix bi put not using right bucket index shard
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/44167
merged
Yuri Weinstein
11:20 AM Bug #53913: rgw: s3website crashes after upgrade from octopus to pacific
I did some tests, tried to curl known fixed issues, like double ending slashes etc, with no rgw crash. Also fresh new... Hubert Niedlich
10:21 AM Backport #53008: octopus: segfault on FIPS enabled server as result of EVP_md5 disabled in openssl
Could we have this backport to octopus? We're facing the same issue with ubuntu 20.04 + fips-updates enabled + secure... Peter Razumovsky
10:10 AM Feature #50611 (Rejected): notification: add conf parameter to allow sending bucket notifications...
Rook now supports setting topics over TLS. This workaround is not needed anymore Yuval Lifshitz
09:13 AM Bug #54016: The decode_json of default placement in zonegroup does not match dump function
https://github.com/ceph/ceph/pull/44785 zhiming zhang
08:59 AM Bug #54016 (Pending Backport): The decode_json of default placement in zonegroup does not match d...
In RGWZoneGroup::dump method
encode_json("default_placement", default_placement, f);
this will call
void e...
zhiming zhang

01/25/2022

07:55 PM Documentation #46779: rgw:doc:IPAddress policy actions are known to work but are not included in ...
Matt, can this be set as "Resolved"? Laura Flores
06:40 PM Bug #53927: rgwlc: version of cls_rgw_lc_get_entry_ret is not updated
Hi Jeegn-Chen,
Ok, on review, let's take a look at rgw_cls_lc_get_entry(...):...
Matt Benjamin
04:27 PM Bug #53927: rgwlc: version of cls_rgw_lc_get_entry_ret is not updated
Hi Jeegn-Chen,
Ok, I'll add that as well in the same PR.
Matt
Matt Benjamin
12:39 PM Bug #54012 (Duplicate): cephadm,ingress: Crash with s3website serving
I am using Ceph v16.2.7 using ceph orch/cephadm using the stock quay.io Docker images.
I am trying to setup radosg...
Manuel Holtgrewe
 

Also available in: Atom