Activity
From 02/07/2018 to 03/08/2018
03/08/2018
- 09:42 PM Backport #23107 (Resolved): luminous: abort early if frontends signal an initialization error
- 09:36 PM Bug #20448 (Pending Backport): rgw: crash with rgw_run_sync_thread=false
- 09:25 PM Bug #20448: rgw: crash with rgw_run_sync_thread=false
- Orit Wasserman wrote:
> https://github.com/ceph/ceph/pull/20769
merged - 06:45 PM Bug #20448: rgw: crash with rgw_run_sync_thread=false
- 07:31 PM Bug #23182: AWS v2 signature does not cover essential parts of admin API
- update: upstream guidance is to introduce an option to require AWS_HMAC_SHA256, which defaults to enabled.
Matt - 07:23 PM Bug #23182 (Triaged): AWS v2 signature does not cover essential parts of admin API
- @robin, thanks for the clarification; both workarounds would appear acceptable; will discuss upstream; it seems cl...
- 07:26 PM Bug #23207 (Fix Under Review): rgw: inefficient buffer usage for PUTs
- 07:23 PM Bug #23203 (Pending Backport): multisite: segfault in radosgw-admin realm pull
- 07:20 PM Bug #23201 (Pending Backport): s3 website: some s3tests are failing because redirects include ind...
- 07:19 PM Bug #23198 (Triaged): osd coredump ClassHandler::ClassMethod::exec
- 07:17 PM Bug #23199 (Triaged): radosgw coredump RGWGC::process
- 07:17 PM Bug #23199: radosgw coredump RGWGC::process
- @yehuda points out that deleting the underlying pool during GC is difficult to support
- 06:54 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- Now that the old bug is solved, try to re-sync missed objects by restarting sync on the specific buckets using:
<pre... - 06:49 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- Just to be clear, note the workaround in comment #3.
- 06:49 PM Bug #22756 (Won't Fix): RGW will not list contents of older buckets at all: reshard makes it show...
- We don't really have any good way to fix it. It's a rados bug, so assigning to josh, but closing it for now.
- 06:44 PM Bug #23261 (Duplicate): Segmentation fault starting multisite radosgw when rgw_run_sync_thread=false
- http://tracker.ceph.com/issues/20448
- 06:34 PM Bug #23170 (Pending Backport): librgw crash when processing request in 12.2.3
- 03:09 PM Documentation #20942 (Pending Backport): Cannot specify multiple ports for civetweb port/listenin...
- 02:18 PM Backport #23274 (In Progress): jewel: abort early if frontends signal an initialization error
- 02:17 PM Backport #23274 (Resolved): jewel: abort early if frontends signal an initialization error
- https://github.com/ceph/ceph/pull/20800
- 01:46 PM Bug #22469 (Resolved): creating a zone with mistake tier-type is not exit
- 01:06 PM Bug #22758 (Fix Under Review): rgw sends garbage meta.compression to ElasticSearch
- https://github.com/ceph/ceph/pull/20796
- 11:51 AM Bug #23257 (Fix Under Review): 'radogw-admin reshard status' command should print text for reshar...
- 11:51 AM Bug #23257: 'radogw-admin reshard status' command should print text for reshard status
- https://github.com/ceph/ceph/pull/20779
- 05:27 AM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- The last log file that you had attached with increased log levels, had two operations:
1. Delete Bucket, which fai...
03/07/2018
- 08:03 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- Abhishek Lekshmanan wrote:
> There was a fix that went in to this in 12.2.3 https://github.com/ceph/ceph/pull/19071,... - 07:38 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- That's really strange, I have a dataset on a rbd device on the same cluster, I just did.
rclone mkdir account:data... - 03:56 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Hi Yoann, I checked all the log files including the latest one, but I dont see any errors there. I also tried to repr...
- 07:12 AM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- here the log file
- 03:54 AM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Hi Yoann, Can you please attach the log files here. Not sure how to get the files that you uploaded using ceph-post-f...
- 01:58 PM Bug #23264 (New): Server side encryption support for s3 COPY operation
- If the source object of a copy operation is encrypted with SSE-C, we should be requiring the x-amz-copy-source-serve...
- 01:05 PM Bug #23261 (Duplicate): Segmentation fault starting multisite radosgw when rgw_run_sync_thread=false
- Configuration...
- 01:02 PM Bug #20448: rgw: crash with rgw_run_sync_thread=false
- https://github.com/ceph/ceph/pull/20769
- 12:38 PM Feature #18855 (Resolved): support "user list" command in radosgw-admin
- 11:17 AM Bug #23257 (Resolved): 'radogw-admin reshard status' command should print text for reshard status
- radosgw-admin reshard status --bucket <bucket>
[
{
"reshard_status": 1,
"new_bucket_instance_... - 08:39 AM Bug #23255 (New): radosgw record wrong data logs when create and delete bucket in seconds repeatedly
- first, enable data log and meta log for multisite.
second, create a bucket and an object, then delete the object and... - 08:00 AM Bug #23201: s3 website: some s3tests are failing because redirects include index doc suffix
- 07:51 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- 02:24 AM Backport #23180 (In Progress): luminous: radosgw-admin data sync run crashes
- https://github.com/ceph/ceph/pull/20762
03/06/2018
- 11:16 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- @Orit & @Greg:
I confirm this fix works for the Benjamin cluster! Now completes in a few seconds. - 06:16 AM Bug #22963 (Fix Under Review): radosgw-admin usage show loops indefinitly - again
- 12:46 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- It kinda looks to me like we should just set "*truncated = false" in the cases where usage_iterate_range() reaches th...
- 12:12 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Okay, dug into the rgw side of things a bit more and I think https://github.com/ceph/ceph/pull/19131 contains some in...
- 08:04 PM Backport #23252 (In Progress): luminous: The return value of auth v2/v4 in RGW is wrong when Expi...
- 08:00 PM Backport #23252 (Resolved): luminous: The return value of auth v2/v4 in RGW is wrong when Expires...
- https://github.com/ceph/ceph/pull/20072
- 07:52 PM Bug #22439 (Pending Backport): The return value of auth v2/v4 in RGW is wrong when Expires/X-Amz-...
- 04:45 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Did you attach any file? Sorry I didnt get it.
- 04:27 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- ceph-post-file: 26c152cf-1fa8-4ef8-9bb8-1621bfcd43bc
- 03:57 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Can you please increase the logging level to 20 and then attach logs.
- 02:51 PM Backport #23245 (Resolved): luminous: multisite: segfault in radosgw-admin realm pull
- https://github.com/ceph/ceph/pull/20722
- 02:41 PM Backport #23244 (Resolved): jewel: multisite: segfault in radosgw-admin realm pull
- https://github.com/ceph/ceph/pull/20749
- 02:38 PM Backport #23243 (Resolved): jewel: possible issue with ssl + libcurl
- https://github.com/ceph/ceph/pull/20749
- 10:36 AM Backport #23239 (Need More Info): luminous: Curl+OpenSSL support in RGW
- 10:31 AM Backport #23239 (Resolved): luminous: Curl+OpenSSL support in RGW
- https://github.com/ceph/ceph/pull/20722
- 10:32 AM Backport #23240 (Resolved): jewel: Curl+OpenSSL support in RGW
- https://github.com/ceph/ceph/pull/20749
- 10:30 AM Bug #23203: multisite: segfault in radosgw-admin realm pull
- Included in https://github.com/ceph/ceph/pull/20390
- 10:28 AM Feature #23162 (Pending Backport): Curl+OpenSSL support in RGW
- 10:12 AM Backport #23179 (In Progress): luminous: rgw: can't download object with range when compression e...
- https://github.com/ceph/ceph/pull/20741
- 08:00 AM Bug #23199: radosgw coredump RGWGC::process
- I have successful got the coredump configure with systemd.
If I get the string external information in the future,wi... - 07:59 AM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- I have successful got the coredump configure with systemd.
If I get the coredump cls and method in the future,will p... - 12:05 AM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- they are in the same env .
but diffrent times. more than 1months.
- 07:45 AM Bug #23232: RGWCopyObj silently corrupts the object that was mulitpart-uploaded in SSE-C
- PR: https://github.com/ceph/ceph/pull/20739
- 02:39 AM Bug #23232: RGWCopyObj silently corrupts the object that was mulitpart-uploaded in SSE-C
- @jeegn chen, that seems potentially plausible, to me.
Matt - 02:31 AM Bug #23232 (Resolved): RGWCopyObj silently corrupts the object that was mulitpart-uploaded in SSE-C
- The issue was found in 12.2.2.
The step to reproduce the issue is that
# Create a bucket (Say A-bucket) and uploa... - 07:26 AM Backport #23237 (Resolved): Corrupted downloads from civetweb when using multipart with slow conn...
- [backport pr https://github.com/ceph/ceph/pull/27982]
We have a problem in civetweb downloads, our clients have sl... - 05:30 AM Backport #23176 (In Progress): luminous: some rgw suites override frontend setting in frontend/be...
- https://github.com/ceph/ceph/pull/20736
- 03:47 AM Feature #22914: admin api --add ability to list objects which is not sync and why on multip site.
- https://github.com/ceph/ceph/pull/20733 is about to list lagging objects,
besides this, we can use sync error list t...
03/05/2018
- 11:57 PM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- the env didn't remove pool。
from communication with my colleague,He just delete objs from s3cmd and run radodgw-admi... - 07:25 PM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- is that the same pool that you removed?
- 11:45 PM Bug #23199: radosgw coredump RGWGC::process
- yes radosgw will coredump every a hours. the period just the gc thread scheduled timestamps.
I delted all omapkeys... - 07:28 PM Bug #23199: radosgw coredump RGWGC::process
- surprising to see this in v10.2.10, n.b., new async gc code is upstreamed to master and planned for backport
Matt - 07:24 PM Bug #23199: radosgw coredump RGWGC::process
- Does it still happen after restarting radosgw?
- 11:07 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Also Sage is asking if missing https://github.com/ceph/ceph/pull/13021 might be causing the repeated loops.
- 10:53 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Looked through the bucket_list.log and do see some inexplicably long op times. Although the watch-notify stuff (3) I ...
- 09:50 PM Backport #23231 (Resolved): luminous: rgw_statfs should report the correct stats
- https://github.com/ceph/ceph/pull/21724
- 09:32 PM Bug #23207: rgw: inefficient buffer usage for PUTs
- Matt suggested that it might be best to apply this fix to master - that way it doesn't matter what behavior a particu...
- 08:47 PM Bug #23207: rgw: inefficient buffer usage for PUTs
- I checked a copy of master. It does not have the problem. I'll need to do more checking; this is a mutant master li...
- 12:15 AM Bug #23207: rgw: inefficient buffer usage for PUTs
- great work, thanks marcus!
Matt - 09:07 PM Bug #23182: AWS v2 signature does not cover essential parts of admin API
- Note: I mentioned this to Yehuda last year, but the decision was that the best we could do would be to force the Admi...
- 08:38 PM Bug #23229 (Resolved): usage trim loops forever: infinite calls to rgw.user_usage_log_trim
- Strong possibility this is related to 22963, but I've got some solid debug logs here. This ALSO triggers issue 22050 ...
- 08:09 PM Bug #22826 (In Progress): "x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD" is not suppo...
- @liu, we have a plan to address this via a general cryptographic token facility that can provide temporary, strong S3...
- 07:32 PM Bug #23216: multi-site: object name should be urlencoded when we put it into ES
- 08:16 AM Bug #23216 (Fix Under Review): multi-site: object name should be urlencoded when we put it into ES
- https://github.com/ceph/ceph/pull/20707
- 07:07 AM Bug #23216 (Resolved): multi-site: object name should be urlencoded when we put it into ES
- 07:28 PM Bug #23220 (In Progress): rgw: return service unavailable when failing to connect to ext. systems
- 01:46 PM Bug #23220 (In Progress): rgw: return service unavailable when failing to connect to ext. systems
- We currently return EINVAL and http error 400 when we fail to connect to keystone for eg. Since this is not an actual...
- 07:22 PM Bug #22312: ERROR: keystone revocation processing returned error r=-22 on keystone v3 openstack o...
- @marcus ping
- 07:21 PM Bug #22331 (Can't reproduce): Luminous 12.2.2 multisite sync failed when objects bigger than 4MB
- 07:17 PM Bug #22202 (Pending Backport): rgw_statfs should report the correct stats
- 07:05 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- There was a fix that went in to this in 12.2.3 https://github.com/ceph/ceph/pull/19071, can you reproduce this in the...
- 06:58 PM Bug #22632: radosgw - s3 keystone integration doesn't work while using civetweb as frontend
- What is the version of openstack keystone that is running?
- 06:54 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- The bug is actually in rados. The problem is that the omap flag was never set on the index objects due to an old rado...
- 06:50 PM Bug #23147: RGW: metrics 'qlen', 'qactive' are not work
- needs to be verified
- 06:41 PM Bug #22951 (Fix Under Review): possible issue with ssl + libcurl
- 05:03 PM Backport #23227 (Resolved): luminous: clang compilation error in BoundedKeyCounter
- https://github.com/ceph/ceph/pull/21083
- 05:03 PM Backport #23225 (Resolved): luminous: rgw: list bilog will loop forever
- https://github.com/ceph/ceph/pull/21054
- 05:02 PM Backport #23223 (Resolved): luminous: rgw: garbage collector removes objects slowly
- https://github.com/ceph/ceph/pull/28784
- 04:48 PM Bug #23197: Bucket Policy - Input Action is compared against all actions, even if a match has bee...
- 04:02 PM Backport #23221 (Resolved): luminous: possible issue with ssl + libcurl
- https://github.com/ceph/ceph/pull/20722
- 01:54 PM Bug #22139 (Pending Backport): clang compilation error in BoundedKeyCounter
03/04/2018
- 10:59 PM Bug #23207: rgw: inefficient buffer usage for PUTs
- I updated my PR. I moved the fill loop down from just the put-object code path to just above mg_read(). I don't bel...
- 06:14 PM Bug #23207 (In Progress): rgw: inefficient buffer usage for PUTs
- https://github.com/ceph/ceph/pull/20698
- 06:29 AM Bug #23207 (Resolved): rgw: inefficient buffer usage for PUTs
- At least in jewel using swift, radosgw is very inefficient about its buffering process. In RGWPutObj_ObjStore::get_d...
03/03/2018
03/02/2018
- 09:46 PM Bug #23203: multisite: segfault in radosgw-admin realm pull
- -https://github.com/ceph/ceph/pull/20694-
- 09:38 PM Bug #23203 (Resolved): multisite: segfault in radosgw-admin realm pull
- The http client in radosgw-admin is trying to use RGWCurlHandles without doing the global init....
- 08:31 PM Bug #23201 (Fix Under Review): s3 website: some s3tests are failing because redirects include ind...
- https://github.com/ceph/ceph/pull/20693
- 07:38 PM Bug #23201 (Resolved): s3 website: some s3tests are failing because redirects include index doc s...
- ...
- 07:54 PM Bug #23202 (Resolved): radosgw-admin user create failing with return code 5
- Tried with master and luminous, Cluster is setup with ceph-ansible.
ubuntu@ovh059:~$ sudo ceph --version
ceph v... - 07:17 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- CentOS doesn't allow the overwrite of resolve.conf, I guess some network manager changes, I need to check with David ...
- 05:17 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- Not sure why this started failing..
http://pulpito.ceph.com/vasu-2018-03-02_02:00:23-rgw-luminous-distro-basic-ovh... - 05:27 PM Bug #23196: rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression enabled
- 05:25 PM Bug #23196: rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression enabled
- https://github.com/ceph/ceph/pull/20686
- 04:07 AM Bug #23196 (Resolved): rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression en...
- Given an object uploaded by 'copy part' without 'x-amz-copy-source-range' when compression enabled.
That object has ... - 08:51 AM Bug #23199: radosgw coredump RGWGC::process
- backtrace please refer to 1111.png
- 08:51 AM Bug #23199 (Resolved): radosgw coredump RGWGC::process
- radosgw coredump RGWGC::process
ceph version :10.2.10
============
The operation may be easy to saw again:
i... - 08:46 AM Bug #23198 (Triaged): osd coredump ClassHandler::ClassMethod::exec
- ceph version 10.2.10 (5dc1e4c05cb68dbf62ae6fce3f0700e4654fdbbe)
1: (()+0x92b18a) [0x7fe6696f018a]
2: (()+0xf370) ... - 05:11 AM Bug #23197: Bucket Policy - Input Action is compared against all actions, even if a match has bee...
- Fixed by: https://github.com/ceph/ceph/pull/20629
- 05:11 AM Bug #23197 (Rejected): Bucket Policy - Input Action is compared against all actions, even if a ma...
- While parsing an input Policy, the Action is compared against all Actions, and even though a match is found, the meth...
- 04:51 AM Backport #23106 (Resolved): luminous: RGW will not list contents of Jewel-era buckets: reshard do...
- 04:29 AM Bug #21942 (Pending Backport): rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/valu...
- 04:28 AM Backport #23192 (In Progress): rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/valu...
- 04:02 AM Backport #23179: luminous: rgw: can't download object with range when compression enabled
- per @Casey
please backport both the original pr and a followup:
https://github.com/ceph/ceph/pull/20226
https:... - 03:19 AM Backport #23180: luminous: radosgw-admin data sync run crashes
- per @Casey
both the original pr and a followup need backport:
https://github.com/ceph/ceph/pull/18898
https:/...
03/01/2018
- 09:27 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- 08:51 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- Testing with: https://github.com/ceph/ceph/pull/20678
- 07:03 PM Bug #22986 (In Progress): hadoop-s3a jobs failing with EPERM
- 07:18 PM Bug #22714: Existing AccessKeys with leading underscore broke in upgrade
- @robbat2 there was some encoding change that we did between jewel and luminous and apparently there was a case we mis...
- 07:08 PM Bug #23149 (Closed): Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied wi...
- 05:33 AM Bug #23149: Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied with reason...
- Was fixed by adding header for nginx-proxy. Nginx rejects '100' headers....
- 07:08 PM Bug #22721 (In Progress): Resharding hangs with versioning-enabled buckets
- 07:06 PM Bug #22737 (Pending Backport): rgw: list bilog will loop forever
- 07:00 PM Bug #23089: rgw: change order of authentication back to local, remote
- 06:59 PM Bug #23193 (New): multsite use of metadata list pagination
- related to issue http://tracker.ceph.com/issues/23099, the rest admin api expects the marker to be url-encoded. multi...
- 06:56 PM Bug #23139 (Pending Backport): rgw: garbage collector removes objects slowly
- 06:52 PM Bug #23139: rgw: garbage collector removes objects slowly
- 06:50 PM Bug #22928 (Resolved): RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- 06:49 PM Bug #23039 (Pending Backport): abort early if frontends signal an initialization error
- 04:07 PM Backport #23192: rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/value pairs
- https://github.com/ceph/ceph/pull/20672
- 04:07 PM Backport #23192 (Resolved): rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/value p...
- Recent changes to rgw_log_op break rgw_lib due to trivially unset keys in RGWEnv. Producer and consumer should be fi...
- 04:06 PM Bug #21942: rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/value pairs
- Report from downstream is we now need backport to L
- 03:14 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- I found plenty of "execute can't find bucket IAM POLICY attr bucket_name = spinn3r.json" in radosgw logs
2018-0... - 03:09 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- some additional logs fronm radosgw
- 01:13 PM Bug #23188 (Closed): Cannot list file in a bucket or delete it : illegal character code U+0001
- Hello,
I created for the first time a new bucket since my upgrade to luminous then upload files using rclone sync.... - 06:45 AM Bug #23147: RGW: metrics 'qlen', 'qactive' are not work
- I guess https://github.com/ceph/ceph/pull/20006 could resolve this problem.
02/28/2018
- 05:50 PM Bug #23183 (New): 'Bad Request' after success when uploading InputStream via swift API
- Hi,
I think I have found an issue in Ceph when connecting via the Swift API and uploading an object with an InputS... - 03:22 PM Bug #23182 (Triaged): AWS v2 signature does not cover essential parts of admin API
- The AWS v2 signature only covers the URL without the query string, but the Admin API passes essential parameters in t...
- 01:28 PM Feature #23162: Curl+OpenSSL support in RGW
- Thanks Marcus for detail explanation.
- 08:13 AM Feature #23162: Curl+OpenSSL support in RGW
- I've made a PR for master with the same changes for CURL* and openssl/ubuntu. https://github.com/ceph/ceph/pull/20635
- 12:21 AM Feature #23162: Curl+OpenSSL support in RGW
- I've made a PR against jewel: https://github.com/ceph/ceph/pull/20623
I will have another PR against master later to... - 01:20 PM Bug #23170: librgw crash when processing request in 12.2.3
- This bug has been fixed by this PR:
https://github.com/ceph/ceph/commit/f9d5528a1990b5bcc33948284136c2d349aa838e - 07:23 AM Bug #23170: librgw crash when processing request in 12.2.3
- fixed with this PR: https://github.com/ceph/ceph/pull/20633
- 06:58 AM Bug #23170 (Duplicate): librgw crash when processing request in 12.2.3
- I'm using librgw with new Ceph release 12.2.3, librgw will always get unexpected crush when it is processing a reques...
- 11:21 AM Backport #23180 (Resolved): luminous: radosgw-admin data sync run crashes
- https://github.com/ceph/ceph/pull/20762
- 11:20 AM Backport #23179 (Resolved): luminous: rgw: can't download object with range when compression enabled
- https://github.com/ceph/ceph/pull/20741
- 11:20 AM Backport #23176 (Resolved): luminous: some rgw suites override frontend setting in frontend/beast...
- https://github.com/ceph/ceph/pull/20736
- 10:01 AM Backport #23171 (Resolved): core dump : recursive lock of RGWKeystoneTokenCache
- When enabling Keystone auth in ceph.conf as below and performing vstart.sh the following core
occurs during rgw sta... - 06:32 AM Bug #23149: Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied with reason...
- Actually this is nginx-proxy issue. Will try to figure out what is problem...
Sorry for the noise.
02/27/2018
- 10:08 PM Feature #23162: Curl+OpenSSL support in RGW
- Well, this needs a description, so here goes.
When rgw makes outbound http requests, it uses libcurl. For keyston... - 01:40 PM Feature #23162 (Resolved): Curl+OpenSSL support in RGW
- Curl+OpenSSL support in RGW
RHT internal bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1522881
https://bugzi... - 06:32 PM Bug #22852 (Pending Backport): rgw: can't download object with range when compression enabled
- 05:29 PM Bug #22852: rgw: can't download object with range when compression enabled
- please backport both the original pr and a followup:
https://github.com/ceph/ceph/pull/20226
https://github.com/c... - 05:52 PM Bug #23002 (Pending Backport): some rgw suites override frontend setting in frontend/beast.yaml
- 05:47 PM Bug #23002: some rgw suites override frontend setting in frontend/beast.yaml
- Casey Bodley wrote:
> https://github.com/ceph/ceph/pull/20440
merged - 05:48 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- Yehuda Sadeh wrote:
> https://github.com/ceph/ceph/pull/20352
merged - 02:25 PM Bug #22080 (Pending Backport): radosgw-admin data sync run crashes
- both the original pr and a followup need backport:
https://github.com/ceph/ceph/pull/18898
https://github.com/cep... - 12:54 PM Bug #23149 (Closed): Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied wi...
- Example code from "Ceph docs":http://docs.ceph.com/docs/master/radosgw/s3/ruby/#ruby-aws-sdk-examples-aws-sdk-gem-2
... - 10:46 AM Bug #23147 (Resolved): RGW: metrics 'qlen', 'qactive' are not work
- Ceph version 12.2.3
I perform a load test rgw, but matrics 'qlen', 'qactive' always equal 0.... - 09:03 AM Bug #23146 (Resolved): rgw: download object might fail for local invariable uninitialized
- https://github.com/ceph/ceph/pull/20612
02/26/2018
- 07:39 PM Bug #23139: rgw: garbage collector removes objects slowly
- https://github.com/ceph/ceph/pull/20546
- 07:39 PM Bug #23139 (Resolved): rgw: garbage collector removes objects slowly
- One rados object at a time. This can take very long when removing a large number of objects.
- 05:26 AM Backport #23102 (In Progress): luminous: Objects only serving first 512K
- https://github.com/ceph/ceph/pull/20586
02/24/2018
- 03:23 PM Bug #23115: fcgi's connection is not closed properly when fastcgi_keep_conn is enabled
- the last FIN is sent until `FCGX_Accept_r` is called on the same FCGX_Request in `QueueRing<FCGX_Request*> qr(max_con...
- 03:13 PM Bug #23115: fcgi's connection is not closed properly when fastcgi_keep_conn is enabled
- https://github.com/ceph/ceph/pull/20579
- 03:06 PM Bug #23115 (Fix Under Review): fcgi's connection is not closed properly when fastcgi_keep_conn is...
- nginx's config:...
02/23/2018
- 09:21 PM Feature #22604 (Resolved): Command to inspect the RGW cache
- 09:18 PM Feature #22603 (Resolved): Command to invalidate cache
- 09:18 PM Backport #22954 (Resolved): luminous: rgw: Command to invalidate cache
- 08:52 PM Backport #22954: luminous: rgw: Command to invalidate cache
- Adam Emerson wrote:
> The original request was to use the radosgw-admin tool to invalidate cache, but this doesn't f... - 09:17 PM Backport #22955 (Resolved): luminous: rgw: Command to inspect the RGW cache
- 08:52 PM Backport #22955: luminous: rgw: Command to inspect the RGW cache
- Adam Emerson wrote:
> Easiest to do first, I'll just walk the cache, print the name of each object and what hangs of... - 07:15 PM Bug #23100: admin/metadata/bucket: weird bucket name in listing, cannot fetch
- I have also reproduced this on a new Jewel-era cluster (since Congress is pre-Argonaut and sometimes causes confusion).
- 06:38 AM Bug #23100 (New): admin/metadata/bucket: weird bucket name in listing, cannot fetch
- listing of bucket names via the admin/metadata/bucket API shows this name for a bucket +@2016-02-26 03:08:16.935351@+...
- 04:22 PM Feature #18227 (Resolved): multisite: trim data changes log as peers zones catch up
- 04:22 PM Backport #18277 (Resolved): jewel: multisite: trim data changes log as peers zones catch up
- 02:58 PM Backport #23107 (Resolved): luminous: abort early if frontends signal an initialization error
- https://github.com/ceph/ceph/pull/20564
- 02:51 PM Backport #23106 (Resolved): luminous: RGW will not list contents of Jewel-era buckets: reshard do...
- https://github.com/ceph/ceph/pull/20565
- 01:50 PM Backport #23102 (Resolved): luminous: Objects only serving first 512K
- This backport got merged twice:
https://github.com/ceph/ceph/pull/20566
https://github.com/ceph/ceph/pull/20586 - 11:53 AM Bug #23091: rgw + OpenLDAP = Failed the auth strategy, reason=-13
- @Matt, thanks. I read ldap_set_option(3), and figure out: lib have server_control and client_control. So my reject is...
- 06:06 AM Bug #23099 (Resolved): REST admin metadata API paging failure bucket & bucket.instance: InvalidAr...
- Trying to use the /admin/metadata/ endpoints to review bucket & bucket.instance metadata for another bug, and found t...
- 01:42 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- Casey Bodley wrote:
> a couple of prs that are potentially related? https://github.com/ceph/ceph/pull/20396 https://... - 12:05 AM Bug #23095 (New): Poor defaults for RGW
- Objecter throttling is apparent from inspecting the admin socket on the RGW host.
Default objecter_inflight_ops sh...
02/22/2018
- 07:08 PM Bug #22982 (Pending Backport): Objects only serving first 512K
- 07:05 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- 06:46 PM Bug #23089: rgw: change order of authentication back to local, remote
- 01:35 PM Bug #23089 (Resolved): rgw: change order of authentication back to local, remote
- from luminous we changed the order of rgw authentication to first try external before attempting locally, this can ha...
- 06:43 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- there's not much that radosgw can do here - workaround is necesary
- 04:33 PM Bug #23092 (New): RGW leaking/orphan data with Jewel
- This was previously reported under #18331 but this popped up again on a Jewel cluster.
Some information:
- Jewe... - 02:39 PM Bug #23091: rgw + OpenLDAP = Failed the auth strategy, reason=-13
- @konstantin, thanks, I'm working on LDAP auth now, I'll take a look.
Matt - 02:24 PM Bug #23091 (Duplicate): rgw + OpenLDAP = Failed the auth strategy, reason=-13
- Hello, recently I posted this question to users ML, but don't receive any feedback.
More at ML: http://lists.ceph.... - 01:40 PM Bug #23090 (New): rgw: both civetweb & libcurl (and boost::asio) may initialize openssl
- currently if rgw is https terminated with civetweb; and if libcurl is linked against openssl both the these will end ...
- 01:29 PM Bug #23088 (Closed): nfs-ganesha repo for Xenial is not signed
- hrw@gossamer:$ docker run -it -u root ubuntu:xenial bash
root@cf9a237f71dc:/# cd /etc/apt/sources.list.d/
root@cf9a... - 09:53 AM Bug #23039: abort early if frontends signal an initialization error
- Suse security assigned CVE-2018-7262 to this
- 04:47 AM Documentation #23081: docs: radosgw: ldap-auth: wrong option name 'rgw_ldap_searchfilter'
- https://github.com/ceph/ceph/pull/20526
- 04:45 AM Documentation #23081 (Resolved): docs: radosgw: ldap-auth: wrong option name 'rgw_ldap_searchfilter'
- ...
- 03:55 AM Bug #22986: hadoop-s3a jobs failing with EPERM
- Running on 7.3 to see why copying to /etc/resolv.conf is failing for 7.4
http://pulpito.ceph.com/vasu-2018-02-22_0...
02/21/2018
02/20/2018
- 04:41 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Hi,
just FYI - I've uploaded some logs in this ticket as well - http://tracker.ceph.com/issues/22314
If you need ... - 09:06 AM Backport #23023 (In Progress): jewel: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20496
- 06:16 AM Backport #23023: jewel: can not set user quota with specific value
- I'm on it.
02/19/2018
- 10:25 PM Bug #23039 (In Progress): abort early if frontends signal an initialization error
- https://github.com/ceph/ceph/pull/20488
- 03:45 PM Bug #23039 (Resolved): abort early if frontends signal an initialization error
- Since some frontends like civetweb can signal an error if non compliant HTTP Headers are passed on request, abort ear...
- 10:42 AM Backport #23021 (In Progress): jewel: The parameter of max-uploads doesn't work when List Multipa...
- https://github.com/ceph/ceph/pull/20479
- 10:17 AM Backport #21637 (Resolved): luminous: encryption: PutObj response does not include sse-kms headers
- 04:34 AM Backport #23022 (In Progress): luminous: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20477
- 04:00 AM Backport #23020 (In Progress): luminous: The parameter of max-uploads doesn't work when List Mult...
- https://github.com/ceph/ceph/pull/20476
02/18/2018
- 11:33 AM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- one mistake - this is a result of : radosgw-admin data sync status --source-zone=zone1 instead of metadata.
- 11:29 AM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- Orit Wasserman wrote:
> can you provide the out put of sync error list commands?
> EBUSY errors are expected to hap... - 06:16 AM Bug #22897: rgw: (jewel) can't delete swift acls with swift command.
- I've made a PR for master with this change,
https://github.com/ceph/ceph/pull/20471
I believe this should apply tri...
02/17/2018
- 05:28 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- The comment by Marcin Gibula in issue 21196 is actually spot on.
This DOES actually complete, eventually, if nothin...
02/16/2018
- 10:24 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Debugging so far:
1. The bug only occurs on the Congress cluster, not the Benjamin cluster (was rebuilt from scratch... - 07:53 PM Feature #22832 (Fix Under Review): SSL support for beast frontend
- https://github.com/ceph/ceph/pull/20464
- 02:24 PM Documentation #23027 (Resolved): document rgw_multipart_min_part_size
- Documentation for rgw_multipart_min_part_size is missing form doc/radosgw/config-ref.rst
- 11:04 AM Backport #23026 (Resolved): jewel: rgw: data sync of versioned objects, note updating bi marker
- https://github.com/ceph/ceph/pull/21213
- 11:04 AM Backport #23025 (Resolved): luminous: rgw: data sync of versioned objects, note updating bi marker
- https://github.com/ceph/ceph/pull/21214
- 11:02 AM Backport #23023 (Resolved): jewel: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20496
- 11:02 AM Backport #23022 (Resolved): luminous: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20477
- 11:02 AM Backport #23021 (Resolved): jewel: The parameter of max-uploads doesn't work when List Multipart ...
- https://github.com/ceph/ceph/pull/20479
- 11:02 AM Backport #23020 (Resolved): luminous: The parameter of max-uploads doesn't work when List Multipa...
- https://github.com/ceph/ceph/pull/20476
02/15/2018
- 09:50 PM Bug #22833: rgw: crash in RGWSyncTraceManager::finish_node()
- Luminous backport to be done in #23018
- 06:35 PM Bug #22833 (Resolved): rgw: crash in RGWSyncTraceManager::finish_node()
- 07:28 PM Bug #22825 (Pending Backport): The parameter of max-uploads doesn't work when List Multipart Uploads
- 07:26 PM Bug #22745 (Pending Backport): can not set user quota with specific value
- 04:59 PM Bug #22745: can not set user quota with specific value
- Casey Bodley wrote:
> https://github.com/ceph/ceph/pull/20063
merged - 07:24 PM Bug #22352 (Duplicate): rados gateway computes wrong AWS4 signature if canonical request contains...
- 07:22 PM Bug #22218 (Duplicate): multisite: rgw crashed during meta sync
- fixed in http://tracker.ceph.com/issues/21097
- 07:14 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- This bug definitely still exists, is affecting billing.
Here's the interesting part from --debug-rgw=20 --debug-ms=1... - 07:12 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- can you provide the out put of sync error list commands?
EBUSY errors are expected to happen, look for other errors. - 07:02 PM Bug #22895: radosgw not work after all rgw `RGWAsyncRadosProcessor had timed out
- the hangs in RGWAsyncRadosProcessor could be due to requests sent to another gateway that's also paused for realm rec...
- 06:58 PM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- a couple of prs that are potentially related? https://github.com/ceph/ceph/pull/20396 https://github.com/ceph/ceph/pu...
- 06:54 PM Bug #22827 (Resolved): rgw_file: FLAG_EXACT_MATCH actually should be enforced
- 06:52 PM Bug #18885 (Pending Backport): rgw: data sync of versioned objects, note updating bi marker
- 06:50 PM Bug #22721: Resharding hangs with versioning-enabled buckets
- 06:44 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- 06:43 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- 06:43 PM Bug #22982: Objects only serving first 512K
- 06:37 PM Backport #23018 (Rejected): rgw multisite sync tracing feature
- https://github.com/ceph/ceph/pull/16492
and fixes in https://github.com/ceph/ceph/pull/20191 - 12:08 PM Bug #20166: RGW: Bad error handling when tail object is missing
- Seems that this is not resolved yet indeed. I'm also failing to see a good way how to handle this error situation. Wo...
02/14/2018
- 07:17 PM Bug #20166: RGW: Bad error handling when tail object is missing
- #20107 is marked as resolved.
Is this issue #20166 also resolved?
I think I am seeing this behavior in Luminous... - 05:23 PM Bug #23002 (Fix Under Review): some rgw suites override frontend setting in frontend/beast.yaml
- 05:23 PM Bug #23002: some rgw suites override frontend setting in frontend/beast.yaml
- https://github.com/ceph/ceph/pull/20440
- 05:21 PM Bug #23002 (Resolved): some rgw suites override frontend setting in frontend/beast.yaml
- for example, a job in the verify suite that includes frontend/beast.yaml:
description: rgw/verify/{clusters/fixe... - 06:44 AM Bug #22982: Objects only serving first 512K
- This change has been deployed at Dreamhost, seems to work so far, watching for other breakage
02/13/2018
- 06:22 PM Bug #22982: Objects only serving first 512K
- a possible fix:
https://github.com/ceph/ceph/pull/20425 - 04:55 PM Bug #22982: Objects only serving first 512K
- Manifest has appropriate raw rados objects there, so we do an unneeded conversion when reading it.
- 04:25 AM Bug #22982: Objects only serving first 512K
- Private attachment made w/ manifest & hexdump of manifest.
- 04:57 PM Bug #22833: rgw: crash in RGWSyncTraceManager::finish_node()
- 03:35 PM Backport #22987 (In Progress): jewel: rgw: user stats increased after bucket reshard
- 10:21 AM Backport #22987 (Resolved): jewel: rgw: user stats increased after bucket reshard
- https://github.com/ceph/ceph/pull/20421
- 10:20 AM Bug #22124 (Pending Backport): rgw: user stats increased after bucket reshard
- Adding jewel backport
02/12/2018
- 09:52 PM Bug #22982: Objects only serving first 512K
- Can you provide the raw manifest somehow? I want to know if the issue is the way we stored the original objects (that...
- 09:08 PM Bug #22982: Objects only serving first 512K
- Ok, Yehuda's underscore could still be correct. This is what the RGW is looking for:
* @default.51695731.142363__sha... - 09:01 PM Bug #22982: Objects only serving first 512K
- It doesn't start with an underscore in this case, but the data is from at least Dumpling, maybe older (Dumpling->Fire...
- 05:06 PM Bug #22982: Objects only serving first 512K
- it's an old object with explicit placement, so maybe pre-firefly (?). The tail objects are starting with underscore, ...
- 04:20 PM Bug #22982: Objects only serving first 512K
- @robin, can you point out the distinguishing characteristics of this? how is our CI testing missing this?
Matt - 07:48 AM Bug #22982 (Resolved): Objects only serving first 512K
- This has be seen in previous releases, but it's back again in Luminous! Too late for 12.2.3 tag, but I'll apply the f...
- 08:49 PM Bug #22986 (Resolved): hadoop-s3a jobs failing with EPERM
- the task modifies /etc/resolv.conf to set up dnsmasq, but something changed and the write to /etc/resolv.conf is now ...
- 07:16 PM Bug #22927 (Fix Under Review): rgw: Location element not returned correctly from Post Upload Object
- 04:28 PM Bug #22985: Buckets missing from rgw_read_user_buckets output
- This output contains the buckets that are fine, but not the missing ones.
The omap header is non-empty.... - 04:25 PM Bug #22985: Buckets missing from rgw_read_user_buckets output
- In the case of account @jerhec@, at least two buckets are known to be missing; let me know what other debug/data expo...
- 04:20 PM Bug #22985 (Can't reproduce): Buckets missing from rgw_read_user_buckets output
- Some buckets that are known to exist have vanished from the listing of buckets.
Dreamhost tracking DHO-1722.
He... - 02:32 PM Bug #22951 (In Progress): possible issue with ssl + libcurl
- 02:32 PM Bug #22951: possible issue with ssl + libcurl
- wip master pr: https://github.com/ceph/ceph/pull/20390
- 07:41 AM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- Workaround fix for this:...
02/09/2018
- 06:52 AM Bug #22963 (Resolved): radosgw-admin usage show loops indefinitly - again
- After updating our cluster from jewel to luminous, @radosgw-admin usage show@ loops. I thought #21196 should have clo...
- 12:57 AM Bug #22951: possible issue with ssl + libcurl
- I was able to reproduce this with libopenssl 1.0.2j-16.1 & curl 7.37 which are shipped with opensuse 42.3. There were...
02/08/2018
- 09:44 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
Another test. dump of bilog:
master merge op(write,delete),but slave miss some op.- 08:24 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- [root@sx-3f3r-ceph-s3-c1-03 my-cluster]# radosgw-admin bucket check --fix --check-objects --bucket=testmix
[]
{
... - 08:23 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- [root@sx-3f3r-ceph-s3-c1-03 my-cluster]# radosgw-admin bucket stats --bucket=testmix
{
... - 08:10 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
#!/bin/sh
usercfg="Ymliu.cfg"
bucket="s3://testmix"
file="s-mix011.txt"
for i in {1..100};
do
{
s3cm...- 03:09 AM Bug #22556: failed to take a lock on datalog.sync-status.f608ce2b-5584-45af-b0c5-f4896995bd22
- It has been confirmed that causes by handles
rgw_num_rados_handles = 100 - 02:31 AM Backport #22938 (In Progress): luminous: system user can't delete bucket completely
- https://github.com/ceph/ceph/pull/20357
- 01:38 AM Bug #22895: radosgw not work after all rgw `RGWAsyncRadosProcessor had timed out
- The timeout may be due to inconsistencies between the two zones, Because that did not reappear again after I deleted ...
02/07/2018
- 09:29 PM Backport #22955 (In Progress): luminous: rgw: Command to inspect the RGW cache
- 09:29 PM Backport #22955 (Resolved): luminous: rgw: Command to inspect the RGW cache
- Easiest to do first, I'll just walk the cache, print the name of each object and what hangs off it. Maybe dump the co...
- 09:27 PM Backport #22954 (In Progress): luminous: rgw: Command to invalidate cache
- 09:27 PM Backport #22954 (Resolved): luminous: rgw: Command to invalidate cache
- The original request was to use the radosgw-admin tool to invalidate cache, but this doesn't fit the RGW architecture...
- 09:21 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- https://github.com/ceph/ceph/pull/20352
- 06:41 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- The problem is that we fail to go to the explicit placement pool for index operations even if one is set. One workaro...
- 09:07 PM Feature #22953 (Rejected): [RFE] Better time slew handling in rgw/keystone integration
- [RFE] Better time slew handling in rgw/keystone integration
RHBZ - https://bugzilla.redhat.com/show_bug.cgi?id=15410... - 07:12 PM Feature #22603 (Pending Backport): Command to invalidate cache
- 07:11 PM Feature #22604 (Pending Backport): Command to inspect the RGW cache
- 05:51 PM Bug #22951 (Resolved): possible issue with ssl + libcurl
- Getting random segmentation faults when radosgw is trying to authenticate with keystone over ssl, while this issue is...
- 02:54 PM Bug #22852: rgw: can't download object with range when compression enabled
- 02:54 PM Bug #22852: rgw: can't download object with range when compression enabled
- s3test case https://github.com/ceph/s3-tests/pull/209
- 01:21 PM Bug #22946: rgw: s3 copy object return 'x-amz-copy-source-version-id' and 'x-amz-version-id
- https://github.com/ceph/ceph/pull/20348
- 01:21 PM Bug #22946 (New): rgw: s3 copy object return 'x-amz-copy-source-version-id' and 'x-amz-version-id
- s3 copy object should return 'x-amz-copy-source-version-id' and 'x-amz-version-id' as AWS S3
- 07:34 AM Bug #22742 (Resolved): rgw: resharding doesn't seem to preserve bucket acls
- 07:34 AM Backport #22922 (Resolved): luminous: rgw: resharding doesn't seem to preserve bucket acls
- 07:32 AM Bug #22124 (Resolved): rgw: user stats increased after bucket reshard
- 07:30 AM Bug #22124 (Pending Backport): rgw: user stats increased after bucket reshard
- 07:31 AM Backport #22434 (Resolved): luminous: rgw: user stats increased after bucket reshard
- Wow, that was fast!
- 05:02 AM Backport #22939 (Resolved): jewel: system user can't delete bucket completely
- https://github.com/ceph/ceph/pull/21212
- 05:02 AM Backport #22938 (Resolved): luminous: system user can't delete bucket completely
- https://github.com/ceph/ceph/pull/20357
- 05:01 AM Backport #22937 (Resolved): luminous: beast: listen on multiple endpoints
- https://github.com/ceph/ceph/pull/21568
- 04:41 AM Backport #22889 (In Progress): luminous: rgw_file: avoid fragging thread_local log buffer
- https://github.com/ceph/ceph/pull/20345
Also available in: Atom