Activity
From 02/13/2018 to 03/14/2018
03/14/2018
- 02:57 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- I just uploaded files of the bucket on a new fresh ceph cluster and everything is fine, I can list bucket files witho...
- 01:55 PM Bug #23335 (In Progress): radosgw-admin: add an option to reset user stats
- 08:56 AM Backport #23357 (In Progress): luminous: Admin API support for bucket quota change
- 08:52 AM Backport #23357 (Resolved): luminous: Admin API support for bucket quota change
- https://github.com/ceph/ceph/pull/20885
- 08:50 AM Feature #21811 (Pending Backport): Admin API support for bucket quota change
- 04:16 AM Backport #23310 (In Progress): luminous: s3 website: some s3tests are failing because redirects i...
- https://github.com/ceph/ceph/pull/20889
03/13/2018
- 09:57 PM Backport #23348 (Resolved): jewel: rgw: inefficient buffer usage for PUTs
- https://github.com/ceph/ceph/pull/21098
- 09:57 PM Backport #23347 (Resolved): luminous: rgw: inefficient buffer usage for PUTs
- https://github.com/ceph/ceph/pull/20953
- 09:57 PM Backport #23346 (Resolved): luminous: RGWCopyObj silently corrupts the object that was mulitpart-...
- https://github.com/ceph/ceph/pull/20937
- 06:09 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- my own further fix here:
https://github.com/dreamhost/ceph/commit/6c0cb1c1491df1f69309ff57a5b0f9d7929e1990
Will be ... - 05:10 PM Bug #23335: radosgw-admin: add an option to reset user stats
- master pr: https://github.com/ceph/ceph/pull/20853
- 02:49 PM Bug #23335 (Resolved): radosgw-admin: add an option to reset user stats
- In cases where radosgw-admin user stats have been clobbered due to an improper reshard etc, add an optiion to recalcu...
- 05:07 PM Backport #23338 (Resolved): jewel: radosgw-admin: add an option to reset user stats
- https://github.com/ceph/ceph/pull/20877
- 04:02 PM Bug #23207 (Pending Backport): rgw: inefficient buffer usage for PUTs
- 03:00 PM Bug #23232 (Pending Backport): RGWCopyObj silently corrupts the object that was mulitpart-uploade...
- 02:54 PM Bug #23232: RGWCopyObj silently corrupts the object that was mulitpart-uploaded in SSE-C
- Jeegn Chen wrote:
> PR: https://github.com/ceph/ceph/pull/20739
merged - 06:19 AM Backport #23302 (In Progress): luminous: rgw: add radosgw-admin sync error trim to trim sync erro...
- https://github.com/ceph/ceph/pull/20859
03/12/2018
- 01:36 PM Bug #23322: radosgw-admin user stats --sync-stats without a user will create an empty object
- master pr: https://github.com/ceph/ceph/pull/20846
- 01:13 PM Bug #23322 (Resolved): radosgw-admin user stats --sync-stats without a user will create an empty ...
- Calling radosgw-admin user stats --sync-stats without any arguments will result in us calling sync-stats with an empt...
- 10:09 AM Bug #23147 (In Progress): RGW: metrics 'qlen', 'qactive' are not work
- Created a PR: https://github.com/ceph/ceph/pull/20842
- 09:42 AM Bug #23301 (Fix Under Review): rgw: 403 error when creating an object with metadata containing se...
- https://github.com/ceph/ceph/pull/20841
- 08:38 AM Bug #23301 (Resolved): rgw: 403 error when creating an object with metadata containing sequence o...
- s3_client.put_object(Bucket=args.bucket,
Key='testdir/test4.txt',
Body='som... - 09:18 AM Backport #23319 (Duplicate): luminous: librgw crash when processing request in 12.2.3
- https://github.com/ceph/ceph/pull/20934
- 09:17 AM Backport #23318 (Resolved): luminous: rgw: crash with rgw_run_sync_thread=false
- https://github.com/ceph/ceph/pull/20932
- 09:17 AM Backport #23317 (Resolved): luminous: Cannot specify multiple ports for civetweb port/listening_p...
- https://github.com/ceph/ceph/pull/20906
- 09:14 AM Backport #23311 (Resolved): jewel: s3 website: some s3tests are failing because redirects include...
- https://github.com/ceph/ceph/pull/21100
- 09:14 AM Backport #23310 (Resolved): luminous: s3 website: some s3tests are failing because redirects incl...
- https://github.com/ceph/ceph/pull/20889
- 09:14 AM Backport #23303 (Resolved): jewel: rgw: add radosgw-admin sync error trim to trim sync error log
- https://github.com/ceph/ceph/pull/21210
- 09:14 AM Backport #23302 (Resolved): luminous: rgw: add radosgw-admin sync error trim to trim sync error log
- https://github.com/ceph/ceph/pull/20859
03/11/2018
- 09:40 AM Backport #23171: core dump : recursive lock of RGWKeystoneTokenCache
- Created pull request:
https://github.com/ceph/ceph/pull/20639 - 12:00 AM Bug #23202: radosgw-admin user create failing with return code 5
- To me this looks like a different code path to #22351 since the "pool or placement group misconfiguration" message is...
03/10/2018
- 09:11 PM Bug #23299 (Fix Under Review): rgw_file: post deadlock fix, direct deleted RGWFileHandle objects ...
- https://github.com/ceph/ceph/pull/20834
(Found using a new workload test and small values for ganesha mdcache size) - 09:07 PM Bug #23299 (Resolved): rgw_file: post deadlock fix, direct deleted RGWFileHandle objects can rema...
- Since 3cf0880, a deleted RGWFileHandle that is NOT being reclaimed (i.e., being disposed) will not be removed from th...
- 04:40 PM Bug #23202: radosgw-admin user create failing with return code 5
- Does rgwrados have an opportunity to know the cause of this error?
Matt - 04:37 PM Bug #23202 (Resolved): radosgw-admin user create failing with return code 5
- Related to pg_num or pgp_num being high for the default mon_max_pg_per_osd, we need to have better error message fro...
- 02:32 AM Bug #21497 (Resolved): boto.exception.S3ResponseError: S3ResponseError: 416 Requested Range Not S...
- For anyone who is hitting this issue
set default pg_num and pgp_num to lower value(8 for example), or set mon_max...
03/09/2018
- 11:36 PM Bug #23202: radosgw-admin user create failing with return code 5
- Casey,
Is this also related to http://tracker.ceph.com/issues/22351?
I think a better error message would help ... - 07:03 PM Bug #21497: boto.exception.S3ResponseError: S3ResponseError: 416 Requested Range Not Satisfiable
- Is this issue fixed? I am having the similar issue. How do I resolve it?
- 06:34 PM rgw-testing Bug #23292 (New): test coverage for rest admin apis
- There's a teuthology task in qa/tasks/radosgw_admin_rest.py with some test coverage, but it's not running in the rgw ...
- 01:15 PM Feature #23287 (Resolved): rgw: add radosgw-admin sync error trim to trim sync error log
- rgw: add radosgw-admin sync error trim to trim sync error log
https://github.com/ceph/ceph/pull/19854 - 10:00 AM Bug #23284: Librgw rename operation returns operation not permitted
- Here is my patch : https://github.com/ceph/ceph/pull/20818
- 08:43 AM Bug #23284 (Fix Under Review): Librgw rename operation returns operation not permitted
- Hi,
Hi, I found the following test can't pass in NFS-Ganesha with NFS3 over RGW:
client side mount option : mou... - 03:39 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- Pengju Niu wrote:
> it beacase that master try to del first write objA, and it has been canceled however bilog reco... - 03:33 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- it beacase that master try to del first write objA, and it has been canceled however bilog record del op behind secon...
03/08/2018
- 09:42 PM Backport #23107 (Resolved): luminous: abort early if frontends signal an initialization error
- 09:36 PM Bug #20448 (Pending Backport): rgw: crash with rgw_run_sync_thread=false
- 09:25 PM Bug #20448: rgw: crash with rgw_run_sync_thread=false
- Orit Wasserman wrote:
> https://github.com/ceph/ceph/pull/20769
merged - 06:45 PM Bug #20448: rgw: crash with rgw_run_sync_thread=false
- 07:31 PM Bug #23182: AWS v2 signature does not cover essential parts of admin API
- update: upstream guidance is to introduce an option to require AWS_HMAC_SHA256, which defaults to enabled.
Matt - 07:23 PM Bug #23182 (Triaged): AWS v2 signature does not cover essential parts of admin API
- @robin, thanks for the clarification; both workarounds would appear acceptable; will discuss upstream; it seems cl...
- 07:26 PM Bug #23207 (Fix Under Review): rgw: inefficient buffer usage for PUTs
- 07:23 PM Bug #23203 (Pending Backport): multisite: segfault in radosgw-admin realm pull
- 07:20 PM Bug #23201 (Pending Backport): s3 website: some s3tests are failing because redirects include ind...
- 07:19 PM Bug #23198 (Triaged): osd coredump ClassHandler::ClassMethod::exec
- 07:17 PM Bug #23199 (Triaged): radosgw coredump RGWGC::process
- 07:17 PM Bug #23199: radosgw coredump RGWGC::process
- @yehuda points out that deleting the underlying pool during GC is difficult to support
- 06:54 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- Now that the old bug is solved, try to re-sync missed objects by restarting sync on the specific buckets using:
<pre... - 06:49 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- Just to be clear, note the workaround in comment #3.
- 06:49 PM Bug #22756 (Won't Fix): RGW will not list contents of older buckets at all: reshard makes it show...
- We don't really have any good way to fix it. It's a rados bug, so assigning to josh, but closing it for now.
- 06:44 PM Bug #23261 (Duplicate): Segmentation fault starting multisite radosgw when rgw_run_sync_thread=false
- http://tracker.ceph.com/issues/20448
- 06:34 PM Bug #23170 (Pending Backport): librgw crash when processing request in 12.2.3
- 03:09 PM Documentation #20942 (Pending Backport): Cannot specify multiple ports for civetweb port/listenin...
- 02:18 PM Backport #23274 (In Progress): jewel: abort early if frontends signal an initialization error
- 02:17 PM Backport #23274 (Resolved): jewel: abort early if frontends signal an initialization error
- https://github.com/ceph/ceph/pull/20800
- 01:46 PM Bug #22469 (Resolved): creating a zone with mistake tier-type is not exit
- 01:06 PM Bug #22758 (Fix Under Review): rgw sends garbage meta.compression to ElasticSearch
- https://github.com/ceph/ceph/pull/20796
- 11:51 AM Bug #23257 (Fix Under Review): 'radogw-admin reshard status' command should print text for reshar...
- 11:51 AM Bug #23257: 'radogw-admin reshard status' command should print text for reshard status
- https://github.com/ceph/ceph/pull/20779
- 05:27 AM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- The last log file that you had attached with increased log levels, had two operations:
1. Delete Bucket, which fai...
03/07/2018
- 08:03 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- Abhishek Lekshmanan wrote:
> There was a fix that went in to this in 12.2.3 https://github.com/ceph/ceph/pull/19071,... - 07:38 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- That's really strange, I have a dataset on a rbd device on the same cluster, I just did.
rclone mkdir account:data... - 03:56 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Hi Yoann, I checked all the log files including the latest one, but I dont see any errors there. I also tried to repr...
- 07:12 AM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- here the log file
- 03:54 AM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Hi Yoann, Can you please attach the log files here. Not sure how to get the files that you uploaded using ceph-post-f...
- 01:58 PM Bug #23264 (New): Server side encryption support for s3 COPY operation
- If the source object of a copy operation is encrypted with SSE-C, we should be requiring the x-amz-copy-source-serve...
- 01:05 PM Bug #23261 (Duplicate): Segmentation fault starting multisite radosgw when rgw_run_sync_thread=false
- Configuration...
- 01:02 PM Bug #20448: rgw: crash with rgw_run_sync_thread=false
- https://github.com/ceph/ceph/pull/20769
- 12:38 PM Feature #18855 (Resolved): support "user list" command in radosgw-admin
- 11:17 AM Bug #23257 (Resolved): 'radogw-admin reshard status' command should print text for reshard status
- radosgw-admin reshard status --bucket <bucket>
[
{
"reshard_status": 1,
"new_bucket_instance_... - 08:39 AM Bug #23255 (New): radosgw record wrong data logs when create and delete bucket in seconds repeatedly
- first, enable data log and meta log for multisite.
second, create a bucket and an object, then delete the object and... - 08:00 AM Bug #23201: s3 website: some s3tests are failing because redirects include index doc suffix
- 07:51 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- 02:24 AM Backport #23180 (In Progress): luminous: radosgw-admin data sync run crashes
- https://github.com/ceph/ceph/pull/20762
03/06/2018
- 11:16 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- @Orit & @Greg:
I confirm this fix works for the Benjamin cluster! Now completes in a few seconds. - 06:16 AM Bug #22963 (Fix Under Review): radosgw-admin usage show loops indefinitly - again
- 12:46 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- It kinda looks to me like we should just set "*truncated = false" in the cases where usage_iterate_range() reaches th...
- 12:12 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Okay, dug into the rgw side of things a bit more and I think https://github.com/ceph/ceph/pull/19131 contains some in...
- 08:04 PM Backport #23252 (In Progress): luminous: The return value of auth v2/v4 in RGW is wrong when Expi...
- 08:00 PM Backport #23252 (Resolved): luminous: The return value of auth v2/v4 in RGW is wrong when Expires...
- https://github.com/ceph/ceph/pull/20072
- 07:52 PM Bug #22439 (Pending Backport): The return value of auth v2/v4 in RGW is wrong when Expires/X-Amz-...
- 04:45 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Did you attach any file? Sorry I didnt get it.
- 04:27 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- ceph-post-file: 26c152cf-1fa8-4ef8-9bb8-1621bfcd43bc
- 03:57 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- Can you please increase the logging level to 20 and then attach logs.
- 02:51 PM Backport #23245 (Resolved): luminous: multisite: segfault in radosgw-admin realm pull
- https://github.com/ceph/ceph/pull/20722
- 02:41 PM Backport #23244 (Resolved): jewel: multisite: segfault in radosgw-admin realm pull
- https://github.com/ceph/ceph/pull/20749
- 02:38 PM Backport #23243 (Resolved): jewel: possible issue with ssl + libcurl
- https://github.com/ceph/ceph/pull/20749
- 10:36 AM Backport #23239 (Need More Info): luminous: Curl+OpenSSL support in RGW
- 10:31 AM Backport #23239 (Resolved): luminous: Curl+OpenSSL support in RGW
- https://github.com/ceph/ceph/pull/20722
- 10:32 AM Backport #23240 (Resolved): jewel: Curl+OpenSSL support in RGW
- https://github.com/ceph/ceph/pull/20749
- 10:30 AM Bug #23203: multisite: segfault in radosgw-admin realm pull
- Included in https://github.com/ceph/ceph/pull/20390
- 10:28 AM Feature #23162 (Pending Backport): Curl+OpenSSL support in RGW
- 10:12 AM Backport #23179 (In Progress): luminous: rgw: can't download object with range when compression e...
- https://github.com/ceph/ceph/pull/20741
- 08:00 AM Bug #23199: radosgw coredump RGWGC::process
- I have successful got the coredump configure with systemd.
If I get the string external information in the future,wi... - 07:59 AM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- I have successful got the coredump configure with systemd.
If I get the coredump cls and method in the future,will p... - 12:05 AM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- they are in the same env .
but diffrent times. more than 1months.
- 07:45 AM Bug #23232: RGWCopyObj silently corrupts the object that was mulitpart-uploaded in SSE-C
- PR: https://github.com/ceph/ceph/pull/20739
- 02:39 AM Bug #23232: RGWCopyObj silently corrupts the object that was mulitpart-uploaded in SSE-C
- @jeegn chen, that seems potentially plausible, to me.
Matt - 02:31 AM Bug #23232 (Resolved): RGWCopyObj silently corrupts the object that was mulitpart-uploaded in SSE-C
- The issue was found in 12.2.2.
The step to reproduce the issue is that
# Create a bucket (Say A-bucket) and uploa... - 07:26 AM Backport #23237 (Resolved): Corrupted downloads from civetweb when using multipart with slow conn...
- [backport pr https://github.com/ceph/ceph/pull/27982]
We have a problem in civetweb downloads, our clients have sl... - 05:30 AM Backport #23176 (In Progress): luminous: some rgw suites override frontend setting in frontend/be...
- https://github.com/ceph/ceph/pull/20736
- 03:47 AM Feature #22914: admin api --add ability to list objects which is not sync and why on multip site.
- https://github.com/ceph/ceph/pull/20733 is about to list lagging objects,
besides this, we can use sync error list t...
03/05/2018
- 11:57 PM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- the env didn't remove pool。
from communication with my colleague,He just delete objs from s3cmd and run radodgw-admi... - 07:25 PM Bug #23198: osd coredump ClassHandler::ClassMethod::exec
- is that the same pool that you removed?
- 11:45 PM Bug #23199: radosgw coredump RGWGC::process
- yes radosgw will coredump every a hours. the period just the gc thread scheduled timestamps.
I delted all omapkeys... - 07:28 PM Bug #23199: radosgw coredump RGWGC::process
- surprising to see this in v10.2.10, n.b., new async gc code is upstreamed to master and planned for backport
Matt - 07:24 PM Bug #23199: radosgw coredump RGWGC::process
- Does it still happen after restarting radosgw?
- 11:07 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Also Sage is asking if missing https://github.com/ceph/ceph/pull/13021 might be causing the repeated loops.
- 10:53 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Looked through the bucket_list.log and do see some inexplicably long op times. Although the watch-notify stuff (3) I ...
- 09:50 PM Backport #23231 (Resolved): luminous: rgw_statfs should report the correct stats
- https://github.com/ceph/ceph/pull/21724
- 09:32 PM Bug #23207: rgw: inefficient buffer usage for PUTs
- Matt suggested that it might be best to apply this fix to master - that way it doesn't matter what behavior a particu...
- 08:47 PM Bug #23207: rgw: inefficient buffer usage for PUTs
- I checked a copy of master. It does not have the problem. I'll need to do more checking; this is a mutant master li...
- 12:15 AM Bug #23207: rgw: inefficient buffer usage for PUTs
- great work, thanks marcus!
Matt - 09:07 PM Bug #23182: AWS v2 signature does not cover essential parts of admin API
- Note: I mentioned this to Yehuda last year, but the decision was that the best we could do would be to force the Admi...
- 08:38 PM Bug #23229 (Resolved): usage trim loops forever: infinite calls to rgw.user_usage_log_trim
- Strong possibility this is related to 22963, but I've got some solid debug logs here. This ALSO triggers issue 22050 ...
- 08:09 PM Bug #22826 (In Progress): "x-amz-content-sha256: STREAMING-AWS4-HMAC-SHA256-PAYLOAD" is not suppo...
- @liu, we have a plan to address this via a general cryptographic token facility that can provide temporary, strong S3...
- 07:32 PM Bug #23216: multi-site: object name should be urlencoded when we put it into ES
- 08:16 AM Bug #23216 (Fix Under Review): multi-site: object name should be urlencoded when we put it into ES
- https://github.com/ceph/ceph/pull/20707
- 07:07 AM Bug #23216 (Resolved): multi-site: object name should be urlencoded when we put it into ES
- 07:28 PM Bug #23220 (In Progress): rgw: return service unavailable when failing to connect to ext. systems
- 01:46 PM Bug #23220 (In Progress): rgw: return service unavailable when failing to connect to ext. systems
- We currently return EINVAL and http error 400 when we fail to connect to keystone for eg. Since this is not an actual...
- 07:22 PM Bug #22312: ERROR: keystone revocation processing returned error r=-22 on keystone v3 openstack o...
- @marcus ping
- 07:21 PM Bug #22331 (Can't reproduce): Luminous 12.2.2 multisite sync failed when objects bigger than 4MB
- 07:17 PM Bug #22202 (Pending Backport): rgw_statfs should report the correct stats
- 07:05 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- There was a fix that went in to this in 12.2.3 https://github.com/ceph/ceph/pull/19071, can you reproduce this in the...
- 06:58 PM Bug #22632: radosgw - s3 keystone integration doesn't work while using civetweb as frontend
- What is the version of openstack keystone that is running?
- 06:54 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- The bug is actually in rados. The problem is that the omap flag was never set on the index objects due to an old rado...
- 06:50 PM Bug #23147: RGW: metrics 'qlen', 'qactive' are not work
- needs to be verified
- 06:41 PM Bug #22951 (Fix Under Review): possible issue with ssl + libcurl
- 05:03 PM Backport #23227 (Resolved): luminous: clang compilation error in BoundedKeyCounter
- https://github.com/ceph/ceph/pull/21083
- 05:03 PM Backport #23225 (Resolved): luminous: rgw: list bilog will loop forever
- https://github.com/ceph/ceph/pull/21054
- 05:02 PM Backport #23223 (Resolved): luminous: rgw: garbage collector removes objects slowly
- https://github.com/ceph/ceph/pull/28784
- 04:48 PM Bug #23197: Bucket Policy - Input Action is compared against all actions, even if a match has bee...
- 04:02 PM Backport #23221 (Resolved): luminous: possible issue with ssl + libcurl
- https://github.com/ceph/ceph/pull/20722
- 01:54 PM Bug #22139 (Pending Backport): clang compilation error in BoundedKeyCounter
03/04/2018
- 10:59 PM Bug #23207: rgw: inefficient buffer usage for PUTs
- I updated my PR. I moved the fill loop down from just the put-object code path to just above mg_read(). I don't bel...
- 06:14 PM Bug #23207 (In Progress): rgw: inefficient buffer usage for PUTs
- https://github.com/ceph/ceph/pull/20698
- 06:29 AM Bug #23207 (Resolved): rgw: inefficient buffer usage for PUTs
- At least in jewel using swift, radosgw is very inefficient about its buffering process. In RGWPutObj_ObjStore::get_d...
03/03/2018
03/02/2018
- 09:46 PM Bug #23203: multisite: segfault in radosgw-admin realm pull
- -https://github.com/ceph/ceph/pull/20694-
- 09:38 PM Bug #23203 (Resolved): multisite: segfault in radosgw-admin realm pull
- The http client in radosgw-admin is trying to use RGWCurlHandles without doing the global init....
- 08:31 PM Bug #23201 (Fix Under Review): s3 website: some s3tests are failing because redirects include ind...
- https://github.com/ceph/ceph/pull/20693
- 07:38 PM Bug #23201 (Resolved): s3 website: some s3tests are failing because redirects include index doc s...
- ...
- 07:54 PM Bug #23202 (Resolved): radosgw-admin user create failing with return code 5
- Tried with master and luminous, Cluster is setup with ceph-ansible.
ubuntu@ovh059:~$ sudo ceph --version
ceph v... - 07:17 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- CentOS doesn't allow the overwrite of resolve.conf, I guess some network manager changes, I need to check with David ...
- 05:17 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- Not sure why this started failing..
http://pulpito.ceph.com/vasu-2018-03-02_02:00:23-rgw-luminous-distro-basic-ovh... - 05:27 PM Bug #23196: rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression enabled
- 05:25 PM Bug #23196: rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression enabled
- https://github.com/ceph/ceph/pull/20686
- 04:07 AM Bug #23196 (Resolved): rgw: fix 'copy part' without 'x-amz-copy-source-range' when compression en...
- Given an object uploaded by 'copy part' without 'x-amz-copy-source-range' when compression enabled.
That object has ... - 08:51 AM Bug #23199: radosgw coredump RGWGC::process
- backtrace please refer to 1111.png
- 08:51 AM Bug #23199 (Resolved): radosgw coredump RGWGC::process
- radosgw coredump RGWGC::process
ceph version :10.2.10
============
The operation may be easy to saw again:
i... - 08:46 AM Bug #23198 (Triaged): osd coredump ClassHandler::ClassMethod::exec
- ceph version 10.2.10 (5dc1e4c05cb68dbf62ae6fce3f0700e4654fdbbe)
1: (()+0x92b18a) [0x7fe6696f018a]
2: (()+0xf370) ... - 05:11 AM Bug #23197: Bucket Policy - Input Action is compared against all actions, even if a match has bee...
- Fixed by: https://github.com/ceph/ceph/pull/20629
- 05:11 AM Bug #23197 (Rejected): Bucket Policy - Input Action is compared against all actions, even if a ma...
- While parsing an input Policy, the Action is compared against all Actions, and even though a match is found, the meth...
- 04:51 AM Backport #23106 (Resolved): luminous: RGW will not list contents of Jewel-era buckets: reshard do...
- 04:29 AM Bug #21942 (Pending Backport): rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/valu...
- 04:28 AM Backport #23192 (In Progress): rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/valu...
- 04:02 AM Backport #23179: luminous: rgw: can't download object with range when compression enabled
- per @Casey
please backport both the original pr and a followup:
https://github.com/ceph/ceph/pull/20226
https:... - 03:19 AM Backport #23180: luminous: radosgw-admin data sync run crashes
- per @Casey
both the original pr and a followup need backport:
https://github.com/ceph/ceph/pull/18898
https:/...
03/01/2018
- 09:27 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- 08:51 PM Bug #22986: hadoop-s3a jobs failing with EPERM
- Testing with: https://github.com/ceph/ceph/pull/20678
- 07:03 PM Bug #22986 (In Progress): hadoop-s3a jobs failing with EPERM
- 07:18 PM Bug #22714: Existing AccessKeys with leading underscore broke in upgrade
- @robbat2 there was some encoding change that we did between jewel and luminous and apparently there was a case we mis...
- 07:08 PM Bug #23149 (Closed): Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied wi...
- 05:33 AM Bug #23149: Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied with reason...
- Was fixed by adding header for nginx-proxy. Nginx rejects '100' headers....
- 07:08 PM Bug #22721 (In Progress): Resharding hangs with versioning-enabled buckets
- 07:06 PM Bug #22737 (Pending Backport): rgw: list bilog will loop forever
- 07:00 PM Bug #23089: rgw: change order of authentication back to local, remote
- 06:59 PM Bug #23193 (New): multsite use of metadata list pagination
- related to issue http://tracker.ceph.com/issues/23099, the rest admin api expects the marker to be url-encoded. multi...
- 06:56 PM Bug #23139 (Pending Backport): rgw: garbage collector removes objects slowly
- 06:52 PM Bug #23139: rgw: garbage collector removes objects slowly
- 06:50 PM Bug #22928 (Resolved): RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- 06:49 PM Bug #23039 (Pending Backport): abort early if frontends signal an initialization error
- 04:07 PM Backport #23192: rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/value pairs
- https://github.com/ceph/ceph/pull/20672
- 04:07 PM Backport #23192 (Resolved): rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/value p...
- Recent changes to rgw_log_op break rgw_lib due to trivially unset keys in RGWEnv. Producer and consumer should be fi...
- 04:06 PM Bug #21942: rgw_log (and rgw_file): don't use undefined/unset RGWEnv key/value pairs
- Report from downstream is we now need backport to L
- 03:14 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- I found plenty of "execute can't find bucket IAM POLICY attr bucket_name = spinn3r.json" in radosgw logs
2018-0... - 03:09 PM Bug #23188: Cannot list file in a bucket or delete it : illegal character code U+0001
- some additional logs fronm radosgw
- 01:13 PM Bug #23188 (Closed): Cannot list file in a bucket or delete it : illegal character code U+0001
- Hello,
I created for the first time a new bucket since my upgrade to luminous then upload files using rclone sync.... - 06:45 AM Bug #23147: RGW: metrics 'qlen', 'qactive' are not work
- I guess https://github.com/ceph/ceph/pull/20006 could resolve this problem.
02/28/2018
- 05:50 PM Bug #23183 (New): 'Bad Request' after success when uploading InputStream via swift API
- Hi,
I think I have found an issue in Ceph when connecting via the Swift API and uploading an object with an InputS... - 03:22 PM Bug #23182 (Triaged): AWS v2 signature does not cover essential parts of admin API
- The AWS v2 signature only covers the URL without the query string, but the Admin API passes essential parameters in t...
- 01:28 PM Feature #23162: Curl+OpenSSL support in RGW
- Thanks Marcus for detail explanation.
- 08:13 AM Feature #23162: Curl+OpenSSL support in RGW
- I've made a PR for master with the same changes for CURL* and openssl/ubuntu. https://github.com/ceph/ceph/pull/20635
- 12:21 AM Feature #23162: Curl+OpenSSL support in RGW
- I've made a PR against jewel: https://github.com/ceph/ceph/pull/20623
I will have another PR against master later to... - 01:20 PM Bug #23170: librgw crash when processing request in 12.2.3
- This bug has been fixed by this PR:
https://github.com/ceph/ceph/commit/f9d5528a1990b5bcc33948284136c2d349aa838e - 07:23 AM Bug #23170: librgw crash when processing request in 12.2.3
- fixed with this PR: https://github.com/ceph/ceph/pull/20633
- 06:58 AM Bug #23170 (Duplicate): librgw crash when processing request in 12.2.3
- I'm using librgw with new Ceph release 12.2.3, librgw will always get unexpected crush when it is processing a reques...
- 11:21 AM Backport #23180 (Resolved): luminous: radosgw-admin data sync run crashes
- https://github.com/ceph/ceph/pull/20762
- 11:20 AM Backport #23179 (Resolved): luminous: rgw: can't download object with range when compression enabled
- https://github.com/ceph/ceph/pull/20741
- 11:20 AM Backport #23176 (Resolved): luminous: some rgw suites override frontend setting in frontend/beast...
- https://github.com/ceph/ceph/pull/20736
- 10:01 AM Backport #23171 (Resolved): core dump : recursive lock of RGWKeystoneTokenCache
- When enabling Keystone auth in ceph.conf as below and performing vstart.sh the following core
occurs during rgw sta... - 06:32 AM Bug #23149: Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied with reason...
- Actually this is nginx-proxy issue. Will try to figure out what is problem...
Sorry for the noise.
02/27/2018
- 10:08 PM Feature #23162: Curl+OpenSSL support in RGW
- Well, this needs a description, so here goes.
When rgw makes outbound http requests, it uses libcurl. For keyston... - 01:40 PM Feature #23162 (Resolved): Curl+OpenSSL support in RGW
- Curl+OpenSSL support in RGW
RHT internal bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=1522881
https://bugzi... - 06:32 PM Bug #22852 (Pending Backport): rgw: can't download object with range when compression enabled
- 05:29 PM Bug #22852: rgw: can't download object with range when compression enabled
- please backport both the original pr and a followup:
https://github.com/ceph/ceph/pull/20226
https://github.com/c... - 05:52 PM Bug #23002 (Pending Backport): some rgw suites override frontend setting in frontend/beast.yaml
- 05:47 PM Bug #23002: some rgw suites override frontend setting in frontend/beast.yaml
- Casey Bodley wrote:
> https://github.com/ceph/ceph/pull/20440
merged - 05:48 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- Yehuda Sadeh wrote:
> https://github.com/ceph/ceph/pull/20352
merged - 02:25 PM Bug #22080 (Pending Backport): radosgw-admin data sync run crashes
- both the original pr and a followup need backport:
https://github.com/ceph/ceph/pull/18898
https://github.com/cep... - 12:54 PM Bug #23149 (Closed): Aws::S3::Errors::SignatureDoesNotMatch (rgw::auth::s3::LocalEngine denied wi...
- Example code from "Ceph docs":http://docs.ceph.com/docs/master/radosgw/s3/ruby/#ruby-aws-sdk-examples-aws-sdk-gem-2
... - 10:46 AM Bug #23147 (Resolved): RGW: metrics 'qlen', 'qactive' are not work
- Ceph version 12.2.3
I perform a load test rgw, but matrics 'qlen', 'qactive' always equal 0.... - 09:03 AM Bug #23146 (Resolved): rgw: download object might fail for local invariable uninitialized
- https://github.com/ceph/ceph/pull/20612
02/26/2018
- 07:39 PM Bug #23139: rgw: garbage collector removes objects slowly
- https://github.com/ceph/ceph/pull/20546
- 07:39 PM Bug #23139 (Resolved): rgw: garbage collector removes objects slowly
- One rados object at a time. This can take very long when removing a large number of objects.
- 05:26 AM Backport #23102 (In Progress): luminous: Objects only serving first 512K
- https://github.com/ceph/ceph/pull/20586
02/24/2018
- 03:23 PM Bug #23115: fcgi's connection is not closed properly when fastcgi_keep_conn is enabled
- the last FIN is sent until `FCGX_Accept_r` is called on the same FCGX_Request in `QueueRing<FCGX_Request*> qr(max_con...
- 03:13 PM Bug #23115: fcgi's connection is not closed properly when fastcgi_keep_conn is enabled
- https://github.com/ceph/ceph/pull/20579
- 03:06 PM Bug #23115 (Fix Under Review): fcgi's connection is not closed properly when fastcgi_keep_conn is...
- nginx's config:...
02/23/2018
- 09:21 PM Feature #22604 (Resolved): Command to inspect the RGW cache
- 09:18 PM Feature #22603 (Resolved): Command to invalidate cache
- 09:18 PM Backport #22954 (Resolved): luminous: rgw: Command to invalidate cache
- 08:52 PM Backport #22954: luminous: rgw: Command to invalidate cache
- Adam Emerson wrote:
> The original request was to use the radosgw-admin tool to invalidate cache, but this doesn't f... - 09:17 PM Backport #22955 (Resolved): luminous: rgw: Command to inspect the RGW cache
- 08:52 PM Backport #22955: luminous: rgw: Command to inspect the RGW cache
- Adam Emerson wrote:
> Easiest to do first, I'll just walk the cache, print the name of each object and what hangs of... - 07:15 PM Bug #23100: admin/metadata/bucket: weird bucket name in listing, cannot fetch
- I have also reproduced this on a new Jewel-era cluster (since Congress is pre-Argonaut and sometimes causes confusion).
- 06:38 AM Bug #23100 (New): admin/metadata/bucket: weird bucket name in listing, cannot fetch
- listing of bucket names via the admin/metadata/bucket API shows this name for a bucket +@2016-02-26 03:08:16.935351@+...
- 04:22 PM Feature #18227 (Resolved): multisite: trim data changes log as peers zones catch up
- 04:22 PM Backport #18277 (Resolved): jewel: multisite: trim data changes log as peers zones catch up
- 02:58 PM Backport #23107 (Resolved): luminous: abort early if frontends signal an initialization error
- https://github.com/ceph/ceph/pull/20564
- 02:51 PM Backport #23106 (Resolved): luminous: RGW will not list contents of Jewel-era buckets: reshard do...
- https://github.com/ceph/ceph/pull/20565
- 01:50 PM Backport #23102 (Resolved): luminous: Objects only serving first 512K
- This backport got merged twice:
https://github.com/ceph/ceph/pull/20566
https://github.com/ceph/ceph/pull/20586 - 11:53 AM Bug #23091: rgw + OpenLDAP = Failed the auth strategy, reason=-13
- @Matt, thanks. I read ldap_set_option(3), and figure out: lib have server_control and client_control. So my reject is...
- 06:06 AM Bug #23099 (Resolved): REST admin metadata API paging failure bucket & bucket.instance: InvalidAr...
- Trying to use the /admin/metadata/ endpoints to review bucket & bucket.instance metadata for another bug, and found t...
- 01:42 AM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- Casey Bodley wrote:
> a couple of prs that are potentially related? https://github.com/ceph/ceph/pull/20396 https://... - 12:05 AM Bug #23095 (New): Poor defaults for RGW
- Objecter throttling is apparent from inspecting the admin socket on the RGW host.
Default objecter_inflight_ops sh...
02/22/2018
- 07:08 PM Bug #22982 (Pending Backport): Objects only serving first 512K
- 07:05 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- 06:46 PM Bug #23089: rgw: change order of authentication back to local, remote
- 01:35 PM Bug #23089 (Resolved): rgw: change order of authentication back to local, remote
- from luminous we changed the order of rgw authentication to first try external before attempting locally, this can ha...
- 06:43 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- there's not much that radosgw can do here - workaround is necesary
- 04:33 PM Bug #23092 (New): RGW leaking/orphan data with Jewel
- This was previously reported under #18331 but this popped up again on a Jewel cluster.
Some information:
- Jewe... - 02:39 PM Bug #23091: rgw + OpenLDAP = Failed the auth strategy, reason=-13
- @konstantin, thanks, I'm working on LDAP auth now, I'll take a look.
Matt - 02:24 PM Bug #23091 (Duplicate): rgw + OpenLDAP = Failed the auth strategy, reason=-13
- Hello, recently I posted this question to users ML, but don't receive any feedback.
More at ML: http://lists.ceph.... - 01:40 PM Bug #23090 (New): rgw: both civetweb & libcurl (and boost::asio) may initialize openssl
- currently if rgw is https terminated with civetweb; and if libcurl is linked against openssl both the these will end ...
- 01:29 PM Bug #23088 (Closed): nfs-ganesha repo for Xenial is not signed
- hrw@gossamer:$ docker run -it -u root ubuntu:xenial bash
root@cf9a237f71dc:/# cd /etc/apt/sources.list.d/
root@cf9a... - 09:53 AM Bug #23039: abort early if frontends signal an initialization error
- Suse security assigned CVE-2018-7262 to this
- 04:47 AM Documentation #23081: docs: radosgw: ldap-auth: wrong option name 'rgw_ldap_searchfilter'
- https://github.com/ceph/ceph/pull/20526
- 04:45 AM Documentation #23081 (Resolved): docs: radosgw: ldap-auth: wrong option name 'rgw_ldap_searchfilter'
- ...
- 03:55 AM Bug #22986: hadoop-s3a jobs failing with EPERM
- Running on 7.3 to see why copying to /etc/resolv.conf is failing for 7.4
http://pulpito.ceph.com/vasu-2018-02-22_0...
02/21/2018
02/20/2018
- 04:41 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Hi,
just FYI - I've uploaded some logs in this ticket as well - http://tracker.ceph.com/issues/22314
If you need ... - 09:06 AM Backport #23023 (In Progress): jewel: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20496
- 06:16 AM Backport #23023: jewel: can not set user quota with specific value
- I'm on it.
02/19/2018
- 10:25 PM Bug #23039 (In Progress): abort early if frontends signal an initialization error
- https://github.com/ceph/ceph/pull/20488
- 03:45 PM Bug #23039 (Resolved): abort early if frontends signal an initialization error
- Since some frontends like civetweb can signal an error if non compliant HTTP Headers are passed on request, abort ear...
- 10:42 AM Backport #23021 (In Progress): jewel: The parameter of max-uploads doesn't work when List Multipa...
- https://github.com/ceph/ceph/pull/20479
- 10:17 AM Backport #21637 (Resolved): luminous: encryption: PutObj response does not include sse-kms headers
- 04:34 AM Backport #23022 (In Progress): luminous: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20477
- 04:00 AM Backport #23020 (In Progress): luminous: The parameter of max-uploads doesn't work when List Mult...
- https://github.com/ceph/ceph/pull/20476
02/18/2018
- 11:33 AM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- one mistake - this is a result of : radosgw-admin data sync status --source-zone=zone1 instead of metadata.
- 11:29 AM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- Orit Wasserman wrote:
> can you provide the out put of sync error list commands?
> EBUSY errors are expected to hap... - 06:16 AM Bug #22897: rgw: (jewel) can't delete swift acls with swift command.
- I've made a PR for master with this change,
https://github.com/ceph/ceph/pull/20471
I believe this should apply tri...
02/17/2018
- 05:28 AM Bug #22963: radosgw-admin usage show loops indefinitly - again
- The comment by Marcin Gibula in issue 21196 is actually spot on.
This DOES actually complete, eventually, if nothin...
02/16/2018
- 10:24 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- Debugging so far:
1. The bug only occurs on the Congress cluster, not the Benjamin cluster (was rebuilt from scratch... - 07:53 PM Feature #22832 (Fix Under Review): SSL support for beast frontend
- https://github.com/ceph/ceph/pull/20464
- 02:24 PM Documentation #23027 (Resolved): document rgw_multipart_min_part_size
- Documentation for rgw_multipart_min_part_size is missing form doc/radosgw/config-ref.rst
- 11:04 AM Backport #23026 (Resolved): jewel: rgw: data sync of versioned objects, note updating bi marker
- https://github.com/ceph/ceph/pull/21213
- 11:04 AM Backport #23025 (Resolved): luminous: rgw: data sync of versioned objects, note updating bi marker
- https://github.com/ceph/ceph/pull/21214
- 11:02 AM Backport #23023 (Resolved): jewel: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20496
- 11:02 AM Backport #23022 (Resolved): luminous: can not set user quota with specific value
- https://github.com/ceph/ceph/pull/20477
- 11:02 AM Backport #23021 (Resolved): jewel: The parameter of max-uploads doesn't work when List Multipart ...
- https://github.com/ceph/ceph/pull/20479
- 11:02 AM Backport #23020 (Resolved): luminous: The parameter of max-uploads doesn't work when List Multipa...
- https://github.com/ceph/ceph/pull/20476
02/15/2018
- 09:50 PM Bug #22833: rgw: crash in RGWSyncTraceManager::finish_node()
- Luminous backport to be done in #23018
- 06:35 PM Bug #22833 (Resolved): rgw: crash in RGWSyncTraceManager::finish_node()
- 07:28 PM Bug #22825 (Pending Backport): The parameter of max-uploads doesn't work when List Multipart Uploads
- 07:26 PM Bug #22745 (Pending Backport): can not set user quota with specific value
- 04:59 PM Bug #22745: can not set user quota with specific value
- Casey Bodley wrote:
> https://github.com/ceph/ceph/pull/20063
merged - 07:24 PM Bug #22352 (Duplicate): rados gateway computes wrong AWS4 signature if canonical request contains...
- 07:22 PM Bug #22218 (Duplicate): multisite: rgw crashed during meta sync
- fixed in http://tracker.ceph.com/issues/21097
- 07:14 PM Bug #22963: radosgw-admin usage show loops indefinitly - again
- This bug definitely still exists, is affecting billing.
Here's the interesting part from --debug-rgw=20 --debug-ms=1... - 07:12 PM Bug #22908: [Multisite] Synchronization works only one way (zone2->zone1)
- can you provide the out put of sync error list commands?
EBUSY errors are expected to happen, look for other errors. - 07:02 PM Bug #22895: radosgw not work after all rgw `RGWAsyncRadosProcessor had timed out
- the hangs in RGWAsyncRadosProcessor could be due to requests sent to another gateway that's also paused for realm rec...
- 06:58 PM Bug #22804: multisite Synchronization failed when read and write delete at the same time
- a couple of prs that are potentially related? https://github.com/ceph/ceph/pull/20396 https://github.com/ceph/ceph/pu...
- 06:54 PM Bug #22827 (Resolved): rgw_file: FLAG_EXACT_MATCH actually should be enforced
- 06:52 PM Bug #18885 (Pending Backport): rgw: data sync of versioned objects, note updating bi marker
- 06:50 PM Bug #22721: Resharding hangs with versioning-enabled buckets
- 06:44 PM Bug #22928: RGW will not list contents of Jewel-era buckets: reshard does NOT fix
- 06:43 PM Bug #22756: RGW will not list contents of older buckets at all: reshard makes it show up again
- 06:43 PM Bug #22982: Objects only serving first 512K
- 06:37 PM Backport #23018 (Rejected): rgw multisite sync tracing feature
- https://github.com/ceph/ceph/pull/16492
and fixes in https://github.com/ceph/ceph/pull/20191 - 12:08 PM Bug #20166: RGW: Bad error handling when tail object is missing
- Seems that this is not resolved yet indeed. I'm also failing to see a good way how to handle this error situation. Wo...
02/14/2018
- 07:17 PM Bug #20166: RGW: Bad error handling when tail object is missing
- #20107 is marked as resolved.
Is this issue #20166 also resolved?
I think I am seeing this behavior in Luminous... - 05:23 PM Bug #23002 (Fix Under Review): some rgw suites override frontend setting in frontend/beast.yaml
- 05:23 PM Bug #23002: some rgw suites override frontend setting in frontend/beast.yaml
- https://github.com/ceph/ceph/pull/20440
- 05:21 PM Bug #23002 (Resolved): some rgw suites override frontend setting in frontend/beast.yaml
- for example, a job in the verify suite that includes frontend/beast.yaml:
description: rgw/verify/{clusters/fixe... - 06:44 AM Bug #22982: Objects only serving first 512K
- This change has been deployed at Dreamhost, seems to work so far, watching for other breakage
02/13/2018
- 06:22 PM Bug #22982: Objects only serving first 512K
- a possible fix:
https://github.com/ceph/ceph/pull/20425 - 04:55 PM Bug #22982: Objects only serving first 512K
- Manifest has appropriate raw rados objects there, so we do an unneeded conversion when reading it.
- 04:25 AM Bug #22982: Objects only serving first 512K
- Private attachment made w/ manifest & hexdump of manifest.
- 04:57 PM Bug #22833: rgw: crash in RGWSyncTraceManager::finish_node()
- 03:35 PM Backport #22987 (In Progress): jewel: rgw: user stats increased after bucket reshard
- 10:21 AM Backport #22987 (Resolved): jewel: rgw: user stats increased after bucket reshard
- https://github.com/ceph/ceph/pull/20421
- 10:20 AM Bug #22124 (Pending Backport): rgw: user stats increased after bucket reshard
- Adding jewel backport
Also available in: Atom