Activity
From 12/11/2014 to 01/09/2015
01/09/2015
- 09:33 AM Bug #10445: Radosgw hang, corrupt manifest?
- Thanks for that, I'll try to test it if I get a chance.
I believe I've narrowed the original problem that gets me ...
01/08/2015
- 12:59 PM Bug #10445: Radosgw hang, corrupt manifest?
- I pushed a change to wip-10445. Now, instead of waiting for the completion, it'll just send it to the garbage collect...
- 12:59 PM Bug #10445: Radosgw hang, corrupt manifest?
- you can run:...
- 09:36 AM Bug #10445: Radosgw hang, corrupt manifest?
- Ah I see what you mean. But I'm not sure what is causing these specific uploads to be so slow to remove. If I start a...
- 09:28 AM Bug #10445: Radosgw hang, corrupt manifest?
- I don't see the gateway stuck in infinite loop. It just goes too slow, so the client retries (while the first operati...
- 07:13 AM Bug #10445: Radosgw hang, corrupt manifest?
- Ok I still don't know how it gets into this state to begin with but right now I have objects or partial uploads where...
- 11:16 AM Bug #10482 (Won't Fix): rgw: older versions need to be able to sign requests with 'versions' subr...
- In firefly and giant we need to make it handle the 'versions' subresource correctly when listing buckets (even thoug...
01/07/2015
- 02:06 PM Bug #10471 (Fix Under Review): rgw: metadata sync doesn't create subusers correctly
- 09:48 AM Bug #10471 (Resolved): rgw: metadata sync doesn't create subusers correctly
- 01:25 PM Bug #10475 (Resolved): rgw: setting bucket metadata via swift api removes existing metadata
- According to http://docs.openstack.org/api/openstack-object-storage/1.0/content/POST_updateContainerMeta__v1__account...
- 01:22 PM Bug #10307 (Pending Backport): rgw: setting bucket attributes (swift POST on bucket) sometimes fail
- 08:44 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- Same problems in run http://pulpito.ceph.com/teuthology-2015-01-06_17:15:01-upgrade:dumpling-firefly-x:parallel-next-...
01/06/2015
- 03:26 PM Bug #10354 (Pending Backport): Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro...
- 09:51 AM Bug #10445: Radosgw hang, corrupt manifest?
- Aaron Bassett wrote:
> https://github.com/ceph/ceph/blob/master/src/rgw/rgw_rados.cc#L589
>
> Reading this sectio... - 09:47 AM Bug #10445: Radosgw hang, corrupt manifest?
- https://github.com/ceph/ceph/blob/master/src/rgw/rgw_rados.cc#L589
Reading this section, it looks to me that if pa... - 09:34 AM Bug #10445: Radosgw hang, corrupt manifest?
- I did some more testing, this time I tried taking the cache tier out of play and set it up with:
a) civetweb, usin... - 09:06 AM Feature #9932 (Resolved): rgw: map swift X-Storage-Policy header to rgw pools
- commit:77c1a35a827f4f285a4a2b47eba62465fc67d126
- 09:03 AM Feature #10191 (In Progress): rgw: object versioning multi-zone support
- 06:42 AM Feature #10453 (In Progress): rgw: sync agent full sync of versioned objects
01/05/2015
- 03:48 PM Bug #10268 (Pending Backport): s3tests.functional.test_s3.test_bucket_create_exists fails with 'S...
- 03:47 PM Bug #10334 (Pending Backport): rgw: s3 need to return InvalidAccessKeyId instead of AccessDenied
- commit:56af795b1046a4c1bfba59e1fefde272bb0e5c1e
- 03:47 PM Bug #10329 (Pending Backport): rgw: s3 need to return SignatureDoesNotMatch instead of AccessDenied
- commit:ef75d720f289ce2e18c0047380a16b7688864560
- 03:45 PM Bug #9478 (Pending Backport): Incorrect content type in response header
- commit:31963459a0a869c4d32f55baa629f36df33eaa90
- 03:45 PM Bug #9479 (Pending Backport): ETag is not included in the XML response to put object copy operation
- commit:b1bfc3a7e0c9088f01f8ff770ae14f569fbc570d
- 03:44 PM Bug #9877 (Pending Backport): In some cases it's possible for rgw to segfault on http COPY
- commit:fe7bf06366adaf787816d1e68f5e3f68e8c91134
- 03:43 PM Bug #9918 (Pending Backport): RGW-Swift: SubUser access permissions, does not seems to work
- commit:5d9f36f757a7272c24d2c9adc31db1ed5e712992
- 03:42 PM Bug #9973 (Pending Backport): Validation of Swift DLO manifest object ETag doesn't match OpenStac...
- commit:ef6d3ad964d34bc526dc4435486bd5c8cdc3b230
- 03:39 PM Bug #5595 (Pending Backport): object has a Content-Type, but its content_type property is not sho...
- commit:f833f12a200ecc2c4f74ddb443d6fa61b7ad14db
- 03:07 PM Feature #7214 (Rejected): radosgw-agent should work with older versions of python-requests
- it no longer uses requests
- 11:01 AM Feature #10454 (Resolved): rgw: sync agent incremental sync of versioned objects
- 11:00 AM Feature #10453 (Resolved): rgw: sync agent full sync of versioned objects
- 08:57 AM Bug #10445: Radosgw hang, corrupt manifest?
- Well I tried the same setup again with civetweb instead of apache. It ran 10-15% slower, which was fine, it made it f...
- 08:28 AM Bug #10445: Radosgw hang, corrupt manifest?
- Not sure what triggers your original issue, althgouh I assume it's similar to what I see in the log. Basically what I...
- 08:41 AM Bug #10354 (Fix Under Review): Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro...
- This is duplicate of 10329, there's a pull request pending. Note though that the pull request is against master, but ...
01/04/2015
- 01:22 AM Bug #10447 (Closed): rx to radosgw is more than tx (more than ten times)
- Hi, I am using radosgw (using civetweb, but happens with apache too), rx to the gateway is more than 10 times the tx ...
01/02/2015
- 06:23 AM Bug #10445 (Resolved): Radosgw hang, corrupt manifest?
- I'm load testing my radosgw setup to get an idea what kind of throughput I can get. I have it set up writing to a cac...
12/26/2014
- 10:46 AM Bug #10354 (New): Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi ...
- Reopening as the initial problem is still there:
Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-1...
12/23/2014
- 02:15 PM Bug #10354 (Resolved): Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-m...
- All passed
- 10:43 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- We were running teuthology pointing to an old path, thus running something old, Zack modified the path and will see h...
- 09:07 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- This suite did not change AFAIK and was passing as is
http://pulpito.front.sepia.ceph.com/teuthology-2014-12-03_17:1... - 08:57 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- It's running the master branch of the s3 tests instead of the next branch.
- 08:44 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- Failed in run http://pulpito.ceph.com/teuthology-2014-12-22_17:10:02-upgrade:dumpling-firefly-x:parallel-next-distro-...
12/22/2014
- 08:50 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- Still failed in run http://pulpito.ceph.com/teuthology-2014-12-21_17:10:01-upgrade:dumpling-firefly-x:parallel-next-d...
12/19/2014
- 11:31 AM Bug #10015 (Resolved): rgw sync agent: 403 when syncing object that has tilde in its name
- Changes included in https://github.com/ceph/radosgw-agent/pull/14
merged commit fbcc438 into master
12/18/2014
- 02:12 PM Bug #10379 (Resolved): rgw: object versioning, remove last version needs to remove olh
- 09:02 AM Documentation #10373 (Closed): Document about S3 POST requests
- Looks like S3 POST requests are supported (atleast for cephx authentication) in rgw. Document about this functionalit...
- 08:55 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- Same in Octo - 5 jobs in run http://pulpito.ceph.redhat.com/teuthology-2014-12-16_22:04:02-rgw-dumpling-distro-basic-...
- 08:49 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- Same issue http://pulpito.ceph.com/teuthology-2014-12-17_17:10:02-upgrade:dumpling-firefly-x:parallel-next-distro-bas...
- 08:01 AM Bug #10354: Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-multi run
- Same issue - http://pulpito.ceph.com/teuthology-2014-12-17_17:15:01-upgrade:dumpling-firefly-x:parallel-next-distro-b...
12/17/2014
- 07:46 PM Bug #10365 (Closed): radosgw-agent: buckets to retry logged by full sync won't be fully synced if...
- When buckets to retry are handled by incremental sync, it uses the bucket index log to figure out which objects withi...
- 02:05 PM Bug #10194: rgw: fcgi connections are not closed when using mod-proxy-fcgi
- The fix was merged to the firefly branch in this commit:...
- 09:39 AM Bug #10354 (Duplicate): Failed s3 tests in upgrade:dumpling-firefly-x:parallel-next-distro-basic-...
- Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-12-16_17:10:02-upgrade:dumpling-firefly-x:parallel-ne...
12/16/2014
- 06:13 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Finally..mini test case.
1. New rgw instance
2. Create bucket + 4 x 10m objects (2 by fastcgi, 2 by civetweb)
3.... - 05:54 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- 1. The issue is reproducible only on objects created by civetweb.
The bucket can be created by either civetweb or... - 05:44 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Weird - the problem occurs with objects *created* by civetweb.
How to reproduce:
1. create new rgw instance
2.... - 05:06 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- The corruption occurs if civetweb is the first thing that touches a new rgw setup.
How to reproduce:
1. rmpool .... - 04:35 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Suddenly civetweb is working, huh!!??
1. Delete 3 objects in succession
[
{ "tag": "default.7460.27",
... - 04:23 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Logs with good gc behaviour:
1. rm 1 object # 3 oids expected
[root@cephadm ~]# radosgw-admin gc list --include-a... - 08:33 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Can you provide logs to both your apache and civetweb gateways, when removing objects? (set 'debug rgw = 20', and 'de...
- 01:36 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- civetweb seems to "corrupt" the gc list.
1. rm one object (10M size) with civetweb, 3 oids in gc list
2. rm 2nd o... - 03:05 PM Feature #10342 (Resolved): rgw: tool to clean up clutter
- ...
- 01:28 PM Bug #9892 (Resolved): radosgw_admin.py: failed len(out['entries']) == 0 on usage show
- merged commit 3abb3ff into master, giant, firefly, and dumpling
Commit hash is actually 47d56d779eb145f5bca1e8e1e... - 01:15 PM Bug #9892 (Fix Under Review): radosgw_admin.py: failed len(out['entries']) == 0 on usage show
- changes had been made by yehuda already, PR opened https://github.com/ceph/ceph-qa-suite/pull/266
After it merges ... - 10:54 AM Bug #10334 (Resolved): rgw: s3 need to return InvalidAccessKeyId instead of AccessDenied
- commit:56af795b1046a4c1bfba59e1fefde272bb0e5c1e...
- 10:07 AM Bug #10329 (Fix Under Review): rgw: s3 need to return SignatureDoesNotMatch instead of AccessDenied
- 08:58 AM Bug #10329 (Resolved): rgw: s3 need to return SignatureDoesNotMatch instead of AccessDenied
- commit:ef75d720f289ce2e18c0047380a16b7688864560
- 10:00 AM Feature #10333 (Resolved): rgw: support AWS4 authentication
- 09:59 AM Bug #10195: s3 java jdk conn.getobject(...) (get s3 object) method fails with latest version of a...
- It seems that the jdk uses a new type of authentication (AWS4), that radosgw is yet to support.
- 09:55 AM Bug #10177: test_multipart_upload failed in upgrade:dumpling-firefly-x:parallel-giant-distro-basi...
- For some reason boto got here a timeout (no real reason that I can see, everything flows nicely):...
- 09:40 AM Bug #10221 (Resolved): Crash in "radosgw-admin" in upgrade:firefly:singleton-firefly-distro-basic...
- Fixed in commit:9d5e4edc15678de675d7a77badcbace0b710d553
- 09:17 AM Bug #10227 (Duplicate): "Segmentation fault" radosgw() in smoke-master-distro-basic-multi suite
- 09:15 AM Bug #10188 (Won't Fix): Can not create new rgw user when specifying an email already assigned to ...
- 09:11 AM Bug #10102 (Resolved): sync agent: does not handle gracefully transient errors
- https://github.com/ceph/radosgw-agent/pull/13
- 09:10 AM Bug #10194: rgw: fcgi connections are not closed when using mod-proxy-fcgi
- fix for master needs to be reviewed / merged.
- 09:03 AM Bug #10251 (Resolved): "Segmentation fault" (radosgw()) in upgrade:dumpling-firefly-x:parallel-ne...
- 09:03 AM Bug #9254 (Pending Backport): rgw: civetweb requires explicit \r\n for http headers
- 09:02 AM Bug #10066 (Pending Backport): rgw: failed md5sum on s3tests-test-readwrite
- 06:19 AM Bug #10268: s3tests.functional.test_s3.test_bucket_create_exists fails with 'S3CreateError not ra...
12/15/2014
- 06:44 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Revert rgw_gc_max_objs to 32, problem remains.
- 06:40 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- civetweb:
when you delete a second object, it does not appear in gc list --incude-all
How to reproduce:
1. add... - 05:35 PM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- The rgw parameters are in the global section.
- 11:18 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Not sure how civetweb could affect gc work. I do see that your rgw_gc_max_objs is non standard. Did you put this conf...
- 09:53 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Summary of bug:
civetweb 0.89 is not correctly preparing the garbage collection list:
1. create 100 rgw objects (... - 09:40 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Should have written:
It is a civetweb bug.
With apache/fastcgi: I added 100 objects and rm'ed - 10M ea,
300 s... - 09:39 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- It is a civetweb bug.
I added 100 objects and rm'ed - 10M ea, 300 shadow objects created.
All 300 shadow objects ... - 09:02 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- No - I am following the exemplary simple configuration
with default zones; only changes are custom cluster name
and... - 08:44 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Do you by any chance have non default zone configuration?
- 08:24 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Reproduced this on a fresh 0.89.
Created 400 objects, deleted 300.
Found 3 objects in gc list --include-all.
... - 12:42 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Tested again: reduced rgw_gc settings to 300s.
On 0.80.7, the shadow object were purged.
On 0.87, 0.89 the garb... - 02:33 PM Feature #10322 (Resolved): rgw: s3-tests, coverage for object versioning
- Some work has already been done (wip-rgw-object-versioning). Need to see if anything is missing, extend tests if needed.
- 02:30 PM Feature #10321 (New): rgw: verify proper tests coverage for bucket sharding
- 02:29 PM Feature #10320 (New): rgw: ability to configure bucket sharding params in rgw teuthology test suite
- 12:40 PM Bug #10268: s3tests.functional.test_s3.test_bucket_create_exists fails with 'S3CreateError not ra...
- And the last one is correct too, using a valid Key ID but a wrong access_key on S3:...
- 11:45 AM Bug #10268: s3tests.functional.test_s3.test_bucket_create_exists fails with 'S3CreateError not ra...
- The second failure I can confirm it is a problem with RGW, it should return 'InvalidAccessKeyId':...
- 07:44 AM Bug #10268 (In Progress): s3tests.functional.test_s3.test_bucket_create_exists fails with 'S3Crea...
- Reverted first test. S3 docs state a duality in behavior depending on US Standard vs Regions and we support the forme...
- 12:51 AM Bug #10311 (Fix Under Review): Object is deleted automatically if conditional PUT returns 412
- Sent out the PR for fix: https://github.com/ceph/ceph/pull/3176
Also committed some functional test cases in: http... - 12:21 AM Documentation #10305: Update documentation on http://ceph.com/docs/master/radosgw/keystone/
- In addition, this issue was pretty cryptic to debug (...a pun... I know) and it would have been pretty straightforwar...
12/14/2014
- 06:59 PM Bug #10311 (Resolved): Object is deleted automatically if conditional PUT returns 412
- After back-porting conditional PUT feature, test shows object will be deleted automatically if conditional PUT return...
12/12/2014
- 05:04 PM Bug #10307 (Fix Under Review): rgw: setting bucket attributes (swift POST on bucket) sometimes fail
- 04:53 PM Bug #10307 (Resolved): rgw: setting bucket attributes (swift POST on bucket) sometimes fail
- Issue related to the way we read the bucket attributes.
- 11:47 AM Bug #9254: rgw: civetweb requires explicit \r\n for http headers
- opened a new pull request:
https://github.com/ceph/ceph/pull/3163 - 11:35 AM Bug #10066 (Fix Under Review): rgw: failed md5sum on s3tests-test-readwrite
- PR opened: https://github.com/ceph/s3-tests/pull/26
- 10:43 AM Feature #8562 (Resolved): rgw: Conditional PUT on ETag
- Merged, commit:7925b82c20c94043dd5e4ca69d3ef55ea43db1e0.
Tests commit:3e1e2ee43893264eb9ca2a3fbc581ebe46c06d0e - 12:42 AM Documentation #10305: Update documentation on http://ceph.com/docs/master/radosgw/keystone/
- Unassigning myself because I'm not the best candidate ;-)
12/11/2014
- 11:33 PM Documentation #10305 (Closed): Update documentation on http://ceph.com/docs/master/radosgw/keystone/
- Hi,
In the URL http://ceph.com/docs/master/radosgw/keystone/, where it mentions about converting the OpenSSL certi... - 01:44 PM Bug #10303 (Can't reproduce): "..s3.test_list_multipart_upload ... FAIL" in upgrade:firefly:newer...
- Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-12-10_17:00:02-upgrade:firefly:newer-firefly-distro-b...
- 12:47 PM Bug #10301 (Resolved): "FATAL ERROR: cannot find a valid <xfs/xfs.h> header file" in powercycle-g...
- A commit was made to ceph-qa-chef to include xfsprogs-devel shortly after this test ran:
038fdda63d363b193b028495e... - 12:40 PM Bug #10301 (Resolved): "FATAL ERROR: cannot find a valid <xfs/xfs.h> header file" in powercycle-g...
- Logs are in http://qa-proxy.ceph.com/teuthology/teuthology-2014-12-09_13:44:11-powercycle-giant-distro-basic-multi/64...
- 08:18 AM Bug #10295: rgw deleting S3 objects leaves __shadow_ objects behind
- Need to wait for gc to run its course. Not all entries are going to show in the gc listing immediately. You need to r...
- 05:55 AM Bug #10295 (Resolved): rgw deleting S3 objects leaves __shadow_ objects behind
- Using 0.80.7, observed also on .87, .89
1. Create 100 objects 10MB ea in rgw.
2. Observe that .rgw.buckets occupi...
Also available in: Atom