https://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2016-06-08T07:17:48ZCeph rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=722252016-06-08T07:17:48ZAlbert Tume@chiehtu.net
<ul></ul><p>PR: <a class="external" href="https://github.com/ceph/ceph/pull/9584">https://github.com/ceph/ceph/pull/9584</a></p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=722892016-06-09T09:16:28ZOrit Wassermanowasserm@redhat.com
<ul></ul><p>In the second stat command you have two objects, maybe someone uploaded another object?<br />can you do reproduce and also run list buckets?</p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=726112016-06-14T11:23:05ZAlbert Tume@chiehtu.net
<ul></ul><p>This bug can reproduce on development environment.<br /><pre>
$ swift -A http://localhost:8000/auth -U test:tester -K 'testing' stat --lh melon
Account: v1
Containers: 1
Objects: 2
Bytes: 20M
Connection: Keep-Alive
X-Timestamp: 1465895380.67945
X-Account-Bytes-Used-Actual: 20971520
X-Trans-Id: tx00000000000000000000c-00575fc9d4-101d-default
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
</pre></p>
<pre>
$ swift -A http://localhost:8000/auth -U test:tester -K 'testing' list --lh melon
10M 2016-06-14 09:04:04 tmp/10M
10M
</pre><br />The same issue before: <a class="external" href="https://github.com/ceph/ceph/pull/6047">https://github.com/ceph/ceph/pull/6047</a> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=729402016-06-17T08:33:49ZPritha Srivastavaprsrivas@redhat.com
<ul></ul><p>I wasnt able to reproduce the problem. I tried the following:</p>
<p>1. In my dev environment, created a container - container1<br />2. Uploaded an object of 16 bytes to container1<br />3. Ran the stat command more than once - in each run I could see the correct number of objects which is 1 and its size which is 16.<br />4. Ran the list command - the ouput of it lists the object once. The size which is listed at the end after the objects is the total size of all objects in the container like</p>
<p>$ swift -A <a class="external" href="http://localhost:8000/auth/1.0">http://localhost:8000/auth/1.0</a> -U test:tester -K testing list --lh container1<br /> 21 2016-06-17 07:05:10 None sample.txt<br /> 16 2016-06-17 07:03:58 None sample1.txt<br /> 21 2016-06-17 07:05:19 None sample2.txt<br /> 58</p>
<p>I also checked the PR mentioned in the comment above: If the needs_stats is set to false, then it supresseses the stats related to objects output by the swift 'stat' command. I am not sure how that will solve the problem that is mentioned in the ticket.</p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=729412016-06-17T08:39:01ZPritha Srivastavaprsrivas@redhat.com
<ul></ul><p>Albert Tu wrote:</p>
<blockquote>
<p>This bug can reproduce on development environment.<br />[...]</p>
<p>[...]<br />The same issue before: <a class="external" href="https://github.com/ceph/ceph/pull/6047">https://github.com/ceph/ceph/pull/6047</a></p>
</blockquote>
<p>Hi Albert, which branch are you working off? I tried the above using the master branch and couldn't see the issue in my dev environment.</p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=731132016-06-22T03:20:44ZAlbert Tume@chiehtu.net
<ul></ul><p>Pritha Srivastava wrote:</p>
<blockquote>
<p>Albert Tu wrote:</p>
<blockquote>
<p>This bug can reproduce on development environment.<br />[...]</p>
<p>[...]<br />The same issue before: <a class="external" href="https://github.com/ceph/ceph/pull/6047">https://github.com/ceph/ceph/pull/6047</a></p>
</blockquote>
<p>Hi Albert, which branch are you working off? I tried the above using the master branch and couldn't see the issue in my dev environment.</p>
</blockquote>
<p>I'm using the master branch (87b5a65). Sorry for typo, the correct stat command shouldn't append container name.</p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=732282016-06-23T06:59:09ZAlbert Tume@chiehtu.net
<ul></ul><p>The stat command returns double space usage because rgw_read_user_buckets() get bucket stats and then count based on it when parameter `need_stats` is true.</p>
<p>Bucket stats are synced by BucketsSyncThread every 3 minutes. That's why we need wait a few minutes to reproduce this bug.</p>
Code links:
<ol>
<li>RGWStatAccount::execute() <a class="external" href="https://github.com/ceph/ceph/blob/master/src/rgw/rgw_op.cc#L1582">https://github.com/ceph/ceph/blob/master/src/rgw/rgw_op.cc#L1582</a></li>
<li>rgw_read_user_buckets() <a class="external" href="https://github.com/ceph/ceph/blob/master/src/rgw/rgw_bucket.cc#L99">https://github.com/ceph/ceph/blob/master/src/rgw/rgw_bucket.cc#L99</a></li>
<li>BucketsSyncThread <a class="external" href="https://github.com/ceph/ceph/blob/master/src/rgw/rgw_quota.cc#L398">https://github.com/ceph/ceph/blob/master/src/rgw/rgw_quota.cc#L398</a></li>
</ol> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=734752016-06-28T03:43:59ZAlbert Tume@chiehtu.net
<ul></ul><p>Update the pull request. <a class="external" href="https://github.com/ceph/ceph/pull/9584">https://github.com/ceph/ceph/pull/9584</a></p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=739152016-07-06T06:16:25ZAlbert Tume@chiehtu.net
<ul></ul><p>Need to backport to hammer. Do I need to create issue or pull request?</p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=739232016-07-06T08:22:48ZNathan Cutlerncutler@suse.cz
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Pending Backport</i></li><li><strong>Backport</strong> set to <i>jewel</i></li></ul> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=739242016-07-06T08:24:03ZNathan Cutlerncutler@suse.cz
<ul><li><strong>Copied to</strong> <i><a class="issue tracker-9 status-3 priority-4 priority-default closed" href="/issues/16599">Backport #16599</a>: jewel: rgw: Swift API returns double space usage and objects of account metadata</i> added</li></ul> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=739282016-07-06T08:43:28ZNathan Cutlerncutler@suse.cz
<ul><li><strong>Subject</strong> changed from <i>jewel: rgw: Swift API returns double space usage and objects of account metadata</i> to <i>rgw: Swift API returns double space usage and objects of account metadata</i></li></ul> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=739292016-07-06T08:45:58ZNathan Cutlerncutler@suse.cz
<ul></ul><p>jewel backport has been staged. Albert, the bug says affected version is 10.2.1 which is jewel. Are you sure this needs to be backported to hammer (0.94.x)?</p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=740032016-07-07T03:59:13ZAlbert Tume@chiehtu.net
<ul></ul><p>Sure, because <a class="issue tracker-1 status-3 priority-4 priority-default closed" title="Bug: rgw: swift API returns more than real object count and bytes used when retrieving account metadata (Resolved)" href="https://tracker.ceph.com/issues/13140">#13140</a> fixes this bug by don't call update_container_stats(). This made radosgw doesn't return immediate stats.</p>
<p>And after backported this, the list command gets fixed too.</p> rgw - Bug #16188: rgw: Swift API returns double space usage and objects of account metadatahttps://tracker.ceph.com/issues/16188?journal_id=770032016-08-22T09:30:55ZLoïc Dacharyloic@dachary.org
<ul><li><strong>Status</strong> changed from <i>Pending Backport</i> to <i>Resolved</i></li></ul>