Bug #16188
closed
rgw: Swift API returns double space usage and objects of account metadata
Added by Albert Tu almost 8 years ago.
Updated over 7 years ago.
Description
When user upload file and run `stat` command, rgw returns correct account metadata immediately.
StorageURL: http://192.168.62.50:8080/swift/v1
Auth Token: AUTH_rgwtk0b0000006d656c6f6e3a7377696674da9bd854682bd73095e75857cd14fb39bd80bae4fc7af565839cdd2075b34cef4bdaff71
Account: v1
Containers: 1
Objects: 1
Bytes: 10M
Connection: Keep-Alive
X-Timestamp: 1465357845.97638
X-Account-Bytes-Used-Actual: 10485760
X-Trans-Id: tx000000000000000000077-0057579615-5e32-default
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
But after a few mintues rgw returns double space usage and objects of account metadata.
StorageURL: http://192.168.62.50:8080/swift/v1
Auth Token: AUTH_rgwtk0b0000006d656c6f6e3a7377696674258a5e645be0252461e85857b32bf422ce1df90fff606d86a86798d2b89ab4fff626a0fe
Account: v1
Containers: 1
Objects: 2
Bytes: 20M
Connection: Keep-Alive
X-Timestamp: 1465358049.58936
X-Account-Bytes-Used-Actual: 20971520
X-Trans-Id: tx000000000000000000079-00575796e1-5e32-default
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
In the second stat command you have two objects, maybe someone uploaded another object?
can you do reproduce and also run list buckets?
This bug can reproduce on development environment.
$ swift -A http://localhost:8000/auth -U test:tester -K 'testing' stat --lh melon
Account: v1
Containers: 1
Objects: 2
Bytes: 20M
Connection: Keep-Alive
X-Timestamp: 1465895380.67945
X-Account-Bytes-Used-Actual: 20971520
X-Trans-Id: tx00000000000000000000c-00575fc9d4-101d-default
Content-Type: text/plain; charset=utf-8
Accept-Ranges: bytes
$ swift -A http://localhost:8000/auth -U test:tester -K 'testing' list --lh melon
10M 2016-06-14 09:04:04 tmp/10M
10M
The same issue before:
https://github.com/ceph/ceph/pull/6047
I wasnt able to reproduce the problem. I tried the following:
1. In my dev environment, created a container - container1
2. Uploaded an object of 16 bytes to container1
3. Ran the stat command more than once - in each run I could see the correct number of objects which is 1 and its size which is 16.
4. Ran the list command - the ouput of it lists the object once. The size which is listed at the end after the objects is the total size of all objects in the container like
$ swift -A http://localhost:8000/auth/1.0 -U test:tester -K testing list --lh container1
21 2016-06-17 07:05:10 None sample.txt
16 2016-06-17 07:03:58 None sample1.txt
21 2016-06-17 07:05:19 None sample2.txt
58
I also checked the PR mentioned in the comment above: If the needs_stats is set to false, then it supresseses the stats related to objects output by the swift 'stat' command. I am not sure how that will solve the problem that is mentioned in the ticket.
Albert Tu wrote:
This bug can reproduce on development environment.
[...]
[...]
The same issue before: https://github.com/ceph/ceph/pull/6047
Hi Albert, which branch are you working off? I tried the above using the master branch and couldn't see the issue in my dev environment.
Pritha Srivastava wrote:
Albert Tu wrote:
This bug can reproduce on development environment.
[...]
[...]
The same issue before: https://github.com/ceph/ceph/pull/6047
Hi Albert, which branch are you working off? I tried the above using the master branch and couldn't see the issue in my dev environment.
I'm using the master branch (87b5a65). Sorry for typo, the correct stat command shouldn't append container name.
Need to backport to hammer. Do I need to create issue or pull request?
- Status changed from New to Pending Backport
- Backport set to jewel
- Copied to Backport #16599: jewel: rgw: Swift API returns double space usage and objects of account metadata added
- Subject changed from jewel: rgw: Swift API returns double space usage and objects of account metadata to rgw: Swift API returns double space usage and objects of account metadata
jewel backport has been staged. Albert, the bug says affected version is 10.2.1 which is jewel. Are you sure this needs to be backported to hammer (0.94.x)?
Sure, because #13140 fixes this bug by don't call update_container_stats(). This made radosgw doesn't return immediate stats.
And after backported this, the list command gets fixed too.
- Status changed from Pending Backport to Resolved
Also available in: Atom
PDF