Ceph » rgw

RFC PR: https://github.com/ceph/ceph/pull/54554

Hi

can you explain what it should count? for what purpose?

the s3select request is a get-request with additional processing (SQL computation)
upon TXT objects such as CSV & JSON, it scanned the whole objects (thus processed-bytes = object-size, like a get-request)

as for the return bytes, it depends on the SQL result.

it must be noted that RGW/s3select contains counters, and it returns them to the client side(boto3 client).
the `aws_response_handler` object responsible for that.

as for Paequet, it is quite different, the s3select engine is processing only parts of the object according to the SQL statement.
thus the processed-bytes is something the engine "knows" (in practice, it is an s3-get-range-request)

Regarding the AWS docs (https://aws.amazon.com/s3/pricing/ - tab "Transform & query") there needs to be an option to know the total "Data scanned" and "Data returned" per bucket and user for billing purposes (the rgw usage log purpose basically).
So regardless of the response body, we need to count them in the cluster for later usage reports.