Project

General

Profile

Actions

Backport #23864

closed

luminous: compression ratio depends on block size, which is much smaller (16K vs 4M) in multisite sync

Added by Nathan Cutler almost 6 years ago. Updated almost 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
Release:
luminous
Pull request ID:
Crash signature (v1):
Crash signature (v2):


Related issues 1 (0 open1 closed)

Copied from rgw - Bug #23547: compression ratio depends on block size, which is much smaller (16K vs 4M) in multisite syncResolvedCasey Bodley04/03/2018

Actions
Actions #1

Updated by Nathan Cutler almost 6 years ago

  • Copied from Bug #23547: compression ratio depends on block size, which is much smaller (16K vs 4M) in multisite sync added
Actions #2

Updated by Prashant D almost 6 years ago

  • Assignee set to Prashant D

Getting below compilation errors while backporting this PR, so need to backport 2 more commits to get this PR into luminous:

[ 97%] Building CXX object src/rgw/CMakeFiles/rgw_a.dir/rgw_rest_log.cc.o
[ 97%] Building CXX object src/rgw/CMakeFiles/rgw_a.dir/rgw_rest_metadata.cc.o
In file included from /home/pdvian/backport/ceph1_lum/src/rgw/rgw_realm_watcher.h:8:0,
from /home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc:72:
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc: In constructor ‘RGWPutObj_Buffer::RGWPutObj_Buffer(RGWPutObjDataProcessor*, unsigned int)’:
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc:7326:12: error: ‘isp2’ was not declared in this scope
assert(isp2(buffer_size)); // must be power of 2
^
/home/pdvian/backport/ceph1_lum/src/include/assert.h:125:5: note: in definition of macro ‘assert’
((expr) \
^~~
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc:7326:12: note: suggested alternative: ‘dup2’
assert(isp2(buffer_size)); // must be power of 2
^
/home/pdvian/backport/ceph1_lum/src/include/assert.h:125:5: note: in definition of macro ‘assert’
((expr) \
^
~~
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc: In member function ‘virtual int RGWPutObj_Buffer::handle_data(ceph::bufferlist&, off_t, void**, rgw_raw_obj*, bool*)’:
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc:7342:24: error: ‘p2align’ was not declared in this scope
const auto count = p2align(buffer.length(), buffer_size);
^~~~~
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc:7342:24: note: suggested alternative: ‘memalign’
const auto count = p2align(buffer.length(), buffer_size);
^~~~~
memalign
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc: In member function ‘virtual int RGWRadosPutObj::flush()’:
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc:7476:46: error: ‘ofs’ was not declared in this scope
return put_data_and_throttle(filter, bl, ofs, false);
^~
/home/pdvian/backport/ceph1_lum/src/rgw/rgw_rados.cc:7476:46: note: suggested alternative: ‘ffs’
return put_data_and_throttle(filter, bl, ofs, false);
^
~
ffs
make3: * [src/rgw/CMakeFiles/rgw_a.dir/build.make:1339: src/rgw/CMakeFiles/rgw_a.dir/rgw_rados.cc.o] Error 1
make3:
Waiting for unfinished jobs....
make2:
[CMakeFiles/Makefile2:23127: src/rgw/CMakeFiles/rgw_a.dir/all] Error 2
make1:
[CMakeFiles/Makefile2:234: CMakeFiles/tests.dir/rule] Error 2
make: *
* [Makefile:175: tests] Error 2

Need to include below commits to backport this PR:

$ git log -n 2 c06b97b3
commit c06b97b3d7e36b5b3be38f1ffa121611bea61a52
Author: Adam C. Emerson <>
Date: Wed Jan 10 17:06:53 2018 -0500

include: Add templates along side macros in intarith
The macros can be removed later.
Signed-off-by: Adam C. Emerson &lt;&gt;

commit 4da29a07cfc4c7c3e6fb499f4f7ddac307292ca0

$ git log -n1 b2143cded
commit b2143cded0e971361cdb089db19a6f69ce5b74dd
Author: Yehuda Sadeh <>
Date: Fri Nov 3 15:57:56 2017 -0700

rgw: rest_client: work towards throttling of http read requests
Adjust the interfaces to provide the ability for the read callback
to pause the reads. While doing that, define a new class interface
for this instead of RGWGetDataCB. This had a butterfly effect that
required modifications to the obj read filters, but the end result
is a bit cleaner.
Signed-off-by: Yehuda Sadeh &lt;&gt;
Actions #3

Updated by Prashant D almost 6 years ago

  • Status changed from New to Need More Info
Actions #4

Updated by Casey Bodley almost 6 years ago

thanks Prashant. i opened https://github.com/ceph/ceph/pull/21758 which resolves the conflicts without additional backports

Actions #5

Updated by Nathan Cutler almost 6 years ago

  • Description updated (diff)
  • Status changed from Need More Info to In Progress
  • Assignee changed from Prashant D to Casey Bodley
Actions #6

Updated by Yuri Weinstein almost 6 years ago

Nathan Cutler wrote:

https://github.com/ceph/ceph/pull/21758

mergedhttps://github.com/ceph/ceph/pull/21792

Actions #7

Updated by Nathan Cutler almost 6 years ago

  • Status changed from In Progress to Resolved
  • Target version set to v12.2.6
Actions

Also available in: Atom PDF