Feature #8475
erasure-code: oversized objects when using the Cauchy technique
100%
Description
When the Cauchy technique is used, the alignment constraint and the default packetsize will inflate a 4KB object to 16KB
Related issues
Associated revisions
erasure-code: implement alignment on chunk sizes
jerasure expects chunk sizes that are aligned on the largest possible
vector size that could be used by SSE instructions, when available (
LARGEST_VECTOR_WORDSIZE == 16 bytes ).
For techniques derived from Cauchy, encoding and decoding is done by
subdividing the chunk into packets of packetsize bytes. The operations
are done w * packetsize bytes at a time. It follows that each chunk must
have a size that is a multiple of w * packetsize bytes.
For techniques derived from Vandermonde, it is enough for a chunk to be
a multiple of w * LARGEST_VECTOR_WORDSIZE.
ErasureCodeJerasure::get_alignment returns a size alignment constraint
that has to be enforced as a multiple of the object size. The resulting
object size then has to match the chunk constraints described above
although they have no relationship with K. For Cauchy, it leads to
excessive padding, making it impossible to set sensible parameters for
when the object size is small.
When the per_chunk_alignement data member is true, the semantic of
ErasureCodeJerasure::get_alignment is changed to return a size alignment
constraint to be enforced as a multiple of the chunk size. The
ErasureCodeJerasure::get_chunk_size method is modified to use the new
semantic when appropriate.
The jerasure-per-chunk-alignement parameter is parsed to set
per_chunk_alignement for the Vandermonde and Cauchy techniques.
The memory address of a chunk is implicitly aligned to a page boundary
because it is allocated with buffer::create_page_aligned.
http://tracker.ceph.com/issues/8475 Fixes: #8475
Signed-off-by: Loic Dachary <loic@dachary.org>
History
#2 Updated by Loïc Dachary almost 9 years ago
- Status changed from Fix Under Review to 7
#3 Updated by Loïc Dachary almost 9 years ago
#4 Updated by Loïc Dachary almost 9 years ago
- Status changed from 7 to Fix Under Review
- % Done changed from 80 to 90
the rados test suite is ok with the exception of 5 failure (none of which involve a configuration that has ec_pool in it) and one job stuck (no ec_pool either).
#5 Updated by Loïc Dachary over 8 years ago
- Tracker changed from Bug to Feature
- Target version set to 0.82
#6 Updated by Ian Colle over 8 years ago
- Target version changed from 0.82 to 0.83 cont.
#7 Updated by Loïc Dachary over 8 years ago
upgrade tests mixed cluster with and without
#8 Updated by Loïc Dachary over 8 years ago
- Status changed from Fix Under Review to 7
#9 Updated by Loïc Dachary over 8 years ago
- Status changed from 7 to Fix Under Review
upgrade/firefly-x/stress-split are working, except for what appears to be an unrelated rgw failure (http://tracker.ceph.com/issues/8698 ).
#10 Updated by Loïc Dachary over 8 years ago
- File upgrade.txt View added
The erasure code workload has been added to firefly-x and tested with one job : upgrade.txt
#11 Updated by Loïc Dachary over 8 years ago
- Status changed from Fix Under Review to 7
A firefly-x suite has been scheduled which includes the erasure coded workload to confirm that it succeeds.
#12 Updated by Samuel Just over 8 years ago
- Target version changed from 0.83 cont. to 0.84
#13 Updated by Loïc Dachary over 8 years ago
Rebased and repushed , running gitbuilder
#14 Updated by Loïc Dachary over 8 years ago
scheduled upgrade:firefly-x:stress-split on plana to only use distro ubuntu
#15 Updated by Loïc Dachary over 8 years ago
- Status changed from 7 to Resolved
- % Done changed from 90 to 100