Project

General

Profile

Bug #876

rbd export rbd image > 2GB, export error: Invalid argument

Added by changping Wu over 9 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
librbd
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

Hi ,
reproduce steps:
1).ceph v0.25.

2) Imported a 10GB image into foo pool
$ rbd ls -p foo
10GBB
10GBrbd
3) export 10GB rbd image

$rbd export 10GBB ./test/10GBBBBB -p foo

logs:

....................................
writing 4194304 bytes at ofs 2105540608
writing 4194304 bytes at ofs 2109734912
writing 4194304 bytes at ofs 2113929216
writing 4194304 bytes at ofs 2118123520
writing 4194304 bytes at ofs 2122317824
writing 4194304 bytes at ofs 2126512128
writing 4194304 bytes at ofs 2130706432
writing 4194304 bytes at ofs 2134900736
writing 4194304 bytes at ofs 2139095040
writing 4194304 bytes at ofs 2143289344
export error: Invalid argument

4) lseek only support < 2GB file.

Jeff,wu

History

#1 Updated by Sage Weil over 9 years ago

  • Category set to librbd
  • Priority changed from Normal to High
  • Target version set to v0.25.1

#2 Updated by Colin McCabe over 9 years ago

  • Assignee set to Colin McCabe

#3 Updated by Colin McCabe over 9 years ago

Hi Jeff,

What version are you using? A commit id would be best.

C.

#4 Updated by Colin McCabe over 9 years ago

Looks like rbdtool goes wacky when you try to create an image of more than 4200 MB (or so). At least on 32 bit.

#6 Updated by Colin McCabe over 9 years ago

  • Status changed from New to Resolved

#7 Updated by changping Wu over 9 years ago

Hi ,
I verified this issue,
use master commit: 4ee75a881ec637e2b0c5b74b16b1e44ac710707c

export file size > 2Gb ,still fail.logs as the below.

./src/rbd export 10GB ./10GB-test -p foo

.....................................
3 writing 4194304 bytes at ofs 2130706432
3 writing 4194304 bytes at ofs 2134900736
3 writing 4194304 bytes at ofs 2139095040
3 writing 4194304 bytes at ofs 2143289344
1 writing 4194304 bytes at ofs 18446744071562067968
export error: Invalid argument


static int export_read_cb(uint64_t ofs, size_t len, const char *buf, void *arg) {
int ret;
int fd = *(int *)arg;

if (!buf) /* a hole */
return 0;
ret = lseek64(fd, ofs, SEEK_SET);
if (ret < 0) {
cerr << "1 writing " << len << " bytes at ofs " << ofs << std::endl;
return -errno; }
ret = write(fd, buf, len);
if (ret < 0) {
cerr << "2 writing " << len << " bytes at ofs " << ofs << std::endl;
return -errno; }
cerr << "3 writing " << len << " bytes at ofs " << ofs << std::endl;
return 0;
}

#8 Updated by changping Wu over 9 years ago

$ ./src/rbd ls -p foo
10GB

10GB is a file created by " dd if=/dev/zero of=./10GB bs=1M count=10240" at local disk,
then import it to foo pool by rbd.

#9 Updated by changping Wu over 9 years ago

$ uname -a
Linux cephhost 2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 02:41:37 UTC 2010 x86_64 GNU/Linux

#10 Updated by changping Wu over 9 years ago

hi ,

commit: 4ee75a881ec637e2b0c5b74b16b1e44ac710707c

modify
librbd.cc:read_iterate:line 1024 :

int total_read = 0;
to
uint64_t total_read = 0;

that can fix this issue.

int read_iterate(ImageCtx *ictx, uint64_t off, size_t len,
int (*cb)(uint64_t, size_t, const char *, void *),
void *arg) {
int r = ictx_check(ictx);
if (r < 0)
return r;

int64_t ret;
uint64_t total_read = 0;
.............

}

#11 Updated by changping Wu over 9 years ago

librbd.cc:line 1217,
modify
int total_write = 0;
to
to uint64_t total_write = 0;

int aio_write(ImageCtx *ictx, uint64_t off, size_t len, const char *buf,
AioCompletion *c) {
if (!len)
return 0;

int r = ictx_check(ictx);
if (r < 0)
return r;

int total_write = 0;
........
}

#12 Updated by Sage Weil over 9 years ago

Hi Jeff,

I've pushed a few different patches cleaning up these type sizes. It's working on our end.. can verify that it's fixed for you?

Thanks!

#13 Updated by Colin McCabe over 9 years ago

for what it's worth, I got these results at ca613786f84d44f352fa4fee4ebd57f4b1ec4cdd

cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ./rbd create -s 10000 10GB
cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ./rbd info 10GB
rbd image '10GB':
        size 10000 MB in 2500 objects
        order 22 (4096 KB objects)
        block_name_prefix: rb.0.0
        parent:  (pool -1)
cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ./rbd export 10GB ./10GB.img
cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ls -lh ./10GB.img 
-rw-r--r-- 1 cmccabe cmccabe 9.8G 2011-03-15 13:20 ./10GB.img

#14 Updated by changping Wu over 9 years ago

Hi ,
i git commit 22241f8daea36df602c9069c311f59e22670918c,
took the following steps to this issue.which should be fixed. thanks.

$ uname -a
Linux cephhost 2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 02:41:37 UTC 2010 x86_64 GNU/Linux

$ du -sh xpsp3
4.7G xpsp3

$ ./rbd import ./source/xpsp3.img xpsp3 --dest-pool foo
............................................
reading 4194304 bytes at offset 4953473024
reading 4194304 bytes at offset 4957667328
reading 4194304 bytes at offset 4961861632
reading 4194304 bytes at offset 4966055936
reading 4194304 bytes at offset 4970250240
reading 4194304 bytes at offset 4974444544
rbd import file_pos=4978638848 extent_len=1179648
reading 1179648 bytes at offset 4978638848

$ ./rbd ls -p foo
10GB
1GB
xpsp3
$./rbd export xpsp3 ./xpsp3 -p foo
..........................
writing 4194304 bytes at ofs 4953473024
writing 4194304 bytes at ofs 4957667328
writing 4194304 bytes at ofs 4961861632
writing 4194304 bytes at ofs 4966055936
writing 4194304 bytes at ofs 4970250240
writing 4194304 bytes at ofs 4974444544
writing 1179648 bytes at ofs 4978638848

Also available in: Atom PDF