Ceph

Hi Jeff,

What version are you using? A commit id would be best.

C.

Looks like rbdtool goes wacky when you try to create an image of more than 4200 MB (or so). At least on 32 bit.

fixed by f294b26746a30960971757c42d679208a99995a7

Hi ,
I verified this issue,
use master commit: 4ee75a881ec637e2b0c5b74b16b1e44ac710707c

export file size > 2Gb ,still fail.logs as the below.

./src/rbd export 10GB ./10GB-test -p foo

.....................................
3 writing 4194304 bytes at ofs 2130706432
3 writing 4194304 bytes at ofs 2134900736
3 writing 4194304 bytes at ofs 2139095040
3 writing 4194304 bytes at ofs 2143289344
1 writing 4194304 bytes at ofs 18446744071562067968
export error: Invalid argument

static int export_read_cb(uint64_t ofs, size_t len, const char *buf, void *arg) {
int ret;
int fd = *(int *)arg;

if (!buf) /* a hole */
    return 0;

ret = lseek64(fd, ofs, SEEK_SET);
  if (ret < 0) {
  cerr << "1 writing " << len << " bytes at ofs " << ofs << std::endl;
    return -errno; }
  ret = write(fd, buf, len);
  if (ret < 0) {
  cerr << "2 writing " << len << " bytes at ofs " << ofs << std::endl;
    return -errno; }

cerr << "3 writing " << len << " bytes at ofs " << ofs << std::endl;
  return 0;
}

$ ./src/rbd ls -p foo
10GB

10GB is a file created by " dd if=/dev/zero of=./10GB bs=1M count=10240" at local disk,
then import it to foo pool by rbd.

$ uname -a
Linux cephhost 2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 02:41:37 UTC 2010 x86_64 GNU/Linux

hi ,

commit: 4ee75a881ec637e2b0c5b74b16b1e44ac710707c

modify
librbd.cc:read_iterate:line 1024 :

int total_read = 0;
to 
  uint64_t total_read = 0;

that can fix this issue.

int read_iterate(ImageCtx *ictx, uint64_t off, size_t len,
int (*cb)(uint64_t, size_t, const char *, void *),
void *arg) {
int r = ictx_check(ictx);
if (r < 0)
return r;

int64_t ret;
  uint64_t total_read = 0;
.............

}

librbd.cc:line 1217,
modify
int total_write = 0;
to
to uint64_t total_write = 0;

int aio_write(ImageCtx *ictx, uint64_t off, size_t len, const char *buf,
AioCompletion *c) {
if (!len)
return 0;

int r = ictx_check(ictx);
  if (r < 0)
    return r;

int total_write = 0;
........
}

Hi Jeff,

I've pushed a few different patches cleaning up these type sizes. It's working on our end.. can verify that it's fixed for you?

Thanks!

for what it's worth, I got these results at ca613786f84d44f352fa4fee4ebd57f4b1ec4cdd

cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ./rbd create -s 10000 10GB
cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ./rbd info 10GB
rbd image '10GB':
        size 10000 MB in 2500 objects
        order 22 (4096 KB objects)
        block_name_prefix: rb.0.0
        parent:  (pool -1)
cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ./rbd export 10GB ./10GB.img
cmccabe@build-ubuntu1010-i386:~/src/ceph/src$ ls -lh ./10GB.img 
-rw-r--r-- 1 cmccabe cmccabe 9.8G 2011-03-15 13:20 ./10GB.img

Hi ,
i git commit 22241f8daea36df602c9069c311f59e22670918c,
took the following steps to this issue.which should be fixed. thanks.

$ uname -a
Linux cephhost 2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 02:41:37 UTC 2010 x86_64 GNU/Linux

$ du -sh xpsp3
4.7G xpsp3

$ ./rbd import ./source/xpsp3.img xpsp3 --dest-pool foo
............................................
reading 4194304 bytes at offset 4953473024
reading 4194304 bytes at offset 4957667328
reading 4194304 bytes at offset 4961861632
reading 4194304 bytes at offset 4966055936
reading 4194304 bytes at offset 4970250240
reading 4194304 bytes at offset 4974444544
rbd import file_pos=4978638848 extent_len=1179648
reading 1179648 bytes at offset 4978638848

$ ./rbd ls -p foo
10GB
1GB
xpsp3
$./rbd export xpsp3 ./xpsp3 -p foo
..........................
writing 4194304 bytes at ofs 4953473024
writing 4194304 bytes at ofs 4957667328
writing 4194304 bytes at ofs 4961861632
writing 4194304 bytes at ofs 4966055936
writing 4194304 bytes at ofs 4970250240
writing 4194304 bytes at ofs 4974444544
writing 1179648 bytes at ofs 4978638848