Ceph

You don't need any special qemu options to use snapshots - the snapshot option is confusingly named. The qemu 'snapshot=on' parameter doesn't enable snapshots - it causes qemu to write to a temporary file instead of the actual disk image (rbd in this case). According to the man page, C-a s will make the data actually be written to the backend, but I haven't tested this with rbd. You don't need any special options to be able to use snapshots.

Hi Josh,

at least my experience showed a different behaviour: no reliable snapshots and even crashes of qemu-system*-processes after "loadvm/savevm" in the qemu-monitor. Snapshots with complete rollback only after "-drive... rbd:...,snapshot=on", sorry. Snapshots visable within monitor and rbd-command. Was implicit perhaps with qemu0.15.[01] running on another node, no problems there.
We are running qemu-1.0rc4 as of my writing. Without param no problems installing a distro, with an invalid partition-table occurs. So, there is a difference.

Thnx for you attention

Oliver.

There's certainly a difference with the snapshot parameter - it doesn't store anything in the rbd image unless you use the C-a s command. It only stores data in a temporary file, so the rbd image will be blank, resulting in the invalid partition error message. The snapshot parameter has nothing to do with rbd (or qcow2 or other disk format) snapshots. Are you using C-a s to save the data to rbd?

Right now rollback and deletion of snapshots aren't wired up in the qemu rbd driver (#390), you have to use the rbd command to do that.

For the loadvm/savevm problems, is this without snapshot=on? If so, can you add extra options to the -drive rbd section: 'rbd:pool/image:debug_rbd=20:log_to_stderr=2' and post the logs and commands leading up to the problem?

Thanks!

Hi Josh,

I have just made a session with savevm/loadvm, once without/with the snapshot-option, now with qemu-1.0. Without, the vm freezes, with, I can "see" the snapshots within monitor-cli via "info snapshots" and via rbd <pool>/<filename>. Everything as expected.
I will come back with a logfile with your debug-switches later. What I don't get is your comment regarding the "invalid partition".
Here my worklog:

0. rados mkpool foo, rbd --size ... create foo/bar.rbd
1. I start a VM with "snapshot=on"-parameter
2. I take the -drive as "/dev/vda" to be partitioned with a debian/ubuntu/suse/whatever distro
3. After complete install - and so even right after partitioning, too - install results in a
"unknown partition table".
4. sigh

If I leave out (1.), everything completes to my expectation. This is the more interesting
point. I did not do any snapshots while installation, of course.

Kind regards,

Oliver.

Hi Oliver,

With snapshot=on data is never saved to the backing device - the original file is not modified unless you use that special shortcut (although I'm not even sure that works - I couldn't get it working on a regular raw disk image). When you're using snapshot=on, your install is stored in a hidden file in $TMP_DIR, but never written to rbd. If you try it with a raw file instead of rbd I think you'll see the same problem.

When you create an rbd image, it is treated as all zeroes until data is written to it. With snapshot=on, rbd doesn't get written to, so the bios just reads all zeroes, which is not a valid partition table.

Thanks for the crash log, it looks like there's some race condition (maybe only with buffered io?) during shutdown. If you could get a core file ('ulimit -c unlimited' before reproducing the crash), then open that up in gdb and post the backtrace that'd be great. I'm not sure yet if the problem is librbd or the way qemu uses it.

Hi Oliver,

That gdb session is actually an entirely different crash - I'll take a closer look at both of these today. The backtrace you got for the second one may be good enough to figure it out. To make the backtraces more useful, you need to install debugging symbols (in librados2-dbg and librbd1-dbg packages) on the machine where you're running qemu. I'd still like a backtrace from the crash you saw when you were using the rbd_writeback_window, if possible. Just doing a gdb bt is enough to start with - from there I can ask for more specific info if needed.

The rbd command line tool has all the snapshot functionality in it, so it's what I recommend. The easiest way to create a snapshot is 'rbd snap create --snap <snapname> pool/image'. Qemu's savevm will do the same thing, but for all the disks in a guest (which you might not want, for example if one is swap).

Note that to make sure the guest filesystem is in a consistent state (doesn't need fsck) when you take a snapshot you need to use the fsfreeze command on the filesystem(s) you're snapshotting, or unmount them. Qemu has a guest agent to make fs freezing and thawing scriptable.

Thanks again for the logs!

Hi Josh,

well, the small fix does it, no more crashes.

But, of course I would love to have back my live-snapshot-capability. Online-snapshot, offline is working with rbd as you wrote. Is there anything I overlooked? If not, s/t in the pipeline?
Cause with our "other" setup with nfs-storage live-snapshots are working.

Thnx for your efforts so far,

Oliver.