Feature #61334
open
cephfs-mirror: use snapdiff api for efficient tree traversal
Added by Venky Shankar 12 months ago.
Updated about 2 months ago.
Category:
Performance/Resource Usage
Component(FS):
cephfs-mirror, tools
Description
With https://github.com/ceph/ceph/pull/43546 merged, cephfs-mirror can make use the snapdiff api (via readdir_snapdiff) to efficiently traverse the directory tree between two snapshots.
This should hugely improve performance when only a handful of files have changed between two consecutive snapshots.
- Assignee set to Jos Collin
- Description updated (diff)
- Status changed from New to In Progress
Jos,
The crux of the changes will be in PeerReplayer::do_synchronize() which if you see does:
sync_stack.emplace(SyncEntry(".", tdirp, tstx)); <<< put "/" on the top of the stack
while (true) {
r = ceph_readdirplus_r(m_local_mount, entry.dirp, &de, &stx,
CEPH_STATX_MODE | CEPH_STATX_UID | CEPH_STATX_GID |
CEPH_STATX_SIZE | CEPH_STATX_ATIME | CEPH_STATX_MTIME,
AT_STATX_DONT_SYNC | AT_SYMLINK_NOFOLLOW, NULL);
if (r < 0) {
derr << ": failed to local read directory=" << entry.epath << dendl;
break;
}
if (r == 0) {
break;
}
auto d_name = std::string(de.d_name);
if (d_name != "." && d_name != "..") {
e_name = d_name;
break;
}
}
which is going to traverse all indoes in the snapshot - note that the mirror daemon does not blindly sync every single inode as it checks the {m,c}time against the "prev"ious snapshot inode. But, the recursive traversal is hitting every single inode and that's what the readdir_diff call would avoid.
- Blocked by Bug #63099: cephfs-mirror: ceph_getxattr call always return -61 (ENODATA) in PeerReplayer::synchronize added
- Pull request ID set to 54633
- Status changed from In Progress to Fix Under Review
- Source set to Community (dev)
- Status changed from Fix Under Review to Resolved
Not a candidate for backport right now to allow more bake time in main branch.
Jos Collin wrote:
Venky Shankar wrote:
Not a candidate for backport right now to allow more bake time in main branch.
What about this BZ 7.1z1 then: https://bugzilla.redhat.com/show_bug.cgi?id=2214930?
I'm aware of that. We need to clear some other test failures in fs:mirror that we are seeing.
- Status changed from Resolved to Pending Backport
- Target version changed from v19.0.0 to v20.0.0
- Backport set to reef,squid
Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and letting the code bake a bit more). Makes sense?
- Copied to Backport #65222: reef: cephfs-mirror: use snapdiff api for efficient tree traversal added
- Copied to Backport #65223: squid: cephfs-mirror: use snapdiff api for efficient tree traversal added
- Tags set to backport_processed
Venky Shankar wrote:
Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and letting the code bake a bit more). Makes sense?
ok
Also available in: Atom
PDF