Project

General

Profile

Actions

Feature #61334

open

cephfs-mirror: use snapdiff api for efficient tree traversal

Added by Venky Shankar 12 months ago. Updated about 1 month ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
Performance/Resource Usage
Target version:
% Done:

0%

Source:
Community (dev)
Tags:
backport_processed
Backport:
reef,squid
Reviewed:
Affected Versions:
Component(FS):
cephfs-mirror, tools
Labels (FS):
Pull request ID:

Description

With https://github.com/ceph/ceph/pull/43546 merged, cephfs-mirror can make use the snapdiff api (via readdir_snapdiff) to efficiently traverse the directory tree between two snapshots.

This should hugely improve performance when only a handful of files have changed between two consecutive snapshots.


Related issues 3 (2 open1 closed)

Blocked by CephFS - Bug #63099: cephfs-mirror: ceph_getxattr call always return -61 (ENODATA) in PeerReplayer::synchronizeResolvedJos Collin

Actions
Copied to CephFS - Backport #65222: reef: cephfs-mirror: use snapdiff api for efficient tree traversalNewJos CollinActions
Copied to CephFS - Backport #65223: squid: cephfs-mirror: use snapdiff api for efficient tree traversalNewJos CollinActions
Actions #1

Updated by Jos Collin 11 months ago

  • Assignee set to Jos Collin
Actions #2

Updated by Jos Collin 9 months ago

  • Description updated (diff)
  • Status changed from New to In Progress
Actions #3

Updated by Venky Shankar 9 months ago

Jos,

The crux of the changes will be in PeerReplayer::do_synchronize() which if you see does:

sync_stack.emplace(SyncEntry(".", tdirp, tstx)); <<< put "/" on the top of the stack

      while (true) {
        r = ceph_readdirplus_r(m_local_mount, entry.dirp, &de, &stx,
                               CEPH_STATX_MODE | CEPH_STATX_UID | CEPH_STATX_GID |
                               CEPH_STATX_SIZE | CEPH_STATX_ATIME | CEPH_STATX_MTIME,
                               AT_STATX_DONT_SYNC | AT_SYMLINK_NOFOLLOW, NULL);
        if (r < 0) {
          derr << ": failed to local read directory=" << entry.epath << dendl;
          break;
        }
        if (r == 0) {
          break;
        }

        auto d_name = std::string(de.d_name);
        if (d_name != "." && d_name != "..") {
          e_name = d_name;
          break;
        }
      }

which is going to traverse all indoes in the snapshot - note that the mirror daemon does not blindly sync every single inode as it checks the {m,c}time against the "prev"ious snapshot inode. But, the recursive traversal is hitting every single inode and that's what the readdir_diff call would avoid.

Actions #4

Updated by Jos Collin 7 months ago

  • Blocked by Bug #63099: cephfs-mirror: ceph_getxattr call always return -61 (ENODATA) in PeerReplayer::synchronize added
Actions #5

Updated by Jos Collin 5 months ago

  • Pull request ID set to 54633
Actions #6

Updated by Jos Collin 5 months ago

  • Status changed from In Progress to Fix Under Review
  • Source set to Community (dev)
Actions #7

Updated by Venky Shankar about 1 month ago

  • Status changed from Fix Under Review to Resolved

Not a candidate for backport right now to allow more bake time in main branch.

Actions #8

Updated by Jos Collin about 1 month ago

Venky Shankar wrote:

Not a candidate for backport right now to allow more bake time in main branch.

What about this BZ 7.1z1 then: https://bugzilla.redhat.com/show_bug.cgi?id=2214930?

Actions #9

Updated by Venky Shankar about 1 month ago

Jos Collin wrote:

Venky Shankar wrote:

Not a candidate for backport right now to allow more bake time in main branch.

What about this BZ 7.1z1 then: https://bugzilla.redhat.com/show_bug.cgi?id=2214930?

I'm aware of that. We need to clear some other test failures in fs:mirror that we are seeing.

Actions #10

Updated by Venky Shankar about 1 month ago

  • Status changed from Resolved to Pending Backport
  • Target version changed from v19.0.0 to v20.0.0
  • Backport set to reef,squid

Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and letting the code bake a bit more). Makes sense?

Actions #11

Updated by Backport Bot about 1 month ago

  • Copied to Backport #65222: reef: cephfs-mirror: use snapdiff api for efficient tree traversal added
Actions #12

Updated by Backport Bot about 1 month ago

  • Copied to Backport #65223: squid: cephfs-mirror: use snapdiff api for efficient tree traversal added
Actions #13

Updated by Backport Bot about 1 month ago

  • Tags set to backport_processed
Actions #14

Updated by Jos Collin about 1 month ago

Venky Shankar wrote:

Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and letting the code bake a bit more). Makes sense?

ok

Actions

Also available in: Atom PDF