Project

General

Profile

Actions
Copy link

Bug #44850

closed

sync on libcephfs and wait for CEPH_CAP_OP_FLUSH_ACK

Added by Jan Fajerski about 4 years ago. Updated about 4 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Client, ceph-fuse, libcephfs
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Opening this due to a conversation on the issue this one was copied from (https://tracker.ceph.com/issues/44744). The relevant question was raised by Luis initially:

when doing a full sync (i.e. when ->sync_fs() is executed), the client will wait for the MDS to send the CEPH_CAP_OP_FLUSH_ACK, acknowledging that everything is safe (in the journal, I believe). And this message is handled in Locker::handle_client_caps() (called from MDSRank::handle_deferrable_message), where it is queued for processing. Finally, it is later sent out to the client from Locker::file_update_finish(). And the time between being queued and sent corresponds to the delay experienced in the kernel client.
The fuse client does not wait for this FLUSH_ACK message from the MDS when 'sync' is executed. In fact, it even allows the filesystem to be umounted before receiving this ACK.

Jeff has proposed a fix to speed up the kernel client.

The question here is if libcephfs clients are actually correct in not waiting for CEPH_CAP_OP_FLUSH_ACK.

Files

Download all files
0001-ceph-request-expedited-service-when-flushing-caps.patch (1.13 KB) 0001-ceph-request-expedited-service-when-flushing-caps.patch Jeff Layton, 03/30/2020 03:50 PM
0001-ceph-request-expedited-service-when-flushing-caps.patch (1.13 KB) 0001-ceph-request-expedited-service-when-flushing-caps.patch Jeff Layton, 03/30/2020 03:51 PM

Related issues 1 (0 open — 1 closed)

Copied from Linux kernel client - Bug #44744: Slow file creation/sync on kernel cephfsResolvedJeff Layton

Actions
Actions
Copy link
 #1

Updated by Jan Fajerski about 4 years ago

  • Copied from Bug #44744: Slow file creation/sync on kernel cephfs added
Actions
Copy link
 #2

Updated by Greg Farnum about 4 years ago

Hmm, looking at the Client, I see:
  • Client::sync_fs() calls Client::_sync_fs()
  • Client::_sync_fs() issues calls wait_unsafe_requests and wait_sync_caps(flush_tid) after flush_caps_sync().

What is that missing?

Actions
Copy link
 #3

Updated by Greg Farnum about 4 years ago

Is it possible that either FUSE or libcephfs allow you to interrupt an in-progress sync when umount is issued and that's the only/actual problem?

Actions
Copy link
 #4

Updated by Luis Henriques about 4 years ago

Greg Farnum wrote:

Hmm, looking at the Client, I see:
  • Client::sync_fs() calls Client::_sync_fs()
  • Client::_sync_fs() issues calls wait_unsafe_requests and wait_sync_caps(flush_tid) after flush_caps_sync().

What is that missing?

Ahem! I believe I have caused this confusion.

I had completely failed to see the sync flag couldn't see where in the code the fuse client was waiting for the FLUSH_ACK from the MDS. Apparently, I'm wrong and mislead Jan in this conclusion as well. Sorry for the mess.

Actions
Copy link
 #5

Updated by Jan Fajerski about 4 years ago

  • Status changed from New to Rejected

What is that missing?

Nothing as it turns out. We weren't sure in the other bug discussion, so I opted to open this potentially invalid item. Closing.

Actions
Copy link

Also available in: Atom PDF