Ceph » CephFS

please try below patch. If it still doesn't work for you(there is no dirty caps), you need to implement mechanism that explicitly asks mds to flush mdlog (modify both client and mds)

diff --git a/src/client/Client.cc b/src/client/Client.cc
index 0a55e73977..aae1a49bd3 100644
--- a/src/client/Client.cc
+++ b/src/client/Client.cc
@@ -5796,6 +5796,7 @@ void Client::unmount()

   ldout(cct, 2) << "unmounting" << dendl;
   unmounting = true;
+  flush_caps_sync();

   while (!mds_requests.empty()) {
     ldout(cct, 10) << "waiting on " << mds_requests.size() << " requests" << dendl;

Zheng Yan wrote:

please try below patch. If it still doesn't work for you(there is no dirty caps), you need to implement mechanism that explicitly asks mds to flush mdlog (modify both client and mds)

[...]

Hi Yan,

I have applied your patch and looks like it worked. would you please explain to me why your solution would work ? cause 
i thought "flush_caps_sync" would only flush the caps, why it would also flush the journal? 
   would this be a final solution for this issue ?

Regards,
Dongdong

the last cap message has CHECK_CAPS_SYNCHRONOUS flag. mds flushes mdlog when it sees the flag. The patch isn't perfect. It does not work for 'no dirty caps' case.

Zheng Yan wrote:

the last cap message has CHECK_CAPS_SYNCHRONOUS flag. mds flushes mdlog when it sees the flag. The patch isn't perfect. It does not work for 'no dirty caps' case.

Thanks yan for your quick reply, i think the perfect solution is to implement a mechanism that explicitly asks mds to flush mdlog
as you told me before. below is my code to implement that mechanism in client side, please take a look and kindly give your advise:

void Client::flush_mdlog_sync()
{
+
+ for (map<mds_rank_t,MetaSession*>::iterator p = mds_sessions.begin();
+ p != mds_sessions.end();
+ +p) {
MetaSession *s = p->second;
+ flush_mdlog(s);
}

void Client::flush_mdlog(MetaSession *session)
{
+ MClientSession *m = new MClientSession(CEPH_SESSION_REQUEST_FLUSH_MDLOG);
+ session->con->send_message(m);
}

void Client::unmount() {
Mutex::Locker lock(client_lock);
@ -5702,9 +5718,10 @ void Client::unmount()

ldout(cct, 2) << "unmounting" << dendl;
   unmounting = true;
-
+ 
   while (!mds_requests.empty()) {
     ldout(cct, 10) << "waiting on " << mds_requests.size() << " requests" << dendl;
+    flush_mdlog_sync(); 
     mount_cond.Wait(client_lock);
   }

as for the mds side:

diff --git a/src/mds/Server.cc b/src/mds/Server.cc
index 1dc5ca2..dea22a3 100644
--- a/src/mds/Server.cc
+++ b/src/mds/Server.cc
@ -390,6 +390,10 @ void Server::handle_client_session(MClientSession *m)
finish_flush_session(session, m->get_seq());
break;

+ case CEPH_SESSION_REQUEST_FLUSH_MDLOG:
+ mdlog->flush();
+ break;
+
default:
ceph_abort();
}

Regards
Dongdong

dongdong tao wrote:

Zheng Yan wrote:

the last cap message has CHECK_CAPS_SYNCHRONOUS flag. mds flushes mdlog when it sees the flag. The patch isn't perfect. It does not work for 'no dirty caps' case.

Thanks yan for your quick reply, i think the perfect solution is to implement a mechanism that explicitly asks mds to flush mdlog
as you told me before. below is my code to implement that mechanism in client side, please take a look and kindly give your advise:

void Client::flush_mdlog_sync()
{
+
+ for (map<mds_rank_t,MetaSession*>::iterator p = mds_sessions.begin();
+ p != mds_sessions.end();
+ +p) {
MetaSession *s = p->second;
+ flush_mdlog(s);
}

void Client::flush_mdlog(MetaSession *session)
{
+ MClientSession *m = new MClientSession(CEPH_SESSION_REQUEST_FLUSH_MDLOG);
+ session->con->send_message(m);
}

void Client::unmount() {
Mutex::Locker lock(client_lock);
@ -5702,9 +5718,10 @ void Client::unmount()

ldout(cct, 2) << "unmounting" << dendl;
unmounting = true;
-
+
while (!mds_requests.empty()) {
ldout(cct, 10) << "waiting on " << mds_requests.size() << " requests" << dendl;
+ flush_mdlog_sync();
mount_cond.Wait(client_lock);
}

as for the mds side:

diff --git a/src/mds/Server.cc b/src/mds/Server.cc
index 1dc5ca2..dea22a3 100644
--- a/src/mds/Server.cc
+++ b/src/mds/Server.cc
@ -390,6 +390,10 @ void Server::handle_client_session(MClientSession *m)
finish_flush_session(session, m->get_seq());
break;

+ case CEPH_SESSION_REQUEST_FLUSH_MDLOG:
+ mdlog->flush();
+ break;
+
default:
ceph_abort();
}

Regards
Dongdong

It should work.

dongdong tao -- could you please open a pull request with your code change once it is working for you?

John Spray wrote:

dongdong tao -- could you please open a pull request with your code change once it is working for you?

ok, John, i will pull a request ASAP.

pull requests: https://github.com/ceph/ceph/pull/15544

John Spray wrote:

dongdong tao -- could you please open a pull request with your code change once it is working for you?

John, is there any thing else i need to do here for this PR?