Ceph » Linux kernel client

Sometimes stuck for longer than 2 mins even 30 minutes after last append to file new append to file do not updating timestamp no mather how long waiting

I can confirm the behavior. I'll note that the time does seem to be updated after the I/O stops, but that's not really the level of cache-coherency that we're going for here. This involves some of the less-traveled paths of the kcephfs client.

In this case, the clients basically don't have the necessary caps to cache the attributes and they are continually issuing GETATTR calls to the MDS. Meanwhile, the client is doing uncached writes since it doesn't have the Fb caps, which are needed to use the pagecache.

I'll have to do a bit of debugging and observation to figure out the cause.

Basically, the mtime only freezes once there are competing clients writing to the file. If you kill one of the writers then the clients will start seeing mtime updates again once the writer is using cached I/O.

So, I think the problem is confined to the uncached codepaths in the kclient. Still looking at the cause. It's not clear to me how the client is intended to update the MDS as to the new mtime, when it doesn't have the caps to flush back an update for that field. Are we supposed to do a SETATTR?

I've been looking at this today. The problem seems to be in the order of operations by the client. It requests Fw caps fairly late in the process, after the inode's i_mtime has already been reset to the current time. We then end up requesting Fw caps from the MDS and that cap update clobbers the mtime. I think the right fix is to ensure that we request Fw caps before doing any changes to the inode, so that we're certain of the point from which the change is being made.