Ceph » RADOS

Steps to Reproduce::
1. creat cache iter of cephfs data pool.
2. copy a file to the mounted cephfs directory.
3. "ceph df" display the OBJECTS of cache pool is zero.
4. read the file with dirctio mode (skiping the client cache).
5. "ceph df" display the OBJECTS of cache pool is still zero.(expecting is objects of the writed files).

I also test reading the file which is exsiting before cache tier created. That can be promoted correctly.

After analysis the log and source code, I found the check condition of CACHEMODE_READONLY is defective.
When reading the file just writed, the r=0, !obc.get()=fasle and obc->obs.exists=0.
So, do_cache_redirect() is called in this situation, although which is the reading operation.

  case pg_pool_t::CACHEMODE_READONLY:
    // TODO: clean this case up
    if (!obc.get() && r == -ENOENT) {
      // we don't have the object and op's a read
      promote_object(obc, missing_oid, oloc, op, promote_obc);
      return cache_result_t::BLOCKED_PROMOTE;
    }
    if (!r) { // it must be a write
      do_cache_redirect(op);
      return cache_result_t::HANDLED_REDIRECT;
    }

I test it in V14.2.23, but the souce code of CACHEMODE_READONLY is not changed in master branch now.