Ceph » rgw

radosgw-admin sync status return this:
realm a4cf2cc1-ce25-49d2-b5de-ffd491e35f7f (Sensetime)
zonegroup 7e478f3f-e2c4-4d9c-902e-95252eaea580 (SENSETIME)
zone d7c248d7-b1f0-4cd0-b4ea-de5a0be675a7 (sensetime.sz)
metadata sync syncing
full sync: 0/64 shards
incremental sync: 64/64 shards
metadata is behind on 4 shards
data sync source: 3c674147-dd68-464a-9d98-354b285ea139 (sensetime.bj)
syncing
full sync: 0/128 shards
incremental sync: 128/128 shards
data is caught up with source
source: c77ed05f-bd51-49e6-be79-0077c3376060 (sensetime.sh)
syncing
full sync: 0/128 shards
incremental sync: 128/128 shards
data is caught up with source

We got this 'metadata behind' wanring after some rgw restart.
At first we thought it's ok, because we observed some rgw thread pool timeout, and hoping the metadata will finally catch up with master when rgw is under low workloads. But after days waiting, the warning is still there.
We list the master's shard object storing meta logs, and find the log is empty, it has been trimmed!
Has anyone experienced this problem? Any suggestion is appreciated!