Project

General

Profile

Actions

Bug #59668

open

cache-flush-evict-all finished with errors - (2) No such file or directory

Added by Pivert Dubuisson 12 months ago. Updated 11 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
librados
Target version:
-
% Done:

0%

Source:
Support
Tags:
tier cache-tiering documentation
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rados
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I can't remove the writeback cache pool.
Problems :
- Documentation seems outdated : procedure to remote the cache mention to use the `ceph osd tier cache-mode cephfs_data_cache proxy` mode, but this option does not exist. So I used the readproxy instead.
- The cache-flush-evict-all reports many errors (check below).
- The cache pool is not empty (46194 objects remains in the cache pool)

root@pve1:~# ceph osd tier cache-mode cephfs_data_cache readproxy
root@pve1:~# rados -p cephfs_data_cache cache-flush-evict-all
...
failed to flush /1000065c15a.00000000: (2) No such file or directory
        1000082419d.00000001
failed to flush /1000082419d.00000001: (2) No such file or directory
        100008437ce.00000000
failed to flush /100008437ce.00000000: (2) No such file or directory
        1000065b25c.00000000
failed to flush /1000065b25c.00000000: (2) No such file or directory
cache-flush-evict-all finished with errors

root@pve1:~# rados -p cephfs_data_cache ls | wc -l
46194
root@pve1:~# rados -p cephfs_data_cache cache-flush-evict-all 2>&1 | grep 'failed to flush' | grep -v 'No such file or directory' | wc -l
0

At this stage, I don't know if it's safe to set the cache-mode to none and remove it.
How should I fix the issue?

Thanks

Actions #1

Updated by Pivert Dubuisson 12 months ago

Well, I removed the cache, and it broke the filesystem.
All files opened by applications and recently updated are corrupt.
Also, all snapshots are corrupt (hourly snapshots from scheduler)
What should have been done to prevent this disaster ?

Actions #2

Updated by Ilya Dryomov 11 months ago

  • Target version deleted (v17.2.6)
Actions

Also available in: Atom PDF