Project

General

Profile

Actions

Bug #23112

closed

rbd kernel client might hang when write to a quota-full pool

Added by dongdong tao about 6 years ago. Updated about 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
rbd
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

the rbd kernel client might hang there when a rbd pool has reached its quota limit.

below is the stack strace i got from dmesg
[1370861.609453] INFO: task prometheus:31028 blocked for more than 120 seconds.
[1370861.617204] Tainted: P OE 4.12.14-coreos #1
[1370861.623813] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[1370861.632509] prometheus D 0 31028 30999 0x00000080
[1370861.638897] Call Trace:
[1370861.642272] __schedule+0x2e0/0x890
[1370861.646604] ? cpumask_next_and+0x30/0x50
[1370861.651424] schedule+0x36/0x80
[1370861.655388] schedule_timeout+0x1e8/0x360
[1370861.660237] ? update_load_avg+0x6cb/0x720
[1370861.665191] ? update_load_avg+0x6cb/0x720
[1370861.670104] __down+0x8a/0xe0
[1370861.673865] ? __down+0x8a/0xe0
[1370861.677830] ? _xfs_buf_find+0x315/0x530 [xfs]
[1370861.683078] down+0x41/0x50
[1370861.686644] ? down+0x41/0x50
[1370861.690379] xfs_buf_lock+0x3c/0xf0 [xfs]
[1370861.695138] _xfs_buf_find+0x315/0x530 [xfs]
[1370861.700162] xfs_buf_get_map+0x2a/0x290 [xfs]
[1370861.705297] xfs_buf_read_map+0x2d/0x180 [xfs]
[1370861.710521] xfs_trans_read_buf_map+0xf4/0x310 [xfs]
[1370861.716242] xfs_bmbt_change_owner+0x1112/0x1e20 [xfs]
[1370861.722157] xfs_bmbt_change_owner+0x11e7/0x1e20 [xfs]
[1370861.728081] xfs_btree_lookup+0xcc/0x3b0 [xfs]
[1370861.733277] ? kmem_cache_alloc+0xdb/0x1a0
[1370861.738104] ? kmem_zone_alloc+0x81/0x100 [xfs]
[1370861.743394] 0xffffffffc080e649
[1370861.747279] 0xffffffffc080ea67
[1370861.751140] xfs_alloc_get_freelist+0xfd0/0x1810 [xfs]
[1370861.757043] xfs_alloc_get_freelist+0x1806/0x1810 [xfs]
[1370861.763031] xfs_alloc_vextent+0x426/0x590 [xfs]
[1370861.768390] xfs_bmap_adjacent+0x591/0xae0 [xfs]
[1370861.773759] xfs_bmap_adjacent+0xade/0xae0 [xfs]
[1370861.779096] xfs_bmapi_write+0x6b6/0xc50 [xfs]
[1370861.784261] xfs_iomap_write_allocate+0x18d/0x380 [xfs]
[1370861.790210] xfs_trans_resv_calc+0x86d/0x1f30 [xfs]
[1370861.795814] xfs_trans_resv_calc+0x181b/0x1f30 [xfs]
[1370861.801507] write_cache_pages+0x254/0x500
[1370861.806270] ? iomap_write_end+0x32/0x70
[1370861.810928] ? xfs_trans_resv_calc+0x15a0/0x1f30 [xfs]
[1370861.816848] xfs_trans_resv_calc+0x1d4b/0x1f30 [xfs]
[1370861.822480] do_writepages+0x1c/0x70
[1370861.826713] _filemap_fdatawrite_range+0xc6/0x100
[1370861.832195] filemap_write_and_wait_range+0x2a/0x70
[1370861.837760] xfs_extent_busy_ag_cmp+0xcc3/0xe30 [xfs]
[1370861.843528] vfs_fsync_range+0x3d/0xb0
[1370861.847974] do_fsync+0x3d/0x70
[1370861.851790] SyS_fsync+0x10/0x20
[1370861.855714] do_syscall_64+0x5a/0x160
[1370861.860076] entry_SYSCALL64_slow_path+0x25/0x25

Actions #1

Updated by dongdong tao about 6 years ago

why the rbd client not just report "quota-full" and then return, when it can not write to a full pool ?

Actions #2

Updated by Jason Dillaman about 6 years ago

  • Project changed from rbd to Linux kernel client
Actions #3

Updated by Ilya Dryomov about 6 years ago

  • Category set to rbd
  • Assignee set to Ilya Dryomov

This isn't specific to the kernel client, I believe other ceph clients behave the same way.

Actions #5

Updated by Ilya Dryomov about 5 years ago

  • Status changed from New to Resolved
Actions

Also available in: Atom PDF