Actions
Bug #58047
openceph-osd: osd numa affinity setting doesn't work
Status:
New
Priority:
Normal
Assignee:
-
Category:
OSD
Target version:
-
% Done:
100%
Source:
Tags:
Backport:
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
knfs
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
After setting osd_numa_node parameter, the osd numa is not as expected.
- cpu numa node
# lscpu | grep NUMA NUMA node(s): 4 NUMA node0 CPU(s): 0-2,6-8,24-26,30-32 NUMA node1 CPU(s): 3-5,9-11,27-29,33-35 NUMA node2 CPU(s): 12-14,18-20,36-38,42-44 NUMA node3 CPU(s): 15-17,21-23,39-41,45-47
- set osd.0 to numa 1
# ceph daemon osd.0 config show | grep osd_numa_node "osd_numa_node": "1",
- check osd numa node
ceph 1121 1 0 Sep25 ? 10:39:11 /usr/bin/ceph-osd --cluster ceph -f -i 0 --setuser ceph --setgroup ceph # ps -L -o pid,lwp,pcpu,cpuid,time,comm 1121 PID LWP %CPU CPUID TIME COMMAND 1121 1121 0.0 19 00:01:30 ceph-osd 1121 1122 0.0 0 00:00:08 log 1121 1124 0.0 12 00:18:17 msgr-worker-0 1121 1125 0.0 28 00:18:26 msgr-worker-1 1121 1126 0.0 38 00:17:23 msgr-worker-2 1121 1127 0.0 28 00:12:26 msgr-worker-3 1121 1128 0.0 34 00:15:27 msgr-worker-4 1121 1129 0.0 14 00:13:55 msgr-worker-5 1121 1130 0.0 27 00:13:26 msgr-worker-6 1121 1131 0.0 41 00:13:33 msgr-worker-7 1121 1132 0.0 11 00:12:09 msgr-worker-8 1121 1133 0.0 34 00:23:33 msgr-worker-9 1121 1134 0.0 1 00:23:36 msgr-worker-10 1121 1135 0.0 35 00:22:15 msgr-worker-11 1121 1136 0.0 39 00:21:30 msgr-worker-12 1121 1137 0.0 30 00:20:10 msgr-worker-13 1121 1138 0.0 17 00:23:41 msgr-worker-14 1121 1139 0.0 11 00:18:31 msgr-worker-15 1121 1140 0.0 38 00:17:47 msgr-worker-16 1121 1141 0.0 29 00:18:01 msgr-worker-17 1121 1142 0.0 23 00:18:40 msgr-worker-18 1121 1143 0.0 35 00:19:35 msgr-worker-19 1121 1144 0.0 4 00:17:39 msgr-worker-20 1121 1145 0.0 43 00:20:52 msgr-worker-21 1121 1146 0.0 12 00:18:14 msgr-worker-22 1121 1147 0.0 31 00:18:33 msgr-worker-23 1121 1151 0.0 10 00:00:24 service 1121 1152 0.0 42 00:12:03 admin_socket 1121 1161 0.0 37 00:00:00 signal_handler 1121 1162 0.0 14 00:00:00 io_context_pool 1121 1163 0.0 7 00:00:00 io_context_pool 1121 1166 0.0 38 00:05:02 OpHistorySvc 1121 1167 0.0 7 00:00:14 ceph_timer 1121 1186 0.0 21 00:02:03 safe_timer 1121 1187 0.0 45 00:05:56 safe_timer 1121 1188 0.0 42 00:00:00 safe_timer 1121 1189 0.0 15 00:00:00 safe_timer 1121 1190 0.0 40 00:00:00 fn_anonymous 1121 1191 0.0 46 00:05:55 bstore_aio 1121 1192 0.0 45 00:00:00 bstore_discard 1121 1199 0.0 16 00:00:08 rocksdb:low0 1121 1200 0.0 23 00:00:04 rocksdb:low1 1121 1201 0.0 16 00:00:04 rocksdb:low2 1121 1202 0.0 47 00:00:00 rocksdb:low3 1121 1203 0.0 21 00:00:05 rocksdb:high0 1121 1204 0.0 23 00:00:08 rocksdb:high1 1121 1386 0.0 42 00:02:24 bstore_aio 1121 1387 0.0 23 00:00:00 bstore_discard 1121 1388 0.0 12 00:02:23 bstore_aio 1121 1389 0.0 40 00:00:00 bstore_discard 1121 1390 0.0 45 00:04:02 bstore_aio 1121 1391 0.0 38 00:00:00 bstore_discard 1121 1392 0.0 44 00:00:00 ceph-osd 1121 1573 0.0 46 00:00:07 rocksdb:dump_st 1121 1574 0.0 23 00:00:00 rocksdb:pst_st 1121 1575 0.0 39 00:00:00 cfin 1121 1576 0.0 16 00:11:41 bstore_kv_sync 1121 1577 0.0 17 00:03:41 bstore_kv_final 1121 1578 0.0 39 00:31:46 bstore_mempool 1121 1579 0.0 46 00:00:59 ms_dispatch 1121 1580 0.0 1 00:00:00 ms_local 1121 1581 0.0 23 00:01:31 safe_timer 1121 1582 0.0 41 00:07:18 safe_timer 1121 1583 0.0 0 00:00:16 ms_dispatch 1121 1584 0.0 32 00:00:00 ms_local 1121 1585 0.0 34 00:00:00 ms_dispatch 1121 1586 0.0 13 00:00:00 ms_local 1121 1587 0.0 40 00:00:00 ms_dispatch 1121 1588 0.0 44 00:00:00 ms_local 1121 1589 0.0 45 00:00:00 ms_dispatch 1121 1590 0.0 44 00:00:00 ms_local 1121 1591 0.0 19 00:00:00 ms_dispatch 1121 1592 0.0 20 00:00:00 ms_local 1121 1593 0.0 12 00:00:00 ms_dispatch 1121 1594 0.0 20 00:00:00 ms_local 1121 1595 0.0 20 00:00:00 fn_anonymous 1121 1596 0.0 44 00:00:00 finisher 1121 1597 0.0 44 00:00:00 safe_timer 1121 1598 0.0 44 00:00:00 safe_timer 1121 1599 0.0 44 00:00:00 ceph-osd 1121 1600 0.0 40 00:00:00 osd_srv_agent 1121 1601 0.0 47 00:15:48 tp_osd_tp 1121 1602 0.0 45 00:14:21 tp_osd_tp 1121 1603 0.0 17 00:10:50 tp_osd_tp 1121 1604 0.0 41 00:20:39 tp_osd_tp 1121 1605 0.0 16 00:13:16 tp_osd_tp 1121 1606 0.0 16 00:26:19 osd_srv_heartbt
No data to display
Actions