Project

General

Profile

Actions

Bug #61557

open

[crimson] rbd map command fails with sysfs write failed | connection timeout

Added by Harsh Kumar 12 months ago. Updated 12 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
crimson
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Encountered during execution of upstream rbd test - https://github.com/ceph/ceph/blob/v18.0.0/qa/workunits/rbd/krbd_exclusive_option.sh

# sudo bash krbd_exclusive_option.sh
+ SYSFS_DIR=/sys/bus/rbd/devices
+ IMAGE_NAME=exclusive-option-test
+ rbd create --size 1 --image-feature '' exclusive-option-test
2023-06-01T04:31:52.827-0400 7f88a6fdc540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-06-01T04:31:52.827-0400 7f88a6fdc540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
++ rbd info --format=json exclusive-option-test
++ python3 -c 'import sys, json; print(json.load(sys.stdin)['\''block_name_prefix'\''].split('\''.'\'')[1])'
2023-06-01T04:31:52.954-0400 7f17b33ff540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-06-01T04:31:52.955-0400 7f17b33ff540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
+ IMAGE_ID=3aa182ed30c2
++ sudo rbd map exclusive-option-test
2023-06-01T04:31:53.069-0400 7f7be62aa540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-06-01T04:31:53.070-0400 7f7be62aa540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
rbd: sysfs write failed
rbd: map failed: (110) Connection timed out
+ DEV='In some cases useful info is found in syslog - try "dmesg | tail".'
[root@dell-r640-087 rbd]# sudo rbd map exclusive-option-test
2023-06-01T04:33:31.232-0400 7f332d254540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-06-01T04:33:31.232-0400 7f332d254540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
rbd: sysfs write failed
In some cases useful info is found in syslog - try "dmesg | tail".
rbd: map failed: (110) Connection timed out
[root@dell-r640-087 rbd]# dmesg | tail
[ 6580.652358]  front: 00001420: 73 3f 66 66 66 3f 0c 12 02 00 00 00 02 00 00 00  s?fff?..........
[ 6580.652359]  front: 00001430: 00 00 00 00 01 00 00 00 02 00 00 00 00 00 00 00  ................
[ 6580.652360]  front: 00001440: 01 00 00 00 00 00 00 00 03 00 00 00 00 00 00 00  ................
[ 6580.652361]  front: 00001450: 01 00 00 00 02 00 00 00 00 00 00 00 01 00 00 00  ................
[ 6580.652362]  front: 00001460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[ 6580.652363]  front: 00001470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
[ 6580.652364]  front: 00001480: 00 01 1d 87 e2 01 01 00 00 00 37 00 00 00 00 00  ..........7.....
[ 6580.652364]  front: 00001490: 00 00                                            ..
[ 6580.652365] footer: 00000000: 00 8d cc 74 00 00 00 00 00 00 00 00 00 00 00 00  ...t............
[ 6580.652366] footer: 00000010: 00 00 00 00 01                                   .....
[root@dell-r640-087 rbd]# rbd list rbd
2023-06-01T04:34:55.015-0400 7f56aab1b540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-06-01T04:34:55.015-0400 7f56aab1b540 -1 WARNING: the following dangerous and experimental features are enabled: crimson
exclusive-option-test
[root@dell-r640-087 rbd]# ceph version
2023-06-01T04:39:15.829-0400 7ffb28aba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-06-01T04:39:15.830-0400 7ffb28aba700 -1 WARNING: the following dangerous and experimental features are enabled: crimson
ceph version 18.0.0-4257-g93b6f716 (93b6f716e647759fdc3b54eb7f372afd826b2e30) reef (dev)

Crimson image used - https://shaman.ceph.com/repos/ceph/main/93b6f716e647759fdc3b54eb7f372afd826b2e30/crimson/280477/

Execution on downstream Quincy ceph version 17.2.6-70.el9cp passed | stdout attached


Files

krbd_exclusive_option.sh_downstream_quincy.txt (63.3 KB) krbd_exclusive_option.sh_downstream_quincy.txt Harsh Kumar, 06/01/2023 08:44 AM
kernel.sh_crimson.txt (4.1 KB) kernel.sh_crimson.txt stdout on crimson cluster Harsh Kumar, 06/01/2023 08:58 AM
kernel.sh_downstream_quincy.txt (6.7 KB) kernel.sh_downstream_quincy.txt stdout on downstream quincy ceph version 17.2.6-70.el9cp Harsh Kumar, 06/01/2023 08:58 AM

Updated by Harsh Kumar 12 months ago

Harsh Kumar wrote:

Encountered during execution of upstream rbd test - https://github.com/ceph/ceph/blob/v18.0.0/qa/workunits/rbd/krbd_exclusive_option.sh

[...]

Crimson image used - https://shaman.ceph.com/repos/ceph/main/93b6f716e647759fdc3b54eb7f372afd826b2e30/crimson/280477/

Execution on downstream Quincy ceph version 17.2.6-70.el9cp passed | stdout attached

Same error is observed when rbd device map command is executed | Upstream test - https://github.com/ceph/ceph/blob/v18.0.0/qa/workunits/rbd/kernel.sh

stdouts have been attached.

Cluster logs - http://magna002.ceph.redhat.com/ceph-qe-logs/harsh/crimson-rbd/tracker-61557/67f6398a-0048-11ee-a993-78ac44395b40/

Actions

Also available in: Atom PDF