Bug #2759
libceph: crush tree algorithm is not understood
% Done:
0%
Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):
Description
After changing the crush algorithm for a bucket to tree, the kernel client cannot parse the OSDMap:
Jul 9 11:12:06 plana29 kernel: [587716.692437] libceph: osdc handle_map corrupt msg Jul 9 11:12:06 plana29 kernel: [587716.716772] header: 00000000: 07 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716777] header: 00000010: 29 00 c4 00 01 00 5d 04 00 00 00 00 00 00 00 00 ).....]......... Jul 9 11:12:06 plana29 kernel: [587716.716781] header: 00000020: 00 00 00 00 01 01 00 00 00 00 00 00 00 01 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716785] header: 00000030: 00 d0 e8 8d ed ..... Jul 9 11:12:06 plana29 kernel: [587716.716788] front: 00000000: ba 9b be 62 86 d5 43 6d a0 82 cd c9 ef 06 8f be ...b..Cm........ Jul 9 11:12:06 plana29 kernel: [587716.716792] front: 00000010: 00 00 00 00 01 00 00 00 07 00 00 00 3d 04 00 00 ............=... Jul 9 11:12:06 plana29 kernel: [587716.716796] front: 00000020: 05 00 ba 9b be 62 86 d5 43 6d a0 82 cd c9 ef 06 .....b..Cm...... Jul 9 11:12:06 plana29 kernel: [587716.716800] front: 00000030: 8f be 07 00 00 00 54 f5 f5 4f d8 71 4e 1c 4d 1d ......T..O.qN.M. Jul 9 11:12:06 plana29 kernel: [587716.716804] front: 00000040: fb 4f 80 7c fa 21 03 00 00 00 00 00 00 00 02 01 .O.|.!.......... Jul 9 11:12:06 plana29 kernel: [587716.716807] front: 00000050: 02 00 02 08 00 00 00 08 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716811] front: 00000060: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716815] front: 00000070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716818] front: 00000080: 00 00 00 01 00 00 00 02 01 02 01 02 08 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716823] front: 00000090: 08 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716828] front: 000000a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716832] front: 000000b0: 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716836] front: 000000c0: 02 01 02 02 02 08 00 00 00 08 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716840] front: 000000d0: 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716844] front: 000000e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716848] front: 000000f0: 00 00 00 00 00 03 00 00 00 00 00 00 00 04 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716853] front: 00000100: 00 64 61 74 61 01 00 00 00 08 00 00 00 6d 65 74 .data........met Jul 9 11:12:06 plana29 kernel: [587716.716857] front: 00000110: 61 64 61 74 61 02 00 00 00 03 00 00 00 72 62 64 adata........rbd Jul 9 11:12:06 plana29 kernel: [587716.716861] front: 00000120: 02 00 00 00 00 00 00 00 02 00 00 00 02 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716866] front: 00000130: 03 03 02 00 00 00 00 00 01 00 00 00 01 00 02 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716870] front: 00000140: 00 00 00 00 00 00 36 54 00 00 00 02 1a 90 0a d6 ......6T........ Jul 9 11:12:06 plana29 kernel: [587716.716874] front: 00000150: 83 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716878] front: 00000160: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716883] front: 00000170: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716887] front: 00000180: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716891] front: 00000190: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716895] front: 000001a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716900] front: 000001b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716904] front: 000001c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 06 4b ...............K Jul 9 11:12:06 plana29 kernel: [587716.716908] front: 000001d0: 00 00 00 02 1a 90 0a d6 83 22 00 00 00 00 00 00 ........."...... Jul 9 11:12:06 plana29 kernel: [587716.716913] front: 000001e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716917] front: 000001f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716921] front: 00000200: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716925] front: 00000210: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716930] front: 00000220: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716934] front: 00000230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716938] front: 00000240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716942] front: 00000250: 00 00 00 00 00 00 03 02 00 00 00 00 01 00 08 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716946] front: 00000260: 00 00 03 00 00 00 02 00 00 00 04 00 00 00 ff ff ................ Jul 9 11:12:06 plana29 kernel: [587716.716951] front: 00000270: ff ff 06 00 04 00 00 00 02 00 01 00 00 00 fd ff ................ Jul 9 11:12:06 plana29 kernel: [587716.716955] front: 00000280: ff ff 00 00 02 00 00 00 01 00 03 00 00 00 fe ff ................ Jul 9 11:12:06 plana29 kernel: [587716.716959] front: 00000290: ff ff 01 00 03 00 00 00 02 00 02 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716963] front: 000002a0: 00 00 01 00 00 00 04 00 00 00 00 00 00 01 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716967] front: 000002b0: 00 02 00 00 00 01 00 04 00 00 00 fd ff ff ff 02 ................ Jul 9 11:12:06 plana29 kernel: [587716.716971] front: 000002c0: 00 04 00 00 00 02 00 01 00 00 00 fe ff ff ff 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716975] front: 000002d0: 00 02 00 00 00 01 00 00 00 00 00 00 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716979] front: 000002e0: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 03 ................ Jul 9 11:12:06 plana29 kernel: [587716.716983] front: 000002f0: 00 00 00 00 01 01 0a 01 00 00 00 ff ff ff ff 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.716988] front: 00000300: 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 04 ................ Jul 9 11:12:06 plana29 kernel: [587716.716992] front: 00000310: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 03 ................ Jul 9 11:12:06 plana29 kernel: [587716.716996] front: 00000320: 00 00 00 01 01 01 0a 01 00 00 00 ff ff ff ff 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.717000] front: 00000330: 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 04 ................ Jul 9 11:12:06 plana29 kernel: [587716.717004] front: 00000340: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 03 ................ Jul 9 11:12:06 plana29 kernel: [587716.717008] front: 00000350: 00 00 00 02 01 01 0a 01 00 00 00 ff ff ff ff 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.717012] front: 00000360: 00 00 00 02 00 00 00 00 00 00 00 00 00 00 00 04 ................ Jul 9 11:12:06 plana29 kernel: [587716.717017] front: 00000370: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 00 ................ Jul 9 11:12:06 plana29 kernel: [587716.717021] front: 00000380: 00 00 00 03 00 00 00 6f 73 64 01 00 00 00 04 00 .......osd...... Jul 9 11:12:06 plana29 kernel: [587716.717026] front: 00000390: 00 00 68 6f 73 74 02 00 00 00 04 00 00 00 72 61 ..host........ra Jul 9 11:12:06 plana29 kernel: [587716.717030] front: 000003a0: 63 6b 03 00 00 00 03 00 00 00 72 6f 77 04 00 00 ck........row... Jul 9 11:12:06 plana29 kernel: [587716.717034] front: 000003b0: 00 04 00 00 00 72 6f 6f 6d 05 00 00 00 0a 00 00 .....room....... Jul 9 11:12:06 plana29 kernel: [587716.717038] front: 000003c0: 00 64 61 74 61 63 65 6e 74 65 72 06 00 00 00 04 .datacenter..... Jul 9 11:12:06 plana29 kernel: [587716.717042] front: 000003d0: 00 00 00 70 6f 6f 6c 05 00 00 00 fd ff ff ff 09 ...pool......... Jul 9 11:12:06 plana29 kernel: [587716.717046] front: 000003e0: 00 00 00 6c 6f 63 61 6c 72 61 63 6b fe ff ff ff ...localrack.... Jul 9 11:12:06 plana29 kernel: [587716.717050] front: 000003f0: 09 00 00 00 6c 6f 63 61 6c 68 6f 73 74 ff ff ff ....localhost... Jul 9 11:12:06 plana29 kernel: [587716.717054] front: 00000400: ff 07 00 00 00 64 65 66 61 75 6c 74 00 00 00 00 .....default.... Jul 9 11:12:06 plana29 kernel: [587716.717058] front: 00000410: 05 00 00 00 6f 73 64 2e 30 01 00 00 00 05 00 00 ....osd.0....... Jul 9 11:12:06 plana29 kernel: [587716.717063] front: 00000420: 00 6f 73 64 2e 31 03 00 00 00 00 00 00 00 04 00 .osd.1.......... Jul 9 11:12:06 plana29 kernel: [587716.717067] front: 00000430: 00 00 64 61 74 61 01 00 00 00 08 00 00 00 6d 65 ..data........me Jul 9 11:12:06 plana29 kernel: [587716.717071] front: 00000440: 74 61 64 61 74 61 02 00 00 00 03 00 00 00 72 62 tadata........rb Jul 9 11:12:06 plana29 kernel: [587716.717075] front: 00000450: 64 02 00 00 00 05 00 00 00 13 00 00 00 d............ Jul 9 11:12:06 plana29 kernel: [587716.717079] footer: 00000000: 9b d3 12 b0 00 00 00 00 00 00 00 00 01 .............
The mount command hangs, as described in http://permalink.gmane.org/gmane.comp.file-systems.ceph.devel/7702
History
#1 Updated by Sage Weil about 10 years ago
- Priority changed from Normal to High
#2 Updated by Ian Colle about 10 years ago
We should be testing CRUSH with multiple algorithms
#3 Updated by Ilya Dryomov over 9 years ago
- Subject changed from crush: tree algorithm is not understood to libceph: crush tree algorithm is not understood
#4 Updated by Ilya Dryomov over 8 years ago
- Status changed from 12 to In Progress
- Assignee set to Ilya Dryomov
- Regression set to No
#5 Updated by Ilya Dryomov over 8 years ago
- Status changed from In Progress to Fix Under Review
#6 Updated by Ilya Dryomov over 8 years ago
- Status changed from Fix Under Review to Resolved
[PATCH] crush: fix a bug in tree bucket decode