Bug #21330
closedLuminous is identifying HP cciss devices as ssd
0%
Description
While testing 12.2.0, I noticed that it is identifying cciss devices as ssd and not as hdd. All devices on hosts ede-c2-osd04 and ede-c2-osd05 are hdd.
ceph osd df tree
ID CLASS WEIGHT REWEIGHT SIZE USE AVAIL %USE VAR PGS TYPE NAME
-1 1.26520 - 1296G 783G 512G 60.44 1.00 - root default
-28 0.33295 - 341G 166G 174G 48.89 0.81 - host ede-c2-osd05
12 ssd 0.06659 1.00000 69872M 33009M 36863M 47.24 0.78 250 osd.12
13 ssd 0.06659 1.00000 69872M 36661M 33211M 52.47 0.87 275 osd.13
14 ssd 0.06659 1.00000 69872M 34839M 35033M 49.86 0.82 275 osd.14
15 ssd 0.06659 1.00000 69872M 35232M 34640M 50.42 0.83 257 osd.15
16 ssd 0.06659 1.00000 69872M 31050M 38822M 44.44 0.74 255 osd.16
-16 0.93225 - 955G 616G 338G 64.57 1.07 - rack rack-1
-17 0.53271 - 545G 338G 207G 62.02 1.03 - chassis chassis-1-101
-5 0.19977 - 204G 148G 57831M 72.41 1.20 - host ede-c2-osd01
1 hdd 0.06659 1.00000 69872M 51702M 18169M 74.00 1.22 340 osd.1
5 hdd 0.06659 1.00000 69872M 49488M 20383M 70.83 1.17 348 osd.5
18 hdd 0.06659 1.00000 69872M 50594M 19278M 72.41 1.20 336 osd.18
-13 0.33295 - 341G 190G 150G 55.78 0.92 - host ede-c2-osd04
3 ssd 0.06659 1.00000 69872M 41725M 28147M 59.72 0.99 258 osd.3
7 ssd 0.06659 1.00000 69872M 35606M 34266M 50.96 0.84 258 osd.7
9 ssd 0.06659 1.00000 69872M 43335M 26537M 62.02 1.03 303 osd.9
10 ssd 0.06659 1.00000 69872M 36595M 33277M 52.37 0.87 44 osd.10
11 ssd 0.06659 1.00000 69872M 37621M 32251M 53.84 0.89 261 osd.11
-22 0.19977 - 204G 145G 60538M 71.12 1.18 - chassis chassis-1-102
-7 0.19977 - 204G 145G 60538M 71.12 1.18 - host ede-c2-osd02
2 hdd 0.06659 1.00000 69872M 47456M 22416M 67.92 1.12 309 osd.2
6 hdd 0.06659 1.00000 69872M 52501M 17371M 75.14 1.24 339 osd.6
17 hdd 0.06659 1.00000 69872M 49121M 20751M 70.30 1.16 332 osd.17
-25 0.19977 - 204G 132G 73771M 64.81 1.07 - chassis chassis-1-103
-3 0.19977 - 204G 132G 73771M 64.81 1.07 - host ede-c2-osd03
0 hdd 0.06659 1.00000 69872M 42666M 27206M 61.06 1.01 325 osd.0
4 hdd 0.06659 1.00000 69872M 46011M 23861M 65.85 1.09 336 osd.4
8 hdd 0.06659 1.00000 69872M 47168M 22704M 67.51 1.12 312 osd.8
TOTAL 1296G 783G 512G 60.44
MIN/MAX VAR: 0.74/1.24 STDDEV: 9.71
mount | grep ceph
/dev/cciss/c0d4p1 on /var/lib/ceph/osd/ceph-16 type xfs (rw,noatime,attr2,inode64,noquota)
/dev/cciss/c0d1p1 on /var/lib/ceph/osd/ceph-12 type xfs (rw,noatime,attr2,inode64,noquota)
/dev/cciss/c0d2p1 on /var/lib/ceph/osd/ceph-14 type xfs (rw,noatime,attr2,inode64,noquota)
/dev/cciss/c0d5p1 on /var/lib/ceph/osd/ceph-15 type xfs (rw,noatime,attr2,inode64,noquota)
/dev/cciss/c0d3p1 on /var/lib/ceph/osd/ceph-13 type xfs (rw,noatime,attr2,inode64,noquota)
lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
cciss/c0d5 104:80 0 68.3G 0 disk
|-cciss/c0d5p2 104:82 0 68.2G 0 part
`-cciss/c0d5p1 104:81 0 100M 0 part /var/lib/ceph/osd/ceph-15
cciss/c0d3 104:48 0 68.3G 0 disk
|-cciss/c0d3p1 104:49 0 100M 0 part /var/lib/ceph/osd/ceph-13
`-cciss/c0d3p2 104:50 0 68.2G 0 part
cciss/c0d1 104:16 0 68.3G 0 disk
|-cciss/c0d1p1 104:17 0 100M 0 part /var/lib/ceph/osd/ceph-12
`-cciss/c0d1p2 104:18 0 68.2G 0 part
sr0 11:0 1 1024M 0 rom
cciss/c0d4 104:64 0 68.3G 0 disk
|-cciss/c0d4p1 104:65 0 100M 0 part /var/lib/ceph/osd/ceph-16
`-cciss/c0d4p2 104:66 0 68.2G 0 part
cciss/c0d2 104:32 0 68.3G 0 disk
|-cciss/c0d2p2 104:34 0 68.2G 0 part
`-cciss/c0d2p1 104:33 0 100M 0 part /var/lib/ceph/osd/ceph-14
cciss/c0d0 104:0 0 68.3G 0 disk
|-cciss/c0d0p5 104:5 0 976M 0 part [SWAP]
|-cciss/c0d0p1 104:1 0 19.1G 0 part /
|-cciss/c0d0p6 104:6 0 48.3G 0 part /var
`-cciss/c0d0p2 104:2 0 1K 0 part
ceph -v
ceph version 12.2.0 (32ce2a3ae5239ee33d6150705cdb24d43bab910c) luminous (rc)
OS: Ubuntu 16.04 using 4.13.0 kernel
uname -a
Linux ede-c2-osd05 4.13.0-041300-generic #201709031731 SMP Sun Sep 3 21:33:09 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Updated by Greg Farnum over 6 years ago
I believe these flags are just set by looking at the rotational media tags. I don't remember off hand if that's a Boolean on/off or keywords, but it wouldn't surprise me if there's a bit of mismatch there with some kind of aggregation chassis. You might see if there are options for setting it on that side.
Updated by Eric Eastman over 6 years ago
The cciss driver is only used for older HP servers. Looking at the Linux driver code and the ROM based device interface, I don't see any options to specify a SSD vs HDD. This is such an old controller that I don't think HP ever supported SSD with it. Per the modinfo
modinfo cciss filename: /lib/modules/4.13.0-041300-generic/kernel/drivers/block/cciss.ko license: GPL version: 3.6.26 description: Driver for HP Smart Array Controllers author: Hewlett-Packard Company srcversion: B5EC856EEA4B3EF7F0E77FB alias: pci:v0000103Cd00003230sv0000103Csd0000323Dbc*sc*i* alias: pci:v0000103Cd00003230sv0000103Csd00003237bc*sc*i* alias: pci:v0000103Cd00003238sv0000103Csd00003215bc*sc*i* alias: pci:v0000103Cd00003238sv0000103Csd00003214bc*sc*i* alias: pci:v0000103Cd00003238sv0000103Csd00003213bc*sc*i* alias: pci:v0000103Cd00003238sv0000103Csd00003212bc*sc*i* alias: pci:v0000103Cd00003238sv0000103Csd00003211bc*sc*i* alias: pci:v0000103Cd00003230sv0000103Csd00003235bc*sc*i* alias: pci:v0000103Cd00003230sv0000103Csd00003234bc*sc*i* alias: pci:v0000103Cd00003230sv0000103Csd00003223bc*sc*i* alias: pci:v0000103Cd00003220sv0000103Csd00003225bc*sc*i* alias: pci:v00000E11d00000046sv00000E11sd0000409Dbc*sc*i* alias: pci:v00000E11d00000046sv00000E11sd0000409Cbc*sc*i* alias: pci:v00000E11d00000046sv00000E11sd0000409Bbc*sc*i* alias: pci:v00000E11d00000046sv00000E11sd0000409Abc*sc*i* alias: pci:v00000E11d00000046sv00000E11sd00004091bc*sc*i* alias: pci:v00000E11d0000B178sv00000E11sd00004083bc*sc*i* alias: pci:v00000E11d0000B178sv00000E11sd00004082bc*sc*i* alias: pci:v00000E11d0000B178sv00000E11sd00004080bc*sc*i* alias: pci:v00000E11d0000B060sv00000E11sd00004070bc*sc*i* depends: intree: Y name: cciss vermagic: 4.13.0-041300-generic SMP mod_unload parm: cciss_tape_cmds:number of commands to allocate for tape devices (default: 6) (int) parm: cciss_simple_mode:Use 'simple mode' rather than 'performant mode' (int) parm: cciss_allow_hpsa:Prevent cciss driver from accessing hardware known to be supported by the hpsa driver (int)
My recommendation would be to have the Ceph code assume anything attached to the cciss device driver be considered a hdd.