Bug #6698
closed
new osds added to the cluster not starting up due to crush lookup failure
Added by Tamilarasi muthamizhan over 10 years ago.
Updated over 10 years ago.
Description
ceph version: next branch [0.72-rc1]
test setup : mira025, mira038
had a cluster running on master branch, upgraded it to next and trying to add new osds to the cluster. The disks[osd.3, osd.4] are prepared and mounted but not starting up.
ubuntu@mira025:~/ceph-dep/ceph-deploy$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 902G 5.7G 850G 1% /
udev 7.8G 12K 7.8G 1% /dev
tmpfs 3.2G 336K 3.2G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 7.9G 0 7.9G 0% /run/shm
cgroup 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/sdb1 931G 43M 931G 1% /var/lib/ceph/osd/ceph-0
/dev/sdc1 931G 33M 931G 1% /var/lib/ceph/osd/ceph-3
ubuntu@mira025:~/ceph-dep/ceph-deploy$ sudo ceph osd tree
# id weight type name up/down reweight
-1 2.73 root default
-2 0.91 host mira025
0 0.91 osd.0 down 1
-3 1.82 host mira038
1 0.91 osd.1 down 1
2 0.91 osd.2 down 1
3 0 osd.3 down 0
4 0 osd.4 down 0
ubuntu@mira025:~/ceph-dep/ceph-deploy$ sudo ceph -v
ceph version 0.72-rc1-83-g592a99c (592a99ccd8da2d7843ebb7ce6439566732580b7a)
- Status changed from New to Fix Under Review
- Assignee deleted (
Alfredo Deza)
- Status changed from Fix Under Review to Resolved
- Status changed from Resolved to In Progress
this bug was caused by crush lookup failure.
I see similar issue with one of my local clusters: mira047 and mira048.
- Subject changed from ceph-deploy: new osds added to the cluster is not starting up to new osds added to the cluster not starting up due to crush lookup failure
- Category deleted (
ceph-deploy)
- Status changed from In Progress to 12
- Assignee set to Dan Mick
- Status changed from 12 to Need More Info
I guess I'm not sure what the expected behavior is here; do we think that the OSDs should start, but not do much, until they're added to the crush map?
Sage suspects this may have been fixed by 1bb5aadbca80eb46d5393bb1ad68af3da025e157
- Status changed from Need More Info to Resolved
tried reproducing this issue by upgrading from dumpling to emperor and then trying to add osds to the cluster but it worked fine.
closing the bug for now.
Also available in: Atom
PDF