Ceph » Orchestrator

OK, here's what's going on: outside the container world, simple OSDs have a unit enabled named something like ceph-volume@simple-19-8ba4de01-ec7a-4b1c-be73-1d89431e3df5.service. This in turn maps to a call to ceph-volume simple activate. Somewhere in ceph_volume/devices/simple/activate.py, we have calls to chown each device that backs the OSD.

Once an OSD is adopted, the constructed unit file never calls ceph-volume simple activate. Instead, it always calls ceph-volume lvm activate, which is effectively a noop for simple OSDs, so when it finally starts the OSD, the chown hasn't happened.

We can't change the constructed unit file to call ceph-volume simple activate for simple OSDs, because that won't work anymore, as the JSON file this command needs (/etc/ceph/osd/19-8ba4de01-ec7a-4b1c-be73-1d89431e3df5.json) has been renamed to 19-8ba4de01-ec7a-4b1c-be73-1d89431e3df5.json.adopted-by-cephadm. And even if we were to bring that file back, it still wouldn't work, because we don't want the OSD to be mounted in the normal fashion, and anyway, all the necessary files have already been moved out of that OSD's data partition, to /var/lib/ceph/$FSID/osd.$ID.

One easy thing we could do is inject something like chown /var/lib/ceph/$FSID/osd.$ID/block* into the generated unit file, before starting the OSD. Does anyone have an opinion on this?