Ceph » RADOS

When a CRUSH text map is encoded, the buckets are read in such a way that they must be defined before they are referenced. For example, in [1], the hosts must be defined before the racks. Each entry also includes an ID number. When the CRUSH map is encoded and then decoded again, the entries are printed out in the order corresponding to their IDs, rather than the order they were originally in the text map.

However, since the order in the text map matters, this means that when non-sequential IDs are used, there are situations where crushtool can successfully encode a map, and later decode it successfully, but fails when you try to re-encoded the decoded result.

It's not at all clear what the IDs are used for; the only mention is a cryptic comment not to change them unnecessarily. It would be nice if they were implicit based on the order of buckets in the file; failing that, crushtool should at least warn when the IDs are non-sequential.

[1] http://ceph.newdream.net/wiki/Custom_data_placement_with_CRUSH#Example_crush_map