Ceph

While working on the libvirt RBD storage driver I noticed the following crash:

osdc/Objecter.cc: In function 'void Objecter::shutdown()' thread 7f45806c6760 time 2012-03-01 15:41:55.273545
osdc/Objecter.cc: 221: FAILED assert(initialized)
 ceph version 0.42.2 (commit:732f3ec94e39d458230b7728b2a936d431e19322)
 1: (Objecter::shutdown()+0x170) [0x7f4580126350]
 2: (librados::RadosClient::shutdown()+0x4b) [0x7f45801068eb]
 3: (librados::RadosClient::connect()+0x4e1) [0x7f4580106fe1]
 4: ./rados_objecter() [0x400875]
 5: (__libc_start_main()+0xfd) [0x7f457f996c4d]
 6: ./rados_objecter() [0x400729]
 ceph version 0.42.2 (commit:732f3ec94e39d458230b7728b2a936d431e19322)
 1: (Objecter::shutdown()+0x170) [0x7f4580126350]
 2: (librados::RadosClient::shutdown()+0x4b) [0x7f45801068eb]
 3: (librados::RadosClient::connect()+0x4e1) [0x7f4580106fe1]
 4: ./rados_objecter() [0x400875]
 5: (__libc_start_main()+0xfd) [0x7f457f996c4d]
 6: ./rados_objecter() [0x400729]
terminate called after throwing an instance of 'ceph::FailedAssertion'
Aborted

I traced this back to 290730ee3e61a4eb680b8e38cee9cb153f83298e

The problem was that my authorization values were wrong. I verified this with the following code:

#include <stdio.h>
#include "rados/librados.h" 

int main(void) {

    rados_t cluster;

    int r;

    r = rados_create(&cluster, "admin");
    if (r < 0) {
        printf("rados_create failed: %d", r);
        return -1;
    }

    rados_conf_set(cluster, "auth_supported", "cephx");
    rados_conf_set(cluster, "mon_host", "[2a00:f10:11b:cef0:230:48ff:fed3:b086]:6789");
    rados_conf_set(cluster, "key", "AQB7jS1PIH2tDxAAda+7fY+khw1VrvKZrrtqsw==");

    r = rados_connect(cluster);
    if (r < 0) {
        printf("rados_connect failed: %d\n", r);
    } else {
        printf("%s\n", "rados_connect succeeded!");
        rados_shutdown(cluster);
    }
}

The key in this case is an old key which is not valid.

This crashes:

root@stack01:~# ./rados_objecter 
osdc/Objecter.cc: In function 'void Objecter::shutdown()' thread 7f571f654760 time 2012-03-01 15:46:55.463435
osdc/Objecter.cc: 221: FAILED assert(initialized)
 ceph version 0.42.2 (commit:732f3ec94e39d458230b7728b2a936d431e19322)
 1: (Objecter::shutdown()+0x170) [0x7f571f0b4350]
 2: (librados::RadosClient::shutdown()+0x4b) [0x7f571f0948eb]
 3: (librados::RadosClient::connect()+0x4e1) [0x7f571f094fe1]
 4: ./rados_objecter() [0x400875]
 5: (__libc_start_main()+0xfd) [0x7f571e924c4d]
 6: ./rados_objecter() [0x400729]
 ceph version 0.42.2 (commit:732f3ec94e39d458230b7728b2a936d431e19322)
 1: (Objecter::shutdown()+0x170) [0x7f571f0b4350]
 2: (librados::RadosClient::shutdown()+0x4b) [0x7f571f0948eb]
 3: (librados::RadosClient::connect()+0x4e1) [0x7f571f094fe1]
 4: ./rados_objecter() [0x400875]
 5: (__libc_start_main()+0xfd) [0x7f571e924c4d]
 6: ./rados_objecter() [0x400729]
terminate called after throwing an instance of 'ceph::FailedAssertion'
Aborted
root@stack01:~#

I'm not sure where this should be fixed, but the objecter seems to be the issue here when the init/connect failed.