Ceph » RADOS

Test LibRadosAio.PoolQuotaPP hanged on
/a/sage-2019-01-28_03:48:46-rados-wip-sage2-testing-2019-01-27-1015-distro-basic-smithi/3518534

There is a single op that is stuck,

2019-01-28 08:31:08.201 7f574f7fe700 20 client.4930.objecter 4  32.be9754b3     osd.1   foo1    [writefull 0~4096]

because the pool is full and full_quota,

pool 32 'test-rados-api-smithi102-47832-3' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode warn last_change 65 flags hashpspool,full,full_quota max_bytes 4096 stripe_width 0 application rados

it appears that the first send attempt went out because the client didn't have the full_quota osdmap yet

2019-01-28 08:30:54.009 7f5766fc57c0  1 -- v1:172.21.15.102:0/2731979199 --> v1:172.21.15.102:6812/35180 -- osd_op(client.4930.0:4 32.3 32:cd2ae97d:::foo1:head [writefull 0~4096] snapc 0=[] ondisk+write+known_if_redirected+full_try e50) v8 -- ?+0 0x561e2a1d5830 con 0x561e2a24ee90

...which is probably why this test usually passes: it's racy.

It doesn't look like the objecter logic takes the full_try flag into account at all:

    ASSERT_EQ(0, ioctx.aio_operate(
    "foo" + stringify(n), completion, &op,
    librados::OPERATION_FULL_TRY));

the only full_try/force logic I see is a global flag on the client. :/