Bug #15240: ceph_test_rados_open_pools_parallel: ENOENT on write - Ceph - Ceph

Actions

Copy link

Bug #15240

closed

ceph_test_rados_open_pools_parallel: ENOENT on write

Added by Samuel Just about 8 years ago. Updated about 8 years ago.

Status:

Resolved

Priority:

Urgent

Assignee:

Kefu Chai

Category:

Target version:

% Done:

Source:

other

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

2016-03-21T17:04:22.198 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_1_⁵⁴⁷⁶: starting.
2016-03-21T17:04:22.198 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_2_⁵⁴⁷⁷: starting.
2016-03-21T17:04:22.198 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_1_⁵⁴⁷⁶: creating pool foo
2016-03-21T17:04:22.199 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_1_⁵⁴⁷⁶: created object 0...
2016-03-21T17:04:22.199 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_1_⁵⁴⁷⁶: created object 25...
2016-03-21T17:04:22.199 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_1_⁵⁴⁷⁶: created object 49...
2016-03-21T17:04:22.199 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_1_⁵⁴⁷⁶: finishing.
2016-03-21T17:04:22.199 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_1_⁵⁴⁷⁶: shutting down.
2016-03-21T17:04:22.200 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_2_⁵⁴⁷⁷: rados_pool_create.
2016-03-21T17:04:22.200 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_2_⁵⁴⁷⁷: rados_ioctx_create.
2016-03-21T17:04:22.200 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_2_⁵⁴⁷⁷: shutting down.
2016-03-21T17:04:22.200 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: ***********************
2016-03-21T17:04:22.200 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_4_⁶¹⁶⁴: starting.
2016-03-21T17:04:22.201 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: ***********************
2016-03-21T17:04:22.201 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_3_⁶¹⁶³: starting.
2016-03-21T17:04:22.201 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_4_⁶¹⁶⁴: rados_pool_create.
2016-03-21T17:04:22.201 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_4_⁶¹⁶⁴: rados_ioctx_create.
2016-03-21T17:04:22.201 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_4_⁶¹⁶⁴: shutting down.
2016-03-21T17:04:22.202 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_3_⁶¹⁶³: creating pool foo
2016-03-21T17:04:22.202 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_3_⁶¹⁶³: created object 0...
2016-03-21T17:04:22.202 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_3_⁶¹⁶³: rados_write error -2
2016-03-21T17:04:22.202 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_3_⁶¹⁶³: finishing.
2016-03-21T17:04:22.202 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: process_3_⁶¹⁶³: shutting down.
2016-03-21T17:04:22.203 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: ***********************
2016-03-21T17:04:22.203 INFO:tasks.workunit.client.0.smithi001.stdout: open_pools_parallel: test2: got error: run_until_finished: runnable process_3: got error: [5355] returned exit_status (254) Unknown error 254

/a/samuelj-2016-03-21_13:58:56-rados-wip-sam-testing-distro-basic-smithi/78465

I think this was hidden by test.sh suppressing errors http://tracker.ceph.com/issues/15165

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Samuel Just about 8 years ago

/a/samuelj-2016-03-21_13:58:56-rados-wip-sam-testing-distro-basic-smithi/78529

Actions

Copy link

Updated by Samuel Just about 8 years ago

/a/samuelj-2016-03-21_13:58:56-rados-wip-sam-testing-distro-basic-smithi/78486

Actions

Copy link

Updated by Samuel Just about 8 years ago

http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2016-03-24_05:00:02-smoke-master-distro-basic-openstack/21869/teuthology.log (failed rados/test.sh)

Actions

Copy link

Updated by Kefu Chai about 8 years ago

ran

$ zgrep '.obj \[write' remote/*/log/ceph-osd.*.log.gz|grep osd_op_reply|grep 'No such file or directory'

in /a/samuelj-2016-03-21_13:58:56-rados-wip-sam-testing-distro-basic-smithi/78529, but it returns nothing.

posting a pr at https://github.com/ceph/ceph/pull/8309 to print the oid which fails the rados_write(), and it gets merged.

Actions

Copy link

Updated by Kefu Chai about 8 years ago

Has duplicate Bug #15284: ceph_test_rados_open_pools_parallel: ENOENT on write added

Actions

Copy link

Updated by Kefu Chai about 8 years ago

in #15284 per sage

there is no osd_op in the osd logs during that time period, which suggests that the client osdmap didn't have the pool yet. objecter bug?

Actions

Copy link

Updated by Kefu Chai about 8 years ago

Subject changed from test.sh: open_pools_parallel failure to ceph_test_rados_open_pools_parallel: ENOENT on write
Assignee set to Kefu Chai

Actions

Copy link