Bug #5240
run_seed_to_range failed, probably fdcache
0%
Description
2013-06-03T04:26:53.232 INFO:teuthology.orchestra.run.err:2013-06-03 04:27:34.948984 7fa652ef5780 0 filestore_diff diff_objects content mismatch on 0.14_head/6808bf5/obj5/head//-1
2013-06-03T04:26:53.233 INFO:teuthology.orchestra.run.err:2013-06-03 04:27:34.949414 7fa652ef5780 0 filestore_diff diff_objects num objs mismatch (A: 2, B: 1)
2013-06-03T04:26:53.237 INFO:teuthology.orchestra.run.err:2013-06-03 04:27:34.953682 7fa652ef5780 0 test_idempotent_sequence diff found an difference
2013-06-03T04:26:53.237 INFO:teuthology.orchestra.run.err:2013-06-03 04:27:34.954077 7fa652ef5780 1 journal close a.10/journal
2013-06-03T04:26:53.238 INFO:teuthology.orchestra.run.err:2013-06-03 04:27:34.954799 7fa652ef5780 1 journal close b.10/journal
2013-06-03T04:26:53.240 INFO:teuthology.orchestra.run.out:FAIL
2013-06-03T04:26:53.240 INFO:teuthology.orchestra.run.out: see:
2013-06-03T04:26:53.240 INFO:teuthology.orchestra.run.out: a.10.fail -- leading up to failure
2013-06-03T04:26:53.240 INFO:teuthology.orchestra.run.out: a.10.recover -- journal replay
2013-06-03T04:26:53.240 INFO:teuthology.orchestra.run.out: b.10.clean -- the clean reference
2013-06-03T04:26:53.240 INFO:teuthology.orchestra.run.out:no dir provided for evidence disposal
2013-06-03T04:26:53.241 INFO:teuthology.orchestra.run.err:./run_seed_to_range.sh: 15: ./run_seed_to_range.sh: d: not found /home/ubuntu/cephtest/30018/data/test.client.0'
2013-06-03T04:26:53.241 DEBUG:teuthology.orchestra.run:Running [10.214.131.21]: 'cp -a /home/ubuntu/cephtest/30018/data/test.client.0 /home/ubuntu/cephtest/30018/archive/idempotent_failure'
2013-06-03T04:26:55.426 DEBUG:teuthology.orchestra.run:Running [10.214.131.21]: 'rm -rf -
2013-06-03T04:26:55.652 ERROR:teuthology.run_tasks:Saw exception from tasks
Traceback (most recent call last):
File "/var/lib/teuthworker/teuthology-master/teuthology/run_tasks.py", line 25, in run_tasks
manager = _run_one_task(taskname, ctx=ctx, config=config)
File "/var/lib/teuthworker/teuthology-master/teuthology/run_tasks.py", line 14, in _run_one_task
return fn(*kwargs)
File "/var/lib/teuthworker/teuthology-master/teuthology/task/filestore_idempotent.py", line 69, in task
raise Exception("./run_seed_to_range.sh errored out")
Exception: ./run_seed_to_range.sh errored out
2013-06-03T04:26:55.666 DEBUG:teuthology.run_tasks:Unwinding manager <contextlib.GeneratorContextManager object at 0x27f1590>
2013-06-03T04:26:55.667 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
File "/var/lib/teuthworker/teuthology-master/teuthology/contextutil.py", line 27, in nested
yield vars
File "/var/lib/teuthworker/teuthology-master/teuthology/task/ceph.py", line 1096, in task
yield
File "/var/lib/teuthworker/teuthology-master/teuthology/run_tasks.py", line 25, in run_tasks
manager = _run_one_task(taskname, ctx=ctx, config=config)
File "/var/lib/teuthworker/teuthology-master/teuthology/run_tasks.py", line 14, in _run_one_task
return fn(*kwargs)
File "/var/lib/teuthworker/teuthology-master/teuthology/task/filestore_idempotent.py", line 69, in task
raise Exception("./run_seed_to_range.sh errored out")
Exception: ./run_seed_to_range.sh errored out
2013-06-03T04:26:55.667 INFO:teuthology.misc:Shutting down mds daemons...
ubuntu@teuthology:/a/teuthology-2013-06-03_01:00:10-rados-master-testing-basic/30018
Related issues
Associated revisions
test_filestore_idempotent: make newly created objects globally unique
The filestore requires hobjects to be globally unique.
Fixes: #5240
Signed-off-by: Samuel Just <sam.just@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
test_filestore_idempotent: use obj name from source coll add
Fixes: #5240
Reviewed-by: David Zafman <david.zafman@inktank.com>
Signed-off-by: Samuel Just <sam.just@inktank.com>
test: idempotent filestore test failure
Remove obsolete use of collection_move()
Allow operations to be skipped if random selections don't make sense
Track total number of possible objects in m_num_objects
BUG: do_remove() was calling _do_touch() instead of _do_remove()
For ops that require an object, select from among existing objects in collection
Initialize m_num_objects unique objects across collections
touch: don't create an object that already exists in another collection
remove: Use remove_obj() to clear object from m_objects to have accurate tracking
clone/clone_range(): Select 2 existing objects in the collection
add: Skip operation if selected target object name exists in target collection
move: Removed this buggy operation that is only present for upgrades
Signed-off-by: David Zafman <david.zafman@inktank.com>
History
#1 Updated by Samuel Just almost 11 years ago
Looks like the tester will place objects with the same name into different collections, fixing test.
#2 Updated by Samuel Just almost 11 years ago
- Status changed from In Progress to Resolved
#3 Updated by Samuel Just almost 11 years ago
- Status changed from Resolved to 12
2013-06-05T04:21:14.657 INFO:teuthology.orchestra.run.err:2013-06-05 04:21:58.389650 7faabeffd700 10 filestore(b) truncate meta/718608c/txn/head//-1 size 0 = 0
2013-06-05T04:21:14.657 INFO:teuthology.orchestra.run.err:2013-06-05 04:21:58.389660 7faabeffd700 15 filestore(b) write meta/718608c/txn/head//-1 0~4
2013-06-05T04:21:14.657 INFO:teuthology.orchestra.run.err:2013-06-05 04:21:58.389703 7faabeffd700 10 filestore(b) write meta/718608c/txn/head//-1 0~4 = 4
2013-06-05T04:21:14.657 INFO:teuthology.orchestra.run.err:2013-06-05 04:21:58.389711 7faabeffd700 15 filestore(b) remove 0.19_head/cbe4aa10/obj4-2/head//-1
2013-06-05T04:21:14.657 INFO:teuthology.orchestra.run.err:2013-06-05 04:21:58.389761 7faabeffd700 10 filestore(b) remove 0.19_head/cbe4aa10/obj4-2/head//-1 = -2
2013-06-05T04:21:14.657 INFO:teuthology.orchestra.run.err:2013-06-05 04:21:58.389768 7faabeffd700 15 filestore(b) collection_add 0.19_head/cbe4aa10/obj4-2/head//-1 from 0.3_head/cbe4aa10/obj4-2/head//-1
2013-06-05T04:21:14.657 INFO:teuthology.orchestra.run.err:2013-06-05 04:21:58.389809 7faabeffd700 10 filestore(b) error opening file b/current/0.3_head/obj4-2__head_CBE4AA10__none with flags=2: (2) No such file or directory
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err:os/FileStore.cc: In function 'int FileStore::_collection_add(coll_t, coll_t, const hobject_t&, const SequencerPosition&)' thread 7faabeffd700 time 2013-06-05 04:21:58.389833
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err:os/FileStore.cc: 4577: FAILED assert(replaying)
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: ceph version 0.63-352-g8c23d8c (8c23d8cb46d432ca3c840336e02a9eed868d78f3)
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 1: (FileStore::_collection_add(coll_t, coll_t, hobject_t const&, SequencerPosition const&)+0x7b6) [0x493266]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 2: (FileStore::_do_transaction(ObjectStore::Transaction&, unsigned long, int)+0x2639) [0x496b89]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 3: (FileStore::_do_transactions(std::list<ObjectStore::Transaction*, std::allocator<ObjectStore::Transaction*> >&, unsigned long, ThreadPool::TPHandle*)+0x71) [0x49b351]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 4: (FileStore::_do_op(FileStore::OpSequencer*, ThreadPool::TPHandle&)+0x274) [0x49b5e4]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 5: (ThreadPool::worker(ThreadPool::WorkThread*)+0x4e6) [0x5107a6]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 6: (ThreadPool::WorkThread::entry()+0x10) [0x5125d0]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 7: (()+0x7e9a) [0x7faac8fede9a]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: 8: (clone()+0x6d) [0x7faac780accd]
2013-06-05T04:21:14.658 INFO:teuthology.orchestra.run.err: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
#4 Updated by Samuel Just almost 11 years ago
- Status changed from 12 to Resolved