Project

General

Profile

Bug #21842

"repair kvstore failed" in qa/workunits/cephtool/test_kvstore_tool.sh

Added by Kefu Chai almost 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
Start date:
10/19/2017
Due date:
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:

Description

2017-10-18T17:18:21.710 INFO:tasks.workunit.client.0.smithi107.stdout:repair rocksdb failed : IO error: stat file: db: No such file or directory
2017-10-18T17:18:21.711 INFO:tasks.workunit.client.0.smithi107.stdout:repair kvstore failed
2017-10-18T17:18:21.740 INFO:tasks.workunit.client.0.smithi107.stderr:2017-10-18 17:18:21.732 7fd721d71380  1 stupidalloc shutdown
2017-10-18T17:18:21.741 INFO:tasks.workunit.client.0.smithi107.stderr:2017-10-18 17:18:21.732 7fd721d71380  1 bdev(0x56084e9d4f40 /tmp/cephtool.UNa/block) close
2017-10-18T17:18:21.982 INFO:tasks.workunit.client.0.smithi107.stderr:2017-10-18 17:18:21.976 7fd721d71380  1 bdev(0x56084e9d4d00 /tmp/cephtool.UNa/block) close
2017-10-18T17:18:22.019 INFO:tasks.workunit.client.0.smithi107.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test_kvstore_tool.sh:1: test_ceph_kvstore_tool:  rm -fr /tmp/cephtool.UNa
2017-10-18T17:18:22.022 INFO:tasks.workunit:Stopping ['cephtool'] on client.0...
2017-10-18T17:18:22.022 INFO:teuthology.orchestra.run.smithi107:Running: 'rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0'
2017-10-18T17:18:22.202 ERROR:teuthology.run_tasks:Saw exception from tasks.

http://qa-proxy.ceph.com/teuthology/kchai-2017-10-18_16:55:07-rados-wip-kefu-testing-2017-10-18-2318-distro-basic-smithi/1746146/teuthology.log

History

#1 Updated by Chang Liu almost 2 years ago

Working on it

#2 Updated by Chang Liu almost 2 years ago

Rocksdb::RepairDB tries to find all files: https://github.com/facebook/rocksdb/blob/master/db/repair.cc#L168, then it will call `env->GetChildren('db', result)`:https://github.com/facebook/rocksdb/blob/master/db/repair.cc#L271.

Our bluefs tells rocksdb that there is not such a directory called `db`.

I'm reproducing it in my env.

#3 Updated by Chang Liu almost 2 years ago

I can't figure out where creates "stat file: db". I guess we use this "stat file: db" as our dbname.

#4 Updated by Kefu Chai almost 2 years ago

Chang, no worries. i am fixing it.

#5 Updated by Kefu Chai almost 2 years ago

  • Assignee changed from Chang Liu to Kefu Chai

#6 Updated by Chang Liu almost 2 years ago

Kefu, I know why repair failed.

rocksdb's Env imports a new member function called AreSameFiles. But our BlueRocksEnv hasn't this function.

see https://github.com/ceph/ceph/pull/18390

#7 Updated by Kefu Chai almost 2 years ago

I can't figure out where creates "stat file: db". I guess we use this "stat file: db" as our dbname.

and it consistently fails the rados qa suite, so i took it away from you.

https://github.com/ceph/ceph/pull/18392

#8 Updated by Kefu Chai almost 2 years ago

  • Status changed from New to Need Review

#9 Updated by Kefu Chai almost 2 years ago

  • Status changed from Need Review to Resolved

Also available in: Atom PDF