Actions
Bug #5815
closedrest/test.py failure: FAILURE: url http://localhost:5000/api/v0.1/mds/setmap?epoch=11
Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
other
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2013-07-31T02:01:51.721 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/compat/show HTTP/1.1" 200 - 2013-07-31T02:01:51.890 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "PUT /api/v0.1/mds/deactivate?who=2 HTTP/1.1" 400 - 2013-07-31T02:01:51.895 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/dump.json HTTP/1.1" 200 - 2013-07-31T02:01:51.901 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/dump.xml HTTP/1.1" 200 - 2013-07-31T02:01:51.906 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/getmap HTTP/1.1" 200 - 2013-07-31T02:01:52.074 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:34] "PUT /api/v0.1/mds/setmap?epoch=11 HTTP/1.1" 400 - 2013-07-31T02:01:52.075 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: FAILURE: url http://localhost:5000/api/v0.1/mds/setmap?epoch=11 2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: expected 200, got 400 2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: Response content: 2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: Error: next mdsmap epoch 12 != 11unrecognized command (-22) 2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: 2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: Headers: {'date': 'Wed, 31 Jul 2013 09:02:34 GMT', 'content-length': '61', 'content-type': 'text/html; charset=utf-8', 'server': 'Werkzeug/0.8.1 Python/2.7.3'} 2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: plain 200 2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export.json: json 200 2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export.xml: xml 200 2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: json 200 2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: xml 200 2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/add?entity=client.xx&caps=mon&caps=allow&caps=osd&caps=allow+*: json 200 2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export?entity=client.xx: plain 200 2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/add?entity=client.xx: plain 200 2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/list: plain 200 2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/list.json: json 200 2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/get-key?entity=client.xx: json 200 2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/print-key?entity=client.xx: json 200 2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/print_key?entity=client.xx: json 200 2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/caps?entity=client.xx&caps=osd&caps=allow+rw: json 200 2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/list.json: json 200 2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: plain 200 2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/import: plain 200 2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: plain 200 2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/del?entity=client.xx: json 200 2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: osd/dump: json 200 2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df: plain 200 2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail=detail: plain 200 2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail: plain 200 2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df: json 200 2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df: xml 200 2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail: json 200 2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail: xml 200 2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: fsid: json 200 2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: health: json 200 2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: health?detail: json 200 2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: health?detail: plain 200 2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_down: 200 2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_down: 200 2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_up: 200 2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_up: 200 2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/rm_incompat?feature=4: 200 2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/rm_incompat?feature=4: 200 2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/show: json 200 2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/show: xml 200 2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/deactivate?who=2: 400 2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/dump.json: json 200 2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/dump.xml: xml 200 2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/getmap: 200 2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/setmap?epoch=11: plain 400 2013-07-31T02:01:52.086 INFO:teuthology.task.workunit:Stopping rest/test.py on client.0... 2013-07-31T02:01:52.086 DEBUG:teuthology.orchestra.run:Running [10.214.131.3]: 'rm -rf -- /home/ubuntu/cephtest/91048/workunits.list /home/ubuntu/cephtest/91048/workunit.client.0'
Updated by Samuel Just over 10 years ago
ubuntu@teuthology:/a/teuthology-2013-07-31_01:00:23-rados-next-testing-basic-plana/91048
Updated by Dan Mick over 10 years ago
OK, this is mds setmap racing against the epoch number we got from mds dump. Possible fixes: retry N times or just skip setmap.
Updated by Dan Mick over 10 years ago
Also, as Sage points out, "unrecognized command" isn't really useful here.
Updated by Dan Mick over 10 years ago
- Status changed from New to Fix Under Review
Fix test failure by adding 10x retry to "get epoch, set epoch+1".
wip-5815
Updated by Ian Colle over 10 years ago
- Status changed from Fix Under Review to Resolved
wip-5815 merged into next
Actions