Project

General

Profile

Bug #5815

rest/test.py failure: FAILURE: url http://localhost:5000/api/v0.1/mds/setmap?epoch=11

Added by Samuel Just over 10 years ago. Updated over 10 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2013-07-31T02:01:51.721 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/compat/show HTTP/1.1" 200 -
2013-07-31T02:01:51.890 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "PUT /api/v0.1/mds/deactivate?who=2 HTTP/1.1" 400 -
2013-07-31T02:01:51.895 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/dump.json HTTP/1.1" 200 -
2013-07-31T02:01:51.901 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/dump.xml HTTP/1.1" 200 -
2013-07-31T02:01:51.906 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:33] "GET /api/v0.1/mds/getmap HTTP/1.1" 200 -
2013-07-31T02:01:52.074 INFO:teuthology.task.rest-api.client.rest0.err:[10.214.131.3]: 127.0.0.1 - - [31/Jul/2013 02:02:34] "PUT /api/v0.1/mds/setmap?epoch=11 HTTP/1.1" 400 -
2013-07-31T02:01:52.075 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: FAILURE: url  http://localhost:5000/api/v0.1/mds/setmap?epoch=11
2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: expected 200, got 400
2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: Response content:
2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: Error: next mdsmap epoch 12 != 11unrecognized command (-22)
2013-07-31T02:01:52.076 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: 
2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.err:[10.214.131.3]: Headers:  {'date': 'Wed, 31 Jul 2013 09:02:34 GMT', 'content-length': '61', 'content-type': 'text/html; charset=utf-8', 'server': 'Werkzeug/0.8.1 Python/2.7.3'}
2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: plain 200
2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export.json: json 200
2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export.xml: xml 200
2013-07-31T02:01:52.077 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: json 200
2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: xml 200
2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/add?entity=client.xx&caps=mon&caps=allow&caps=osd&caps=allow+*: json 200
2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export?entity=client.xx: plain 200
2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/add?entity=client.xx: plain 200
2013-07-31T02:01:52.078 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/list: plain 200
2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/list.json: json 200
2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/get-key?entity=client.xx: json 200
2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/print-key?entity=client.xx: json 200
2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/print_key?entity=client.xx: json 200
2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/caps?entity=client.xx&caps=osd&caps=allow+rw: json 200
2013-07-31T02:01:52.079 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/list.json: json 200
2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: plain 200
2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/import: plain 200
2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/export: plain 200
2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: auth/del?entity=client.xx: json 200
2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: osd/dump: json 200
2013-07-31T02:01:52.080 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df: plain 200
2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail=detail: plain 200
2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail: plain 200
2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df: json 200
2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df: xml 200
2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail: json 200
2013-07-31T02:01:52.081 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: df?detail: xml 200
2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: fsid: json 200
2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: health: json 200
2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: health?detail: json 200
2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: health?detail: plain 200
2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_down:  200
2013-07-31T02:01:52.082 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_down:  200
2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_up:  200
2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/cluster_up:  200
2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/rm_incompat?feature=4:  200
2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/rm_incompat?feature=4:  200
2013-07-31T02:01:52.083 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/show: json 200
2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/compat/show: xml 200
2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/deactivate?who=2:  400
2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/dump.json: json 200
2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/dump.xml: xml 200
2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/getmap:  200
2013-07-31T02:01:52.084 INFO:teuthology.task.workunit.client.0.out:[10.214.131.3]: mds/setmap?epoch=11: plain 400
2013-07-31T02:01:52.086 INFO:teuthology.task.workunit:Stopping rest/test.py on client.0...
2013-07-31T02:01:52.086 DEBUG:teuthology.orchestra.run:Running [10.214.131.3]: 'rm -rf -- /home/ubuntu/cephtest/91048/workunits.list /home/ubuntu/cephtest/91048/workunit.client.0'

Associated revisions

Revision c9e8ff2b (diff)
Added by Dan Mick over 10 years ago

rest/test.py: retry mds setmap in case epoch has changed

Loop 10 times retrying mds dump/getmap/setmap.

Fixes: #5815
Signed-off-by: Dan Mick <>

History

#1 Updated by Samuel Just over 10 years ago

ubuntu@teuthology:/a/teuthology-2013-07-31_01:00:23-rados-next-testing-basic-plana/91048

#2 Updated by Dan Mick over 10 years ago

OK, this is mds setmap racing against the epoch number we got from mds dump. Possible fixes: retry N times or just skip setmap.

#3 Updated by Dan Mick over 10 years ago

Also, as Sage points out, "unrecognized command" isn't really useful here.

#4 Updated by Dan Mick over 10 years ago

  • Status changed from New to Fix Under Review

Fix test failure by adding 10x retry to "get epoch, set epoch+1".

wip-5815

#5 Updated by Dan Mick over 10 years ago

  • Description updated (diff)

#6 Updated by Ian Colle over 10 years ago

  • Status changed from Fix Under Review to Resolved

wip-5815 merged into next

Also available in: Atom PDF