Actions
Bug #5251
closedwrong node messages in mds log
Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
when upgrading from bobtail to next branch, seeing repeated wrong node messages in the osd logs.
2013-06-04 12:21:01.505350 7f351729a700 1 mds.0.2 waiting for osdmap 7 (which blacklists prior instance) 2013-06-04 12:21:01.522313 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53185 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.1 31.32:6803/20680 - wrong node! 2013-06-04 12:21:01.522388 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53185 s=1 pgs=0 cs=0 l=1).fault 2013-06-04 12:21:01.522680 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :45979 s=1 pgs=0 cs=0 l=1).connect claims to be 0.0.0.0:6803/26492 not 10.214.131.31: 6803/25734 - wrong node! 2013-06-04 12:21:01.522742 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :45979 s=1 pgs=0 cs=0 l=1).fault 2013-06-04 12:21:01.522823 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53188 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.131.32:6803/20680 - wrong node! 2013-06-04 12:21:01.522876 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44142 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6800/26492 not 10.214.131.31:6800/25733 - wrong node! 2013-06-04 12:21:01.522951 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44142 s=1 pgs=0 cs=0 l=1).fault 2013-06-04 12:21:01.523499 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44145 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6800/26492 not 10.214.131.31:6800/25733 - wrong node! 2013-06-04 12:21:01.523541 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :45982 s=1 pgs=0 cs=0 l=1).connect claims to be 0.0.0.0:6803/26492 not 10.214.131.31:6803/25734 - wrong node! 2013-06-04 12:21:01.723426 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53191 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.131.32:6803/20680 - wrong node! 2013-06-04 12:21:01.724294 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44147 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6800/26492 not 10.214.131.31:6800/25733 - wrong node! 2013-06-04 12:21:01.724380 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :45986 s=1 pgs=0 cs=0 l=1).connect claims to be 0.0.0.0:6803/26492 not 10.214.131.31:6803/25734 - wrong node! 2013-06-04 12:21:02.124057 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53198 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.131.32:6803/20680 - wrong node! 2013-06-04 12:21:02.125059 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :45993 s=1 pgs=0 cs=0 l=1).connect claims to be 0.0.0.0:6803/26492 not 10.214.131.31:6803/25734 - wrong node! 2013-06-04 12:21:02.125126 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44154 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6800/26492 not 10.214.131.31:6800/25733 - wrong node! 2013-06-04 12:21:02.924607 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53206 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.131.32:6803/20680 - wrong node! 2013-06-04 12:21:02.925946 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :46000 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6803/26492 not 10.214.131.31:6803/25734 - wrong node! 2013-06-04 12:21:02.925995 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44163 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6800/26492 not 10.214.131.31:6800/25733 - wrong node! 2013-06-04 12:21:04.525170 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53216 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.131.32:6803/20680 - wrong node! 2013-06-04 12:21:04.526761 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :46010 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6803/26492 not 10.214.131.31:6803/25734 - wrong node! 2013-06-04 12:21:04.526810 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44173 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6800/26492 not 10.214.131.31:6800/25733 - wrong node! 2013-06-04 12:21:07.725852 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53232 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.131.32:6803/20680 - wrong node! 2013-06-04 12:21:07.727634 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :46026 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6803/26492 not 10.214.131.31:6803/25734 - wrong node! 2013-06-04 12:21:07.727640 7f3514792700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6800/25733 pipe(0x16a6780 sd=19 :44189 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6800/26492 not 10.214.131.31:6800/25733 - wrong node! 2013-06-04 12:21:14.126429 7f351ba16700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.32:6803/20680 pipe(0x16a6280 sd=17 :53241 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.32:6803/21481 not 10.214.131.32:6803/20680 - wrong node! 2013-06-04 12:21:14.128540 7f3514893700 0 -- 10.214.131.32:6800/21468 >> 10.214.131.31:6803/25734 pipe(0x168b000 sd=18 :46035 s=1 pgs=0 cs=0 l=1).connect claims to be 10.214.131.31:6803/26492 not 10.214.131.31:6803/25734 - wrong node! to reproduce: tamil@ubuntu:/tmp/up$ cat config.yaml roles: - - mon.a - mds.a - osd.0 - osd.1 - - mon.b - mon.c - osd.2 - osd.3 - client.0 targets: ubuntu@plana08.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCqqzUa6OfNvqirlipO2jY32KanSfk2mSstJJdakPuKVijDoaDHvC7L14A9iNMChI/+OgVJRpBF4DI+QwrO00yj4jSGSuq/dtp6vxkX5fw1+g21uqZG7Sl6S89ytRkRs+NFoNY1jhWR0Qo4opEim9qApVSxlouG61L++IEv7zhZ62ogpknTMQhkgpHJ4w146silaCh6vnmoNsTBt+eIuVE/7vhMQep4REpw5uVZR6PVUBsJDJAkJuyAkNu3Xva7KC4W22sjzSqHKWqzNeAmPxQ8Ywvu5PWQulOtA/LF9gAVsJjbKE7+ZsXVYvTfpZtEKfOduss8dB3lP8Xez6CbUzsv ubuntu@plana09.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsROY/kuvJRrw3yOos6aHDtveOLPgZ+mBs4ul/O6LTm0swWIpeiilmIeILba12z98XmkoqIGTAgZqcGx6FfNpwrOg8P2pdDg9YKT9YrdESlFZryHyu3Rjyn+lZcskMdHAoXhTUzRkHh7t9/vM5cuhq+BoO5+nAebtFCQ1mcn9w8jn6ZqnXkQiplB93UBmXqGKhbqnaGok6xJfYtV3NtmXOFBJ0keja0rOT4ylnSBExG3YBdZ4qSEaWVNeBtoDslu6K0hq5/mmNtZwJf2iXQWOkhfM3tVUbcMfwaGVEe77gzg3CeU0lT7ag1oJan9MAxzVBqOXO1ZS2h8w4zBrX7eYt tasks: - internal.save_config: null - internal.check_lock: null - internal.connect: null - internal.check_conflict: null - internal.check_ceph_data: null - internal.vm_setup: null - internal.base: null - internal.archive: null - internal.coredump: null - internal.syslog: null - internal.timer: null - install: branch: bobtail - ceph: null - workunit: clients: all: - misc/trivial_sync.sh - install.upgrade: all: branch: next - ceph.restart: - mon.a - mon.b - mon.c - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - workunit: clients: all: - misc/trivial_sync.sh
Updated by Sage Weil over 10 years ago
- Status changed from New to Can't reproduce
Actions