Project

General

Profile

Actions
Copy link

Bug #1279

closed

osd: [WRN] bad locator

Added by Wido den Hollander almost 13 years ago. Updated almost 13 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
OSD
Target version:
-
% Done:

0%

Spent time:
2:00 h
Source:
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Like I mentioned yesterday, I'm seeing bad locator messages on my OSD's after uploading a file through the RADOS gateway (not using the FS right now).

I get these messages when I do a bucket list on the bucket.

s3 -u put wido-dump/EE20Final.rar filename=EE20Final.rar contentType=$(mimetype EE20Final.rar|awk '{print $2}') cannedAcl=public-read

My 'ceph -w' shows me:

2011-07-06 14:44:32.369314    pg v8839: 7992 pgs: 7992 active+clean; 38656 MB data, 113 GB used, 74326 GB / 74520 GB avail
2011-07-06 14:44:36.016068   log 2011-07-06 14:43:39.035756 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1535 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4192.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
2011-07-06 14:44:36.016068   log 2011-07-06 14:43:39.037881 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1536 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4192.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
2011-07-06 14:44:36.300930    pg v8840: 7992 pgs: 7992 active+clean; 38656 MB data, 113 GB used, 74326 GB / 74520 GB avail

My logging is a bit low at the moment, so my OSD log doesn't reveal that much useful information.

Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4191 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 1 ==== osd_op(client4191.0:9569 EE20Final.rar [writefull 0~3963052] 11.1fcf) v2 ==== 128+0+3963052 (3558807836 0 2649087991) 0x1794240 con 0x14ce000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 -- osd_sub_op(client4191.0:9569 11.7 EE20Final.rar/head [] v 19'1 snapset=0=[]:[] snapc=0=[]) v1 -- ?+3963438 0x1986600
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 -- osd_sub_op(client4191.0:9569 11.7 EE20Final.rar/head [] v 19'1 snapset=0=[]:[] snapc=0=[]) v1 -- ?+3963438 0x1b62100
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 181 ==== osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ack = 0) v1 ==== 120+0+0 (1427496518 0 0) 0x1a1a6c0 con 0x1b34280
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd69e0700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 615 ==== osd_sub_op(client4191.0:9569 11.7 EE20Final.rar/head [] v 19'1 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+3963438 (1223011242 0 1487582381) 0x184e000 con 0x1f90c80
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024981d700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 183 ==== osd_sub_op(client4191.0:9569 11.7 EE20Final.rar/head [] v 19'1 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+3963438 (1223011242 0 1487582381) 0x240c080 con 0x277bdc0
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024b821700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ack = 0) v1 -- ?+0 0x1837000
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd89e4700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ack = 0) v1 -- ?+0 0x1fc9240
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 666 ==== osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ack = 0) v1 ==== 120+0+0 (371651089 0 0) 0x1a1a900 con 0x1470a00
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 182 ==== osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (3447005260 0 0) 0x1794480 con 0x1b34280
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024e026700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0x2072240
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb39e81700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 -- osd_op_reply(9569 EE20Final.rar [writefull 0~3963052] ack = 0) v1 -- ?+0 0x1bc1700 con 0x14ce000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4191 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 2 ==== osd_op(client4191.0:9570 EE20Final.rar [setxattr user.rgw.acl (239)] 11.1fcf) v2 ==== 128+0+251 (1288368260 0 3629669914) 0xfded80 con 0x14ce000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 -- osd_sub_op(client4191.0:9570 11.7 EE20Final.rar/head [] v 19'2 snapset=0=[]:[] snapc=0=[]) v1 -- ?+626 0x173b100
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 -- osd_sub_op(client4191.0:9570 11.7 EE20Final.rar/head [] v 19'2 snapset=0=[]:[] snapc=0=[]) v1 -- ?+626 0x15f3580
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd69e0700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 616 ==== osd_sub_op(client4191.0:9570 11.7 EE20Final.rar/head [] v 19'2 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+626 (2560212998 0 3993394667) 0x1adb080 con 0x1f90c80
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024981d700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 184 ==== osd_sub_op(client4191.0:9570 11.7 EE20Final.rar/head [] v 19'2 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+626 (2560212998 0 3993394667) 0x240b580 con 0x277bdc0
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd89e4700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9570 11.7 EE20Final.rar/head [] ack = 0) v1 -- ?+0 0xd9bb40
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024e026700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9570 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0x1bfd900
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 667 ==== osd_sub_op_reply(client4191.0:9570 11.7 EE20Final.rar/head [] ack = 0) v1 ==== 120+0+0 (3600838188 0 0) 0x1a1ad80 con 0x1470a00
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 183 ==== osd_sub_op_reply(client4191.0:9570 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (2842452789 0 0) 0x188eb40 con 0x1b34280
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 -- osd_op_reply(9570 EE20Final.rar [setxattr user.rgw.acl (239)] ack = 0) v1 -- ?+0 0x1bd0540 con 0x14ce000
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd69e0700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 617 ==== osd_sub_op(client4191.0:9571 11.7 EE20Final.rar/head [] v 19'3 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+414 (2429227266 0 800439570) 0x1b1d600 con 0x1f90c80
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024981d700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 185 ==== osd_sub_op(client4191.0:9571 11.7 EE20Final.rar/head [] v 19'3 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+414 (2429227266 0 800439570) 0x25e7b00 con 0x277bdc0
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4191 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 3 ==== osd_op(client4191.0:9571 EE20Final.rar [setxattr user.rgw.content_type (18)] 11.1fcf) v2 ==== 128+0+39 (4090768943 0 1872184384) 0x1ac0b40 con 0x14ce000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 -- osd_sub_op(client4191.0:9571 11.7 EE20Final.rar/head [] v 19'3 snapset=0=[]:[] snapc=0=[]) v1 -- ?+414 0x15f3000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 -- osd_sub_op(client4191.0:9571 11.7 EE20Final.rar/head [] v 19'3 snapset=0=[]:[] snapc=0=[]) v1 -- ?+414 0x1a95080
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd89e4700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9571 11.7 EE20Final.rar/head [] ack = 0) v1 -- ?+0 0x1fc9000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 668 ==== osd_sub_op_reply(client4191.0:9571 11.7 EE20Final.rar/head [] ack = 0) v1 ==== 120+0+0 (45013439 0 0) 0x188e900 con 0x1470a00
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024e026700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9571 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0x2072480
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 184 ==== osd_sub_op_reply(client4191.0:9571 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (2487074729 0 0) 0x188e6c0 con 0x1b34280
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 -- osd_op_reply(9571 EE20Final.rar [setxattr user.rgw.content_type (18)] ack = 0) v1 -- ?+0 0x1c518c0 con 0x14ce000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4191 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 4 ==== osd_op(client4191.0:9572 EE20Final.rar [setxattr user.rgw.etag (33)] 11.1fcf) v2 ==== 128+0+46 (1664136672 0 3937289081) 0xfe8000 con 0x14ce000
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb35d78700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 -- osd_sub_op(client4191.0:9572 11.7 EE20Final.rar/head [] v 19'4 snapset=0=[]:[] snapc=0=[]) v1 -- ?+421 0x1871b00
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb35d78700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 -- osd_sub_op(client4191.0:9572 11.7 EE20Final.rar/head [] v 19'4 snapset=0=[]:[] snapc=0=[]) v1 -- ?+421 0x1a95b80
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024981d700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 186 ==== osd_sub_op(client4191.0:9572 11.7 EE20Final.rar/head [] v 19'4 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+421 (1401358560 0 135791029) 0x1ea5b00 con 0x277bdc0
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024b821700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ack = 0) v1 -- ?+0 0x1bfd900
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd69e0700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 <== osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 618 ==== osd_sub_op(client4191.0:9572 11.7 EE20Final.rar/head [] v 19'4 snapset=0=[]:[] snapc=0=[]) v3 ==== 518+0+421 (1401358560 0 135791029) 0x1a2a680 con 0x1f90c80
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 185 ==== osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ack = 0) v1 ==== 120+0+0 (3742706826 0 0) 0xfde6c0 con 0x1b34280
Jul  6 14:43:18 atom9 osd.38[5269]: 7f024e026700 -- [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0x206e240
Jul  6 14:43:18 atom3 osd.12[4738]: 7fddd89e4700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ack = 0) v1 -- ?+0 0x160e240
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd38 [2a00:f10:113:1:225:90ff:fe33:49a4]:6807/5268 186 ==== osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (396533784 0 0) 0x1aa1000 con 0x1b34280
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 669 ==== osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ack = 0) v1 ==== 120+0+0 (626995876 0 0) 0x188eb40 con 0x1470a00
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1004881 -- osd_op_reply(9572 EE20Final.rar [setxattr user.rgw.etag (33)] ack = 0) v1 -- ?+0 0x1c81700 con 0x14ce000
Jul  6 14:43:18 atom3 osd.12[4738]: 7fdddb1e9700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0x19f8480
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 670 ==== osd_sub_op_reply(client4191.0:9569 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (3646652921 0 0) 0x1a1a900 con 0x1470a00
Jul  6 14:43:18 atom3 osd.12[4738]: 7fdddb1e9700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9570 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0x1774d80
Jul  6 14:43:18 atom3 osd.12[4738]: 7fdddb1e9700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9571 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0x1fc9240
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 671 ==== osd_sub_op_reply(client4191.0:9570 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (3175268992 0 0) 0x1542480 con 0x1470a00
Jul  6 14:43:18 atom3 osd.12[4738]: 7fdddb1e9700 -- [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 --> osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 -- osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ondisk = 0) v1 -- ?+0 0xd9bb40
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 672 ==== osd_sub_op_reply(client4191.0:9571 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (2148736540 0 0) 0x1542d80 con 0x1470a00
Jul  6 14:43:18 atom8 osd.32[5156]: 7fbb37e7d700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6801/5155 <== osd12 [2a00:f10:113:1:225:90ff:fe33:49b0]:6801/4737 673 ==== osd_sub_op_reply(client4191.0:9572 11.7 EE20Final.rar/head [] ondisk = 0) v1 ==== 120+0+0 (59507117 0 0) 0x1542000 con 0x1470a00
Jul  6 14:43:39 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4192 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005537 2 ==== osd_op(client4192.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2 ==== 141+0+0 (3514418091 0 0) 0xfc16c0 con 0x1abb000
Jul  6 14:43:39 atom8 osd.32[5156]: 7fbb36579700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4192.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:43:39 atom8 osd.32[5156]: 2011-07-06 14:43:39.035756 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1535 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4192.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:43:39 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005537 -- osd_op_reply(25 EE20Final.rar [stat 0~0] = 0) v1 -- ?+16 0x169d700 con 0x1abb000
Jul  6 14:43:39 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4192 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005537 3 ==== osd_op(client4192.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2 ==== 141+0+0 (2823864012 0 0) 0x1661480 con 0x1abb000
Jul  6 14:43:39 atom8 osd.32[5156]: 7fbb35d78700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4192.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:43:39 atom8 osd.32[5156]: 2011-07-06 14:43:39.037881 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1536 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4192.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:43:39 atom8 osd.32[5156]: 7fbb35d78700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005537 -- osd_op_reply(26 EE20Final.rar [getxattrs] = 0) v1 -- ?+364 0x1a17e00 con 0x1abb000
Jul  6 14:47:45 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4194 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005565 2 ==== osd_op(client4194.0:9565 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2 ==== 141+0+0 (3514418091 0 0) 0x171f240 con 0x1abbb40
Jul  6 14:47:45 atom8 osd.32[5156]: 7fbb35d78700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4194.0:9565 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:45 atom8 osd.32[5156]: 2011-07-06 14:47:45.696755 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1537 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4194.0:9565 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:45 atom8 osd.32[5156]: 7fbb35d78700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005565 -- osd_op_reply(9565 EE20Final.rar [stat 0~0] = 0) v1 -- ?+16 0x194d8c0 con 0x1abbb40
Jul  6 14:47:45 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4194 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005565 3 ==== osd_op(client4194.0:9566 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2 ==== 141+0+0 (2823864012 0 0) 0x171fb40 con 0x1abbb40
Jul  6 14:47:45 atom8 osd.32[5156]: 7fbb36579700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4194.0:9566 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:45 atom8 osd.32[5156]: 2011-07-06 14:47:45.698104 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1538 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4194.0:9566 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:45 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005565 -- osd_op_reply(9566 EE20Final.rar [getxattrs] = 0) v1 -- ?+364 0x1bb3a80 con 0x1abbb40
Jul  6 14:47:51 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4196 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005586 2 ==== osd_op(client4196.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2 ==== 141+0+0 (3514418091 0 0) 0x19dfb40 con 0x1abb140
Jul  6 14:47:51 atom8 osd.32[5156]: 7fbb36579700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4196.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:51 atom8 osd.32[5156]: 2011-07-06 14:47:51.369236 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1539 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4196.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:51 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005586 -- osd_op_reply(25 EE20Final.rar [stat 0~0] = 0) v1 -- ?+16 0x1a87380 con 0x1abb140
Jul  6 14:47:51 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4196 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005586 3 ==== osd_op(client4196.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2 ==== 141+0+0 (2823864012 0 0) 0x171f480 con 0x1abb140
Jul  6 14:47:51 atom8 osd.32[5156]: 7fbb35d78700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4196.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:51 atom8 osd.32[5156]: 2011-07-06 14:47:51.370278 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1540 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4196.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:51 atom8 osd.32[5156]: 7fbb35d78700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005586 -- osd_op_reply(26 EE20Final.rar [getxattrs] = 0) v1 -- ?+364 0x1a21a80 con 0x1abb140
Jul  6 14:47:52 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4198 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005604 2 ==== osd_op(client4198.0:34 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2 ==== 141+0+0 (3514418091 0 0) 0x19df240 con 0x1abb3c0
Jul  6 14:47:52 atom8 osd.32[5156]: 7fbb35d78700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4198.0:34 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:52 atom8 osd.32[5156]: 2011-07-06 14:47:52.213915 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1541 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4198.0:34 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:52 atom8 osd.32[5156]: 7fbb35d78700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005604 -- osd_op_reply(34 EE20Final.rar [stat 0~0] = 0) v1 -- ?+16 0x1c45000 con 0x1abb3c0
Jul  6 14:47:52 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4198 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005604 3 ==== osd_op(client4198.0:35 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2 ==== 141+0+0 (2823864012 0 0) 0x19df480 con 0x1abb3c0
Jul  6 14:47:52 atom8 osd.32[5156]: 7fbb36579700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4198.0:35 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:52 atom8 osd.32[5156]: 2011-07-06 14:47:52.215176 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1542 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4198.0:35 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:52 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005604 -- osd_op_reply(35 EE20Final.rar [getxattrs] = 0) v1 -- ?+364 0x1c7b700 con 0x1abb3c0
Jul  6 14:47:53 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4203 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005682 2 ==== osd_op(client4203.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2 ==== 141+0+0 (3514418091 0 0) 0xf40900 con 0x1abbc80
Jul  6 14:47:53 atom8 osd.32[5156]: 7fbb36579700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4203.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:53 atom8 osd.32[5156]: 2011-07-06 14:47:53.206716 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1543 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4203.0:25 EE20Final.rar @11:EE20Final.rar [stat 0~0] 11.1fcf) v2
Jul  6 14:47:53 atom8 osd.32[5156]: 7fbb36579700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005682 -- osd_op_reply(25 EE20Final.rar [stat 0~0] = 0) v1 -- ?+16 0x19361c0 con 0x1abbc80
Jul  6 14:47:53 atom8 osd.32[5156]: 7fbb3867e700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 <== client4203 [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005682 3 ==== osd_op(client4203.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2 ==== 141+0+0 (2823864012 0 0) 0xf40d80 con 0x1abbc80
Jul  6 14:47:53 atom8 osd.32[5156]: 7fbb35d78700 log [WRN] : bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4203.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:53 atom8 osd.32[5156]: 2011-07-06 14:47:53.207746 osd32 [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 1544 : [WRN] bad locator @11:EE20Final.rar on object @11 loc @11:EE20Final.rar op osd_op(client4203.0:26 EE20Final.rar @11:EE20Final.rar [getxattrs] 11.1fcf) v2
Jul  6 14:47:53 atom8 osd.32[5156]: 7fbb35d78700 -- [2a00:f10:113:1:225:90ff:fe33:498e]:6800/5155 --> [2a00:f10:113:1:230:48ff:feba:fbc0]:0/1005682 -- osd_op_reply(26 EE20Final.rar [getxattrs] = 0) v1 -- ?+364 0x16e88c0 con 0x1abbc80

I manually verified the md5sum of the object on osd32, that is correct.

I'm running version 0.30 on all my nodes.

Actions
Copy link
 #1

Updated by Yehuda Sadeh almost 13 years ago

I can't reproduce (this specific problem), what's your rgw commit version?

Actions
Copy link
 #2

Updated by Wido den Hollander almost 13 years ago 

root@gateway:~# radosgw --version
ceph version 0.30 (commit:64b1b2c70f0cde39c72d5d724c65ea8afaaa00b9)
root@gateway:~#
Actions
Copy link
 #3

Updated by Yehuda Sadeh almost 13 years ago

  • Status changed from New to Resolved

Ok, the issue happened actually when listing objects inside a bucket. From what I understand this warning in this situation is harmless. Anyway, pushed a fix: 8f9eaf0de30c2a1a5339f17d9170ba049acd1835.

Actions
Copy link

Also available in: Atom PDF