Project

General

Profile

Actions

Bug #21327

closed

ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg

Added by Robin Johnson over 6 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
David Zafman
Category:
OSD
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
jewel, luminous
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The ceph-objectstore-tool positional commands scan all PGs instead of taking the specified PG.

As a workaround, you can use --op list to get the JSON, and pass that.

Command:

pg=5.e53f osd=166 obj=.dir.default.200379345.14409.21
journal=/srv/ceph/devices/osd.${osd}.journal
datapath=/srv/ceph/osd/$osd/
ceph-objectstore-tool --data-path $datapath --pgid $pg --journal /srv/ceph/devices/osd.${osd}.journal --debug $obj get-omaphdr | tee /tmp/omaphdr

Output:

2017-09-08 22:36:26.163881 7f4cecdd9980  0 filestore(/srv/ceph/osd/166/) backend xfs (magic 0x58465342)
2017-09-08 22:36:26.164422 7f4cecdd9980  0 genericfilestorebackend(/srv/ceph/osd/166/) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2017-09-08 22:36:26.164433 7f4cecdd9980  0 genericfilestorebackend(/srv/ceph/osd/166/) detect_features: SEEK_DATA/SEEK_HOLE is disabled via 'filestore seek data hole' config option
2017-09-08 22:36:26.164453 7f4cecdd9980  0 genericfilestorebackend(/srv/ceph/osd/166/) detect_features: splice is supported
2017-09-08 22:36:26.180020 7f4cecdd9980  0 genericfilestorebackend(/srv/ceph/osd/166/) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2017-09-08 22:36:26.180146 7f4cecdd9980  0 xfsfilestorebackend(/srv/ceph/osd/166/) detect_feature: extsize is disabled by conf
#### COMMENT: this long wait is scanning LevelDB ####
#### COMMENT: this long wait is scanning LevelDB ####
#### COMMENT: this long wait is scanning LevelDB ####
2017-09-08 22:38:49.192232 7f4cecdd9980  0 filestore(/srv/ceph/osd/166/) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 28 00 00 00 00 20 00 00 00 00 00 00 85 01 03 0a 00 00 00 00 00 00 00 00 00
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 28 00 00 00 00 20 00 00 00 00 00 00 85 01 03 0a 00 00 00 00 00 00 00 00 00
 HDIO_DRIVE_CMD(identify) failed: Input/output error
2017-09-08 22:38:49.226694 7f4cecdd9980  1 journal _open /srv/ceph/devices/osd.166.journal fd 14: 10736369664 bytes, block size 4096 bytes, directio = 1, aio = 1
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 28 00 00 00 00 20 00 00 00 00 00 00 85 01 03 0a 00 00 00 00 00 00 00 00 00
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 28 00 00 00 00 20 00 00 00 00 00 00 85 01 03 0a 00 00 00 00 00 00 00 00 00
 HDIO_DRIVE_CMD(identify) failed: Input/output error
2017-09-08 22:38:49.281060 7f4cecdd9980  1 journal _open /srv/ceph/devices/osd.166.journal fd 14: 10736369664 bytes, block size 4096 bytes, directio = 1, aio = 1
2017-09-08 22:38:49.283207 7f4cecdd9980  1 filestore(/srv/ceph/osd/166/) upgrade
Cluster fsid=31b8be2f-ac05-4e56-96b7-e702df166e29
Supported features: compat={},rocompat={},incompat={1=initial feature set(~v.18),2=pginfo object,3=object locator,4=last_epoch_clean,5=categories,6=hobjectpool,7=biginfo,8=leveldbinfo,9=leveldblog,10=snapmapper,11=sharded objects,12=transaction hints,13=pg meta object}
On-disk features: compat={},rocompat={},incompat={1=initial feature set(~v.18),2=pginfo object,3=object locator,4=last_epoch_clean,5=categories,6=hobjectpool,7=biginfo,8=leveldbinfo,9=leveldblog,10=snapmapper,12=transaction hints,13=pg meta object}
200 pgs to scan
Scanning 5.ea22_head, 0/200 completed
Error getting attr on : 5.ea22_head,#-7:44570000:::scrub_5.ea22:head#, (61) No data available
Scanning 5.cb76_head, 1/200 completed
Scanning 5.89e3_head, 2/200 completed
Error getting attr on : 5.89e3_head,#-7:c7910000:::scrub_5.89e3:head#, (61) No data available
Scanning 5.3d51_head, 3/200 completed
Scanning 5.23fb_head, 4/200 completed
Scanning 5.ef12_head, 5/200 completed
Scanning 5.3995_head, 6/200 completed
Error getting attr on : 5.3995_head,#-7:a99c0000:::scrub_5.3995:head#, (61) No data available

Workaround example:

pg=5.e53f osd=166 obj=.dir.default.200379345.14409.21
journal=/srv/ceph/devices/osd.${osd}.journal
datapath=/srv/ceph/osd/$osd/
ceph-objectstore-tool --data-path $datapath --pgid $pg --journal /srv/ceph/devices/osd.${osd}.journal --debug --op list | fgrep $obj | tee  /tmp/obj.json
cat /tmp/obj.json
ceph-objectstore-tool --data-path $datapath --pgid $pg --journal /srv/ceph/devices/osd.${osd}.journal --debug "$(cat /tmp/obj.json)" get-omaphdr | tee /tmp/omaphdr

/tmp/obj.json:
["5.e53f",{"oid":".dir.default.200379345.14409.21","key":"","snapid":-2,"hash":3842893119,"max":0,"pool":5,"namespace":"","max":0}]

Related issues 2 (0 open2 closed)

Copied to Ceph - Backport #22911: luminous: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pgResolvedDavid ZafmanActions
Copied to Ceph - Backport #22912: jewel: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pgResolvedDavid ZafmanActions
Actions #1

Updated by David Zafman over 6 years ago

  • Assignee set to David Zafman
Actions #2

Updated by David Zafman over 6 years ago

  • Status changed from New to In Progress
Actions #3

Updated by David Zafman over 6 years ago

  • Backport set to Jewel, Luminous
Actions #4

Updated by David Zafman over 6 years ago

  • Status changed from In Progress to 17
Actions #5

Updated by David Zafman about 6 years ago

  • Status changed from 17 to Pending Backport
Actions #6

Updated by Nathan Cutler about 6 years ago

  • Backport changed from Jewel, Luminous to jewel, luminous

Using lower-case "jewel", "luminous" to appease backport tooling.

Actions #7

Updated by Nathan Cutler about 6 years ago

  • Copied to Backport #22911: luminous: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg added
Actions #8

Updated by Nathan Cutler about 6 years ago

  • Copied to Backport #22912: jewel: ceph-objectstore-tool: "$OBJ get-omaphdr" and "$OBJ list-omap" scan all pgs instead of using specific pg added
Actions #9

Updated by David Zafman about 6 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF