Project

General

Profile

Bug #42477

Rados should use the '-o outfile' convention

Added by Christian Huebner almost 4 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Brad Hubbard
Category:
Administration/Usability
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus, mimic
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
31310
Crash signature (v1):
Crash signature (v2):

Description

I have a healthy Ceph cluster with Nautilus 14.2.4. When I issue 'rados df' on it, I get the correct result, both with plain text and JSON. If I try to write the output to disk, though, the rados tool throws the following error:

crh@node1:~$ rados df -o rados_df.json
Invalid value for object-size: strict_iecstrtoll: illegal prefix (length > 2)

No data is written to the outfile. Without the -o flag, I get valid text or JSON output.

This syntax works with other Ceph commands (e.g. ceph osd df -f json -o test.json) and I infer that it should work with rados df, too.

Related issues

Copied to RADOS - Backport #44360: nautilus: Rados should use the '-o outfile' convention Resolved
Copied to RADOS - Backport #44368: mimic: Rados should use the '-o outfile' convention Rejected

History

#1 Updated by Brad Hubbard almost 4 years ago

  • Project changed from Ceph to RADOS
  • Category deleted (ceph cli)

#2 Updated by Brad Hubbard almost 4 years ago

  • Category set to Administration/Usability

#3 Updated by Brad Hubbard almost 4 years ago

'-o' does not do what you inferred.

# rados --help|grep -A1 "\-o "                                                                                                                                                                                      
   -o object_size                                                                                                     
        set the object size for put/get ops and for write benchmarking

#4 Updated by Christian Huebner almost 4 years ago

According to the manpage for the rados tool, -o denotes the output file name:

NAME
rados - rados object storage utility

SYNOPSIS
rados [ options ] [ command ]

DESCRIPTION
[...]
-o outfile
will write any payload returned by the monitor cluster with its reply to outfile. Only specific monitor com‐
mands (e.g. osd getmap) return a payload.

The manpage for any Unix/Linux tool should be authoritative for its use and thus the manpage for rados should be fixed. You are right, though, --help shows a different usage of -o.

However, as the other ceph tooling as well as most unix tooling uses -o as output file I recommend denoting a different parameter for object size and assigning -o to output file name as is customary. At the moment the rados tool does not have an output file option. It is a clunky workaround to use redirection to force the JSON into a file.

#5 Updated by Brad Hubbard almost 4 years ago

  • Subject changed from Rados df throws 'illegal prefix' when writing output to disk to Rados should use the '-o outfile' convention

Christian Huebner wrote:

According to the manpage for the rados tool, -o denotes the output file name:

Oops, that's unfortunate. I've created https://github.com/ceph/ceph/pull/31204 to try and rectify that.

NAME
rados - rados object storage utility

SYNOPSIS
rados [ options ] [ command ]

DESCRIPTION
[...]
-o outfile
will write any payload returned by the monitor cluster with its reply to outfile. Only specific monitor com‐
mands (e.g. osd getmap) return a payload.

The manpage for any Unix/Linux tool should be authoritative for its use and thus the manpage for rados should be fixed. You are right, though, --help shows a different usage of -o.

I've opened https://tracker.ceph.com/issues/42518 to address other inconsistencies between the man page and actual usage.

However, as the other ceph tooling as well as most unix tooling uses -o as output file I recommend denoting a different parameter for object size and assigning -o to output file name as is customary. At the moment the rados tool does not have an output file option. It is a clunky workaround to use redirection to force the JSON into a file.

I understand your view and it does have merit. I've modified the title of this tracker so it reflects the request more accurately and will leave this open in the hope someone can attend to it in the not too distant future. Thanks for the report.

#6 Updated by Brad Hubbard almost 4 years ago

  • Tags set to low-hanging-fruit

#7 Updated by Brad Hubbard almost 4 years ago

  • Assignee set to Brad Hubbard

#8 Updated by Brad Hubbard almost 4 years ago

  • Tags deleted (low-hanging-fruit)

#9 Updated by Brad Hubbard almost 4 years ago

  • Pull request ID set to 31310

#10 Updated by Nathan Cutler almost 4 years ago

  • Status changed from New to Fix Under Review
  • Backport set to nautilus

#11 Updated by Nathan Cutler almost 4 years ago

  • Backport changed from nautilus to nautilus, mimic

#12 Updated by Kefu Chai almost 4 years ago

  • Status changed from Fix Under Review to Pending Backport

#15 Updated by Brad Hubbard almost 4 years ago

Note that the reason I did not set this for backport is that it has the potential to break existing scripts and functionality. I did not think it was worth that risk for previous releases where the current behaviour of '-o' is currently accepted (although arguably incorrect).

#16 Updated by Nathan Cutler almost 4 years ago

  • Backport changed from nautilus, mimic to nautilus

#17 Updated by Nathan Cutler almost 4 years ago

  • Status changed from Pending Backport to Resolved
  • Backport deleted (nautilus)

#18 Updated by Nathan Cutler almost 4 years ago

@Brad - got it, thanks. So, the issue is fixed as of Octopus and the fix will not be backported for the reason you stated.

#19 Updated by Brad Hubbard almost 4 years ago

@Nathan, I think that's the right decision in this case mate. It should be less disruptive hopefully.

#20 Updated by Kefu Chai over 3 years ago

  • Status changed from Resolved to Pending Backport
  • Backport set to nautilus, mimic

we have to backport this change, otherwise we have 

2020-02-29T19:36:14.417 INFO:teuthology.orchestra.run.smithi076:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 -O 65536 -p unique_pool_2 bench 150 write'
2020-02-29T19:36:14.419 INFO:tasks.radosbench:joining radosbench (timing out after 4800s)
2020-02-29T19:36:14.420 INFO:teuthology.orchestra.run:waiting for 4800
2020-02-29T19:36:14.424 INFO:tasks.ceph.osd.3.smithi183.stderr:2020-02-29T19:36:14.418+0000 7ff03012f700 -1 received  signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3  (PID: 87809) UID: 0
2020-02-29T19:36:14.467 INFO:tasks.radosbench.radosbench.0.smithi076.stderr:unrecognized command -O; -h or --help for usage
2020-02-29T19:36:14.467 INFO:tasks.radosbench.radosbench.0.smithi076.stderr:error 22: (22) Invalid argument
2020-02-29T19:36:14.470 DEBUG:teuthology.orchestra.run:got remote process result: 1

when doing upgrade tests

#21 Updated by Kefu Chai over 3 years ago

  • Copied to Backport #44360: nautilus: Rados should use the '-o outfile' convention added

#22 Updated by Nathan Cutler over 3 years ago

  • Copied to Backport #44368: mimic: Rados should use the '-o outfile' convention added

#23 Updated by Nathan Cutler over 2 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Also available in: Atom PDF