Ceph » RADOS

http://qa-proxy.ceph.com/teuthology/dzafman-2019-02-19_08:38:36-rados-wip-zafman-testing-38344-distro-basic-smithi/3612917/teuthology.log

I also ran ceph --help mon manually after a vstart and it produced the same error. This is blocking other tests in test.sh.

This may have been caused by:

commit 19fbdb3ffde5bcb73792b1d822fe3f8d0407d490
Author: Patrick Donnelly <pdonnell@redhat.com>
Date:   Mon Jan 21 14:45:07 2019 -0800

    ceph: omit hidden commands from suggestions

    Fixes: http://tracker.ceph.com/issues/37955
    Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

2019-02-19T17:46:45.314 INFO:tasks.workunit.client.0.smithi053.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh:1156: test_mon_mon:  ceph --help mon
2019-02-19T17:46:45.456 INFO:tasks.workunit.client.0.smithi053.stdout:
2019-02-19T17:46:45.456 INFO:tasks.workunit.client.0.smithi053.stdout: General usage:
2019-02-19T17:46:45.457 INFO:tasks.workunit.client.0.smithi053.stdout: ==============
2019-02-19T17:46:45.457 INFO:tasks.workunit.client.0.smithi053.stdout:usage: ceph [-h] [-c CEPHCONF] [-i INPUT_FILE] [-o OUTPUT_FILE]
2019-02-19T17:46:45.457 INFO:tasks.workunit.client.0.smithi053.stdout:            [--id CLIENT_ID] [--name CLIENT_NAME] [--cluster CLUSTER]
2019-02-19T17:46:45.457 INFO:tasks.workunit.client.0.smithi053.stdout:            [--admin-daemon ADMIN_SOCKET] [-s] [-w] [--watch-debug]
2019-02-19T17:46:45.458 INFO:tasks.workunit.client.0.smithi053.stdout:            [--watch-info] [--watch-sec] [--watch-warn] [--watch-error]
2019-02-19T17:46:45.458 INFO:tasks.workunit.client.0.smithi053.stdout:            [--watch-channel {cluster,audit,*}] [--version] [--verbose]
2019-02-19T17:46:45.458 INFO:tasks.workunit.client.0.smithi053.stdout:            [--concise] [-f {json,json-pretty,xml,xml-pretty,plain}]
2019-02-19T17:46:45.459 INFO:tasks.workunit.client.0.smithi053.stdout:            [--connect-timeout CLUSTER_TIMEOUT] [--block] [--period PERIOD]
2019-02-19T17:46:45.459 INFO:tasks.workunit.client.0.smithi053.stdout:
2019-02-19T17:46:45.459 INFO:tasks.workunit.client.0.smithi053.stdout:Ceph administration tool
2019-02-19T17:46:45.459 INFO:tasks.workunit.client.0.smithi053.stdout:
2019-02-19T17:46:45.459 INFO:tasks.workunit.client.0.smithi053.stdout:optional arguments:
2019-02-19T17:46:45.459 INFO:tasks.workunit.client.0.smithi053.stdout:  -h, --help            request mon help
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:  -c CEPHCONF, --conf CEPHCONF
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:                        ceph configuration file
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:  -i INPUT_FILE, --in-file INPUT_FILE
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:                        input file, or "-" for stdin
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:  -o OUTPUT_FILE, --out-file OUTPUT_FILE
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:                        output file, or "-" for stdout
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:  --id CLIENT_ID, --user CLIENT_ID
2019-02-19T17:46:45.460 INFO:tasks.workunit.client.0.smithi053.stdout:                        client id for authentication
2019-02-19T17:46:45.461 INFO:tasks.workunit.client.0.smithi053.stdout:  --name CLIENT_NAME, -n CLIENT_NAME
2019-02-19T17:46:45.461 INFO:tasks.workunit.client.0.smithi053.stdout:                        client name for authentication
2019-02-19T17:46:45.461 INFO:tasks.workunit.client.0.smithi053.stdout:  --cluster CLUSTER     cluster name
2019-02-19T17:46:45.461 INFO:tasks.workunit.client.0.smithi053.stdout:  --admin-daemon ADMIN_SOCKET
2019-02-19T17:46:45.461 INFO:tasks.workunit.client.0.smithi053.stdout:                        submit admin-socket commands ("help" for help
2019-02-19T17:46:45.461 INFO:tasks.workunit.client.0.smithi053.stdout:  -s, --status          show cluster status
2019-02-19T17:46:45.461 INFO:tasks.workunit.client.0.smithi053.stdout:  -w, --watch           watch live cluster changes
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:  --watch-debug         watch debug events
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:  --watch-info          watch info events
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:  --watch-sec           watch security events
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:  --watch-warn          watch warn events
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:  --watch-error         watch error events
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:  --watch-channel {cluster,audit,*}
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:                        which log channel to follow when using -w/--watch. One
2019-02-19T17:46:45.462 INFO:tasks.workunit.client.0.smithi053.stdout:                        of ['cluster', 'audit', '*']
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:  --version, -v         display version
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:  --verbose             make verbose
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:  --concise             make less verbose
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:  -f {json,json-pretty,xml,xml-pretty,plain}, --format {json,json-pretty,xml,xml-pretty,plain}
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:  --connect-timeout CLUSTER_TIMEOUT
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:                        set a timeout for connecting to the cluster
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:  --block               block until completion (scrub and deep-scrub only)
2019-02-19T17:46:45.463 INFO:tasks.workunit.client.0.smithi053.stdout:  --period PERIOD, -p PERIOD
2019-02-19T17:46:45.464 INFO:tasks.workunit.client.0.smithi053.stdout:                        polling period, default 1.0 second (for polling
2019-02-19T17:46:45.464 INFO:tasks.workunit.client.0.smithi053.stdout:                        commands only)
2019-02-19T17:46:45.464 INFO:tasks.workunit.client.0.smithi053.stdout:
2019-02-19T17:46:45.464 INFO:tasks.workunit.client.0.smithi053.stdout: Local commands:
2019-02-19T17:46:45.464 INFO:tasks.workunit.client.0.smithi053.stdout: ===============
2019-02-19T17:46:45.464 INFO:tasks.workunit.client.0.smithi053.stdout:
2019-02-19T17:46:45.464 INFO:tasks.workunit.client.0.smithi053.stdout:ping <mon.id>           Send simple presence/life test to a mon
2019-02-19T17:46:45.465 INFO:tasks.workunit.client.0.smithi053.stdout:                        <mon.id> may be 'mon.*' for all mons
2019-02-19T17:46:45.465 INFO:tasks.workunit.client.0.smithi053.stdout:daemon {type.id|path} <cmd>
2019-02-19T17:46:45.465 INFO:tasks.workunit.client.0.smithi053.stdout:                        Same as --admin-daemon, but auto-find admin socket
2019-02-19T17:46:45.465 INFO:tasks.workunit.client.0.smithi053.stdout:daemonperf {type.id | path} [stat-pats] [priority] [<interval>] [<count>]
2019-02-19T17:46:45.465 INFO:tasks.workunit.client.0.smithi053.stdout:daemonperf {type.id | path} list|ls [stat-pats] [priority]
2019-02-19T17:46:45.465 INFO:tasks.workunit.client.0.smithi053.stdout:                        Get selected perf stats from daemon/admin socket
2019-02-19T17:46:45.465 INFO:tasks.workunit.client.0.smithi053.stdout:                        Optional shell-glob comma-delim match string stat-pats
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout:                        Optional selection priority (can abbreviate name):
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout:                         critical, interesting, useful, noninteresting, debug
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout:                        List shows a table of all available stats
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout:                        Run <count> times (default forever),
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout:                         once per <interval> seconds (default 1)
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout:
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout:
2019-02-19T17:46:45.466 INFO:tasks.workunit.client.0.smithi053.stdout: Monitor commands:
2019-02-19T17:46:45.467 INFO:tasks.workunit.client.0.smithi053.stdout: =================
2019-02-19T17:46:45.467 INFO:tasks.workunit.client.0.smithi053.stderr:Traceback (most recent call last):
2019-02-19T17:46:45.467 INFO:tasks.workunit.client.0.smithi053.stderr:  File "/usr/bin/ceph", line 1242, in <module>
2019-02-19T17:46:45.467 INFO:tasks.workunit.client.0.smithi053.stderr:    retval = main()
2019-02-19T17:46:45.467 INFO:tasks.workunit.client.0.smithi053.stderr:  File "/usr/bin/ceph", line 1039, in main
2019-02-19T17:46:45.467 INFO:tasks.workunit.client.0.smithi053.stderr:    return do_extended_help(parser, childargs, ('mon', ''), ' '.join(childargs))
2019-02-19T17:46:45.467 INFO:tasks.workunit.client.0.smithi053.stderr:  File "/usr/bin/ceph", line 407, in do_extended_help
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:    return help_for_target(target, partial)
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:  File "/usr/bin/ceph", line 404, in help_for_target
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:    return help_for_sigs(outbuf.decode('utf-8'), partial)
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:  File "/usr/bin/ceph", line 383, in help_for_sigs
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:    partial=partial))
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:  File "/usr/bin/ceph", line 474, in format_help
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:    if flags & (argparse.FLAG_OBSOLETE | argparse.FLAG_DEPRECATED | argparse.FLAG_HIDDEN):
2019-02-19T17:46:45.468 INFO:tasks.workunit.client.0.smithi053.stderr:AttributeError: 'module' object has no attribute 'FLAG_OBSOLETE'
2019-02-19T17:46:45.469 INFO:tasks.workunit.client.0.smithi053.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh:1: test_mon_mon:  rm -fr /tmp/cephtool.Lv4
2019-02-19T17:46:45.470 DEBUG:teuthology.orchestra.run:got remote process result: 1
2019-02-19T17:46:45.470 INFO:tasks.workunit:Stopping ['cephtool'] on client.0...
2019-02-19T17:46:45.470 INFO:teuthology.orchestra.run.smithi053:Running:
2019-02-19T17:46:45.470 INFO:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0
2019-02-19T17:46:45.593 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 86, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 65, in run_one_task
    return task(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-zafman-testing-38344/qa/tasks/workunit.py", line 136, in task
    cleanup=cleanup)
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-zafman-testing-38344/qa/tasks/workunit.py", line 286, in _spawn_on_all_clients
    timeout=timeout)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 85, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 99, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/parallel.py", line 22, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-zafman-testing-38344/qa/tasks/workunit.py", line 409, in _run_tests
    label="workunit test {workunit}".format(workunit=workunit)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 193, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 435, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 162, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 184, in _raise_for_status
    node=self.hostname, label=self.label
CommandFailedError: Command failed (workunit test cephtool/test.sh) on smithi053 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=de422f77a7271f8af9861c940b1c7879d2e2ee10 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh'