Project

General

Profile

Actions

Bug #45356

closed

nautilus: rados/upgrade/mimic-x-singleton failures due to mon_client_directed_command_retry

Added by Neha Ojha almost 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2020-04-30T22:35:17.788 INFO:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog
2020-04-30T22:35:18.221 INFO:teuthology.orchestra.run.smithi002.stderr:no valid command found; 10 closest matches:
2020-04-30T22:35:18.221 INFO:teuthology.orchestra.run.smithi002.stderr:osd utilization
2020-04-30T22:35:18.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool application get {<poolname>} {<app>} {<key>}
2020-04-30T22:35:18.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool application rm <poolname> <app> <key>
2020-04-30T22:35:18.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool application set <poolname> <app> <key> <value>
2020-04-30T22:35:18.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool application disable <poolname> <app> {--yes-i-really-mean-it}
2020-04-30T22:35:18.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool application enable <poolname> <app> {--yes-i-really-mean-it}
2020-04-30T22:35:18.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool get-quota <poolname>
2020-04-30T22:35:18.223 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool set-quota <poolname> max_objects|max_bytes <val>
2020-04-30T22:35:18.223 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool set <poolname> size|min_size|pg_num|pgp_num|crush_rule|hashpspool|nodelete|nopgchange|nosizechange|write_fadvise_dontneed|noscrub|nodeep-scrub|hit_set_type|hit_set_period|hit_set_count|hit_set_fpp|use_gmt_hitset|target_max_bytes|target_max_objects|cache_target_dirty_ratio|cache_target_dirty_high_ratio|cache_target_full_ratio|cache_min_flush_age|cache_min_evict_age|auid|min_read_recency_for_promote|min_write_recency_for_promote|fast_read|hit_set_grade_decay_rate|hit_set_search_last_n|scrub_min_interval|scrub_max_interval|deep_scrub_interval|recovery_priority|recovery_op_priority|scrub_priority|compression_mode|compression_algorithm|compression_required_ratio|compression_max_blob_size|compression_min_blob_size|csum_type|csum_min_block|csum_max_block|allow_ec_overwrites <val> {--yes-i-really-mean-it}
2020-04-30T22:35:18.223 INFO:teuthology.orchestra.run.smithi002.stderr:osd pool get <poolname> size|min_size|pg_num|pgp_num|crush_rule|hashpspool|nodelete|nopgchange|nosizechange|write_fadvise_dontneed|noscrub|nodeep-scrub|hit_set_type|hit_set_period|hit_set_count|hit_set_fpp|use_gmt_hitset|auid|target_max_objects|target_max_bytes|cache_target_dirty_ratio|cache_target_dirty_high_ratio|cache_target_full_ratio|cache_min_flush_age|cache_min_evict_age|erasure_code_profile|min_read_recency_for_promote|all|min_write_recency_for_promote|fast_read|hit_set_grade_decay_rate|hit_set_search_last_n|scrub_min_interval|scrub_max_interval|deep_scrub_interval|recovery_priority|recovery_op_priority|scrub_priority|compression_mode|compression_algorithm|compression_required_ratio|compression_max_blob_size|compression_min_blob_size|csum_type|csum_min_block|csum_max_block|allow_ec_overwrites
2020-04-30T22:35:18.223 INFO:teuthology.orchestra.run.smithi002.stderr:Error EINVAL: invalid command
2020-04-30T22:35:18.232 DEBUG:teuthology.orchestra.run:got remote process result: 22
2020-04-30T22:35:18.232 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/contextutil.py", line 34, in nested
    yield vars
  File "/home/teuthworker/src/github.com_ceph_ceph-c_wip-yuri8-testing-2020-04-29-1603-nautilus/qa/tasks/ceph.py", line 1909, in task
    '--no-mon-health-to-clog',
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/remote.py", line 203, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 473, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 162, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_py2/teuthology/orchestra/run.py", line 184, in _raise_for_status
    node=self.hostname, label=self.label
CommandFailedError: Command failed on smithi002 with status 22: "sudo ceph --cluster ceph --mon-client-directed-command-retry 5 tell 'mon.*' injectargs -- --no-mon-health-to-clog" 

/a/yuriw-2020-04-30_19:41:52-rados-wip-yuri8-testing-2020-04-29-1603-nautilus-distro-basic-smithi/5003022

mimic does not understand mon_client_directed_command_retry and the upgrade tests use the nautilus ceph task which uses this parameter(since https://github.com/ceph/ceph/pull/31037) after installing mimic.


Related issues 1 (0 open1 closed)

Related to Ceph - Bug #45379: nautilus: upgrade:luminous-x-nautilus failures due to mon_client_directed_command_retry addedDuplicate

Actions
Actions #1

Updated by Neha Ojha almost 4 years ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 34884
Actions #2

Updated by Neha Ojha almost 4 years ago

  • Related to Bug #45379: nautilus: upgrade:luminous-x-nautilus failures due to mon_client_directed_command_retry added added
Actions #4

Updated by Neha Ojha almost 4 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF