Project

General

Profile

Bug #40773

qa: 'ceph osd require-osd-release nautilus' fails

Added by Patrick Donnelly 3 months ago. Updated 2 months ago.

Status:
Pending Backport
Priority:
High
Category:
-
Target version:
Start date:
Due date:
% Done:

0%

Source:
Q/A
Tags:
Backport:
nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
qa-suite
Labels (FS):
Pull request ID:
Crash signature:

Description

2019-07-12T02:36:23.188 INFO:teuthology.orchestra.run.smithi118:Running:
2019-07-12T02:36:23.188 INFO:teuthology.orchestra.run.smithi118:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release nautilus'
...
2019-07-12T02:36:23.505 INFO:teuthology.orchestra.run.smithi118.stderr:Error EPERM: not all up OSDs have CEPH_FEATURE_SERVER_NAUTILUS feature
...
2019-07-12T02:36:24.180 DEBUG:teuthology.orchestra.run:got remote process result: 1
2019-07-12T02:36:24.181 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 86, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 65, in run_one_task
    return task(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/exec.py", line 54, in task
    c],
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 205, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 437, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 162, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 184, in _raise_for_status
    node=self.hostname, label=self.label
CommandFailedError: Command failed on smithi118 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release nautilus'" 

From: /ceph/teuthology-archive/pdonnell-2019-07-11_22:52:33-fs-wip-pdonnell-testing-20190711.203149-distro-basic-smithi/4111924/teuthology.log

Neha suggested the fix would be to set the nautilus required release flag before turning on msgr2.


Related issues

Copied to fs - Backport #41495: nautilus: qa: 'ceph osd require-osd-release nautilus' fails In Progress

History

#1 Updated by Patrick Donnelly 2 months ago

http://pulpito.ceph.com/pdonnell-2019-08-15_13:19:31-fs-wip-pdonnell-testing-20190814.222632-distro-basic-smithi/

All OSDs are upgraded to the nautilus binary according to their logs:

pdonnell@teuthology /a/pdonnell-2019-08-15_13:19:31-fs-wip-pdonnell-testing-20190814.222632-distro-basic-smithi/4216926$ find -name \*osd\* -execdir zgrep '^2019-08-15T14:56:4' {} + | grep 'ceph version'
./ceph-osd.3.log.gz:2019-08-15T14:56:40.863+0000 7f75c08ef100  0 ceph version 15.0.0-3917-g102d1ea (102d1ea218d46368292ef9c9e2df49adf1d27708) octopus (dev), process ceph-osd, pid 16694
./ceph-osd.2.log.gz:2019-08-15T14:56:40.935+0000 7f2d7b751100  0 ceph version 15.0.0-3917-g102d1ea (102d1ea218d46368292ef9c9e2df49adf1d27708) octopus (dev), process ceph-osd, pid 16712
./ceph-osd.1.log.gz:2019-08-15T14:56:40.699+0000 7fb8c3267100  0 ceph version 15.0.0-3917-g102d1ea (102d1ea218d46368292ef9c9e2df49adf1d27708) octopus (dev), process ceph-osd, pid 16658
./ceph-osd.0.log.gz:2019-08-15T14:56:40.767+0000 7f0462548100  0 ceph version 15.0.0-3917-g102d1ea (102d1ea218d46368292ef9c9e2df49adf1d27708) octopus (dev), process ceph-osd, pid 16676
./ceph-osd.0.log.gz:2019-08-15T14:56:48.418+0000 7f0457199700 10 osd.0 17 _collect_metadata {arch=x86_64,back_addr=v1:172.21.15.172:6806/16676,back_iface=enp3s0f1,bluefs=1,bluefs_single_shared_device=1,bluestore_bdev_access_mode=file,bluestore_bdev_block_size=4096,bluestore_bdev_devices=nvme0n1,bluestore_bdev_driver=KernelDevice,bluestore_bdev_path=/var/lib/ceph/osd/ceph-0/block,bluestore_bdev_rotational=0,bluestore_bdev_size=96636764160,bluestore_bdev_support_discard=1,bluestore_bdev_type=ssd,ceph_release=octopus,ceph_version=ceph version 15.0.0-3917-g102d1ea (102d1ea218d46368292ef9c9e2df49adf1d27708) octopus (dev),ceph_version_short=15.0.0-3917-g102d1ea,cpu=Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz,default_device_class=ssd,device_ids=nvme0n1=INTEL_SSDPEDMD400G4_CVFT6233005M400BGN,devices=nvme0n1,distro=ubuntu,distro_description=Ubuntu 18.04.3 LTS,distro_version=18.04,front_addr=v1:172.21.15.172:6805/16676,front_iface=enp3s0f1,hb_back_addr=v1:172.21.15.172:6808/16676,hb_front_addr=v1:172.21.15.172:6807/16676,hostname=smithi172,journal_rotational=0,kernel_description=#64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019,kernel_version=4.15.0-58-generic,mem_swap_kb=2097148,mem_total_kb=32824384,network_numa_node=0,network_numa_nodes=0,numa_node=0,numa_node_cpus=0-7,objectstore_numa_node=0,objectstore_numa_nodes=0,os=Linux,osd_data=/var/lib/ceph/osd/ceph-0,osd_objectstore=bluestore,rotational=0}

#3 Updated by Patrick Donnelly 2 months ago

  • Status changed from New to Need Review
  • Pull request ID set to 29715

#4 Updated by Patrick Donnelly 2 months ago

  • Status changed from Need Review to Pending Backport

#5 Updated by Nathan Cutler about 2 months ago

  • Copied to Backport #41495: nautilus: qa: 'ceph osd require-osd-release nautilus' fails added

Also available in: Atom PDF