Project

General

Profile

Actions

Bug #49484

closed

rados/upgrade/pacific-x/parallel fails due to more than 1 version shown after uprade

Added by Kefu Chai about 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
cephadm
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

rados/upgrade/pacific-x/parallel/{0-start 1-tasks distro1$/{rhel_latest} mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}}
2021-02-25T05:01:49.334 INFO:teuthology.orchestra.run.smithi115.stdout:{
2021-02-25T05:01:49.334 INFO:teuthology.orchestra.run.smithi115.stdout:    "mon": {
2021-02-25T05:01:49.335 INFO:teuthology.orchestra.run.smithi115.stdout:        "ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc)": 3
2021-02-25T05:01:49.335 INFO:teuthology.orchestra.run.smithi115.stdout:    },
2021-02-25T05:01:49.336 INFO:teuthology.orchestra.run.smithi115.stdout:    "mgr": {
2021-02-25T05:01:49.336 INFO:teuthology.orchestra.run.smithi115.stdout:        "ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc)": 1,
2021-02-25T05:01:49.337 INFO:teuthology.orchestra.run.smithi115.stdout:        "ceph version 17.0.0-1167-g2f056a0e (2f056a0e787f6a028746e1f5bc82235dcee9d07e) quincy (dev)": 1
2021-02-25T05:01:49.337 INFO:teuthology.orchestra.run.smithi115.stdout:    },
2021-02-25T05:01:49.337 INFO:teuthology.orchestra.run.smithi115.stdout:    "osd": {
2021-02-25T05:01:49.338 INFO:teuthology.orchestra.run.smithi115.stdout:        "ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc)": 8
2021-02-25T05:01:49.338 INFO:teuthology.orchestra.run.smithi115.stdout:    },
2021-02-25T05:01:49.339 INFO:teuthology.orchestra.run.smithi115.stdout:    "mds": {},
2021-02-25T05:01:49.339 INFO:teuthology.orchestra.run.smithi115.stdout:    "overall": {
2021-02-25T05:01:49.339 INFO:teuthology.orchestra.run.smithi115.stdout:        "ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc)": 12,
2021-02-25T05:01:49.340 INFO:teuthology.orchestra.run.smithi115.stdout:        "ceph version 17.0.0-1167-g2f056a0e (2f056a0e787f6a028746e1f5bc82235dcee9d07e) quincy (dev)": 1
2021-02-25T05:01:49.340 INFO:teuthology.orchestra.run.smithi115.stdout:    }
2021-02-25T05:01:49.341 INFO:teuthology.orchestra.run.smithi115.stdout:}
2021-02-25T05:01:49.881 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest-pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5c7db87a-7725-11eb-9039-001a4aab830c -e sha1=
2f056a0e787f6a028746e1f5bc82235dcee9d07e -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"''
$ zgrep 'ceph version' remote/ubuntu@smithi149.front.sepia.ceph.com/log/5c7db87a-7725-11eb-9039-001a4aab830c/ceph-mgr.x.log.gz
2021-02-25T04:55:49.459+0000 7f4d95b6b380  0 ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc), process ceph-mgr, pid 1
2021-02-25T05:01:30.172+0000 7f73c513d500  0 ceph version 17.0.0-1167-g2f056a0e (2f056a0e787f6a028746e1f5bc82235dcee9d07e) quincy (dev), process ceph-mgr, pid 1
$ zgrep 'ceph version' remote/ubuntu@smithi115.front.sepia.ceph.com/log/5c7db87a-7725-11eb-9039-001a4aab830c/ceph-mgr.y.log.gz
2021-02-25T04:53:53.469+0000 7f6f31444380  0 ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc), process ceph-mgr, pid 1
2021-02-25T04:54:04.489+0000 7f99e38df380  0 ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc), process ceph-mgr, pid 1
2021-02-25T04:54:20.425+0000 7f175b718380  0 ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc), process ceph-mgr, pid 1
    "ceph_version": "ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc)",
2021-02-25T05:00:05.388+0000 7f171bcf1700  0 [cephadm DEBUG cephadm.serve] image docker.io/ceph/daemon-base:latest-pacific -> ContainerInspectInfo(image_id='25b7284a02c8597227ac78196207ee05b8cb7ded7fa1584ae886321db63ed39b', ceph_version='ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc)', repo_digests=['docker.io/ceph/daemon-base@sha256:33734f859be10ce6263e70ee0c2f2456b5a7349614b8ea8a1c918b975061deb7'])
    "ceph_version": "ceph version 17.0.0-1167-g2f056a0e (2f056a0e787f6a028746e1f5bc82235dcee9d07e) quincy (dev)",
2021-02-25T05:00:41.600+0000 7f171bcf1700  0 [cephadm DEBUG cephadm.serve] image quay.ceph.io/ceph-ci/ceph:2f056a0e787f6a028746e1f5bc82235dcee9d07e -> ContainerInspectInfo(image_id='87de2cc64f7400c7c746635c8805cbae11542b57b01e782b7b5de6cad4edcd70', ceph_version='ceph version 17.0.0-1167-g2f056a0e (2f056a0e787f6a028746e1f5bc82235dcee9d07e) quincy (dev)', repo_digests=['quay.ceph.io/ceph-ci/ceph@sha256:ca487aa45e80571b4a4d1d17cb6588105b4b1bd82a2592e7d2bdb8839894761b'])
    "ceph_version": "ceph version 17.0.0-1167-g2f056a0e (2f056a0e787f6a028746e1f5bc82235dcee9d07e) quincy (dev)",
2021-02-25T05:01:35.555+0000 7f0d82286380  0 ceph version 16.1.0-342-g9ed1c344 (9ed1c3449f8346134d787dc6c03dc8c4d4028e96) pacific (rc), process ceph-mgr, pid 1
grep 'container create' teuthology.log | grep -- '-mgr' | cut -d',' --output-delimiter='===' -f1,2,15
2021-02-25T04:53:53.443 INFO:journalctl@ceph.mgr.y.smithi115.stdout:Feb 25 04:53:53 smithi115 podman[30649]: 2021-02-25 04:53:53.074515748 +0000 UTC m=+0.132529187 container create a345a42f6ebcf39718a32dc8b79f2c18602d983e6dee7a7a6b06862ed2c8f30b (image=docker.io/ceph/daemon-base:latest-pacific=== name=ceph-5c7db87a-7725-11eb-9039-001a4aab830c-mgr.y=== org.label-schema.name=CentOS Base Image)
2021-02-25T04:55:49.433 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Feb 25 04:55:49 smithi149 podman[31809]: 2021-02-25 04:55:49.082873749 +0000 UTC m=+0.160610226 container create 5a941b87324f932c37bb4bf2cf3d73a0bae335428c28996dbf9aba51b88fdea9 (image=docker.io/ceph/daemon-base:latest-pacific=== name=ceph-5c7db87a-7725-11eb-9039-001a4aab830c-mgr.x=== RELEASE=master-908e75b)
2021-02-25T05:01:30.053 INFO:journalctl@ceph.mgr.x.smithi149.stdout:Feb 25 05:01:29 smithi149 podman[54135]: 2021-02-25 05:01:29.796327416 +0000 UTC m=+0.165272119 container create 4772a4a35446a791c7150d856d4c293ca81915449016046d47ffc5512074366d (image=quay.ceph.io/ceph-ci/ceph@sha256:ca487aa45e80571b4a4d1d17cb6588105b4b1bd82a2592e7d2bdb8839894761b=== name=ceph-5c7db87a-7725-11eb-9039-001a4aab830c-mgr.x=== GIT_BRANCH=HEAD

mgr.y stayed at 16.1.0. while mgr.x was upgraded.

/a/kchai-2021-02-25_04:36:04-rados-wip-kefu-testing-2021-02-24-2230-distro-basic-smithi/5912965

/a/kchai-2021-02-25_05:44:50-rados:upgrade-master-distro-basic-smithi/5913155/


Related issues 1 (0 open1 closed)

Has duplicate RADOS - Bug #49461: rados/upgrade/pacific-x/parallel: upgrade incompleteDuplicate

Actions
Actions #1

Updated by Kefu Chai about 3 years ago

  • Description updated (diff)
Actions #2

Updated by Kefu Chai about 3 years ago

  • Description updated (diff)
Actions #3

Updated by Kefu Chai about 3 years ago

  • Description updated (diff)
Actions #4

Updated by Kefu Chai about 3 years ago

  • Project changed from RADOS to Orchestrator
  • Category set to cephadm
Actions #5

Updated by Neha Ojha about 3 years ago

  • Has duplicate Bug #49461: rados/upgrade/pacific-x/parallel: upgrade incomplete added
Actions #6

Updated by Kefu Chai about 3 years ago

  • Status changed from New to Resolved
  • Pull request ID set to 39788
Actions

Also available in: Atom PDF