Project

General

Profile

Actions
Copy link

Bug #49293

closed

podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,metacopy=on" options: invalid argument

Added by Deepika Upadhyay about 3 years ago. Updated almost 3 years ago.

Status:
Rejected
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
pacific, octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

cephadm faile to pull quay image using podman:

2021-02-12T17:10:27.400 INFO:tasks.workunit.client.0.gibba025.stderr:Pulling container image docker.io/ceph/ceph:v15...
2021-02-12T17:10:30.180 INFO:tasks.workunit.client.0.gibba025.stderr:Non-zero exit code 125 from /usr/bin/podman pull docker.io/ceph/ceph:v15
2021-02-12T17:10:30.180 INFO:tasks.workunit.client.0.gibba025.stderr:/usr/bin/podman: stderr Error: Cannot connect to the Podman socket, make sure there is a Podman REST API
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:Traceback (most recent call last):
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:  File "/tmp/tmp.oHhd7otVbz/cephadm", line 6152, in <module>
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:    r = args.func()
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:  File "/tmp/tmp.oHhd7otVbz/cephadm", line 1411, in _default_image
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:    return func()
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:  File "/tmp/tmp.oHhd7otVbz/cephadm", line 3933, in command_adopt
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:    _pull_image(args.image)
2021-02-12T17:10:30.181 INFO:tasks.workunit.client.0.gibba025.stderr:  File "/tmp/tmp.oHhd7otVbz/cephadm", line 2713, in _pull_image
2021-02-12T17:10:30.182 INFO:tasks.workunit.client.0.gibba025.stderr:    raise RuntimeError('Failed command: %s' % cmd_str)
2021-02-12T17:10:30.182 INFO:tasks.workunit.client.0.gibba025.stderr:RuntimeError: Failed command: /usr/bin/podman pull docker.io/ceph/ceph:v15
2021-02-12T17:10:30.188 INFO:tasks.workunit.client.0.gibba025.stderr:+ sudo rm -rf /tmp/tmp.DOfjW1jH4Y
2021-02-12T17:10:30.196 INFO:tasks.workunit.client.0.gibba025.stderr:+ rm -rf /tmp/tmp.oHhd7otVbz
2021-02-12T17:10:30.196 DEBUG:teuthology.orchestra.run:got remote process result: 1
2021-02-12T17:10:30.197 INFO:tasks.workunit:Stopping ['cephadm/test_adoption.sh'] on client.0...

/ceph/teuthology-archive/yuriw-2021-02-12_16:01:06-rados:cephadm-wip-yuri2-testing-2021-02-08-1048-octopus-distro-basic-gibba/5877761/teuthology.log
/ceph/teuthology-archive/yuriw-2021-02-12_16:01:06-rados:cephadm-wip-yuri2-testing-2021-02-08-1048-octopus-distro-basic-gibba/5877759/teuthology.log

Related issues 1 (0 open — 1 closed)

Related to Orchestrator - Bug #49603: cephadm/podman: 3 stray daemon(s) not managed by cephadm","count":3}Can't reproduce

Actions
Actions
Copy link
 #1

Updated by Deepika Upadhyay about 3 years ago

  • Backport set to octopus
Actions
Copy link
 #2

Updated by Deepika Upadhyay about 3 years ago

  • Backport changed from octopus to pacific, octopus
Actions
Copy link
 #4

Updated by Sebastian Wagner about 3 years ago

  • Project changed from teuthology to Orchestrator
  • Subject changed from cephadm/test_adoption: cannot connect to the Podman socket, missing Podman REST API to podman 3.0: cephadm/test_adoption: cannot connect to the Podman socket, missing Podman REST API
  • Priority changed from Normal to High
Actions
Copy link
 #6

Updated by Neha Ojha about 3 years ago

/a/yuriw-2021-02-15_19:34:09-rados-octopus-distro-basic-smithi/5884688/

Actions
Copy link
 #7

Updated by Sebastian Wagner about 3 years ago 

2021-02-15T20:01:39.580 INFO:teuthology.orchestra.run.smithi160.stderr:/usr/bin/podman: stderr Error: Cannot connect to the Podman socket, make sure there is a Podman REST API service running.: failed to mount overlay for metacopy check with "nodev,metacopy=on" options: invalid argument
2021-02-15T20:01:39.584 INFO:teuthology.orchestra.run.smithi160.stderr:Non-zero exit code 125 from /usr/bin/podman pull quay.ceph.io/ceph-ci/ceph:987e8a77577b54cebb1dd5f0a9500844d43b8602
2021-02-15T20:01:39.584 INFO:teuthology.orchestra.run.smithi160.stderr:/usr/bin/podman: stderr Error: Cannot connect to the Podman socket, make sure there is a Podman REST API service running.: failed to mount overlay for metacopy check with "nodev,metacopy=on" options: invalid argument
2021-02-15T20:01:39.588 INFO:teuthology.orchestra.run.smithi160.stderr:Traceback (most recent call last):
2021-02-15T20:01:39.588 INFO:teuthology.orchestra.run.smithi160.stderr:  File "/home/ubuntu/cephtest/cephadm", line 6151, in <module>
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:    r = args.func()
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:  File "/home/ubuntu/cephtest/cephadm", line 1410, in _default_image
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:    return func()
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:  File "/home/ubuntu/cephtest/cephadm", line 2911, in command_bootstrap
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:    _pull_image(args.image)
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:  File "/home/ubuntu/cephtest/cephadm", line 2712, in _pull_image
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:    raise RuntimeError('Failed command: %s' % cmd_str)
2021-02-15T20:01:39.589 INFO:teuthology.orchestra.run.smithi160.stderr:RuntimeError: Failed command: /usr/bin/podman pull quay.ceph.io/ceph-ci/ceph:987e8a77577b54cebb1dd5f0a9500844d43b8602
Actions
Copy link
 #8

Updated by Deepika Upadhyay about 3 years ago

/ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-octopus-distro-basic-smithi/
/ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-octopus-distro-basic-smithi/5904346/teuthology.log"

Actions
Copy link
 #9

Updated by Sebastian Wagner about 3 years ago

Deepika Upadhyay wrote:

/ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-octopus-distro-basic-smithi/
/ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-octopus-distro-basic-smithi/5904346/teuthology.log"

rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} indeed

Actions
Copy link
 #10

Updated by Sebastian Wagner about 3 years ago

Sebastian Wagner wrote:

Deepika Upadhyay wrote:

/ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-octopus-distro-basic-smithi/
/ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-octopus-distro-basic-smithi/5904346/teuthology.log"

rados/cephadm/workunits/{distro/ubuntu_18.04_podman task/test_orch_cli} indeed

pinged them again in https://github.com/containers/podman/issues/9363#issuecomment-788807572

Actions
Copy link
 #11

Updated by Deepika Upadhyay about 3 years ago

  • Related to Bug #49603: cephadm/podman: 3 stray daemon(s) not managed by cephadm","count":3} added
Actions
Copy link
 #12

Updated by Sebastian Wagner about 3 years ago

  • Status changed from New to Rejected

Podman dropped support for ubuntu 18.04.

Nothing we can do about this

Actions
Copy link
 #13

Updated by Deepika Upadhyay about 3 years ago

@Sebastian I. should we use `--docker` instead of podman just for ubuntu
18.04, as a workaround?

Actions
Copy link
 #15

Updated by Sebastian Wagner about 3 years ago

for SEO: the new error message is:

Pulling container image quay.ceph.io/ceph-ci/ceph:381387f151822ee4d4927f3fc584ff1af9deb9ce...
Running command: /usr/bin/podman pull quay.ceph.io/ceph-ci/ceph:381387f151822ee4d4927f3fc584ff1af9deb9ce
/usr/bin/podman: stderr Error: failed to mount overlay for metacopy check with "nodev,metacopy=on" options: invalid argument
Non-zero exit code 125 from /usr/bin/podman pull quay.ceph.io/ceph-ci/ceph:381387f151822ee4d4927f3fc584ff1af9deb9ce
/usr/bin/podman: stderr Error: failed to mount overlay for metacopy check with "nodev,metacopy=on" options: invalid argument
Traceback (most recent call last):
  File "/home/ubuntu/cephtest/cephadm", line 6151, in <module>
    r = args.func()
  File "/home/ubuntu/cephtest/cephadm", line 1410, in _default_image
    return func()
  File "/home/ubuntu/cephtest/cephadm", line 2910, in command_bootstrap
    _pull_image(args.image)
  File "/home/ubuntu/cephtest/cephadm", line 2711, in _pull_image
    raise RuntimeError('Failed command: %s' % cmd_str)
RuntimeError: Failed command: /usr/bin/podman pull quay.ceph.io/ceph-ci/ceph:381387f151822ee4d4927f3fc584ff1af9deb9ce
Actions
Copy link
 #16

Updated by Sebastian Wagner about 3 years ago

  • Subject changed from podman 3.0: cephadm/test_adoption: cannot connect to the Podman socket, missing Podman REST API to podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,metacopy=on" options: invalid argument
Actions
Copy link
 #17

Updated by Deepika Upadhyay almost 3 years ago

/ceph/teuthology-archive/yuriw-2021-05-03_16:25:32-rados-wip-yuri-testing-2021-04-29-1033-octopus-distro-basic-smithi/6092801/teuthology.log

Actions
Copy link
 #18

Updated by Deepika Upadhyay almost 3 years ago

Deepika Upadhyay wrote:

/ceph/teuthology-archive/yuriw-2021-05-03_16:25:32-rados-wip-yuri-testing-2021-04-29-1033-octopus-distro-basic-smithi/6092801/teuthology.log

@Sage Weil @Sebastian I.: should we use docker instead of podman for 18.04 bionic? (has been around failing consistently in octopus)

Actions
Copy link

Also available in: Atom PDF