Project

General

Profile

Activity

From 02/07/2021 to 03/08/2021

03/08/2021

07:10 PM Bug #49659 (Fix Under Review): Exception found on "ceph orch osd rm stop" command
Daniel Pivonka
07:10 PM Bug #49659 (Resolved): Exception found on "ceph orch osd rm stop" command
Exception found using "ceph orch osd rm stop" command
>>>[ceph: root@ceph-suniladm-1612441365162-node1-monmgrins...
Daniel Pivonka
03:59 PM Support #49497: Cephadm fails to upgrade from 15.2.8 to 15.2.9
I have the same issue on hosts running CentOS 8.3 with podman 2.2.1, only difference is that I run cephadm as root. Jimmy Spets
03:05 PM Bug #49654 (Resolved): iSCSI stops working after Upgrade 15.2.4 -> 15.2.9
We have four iSCSI-Gateways deployed with "ceph orch". two gateways each form an IQN sharing one /23 network.
In U...
Frank Holtz
01:40 PM Bug #49484 (Resolved): rados/upgrade/pacific-x/parallel fails due to more than 1 version shown af...
Kefu Chai
10:53 AM Bug #49633: podman: ERROR (catatonit:2): failed to exec pid1: No such file or directory
https://github.com/containers/podman/issues/9660 Sebastian Wagner
10:50 AM Bug #49633: podman: ERROR (catatonit:2): failed to exec pid1: No such file or directory
https://pulpito.ceph.com/swagner-2021-03-05_12:04:33-rados:cephadm-wip-swagner2-testing-2021-03-05-1040-pacific-distr... Sebastian Wagner

03/07/2021

10:30 AM Bug #49638 (Resolved): qa/workunits/cephadm/create_iscsi_disks.sh: The script only supports CentOS
... Kefu Chai

03/05/2021

08:44 PM Bug #49633: podman: ERROR (catatonit:2): failed to exec pid1: No such file or directory
/a/sage-2021-03-05_17:57:02-rados:cephadm-wip-sage-testing-2021-03-05-0948-distro-basic-gibba/5939780
distro/ubunt...
Sage Weil
08:43 PM Bug #49633 (Can't reproduce): podman: ERROR (catatonit:2): failed to exec pid1: No such file or d...
... Sage Weil
06:05 PM Bug #49348 (Pending Backport): cephadm: giving nonexistent service for service action command sho...
Adam King
06:03 PM Bug #49348 (Duplicate): cephadm: giving nonexistent service for service action command should ret...
Adam King
06:03 PM Feature #49246 (Duplicate): cephadm: Display error message when given service name is wrong
Adam King
04:37 PM Bug #49435 (Fix Under Review): cephadm: rgw not getting deployed due to HEALTH_WARN
Sage Weil
03:18 PM Bug #49626 (Fix Under Review): cephadm: remove duplicate labels when adding a host
Adam King
03:10 PM Bug #49626 (Resolved): cephadm: remove duplicate labels when adding a host
right now, the "ceph orch host label add" command with cephadm will only add a label if it is not already attached to... Adam King
02:57 PM Bug #48373 (Resolved): cephadm iscsi: required file missing from config-json: iscsi-gateway.cfg
Adam King
12:01 PM Bug #49622: cephad orchestrator allows to delete hosts with ceph daemons running
https://github.com/ceph/ceph/pull/39850 Juan Miguel Olmo Martínez
11:49 AM Bug #49622 (Resolved): cephad orchestrator allows to delete hosts with ceph daemons running
This can drive the user to a difficult situation Juan Miguel Olmo Martínez

03/04/2021

06:39 PM Bug #49467 (Resolved): cephadm bootstrap process fails in open_ports on octopus branch: TypeError...
Nathan Cutler
04:17 PM Bug #49573: cephadm: rgw-nfs: stray daemons(s) not managed by cephadm
nfs-ganesha is deployed in such a way that it can host both cephfs and rgw exports from a single daemon.
rgw appea...
Michael Fritch
04:09 PM Bug #49522: cephadm: ValueError: not enough values to unpack (expected 2, got 1)
https://github.com/ceph/ceph/pull/39715 merged Yuri Weinstein
02:29 PM Bug #49603 (Can't reproduce): cephadm/podman: 3 stray daemon(s) not managed by cephadm","count":3}
... Deepika Upadhyay
02:01 PM Bug #45327: cephadm: Orch daemon add is not idempotent
... Deepika Upadhyay
10:35 AM Bug #49411 (Resolved): Rook: "ceph orch ls" command fails with KeyError: 'crashcollector'
Juan Miguel Olmo Martínez
10:28 AM Feature #47145: cephadm: Multiple daemons of the same service on single host
Kubernetes:
> The below yaml snippet of the webserver deployment has podAntiAffinity and podAffinity configured. T...
Sebastian Wagner

03/03/2021

11:18 PM Bug #49573 (Fix Under Review): cephadm: rgw-nfs: stray daemons(s) not managed by cephadm
during the an nfs mount of an rgw bucket, librgw will register a daemon with type `rgw-nfs`:
https://github.com/ceph...
Michael Fritch
08:51 PM Bug #49590: Error: error parsing host port: invalid port number: strconv.Atoi: parsing " 9093": i...
introduced by usage of shlex in https://github.com/ceph/ceph/pull/39730
quoted args are passed to podman:...
Michael Fritch
08:18 PM Bug #49590 (Fix Under Review): Error: error parsing host port: invalid port number: strconv.Atoi:...
Michael Fritch
08:15 PM Bug #49590 (Resolved): Error: error parsing host port: invalid port number: strconv.Atoi: parsing...
a/mgfritch-2021-03-02_23:06:05-rados:cephadm-wip-mgfritch-testing-2021-03-02-1408-distro-basic-smithi/5928333/
a/mgf...
Michael Fritch
08:07 PM Bug #49277 (Can't reproduce): cephadm bootstrap --apply-spec <cluster.yaml> hangs
tested on 15.2.8 and 15.2.9 cant reproduce Daniel Pivonka
02:27 PM Bug #49277: cephadm bootstrap --apply-spec <cluster.yaml> hangs
... Sebastian Wagner
03:10 PM Bug #48924 (Resolved): cephadm: upgrade process failed to pull target image: not enough values to...
Sage Weil
03:09 PM Bug #48924: cephadm: upgrade process failed to pull target image: not enough values to unpack (ex...
upstream issue: https://github.com/containers/podman/issues/8882 Sage Weil
02:49 PM Bug #49483 (Can't reproduce): CommandFailedError: Command failed on smithi104 with status 1: 'sud...
Sebastian Wagner
02:49 PM Bug #49483: CommandFailedError: Command failed on smithi104 with status 1: 'sudo /home/ubuntu/cep...
caused by ... Sebastian Wagner
02:49 PM Documentation #48267 (New): orchestrator docs should at least mention keyrings
Sebastian Wagner
02:49 PM Documentation #48267 (Can't reproduce): orchestrator docs should at least mention keyrings
Sebastian Wagner
09:52 AM Bug #46745 (Pending Backport): cephadm does not set systemd dependencies when using docker
Sebastian Wagner
09:48 AM Bug #49456: cephadm dashboard test: failed to connect to the server
https://pulpito.ceph.com/swagner-2021-03-02_17:23:47-rados:cephadm-wip-swagner3-testing-2021-03-02-1313-distro-basic-... Sebastian Wagner
09:40 AM Bug #49506 (Pending Backport): cephadm: `cephadm ls` broken for SUSE's downstream alertmanager co...
Octopus Backport PR: https://github.com/ceph/ceph/pull/39802/ Patrick Seidensal

03/02/2021

11:56 PM Bug #49348 (Pending Backport): cephadm: giving nonexistent service for service action command sho...
Ken Dreyer
11:52 PM Bug #49366 (Pending Backport): 'cephadm version' will show a trace back if image comes from a aut...
Ken Dreyer
11:41 PM Bug #49056 (Resolved): faulty behaviour running ceph orch apply mds with missing fsname
Merged to pacific as a part of https://github.com/ceph/ceph/pull/39623 Ken Dreyer
11:41 PM Bug #49056 (Pending Backport): faulty behaviour running ceph orch apply mds with missing fsname
Ken Dreyer
07:25 PM Bug #49522: cephadm: ValueError: not enough values to unpack (expected 2, got 1)
workaround by not testing on centos/rhel for now: https://github.com/ceph/ceph/pull/39791
Sage Weil
05:21 PM Bug #49287: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope n...
when running tests on gibba, it's always 20.04, and podman 3.0.1-2. and it seems pretty consistent.
error is some...
Sage Weil
04:59 PM Bug #49573 (Resolved): cephadm: rgw-nfs: stray daemons(s) not managed by cephadm
... Sebastian Wagner
04:33 PM Fix #49336 (Pending Backport): re-enable coredumps for cephadm
Sage Weil
04:23 PM Bug #49571 (Resolved): cephadm: same OSD one two host + daemon_id not unique
OSD.1 on both node06 and node01. The node01 is not managing a drive and is not working. Any idea on how to delete thi... Sebastian Wagner
04:19 PM Bug #49570 (Can't reproduce): cephadm: UnicodeDecodeError when reading sysctl
cephadm throws an UnicodeDecodeError when reading systctl data which contains unreadable errors.... Patrick Seidensal
10:37 AM Bug #49293: podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,me...
Sebastian Wagner wrote:
> Deepika Upadhyay wrote:
> > /ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-...
Sebastian Wagner
09:11 AM Bug #49456: cephadm dashboard test: failed to connect to the server
How often this one happens? We rarely see that in the none-cephadm dashboard e2e tests... It might be due to the diff... Ernesto Puerta

03/01/2021

10:42 PM Bug #46745 (Fix Under Review): cephadm does not set systemd dependencies when using docker
Sage Weil
09:17 PM Bug #49287: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope n...
/a/sage-2021-03-01_20:25:17-rados-wip-sage4-testing-2021-03-01-1042-distro-basic-gibba/5924180
description: rados/...
Sage Weil
07:16 PM Bug #49293: podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,me...
Deepika Upadhyay wrote:
> /ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-...
Sebastian Wagner
06:19 PM Bug #49293: podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,me...
/ceph/teuthology-archive/yuriw-2021-02-22_20:19:50-rados-wip-yuri-testing-2021-02-22-0812-octopus-distro-basic-smithi... Deepika Upadhyay
04:50 PM Bug #49539 (Resolved): test_cephadm.sh failure: container.alertmanager.a does'nt start
the problem was that the container image was being pulled from docker.io and was a day old and didn't have the needed... Sage Weil
04:49 PM Bug #49539: test_cephadm.sh failure: container.alertmanager.a does'nt start
Sage Weil wrote:
> [...]
> /a/sage-2021-02-28_18:35:15-rados-wip-sage-testing-2021-02-28-1217-distro-basic-smithi/5...
Sebastian Wagner
11:38 AM Bug #49551: cephadm journald logs are mangled
I think use std::clog instead of std::cerr to write the log can mitigate this issue. Because clog buffers content but... 玮文 胡
11:13 AM Bug #49551 (Resolved): cephadm journald logs are mangled
Regression introduced by https://github.com/ceph/ceph/pull/37729
to fix a regression introduced by https://github....
Sebastian Wagner
11:23 AM Fix #49336 (Fix Under Review): re-enable coredumps for cephadm
Sebastian Wagner

02/28/2021

09:23 PM Bug #49539 (Resolved): test_cephadm.sh failure: container.alertmanager.a does'nt start
... Sage Weil

02/26/2021

10:07 PM Bug #49522 (Resolved): cephadm: ValueError: not enough values to unpack (expected 2, got 1)
... Sage Weil
04:43 PM Bug #49506 (Fix Under Review): cephadm: `cephadm ls` broken for SUSE's downstream alertmanager co...
Patrick Seidensal
04:33 PM Bug #49506 (Resolved): cephadm: `cephadm ls` broken for SUSE's downstream alertmanager container
... Patrick Seidensal
11:01 AM Support #49499: new osds created by orchestrator running different image version
Already solved with help from mailing list, sorry for the noise.
The fix (thanks to Tobias Fisher):
Hi Kenneth,
...
Kenneth Waegeman

02/25/2021

07:01 PM Bug #49287: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope n...
... Sage Weil
04:21 PM Support #49499 (Resolved): new osds created by orchestrator running different image version
Hi,
I installed new host for OSDS today, they were automatically created using drivegroups service specs (https://do...
Kenneth Waegeman
03:16 PM Support #49497 (Resolved): Cephadm fails to upgrade from 15.2.8 to 15.2.9
I'm trying to upgrade cluster from 15.2.8 to 15.2.9 by running... Illya S.
11:24 AM Feature #49492 (Resolved): cepham: Spine-Leaf network architecture
https://access.redhat.com/webassets/avalon/d/Red_Hat_OpenStack_Platform-13-Spine_Leaf_Networking-en-US/images/0bd8baa... Sebastian Wagner
10:46 AM Documentation #49488: Document service specs for iSCSI deployment
fixed by https://github.com/ceph/ceph/pull/39551
https://ceph--39551.org.readthedocs.build/en/39551/cephadm/iscsi/
Sebastian Wagner
08:58 AM Documentation #49488 (Resolved): Document service specs for iSCSI deployment
There's currently no documentation on how to deploy iSCSI gateways (or, if there is, I can't find it beyond what's li... Tim Serong
10:34 AM Tasks #49490: cephadm additions/changes to support everything rgw.py needs
config key/value: https://github.com/ceph/ceph/pull/39648 Sebastian Wagner
10:32 AM Tasks #49490 (Resolved): cephadm additions/changes to support everything rgw.py needs
From: https://pad.ceph.com/p/rgw-cephadm
- adopt realm.zone.id naming for tests
- make sure PlacementSpec nam...
Sebastian Wagner
10:25 AM Bug #49467 (Fix Under Review): cephadm bootstrap process fails in open_ports on octopus branch: T...
Sebastian Wagner
01:29 AM Bug #49467 (Resolved): cephadm bootstrap process fails in open_ports on octopus branch: TypeError...
There is no problem until v15.2.8, but an error occurs during bootstrap process in open_ports of the latest octopus b... Donggyu Park
08:07 AM Bug #49456: cephadm dashboard test: failed to connect to the server
qa/workunits/cephadm/test_dashboard_e2e.sh (for SEO) Kefu Chai
06:36 AM Bug #49484 (Resolved): rados/upgrade/pacific-x/parallel fails due to more than 1 version shown af...
... Kefu Chai
06:33 AM Bug #49483 (Can't reproduce): CommandFailedError: Command failed on smithi104 with status 1: 'sud...
https://sentry.ceph.com/organizations/ceph/issues/4720/... Sentry Bot
03:40 AM Bug #49436 (In Progress): cephadm bootstrap fails to create /etc/ceph directory
the creation of the missing dir is there in the src for octopus and pacific. In both examples provided, the source is... Paul Cuzner

02/24/2021

10:50 PM Bug #49462 (Fix Under Review): cephadm: add iscsi and nfs to upgrade
Adam King
09:44 PM Bug #49462 (Resolved): cephadm: add iscsi and nfs to upgrade
these daemons are not upgraded during the upgrade process but they should be since they use the Ceph image. Adam King
05:19 PM Bug #49191 (Duplicate): cephadm: service_type: osd: Failed to apply: ''NoneType'' object has no a...
already fixed Sebastian Wagner
02:35 PM Bug #49456: cephadm dashboard test: failed to connect to the server
/a/kchai-2021-02-24_13:00:59-rados-wip-kefu-testing-2021-02-24-1742-distro-basic-smithi/5911838/ Kefu Chai
02:28 PM Bug #49456 (Can't reproduce): cephadm dashboard test: failed to connect to the server
https://pulpito.ceph.com/swagner-2021-02-22_13:53:31-rados:cephadm-wip-swagner3-testing-2021-02-22-1135-distro-basic-... Sebastian Wagner
12:42 PM Bug #49280 (Duplicate): mds/orch: bare/short hostname as a number is not supported
will be fixed by #46219 Sebastian Wagner
11:44 AM Bug #49449 (Won't Fix): cephadm: synchronize container timezone with host
Right now, the timezone within the container differs from the host. Timestamps in the future (or past) will hinder tr... Sebastian Wagner
04:06 AM Feature #47711 (Fix Under Review): mgr/cephadm: add a feature to examine the host facts to look f...
Paul Cuzner

02/23/2021

11:25 PM Bug #48870: cephadm: Several services in error status after upgrade to 15.2.8: unrecognized argum...
Sebastian, I am seeing something similar in pacific upgrade tests, want me create a new tracker issue?
rados/cepha...
Neha Ojha
09:26 PM Bug #49439: logrotate should clean up container logs
is this caused by cephadm? Sebastian Wagner
06:37 PM Bug #49439: logrotate should clean up container logs
See also https://bugzilla.redhat.com/show_bug.cgi?id=1892170
https://access.redhat.com/solutions/5434641
David Galloway
06:35 PM Bug #49439 (Resolved): logrotate should clean up container logs
... David Galloway
05:19 PM Bug #48598 (Can't reproduce): "ceph orch daemon redeploy" fails with [errno 13] RADOS permission ...
these tests pass now. there have been a ton of changes/fixes in hte last 2 months Sage Weil
04:18 PM Bug #49436 (Can't reproduce): cephadm bootstrap fails to create /etc/ceph directory
In trials of Octopus and Pacific, cephadm fails to create /etc/ceph during the bootstrapping process.
These two et...
Zac Dover
04:16 PM Feature #47145: cephadm: Multiple daemons of the same service on single host
... Sebastian Wagner
04:10 PM Fix #49336: re-enable coredumps for cephadm
two issues:
1. messanger v2 bug
2. "wait for snap complete" hangs
Sebastian Wagner
04:09 PM Fix #49336: re-enable coredumps for cephadm
sage: sending signals seemed to work find. Sebastian Wagner
04:09 PM Fix #49336: re-enable coredumps for cephadm
https://github.com/ceph/ceph/pull/39530 Sebastian Wagner
03:43 PM Bug #49435 (Closed): cephadm: rgw not getting deployed due to HEALTH_WARN
We should provide a way for users to deploy RGW anyway and at the same time prevent radosgw-admin to block indefinite... Sebastian Wagner
03:16 PM Bug #49411 (Fix Under Review): Rook: "ceph orch ls" command fails with KeyError: 'crashcollector'
Sebastian Wagner
03:16 PM Feature #47261 (Fix Under Review): cephadm integration for cephfs-mirror daemon
Sebastian Wagner
11:25 AM Feature #49159 (Fix Under Review): "cephadm ceph-volume activate" does not support cephadm
Sebastian Wagner

02/22/2021

02:33 PM Bug #49126 (Pending Backport): rook: 'ceph orch ls' throws type error
Varsha Rao
11:19 AM Bug #49411 (Resolved): Rook: "ceph orch ls" command fails with KeyError: 'crashcollector'
... Juan Miguel Olmo Martínez
09:50 AM Feature #49269: cephadm: upgrade stuck in repeating sleep when a host is offline
Gunther Heinrich wrote:
> There might be some cases where either the host is offline for a longer amount of time or ...
Sebastian Wagner
08:15 AM Feature #49269: cephadm: upgrade stuck in repeating sleep when a host is offline
Sebastian Wagner wrote:
> Did you verify that the upgrade continues, if the host is online again?
Yes, the upgrade...
Gunther Heinrich
01:43 AM Feature #49407 (Fix Under Review): Enable the ability of cephadm to trigger libstoragemgmt info f...
When this information is gathered, cephadm configuration checks will be able to;
- raise a healthcheck for any dis...
Paul Cuzner
01:41 AM Feature #49407 (Resolved): Enable the ability of cephadm to trigger libstoragemgmt info from ceph...
The default in ceph-volume inventory runs without libstoragemgmt integration, so by default this enhanced information... Paul Cuzner

02/20/2021

02:40 PM Bug #49143 (Resolved): rados/upgrade/pacific-x/parallel: monclient(hunting): authenticate timed o...
Sage Weil

02/19/2021

08:48 PM Bug #48695 (Closed): cephadm: upgrade should not get blocked, on intermediate UPGRADE_FAILED_PULL
The problem only is an issue when upgrading from older versions (that don't include https://github.com/ceph/ceph/comm... Adam King
06:52 PM Bug #49143 (Fix Under Review): rados/upgrade/pacific-x/parallel: monclient(hunting): authenticate...
Sage Weil
12:12 PM Bug #49293: podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,me...
... Sebastian Wagner

02/18/2021

09:27 PM Bug #49366 (In Progress): 'cephadm version' will show a trace back if image comes from a authenti...
Daniel Pivonka
09:26 PM Bug #49366 (Resolved): 'cephadm version' will show a trace back if image comes from a authenticat...
@
[root@magna081 sbin]# sudo cephadm version
Non-zero exit code 125 from /bin/podman run --rm --ipc=host --net=host...
Daniel Pivonka
07:13 PM Bug #49239: cephadm cannot deploy OSDs with selinux-policy-minimum
JuanMi's backport at https://github.com/ceph/ceph/pull/39636 Ken Dreyer
05:39 PM Bug #48754 (Pending Backport): "failed xx 'sudo systemctl start ceph-None@rgw.client.1'" in upgra...
Sage Weil
04:22 PM Bug #49350 (Duplicate): cephadm/test_dashboard_e2e.sh: AssertionError: Timed out retrying: expect...
Sebastian Wagner
03:40 PM Bug #48695: cephadm: upgrade should not get blocked, on intermediate UPGRADE_FAILED_PULL
https://github.com/ceph/ceph/blob/13c0f27837d79655dc27f0f15554cf3e501fa7e4/src/cephadm/cephadm#L3099-L3103 Sebastian Wagner
06:26 AM Feature #47711 (In Progress): mgr/cephadm: add a feature to examine the host facts to look for co...
Paul Cuzner
05:00 AM Bug #49348: cephadm: giving nonexistent service for service action command should return error
Thanks a lot for working on this. Is this linked to the feature request / bug #49246 I issued some days ago? In that ... Gunther Heinrich
04:12 AM Tasks #47369: Ceph scales to 100's of hosts, 1000's of OSDs....can orchestrator?
There was an issue with bucket aggregation in the heatmap panel, so instead of 7 clusters with 4,096 - 5,792 OSDs eac... Yaarit Hatuka
12:26 AM Bug #49293: podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,me...
/a/yuriw-2021-02-15_19:34:09-rados-octopus-distro-basic-smithi/5884688/ Neha Ojha

02/17/2021

11:19 PM Bug #49350 (Duplicate): cephadm/test_dashboard_e2e.sh: AssertionError: Timed out retrying: expect...
... Neha Ojha
11:14 PM Bug #49143: rados/upgrade/pacific-x/parallel: monclient(hunting): authenticate timed out after 30...
/a/nojha-2021-02-16_17:14:48-rados-master-distro-basic-smithi/5887669 Neha Ojha
10:10 PM Bug #49013: cephadm: Service definition causes some container startups to fail
Note: https://github.com/ceph/ceph/pull/39385 is intended to allow redeploy of daemons put into the state described i... Adam King
10:04 PM Bug #49013 (Fix Under Review): cephadm: Service definition causes some container startups to fail
Adam King
10:02 PM Bug #49348 (Fix Under Review): cephadm: giving nonexistent service for service action command sho...
Adam King
09:46 PM Bug #49348 (Resolved): cephadm: giving nonexistent service for service action command should retu...
Right now, if a user runs a command like "ceph orch redeploy . . ." and gives an incorrect service name no output is ... Adam King
10:00 PM Bug #48142 (Pending Backport): rados:cephadm/upgrade/mon_election tests are failing: CapAdd and p...
Sage Weil
09:41 PM Bug #49339 (Fix Under Review): cephadm/osd: OSD draining: don't mark OSDs out
Sage Weil
04:29 PM Bug #49339 (Resolved): cephadm/osd: OSD draining: don't mark OSDs out
there is already a --replace flag; we just need to fix the behavior in teh non-replace case to crush weight o 0 inste... Sebastian Wagner
04:21 PM Feature #47261: cephadm integration for cephfs-mirror daemon
Hey Sebastian,
This tracker is currently unassigned. Would we want to add an assignee? (and tho whom)
Venky Shankar
03:20 PM Fix #49336 (Resolved): re-enable coredumps for cephadm
we reverted the podman --init Pr. we need to find out why we have a problem there Sebastian Wagner
12:14 PM Bug #45452 (Closed): cephadm: while removing ceph-common, unable to remove directory '/var/lib/ce...
Juan Miguel Olmo Martínez
12:11 PM Bug #45907 (Resolved): cepham: daemon rm for managed services is completely broken
Juan Miguel Olmo Martínez
11:38 AM Bug #47381 (Can't reproduce): "ceph orch apply --dry-run" reports empty osdspec even though OSDs ...
Juan Miguel Olmo Martínez
11:25 AM Bug #48930: when removing the iscsi service, the gateway config object remains
this needs to be implemented in on of the methods here for iscsi:
https://github.com/ceph/ceph/blob/9bdc3b9f6fad2b...
Sebastian Wagner
10:23 AM Bug #45725 (Can't reproduce): cephadm: Further improve "Failed to infer CIDR network for mon ip"
Juan Miguel Olmo Martínez

02/16/2021

05:27 PM Bug #48930: when removing the iscsi service, the gateway config object remains
but who is creating the object in the first place? Sebastian Wagner
03:41 PM Bug #48930: when removing the iscsi service, the gateway config object remains
Sebastian Wagner wrote:
> who's creating the gateway.conf object? I can't find it in https://github.com/ceph/ceph/bl...
Juan Miguel Olmo Martínez
03:35 PM Bug #48930: when removing the iscsi service, the gateway config object remains
who's creating the gateway.conf object? I can't find it in https://github.com/ceph/ceph/blob/master/src/pybind/mgr/ce... Sebastian Wagner
03:30 PM Bug #48597 (Pending Backport): pybind/mgr/cephadm: mds_join_fs not cleaned up
Sebastian Wagner
03:05 PM Tasks #49306 (New): cephadm teuthology: Add RGW client workload
The current test suite is rather limited as we only verify the deployment of the cluster.
Would be great to have ...
Sebastian Wagner
03:01 PM Cleanup #49305 (New): cephadm: don't block the CLI handler thread
We still have a lot of places where we run remote SSH calls form within the CLI hander thread:... Sebastian Wagner
01:14 PM Bug #48598: "ceph orch daemon redeploy" fails with [errno 13] RADOS permission denied
reminds me of
https://www.reddit.com/r/ceph/comments/kv3z7h/cephadm_osd_deploy_errors/
but this doesn't make ...
Sebastian Wagner

02/15/2021

04:41 PM Bug #49293: podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,me...
https://github.com/containers/podman/issues/9382 Sebastian Wagner
04:04 PM Bug #49293: podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check with "nodev,me...
* https://github.com/containers/podman/blob/30607d727895036d32aede44c1b4375849566433/cmd/podman/root.go#L161
* https...
Sebastian Wagner
03:25 PM Bug #49293 (Rejected): podman 3.0 on ubuntu 18.04: failed to mount overlay for metacopy check wit...
cephadm faile to pull quay image using podman:... Deepika Upadhyay
03:55 PM Cleanup #45118 (Closed): orch (pacific): cleanup CLI
I think the cli is "done" for now Sebastian Wagner
01:02 PM Documentation #45767 (In Progress): documentation: disable the scheduler: unmanaged=True + ceph o...
Sebastian Wagner
12:35 PM Documentation #45833 (In Progress): cephadm: properly document labels
Sebastian Wagner
12:04 PM Feature #48560: Spec files for each daemon in the monitoring stack
This is the current **monitoring.yaml**:... Sebastian Wagner
12:01 PM Documentation #49214 (In Progress): Docs: howto Restore MON quorum
Sebastian Wagner
10:56 AM Documentation #48974: document repo_digest
it's enabled by default now Sebastian Wagner

02/13/2021

01:01 AM Bug #47921: Bad auth caps for orchestrated mds daemon
the problem is probably not related to the missing caps. Are you *really* sure this is the correct solution? Sebastian Wagner
12:54 AM Bug #49287 (New): podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.s...
... Sebastian Wagner

02/12/2021

05:24 PM Tasks #47369: Ceph scales to 100's of hosts, 1000's of OSDs....can orchestrator?
I was talking with Yaarit for getting real figures from Telemetry, and she mentioned the following ones:
* RBD ima...
Ernesto Puerta
04:15 PM Bug #49280: mds/orch: bare/short hostname as a number is not supported
... David Casier
04:13 PM Bug #49280 (Duplicate): mds/orch: bare/short hostname as a number is not supported
If the bare/short hostname is a simple id (example: 2.storage.domain):
----...
David Casier
03:46 PM Bug #49277: cephadm bootstrap --apply-spec <cluster.yaml> hangs
Reproduced just now and attaching logs and versions.
using latest cephadm from https://download.ceph.com/rpm-octo...
John Fulton
02:40 PM Bug #49277 (Duplicate): cephadm bootstrap --apply-spec <cluster.yaml> hangs
The feature introduced by https://tracker.ceph.com/issues/44873 seems to have the following flaw.
If I bootstrap...
John Fulton
01:48 PM Bug #49276 (Duplicate): Create multiple RGW instances in the same realm , same zone fails using c...
The scenario is to create 2 rgw instances on the same node (different ports) under the same realm and same zone.
Cu...
Juan Miguel Olmo Martínez
01:13 PM Bug #49273 (Resolved): cephadm fails deployment of node-exporter when ipv6 is disabled
Which is due to the check in `port_in_use` method checking for both, ipv4 and ipv6 support in the same try except block. Patrick Seidensal
12:45 PM Feature #49249 (Duplicate): cephadm: Automatically create OSDs after reinstalling base os
Sebastian Wagner
12:43 PM Feature #49159: "cephadm ceph-volume activate" does not support cephadm
Great! Please verify that the container image used is consistent across the cluster after running the adoption process. Sebastian Wagner
09:31 AM Feature #49159: "cephadm ceph-volume activate" does not support cephadm
Hi Sebastian,
Since I would need to do this for +500 OSDs, doing this manually is not really an option..
I guess ...
Kenneth Waegeman
12:30 PM Feature #49269: cephadm: upgrade stuck in repeating sleep when a host is offline
Did you verify that the upgrade continues, if the host is online again?
I'm a bit inclined to close this as works...
Sebastian Wagner
09:51 AM Feature #49269 (New): cephadm: upgrade stuck in repeating sleep when a host is offline
Even though the documentation clearly mentions that all hosts should be online before you initiate an upgrade I never... Gunther Heinrich
12:01 AM Feature #47885: Add networking checks
Currently preparing a PR which deals with requirements 1 and 4. Paul Cuzner

02/11/2021

07:34 PM Bug #49239: cephadm cannot deploy OSDs with selinux-policy-minimum
Follow-on fix for systems that do not have /usr/share/empty (eg. SUSE): https://github.com/ceph/ceph/pull/39424
An...
Ken Dreyer
01:34 AM Bug #49239 (Pending Backport): cephadm cannot deploy OSDs with selinux-policy-minimum
Ken Dreyer
07:12 PM Bug #48142 (Fix Under Review): rados:cephadm/upgrade/mon_election tests are failing: CapAdd and p...
Sage Weil
06:24 PM Feature #49159: "cephadm ceph-volume activate" does not support cephadm
does https://tracker.ceph.com/issues/46691#note-1 help? Sebastian Wagner
05:55 PM Feature #49159: "cephadm ceph-volume activate" does not support cephadm
Hi,
Thanks for looking into this!
Well, it's a chicken-egg problem:
This is a node that I just reinstalled to a...
Kenneth Waegeman
10:22 AM Feature #49159: "cephadm ceph-volume activate" does not support cephadm
there should not be such a big difference between running ceph-volume naively vs in a container.
Please have a lo...
Sebastian Wagner
04:49 PM Feature #49249 (Duplicate): cephadm: Automatically create OSDs after reinstalling base os
#46691 provides the manual process of deploying cephadm OSDs.
we should probably provide an automated way to do t...
Sebastian Wagner
03:51 PM Bug #48598: "ceph orch daemon redeploy" fails with [errno 13] RADOS permission denied
might be related to a wrong or non-upt-do-date ceph.conf??? Sebastian Wagner
12:25 PM Feature #43696 (Rejected): cephadm: check that units start
this would make the daemon deployment of cephadm super slow Sebastian Wagner
12:23 PM Documentation #45383 (Can't reproduce): Cephadm.py OSD deployment fails: full device path or just...
Sebastian Wagner
12:19 PM Support #49247 (Resolved): cephadm: Add support for single daemon redeployment
already done: https://docs.ceph.com/en/latest/api/mon_command_api/#orch-daemon-redeploy Sebastian Wagner
12:11 PM Support #49247 (Resolved): cephadm: Add support for single daemon redeployment
Currently, cephadm allows to redeploy services as a whole which sometimes might be a little over the top if only one ... Gunther Heinrich
12:17 PM Feature #45770 (Rejected): cephadm: allow count=0 to have services without daemons
why? Sebastian Wagner
12:15 PM Feature #43691 (Resolved): cephadm: upgrade major releases
done in the meantime Sebastian Wagner
12:14 PM Feature #44606 (Resolved): cephadm: RGW firewall + static port
resolved in the meantime Sebastian Wagner
12:14 PM Bug #46134 (Can't reproduce): ceph mgr should fail if it cannot add osd
Sebastian Wagner
12:13 PM Feature #45769 (Resolved): cephadm: Don't deploy on offline hosts
done in the meantime Sebastian Wagner
12:12 PM Feature #46265 (Duplicate): test cephadm MDS deployment
Sebastian Wagner
12:11 PM Documentation #46335: Document "Using cephadm to set up rgw-nfs"
thing is, I don't want users to use nfs-rgw. performance and usability is just poor. Sebastian Wagner
12:06 PM Bug #46568 (Can't reproduce): cephadm: Sometimes setting global container_image does not work
please reopen, if it is still reproducible Sebastian Wagner
12:05 PM Support #46547 (Resolved): cephadm: Exception adding host via FQDN if host was already added
Sebastian Wagner
12:01 PM Documentation #46691: Document manually deploment of OSDs
h3. How to manually (re-)deploy OSDs
In order to manually deploy cephadm OSDs, first run ceph-volume (skip this ...
Sebastian Wagner
11:56 AM Documentation #44354 (Duplicate): cephadm: Log messages are missing
fixed by using journald as log driver Sebastian Wagner
11:55 AM Bug #46990 (Can't reproduce): execnet: EOFError: couldnt load message header, expected 9 bytes, g...
Sebastian Wagner
11:54 AM Bug #44673 (Rejected): cephadm: `orch apply` and `orch daemon add` use completely different code ...
Sebastian Wagner
11:53 AM Bug #47401 (Can't reproduce): improve drive group validation
workaround: Use *ceph orch apply -i* instead of *ceph orch apply osd -i* Sebastian Wagner
11:52 AM Feature #47533 (Rejected): Scan for dangling ceph auth entries
we now clean up auth entities. Sebastian Wagner
11:50 AM Bug #47702 (Can't reproduce): upgrading via ceph orch upgrade start results in partial applicatio...
Sebastian Wagner
11:49 AM Bug #47694 (Won't Fix): downgrading via ceph orch upgrade start results in partial application an...
we have to support downgrades to some degree. closing as it worked eventually Sebastian Wagner
11:48 AM Bug #47726 (Resolved): disk selector should pass all devices to ceph-volume (available and unavai...
Sebastian Wagner
11:45 AM Bug #46665 (Resolved): cephadm plugin: Failure to start service stops service loop; no other inst...
Sebastian Wagner
11:45 AM Bug #47916 (Pending Backport): podman containers running in a detached state do not output logs t...
Sebastian Wagner
11:44 AM Bug #48105 (Can't reproduce): cephadm.py: failure on interactive on error for archive file handling
that code changed in the meantime Sebastian Wagner
11:42 AM Bug #48171 (Resolved): catatonit not available on CentOS
pacific now uses... Sebastian Wagner
11:41 AM Bug #45808 (Resolved): cephadm/test_adoption.sh: Error parsing image configuration: Invalid statu...
Sebastian Wagner
11:36 AM Bug #47500 (Won't Fix): Feature <encryption> is not supported" with having it set it to "False"
downstream issue not upstream Sebastian Wagner
11:36 AM Tasks #47369: Ceph scales to 100's of hosts, 1000's of OSDs....can orchestrator?
yes, we have users with > 1000 osds. that works already :-) Sebastian Wagner
11:27 AM Feature #45712 (Duplicate): Add 'state' attribute to ServiceSpec
Sebastian Wagner
11:25 AM Bug #46247 (Can't reproduce): cephadm mon failure: Error: no container with name or ID ... no suc...
This was fixed in the meantime Sebastian Wagner
11:12 AM Feature #47261 (New): cephadm integration for cephfs-mirror daemon
Sebastian Wagner
11:10 AM Feature #47261: cephadm integration for cephfs-mirror daemon
cephfs-mirror daemon
https://github.com/ceph/ceph/blob/72c3b5e6a3a88c40f6b8286cd4b2d6f1a335ed63/doc/man/8/cephfs-mir...
Sebastian Wagner
11:11 AM Feature #48560 (Need More Info): Spec files for each daemon in the monitoring stack
Sebastian Wagner
11:11 AM Feature #48560 (New): Spec files for each daemon in the monitoring stack
Sebastian Wagner
11:06 AM Feature #48560 (Need More Info): Spec files for each daemon in the monitoring stack
Sebastian Wagner
11:10 AM Bug #47107 (Resolved): device-health-metrics unavailable because image ceph/ceph:latest has smart...
resolved in ceph-container Sebastian Wagner
11:08 AM Feature #48822: Add proper port management to mgr/cephadm
... Sebastian Wagner
11:05 AM Feature #49246 (Duplicate): cephadm: Display error message when given service name is wrong
When executing a service command with a wrong service name like... Gunther Heinrich
11:05 AM Feature #47145: cephadm: Multiple daemons of the same service on single host
in order to co-locate daemons, we have to use different ports for those new daemons. Sebastian Wagner
11:02 AM Bug #48656: cephadm botched install of ceph-fuse (symbol lookup error)
tbh, this is somewhat out of scope for cephadm. cephadm mainly cares about containers, not so much about keeping pack... Sebastian Wagner
10:57 AM Bug #48442: cephadm: upgrade loops on mixed x86_64/arm64 cluster
right now, this is somewhat low on our priority list. But in pacific, this should be improved by using repo_digest fo... Sebastian Wagner
10:55 AM Bug #48261 (Won't Fix): cephadm ceph-volume inventory -- --format json-pretty: INFO:cephadm:/usr...
*workaround*: ... Sebastian Wagner
10:54 AM Bug #48799 (Can't reproduce): test_cephadm: stderr Job for container.alertmanager.a.service faile...
gone Sebastian Wagner
10:54 AM Subtask #45116 (Resolved): cephadm: RGW Load balancer using HAproxy
Sebastian Wagner
10:54 AM Documentation #48333 (Rejected): cephadm: document the image used by cephadm to call ceph-volume ...
Sebastian Wagner
10:53 AM Bug #48894: cephadm e2e: ceph device monitoring off: Error EINVAL
somehow this is gone now Sebastian Wagner
10:52 AM Bug #48894 (Can't reproduce): cephadm e2e: ceph device monitoring off: Error EINVAL
Sebastian Wagner
10:51 AM Bug #48694 (Resolved): ceph-volume: unrecognized arguments: --filter-for-batch
Sebastian Wagner
10:50 AM Bug #45973: Adopted MDS daemons are removed by the orchestrator because they're orphans
prio=low. probably easier to simply redeploy MDS for upstream and find a typical downstream solution for downstream. Sebastian Wagner
10:49 AM Bug #45465 (Resolved): cephadm: `ceph orch restart osd` has the potential to break your cluster
Sebastian Wagner
10:48 AM Support #48630 (Resolved): non-LVM OSD do not start after upgrade from 15.2.4 -> 15.2.7
Sebastian Wagner
10:47 AM Bug #45628 (Resolved): cephadm qa: smoke should verify daemons are actually running
Sebastian Wagner
10:47 AM Bug #48925 (Resolved): cephadm: iscsi missing mgr permissions
Sebastian Wagner
10:47 AM Bug #48947 (Resolved): cephadm: fix rgw osd cap tag
Sebastian Wagner
10:46 AM Bug #48594 (Resolved): cephadm: too many osd privileges for osd caps
Sebastian Wagner
10:45 AM Bug #48870 (Resolved): cephadm: Several services in error status after upgrade to 15.2.8: unrecog...
https://github.com/ceph/ceph/pull/39300 Sebastian Wagner
10:44 AM Bug #44559 (Can't reproduce): cephadm logs an invalid stat command
please reopen, if you see this again Sebastian Wagner
10:42 AM Bug #49016 (Resolved): find multiple coredumps of conmon
resolved upstream Sebastian Wagner
10:41 AM Bug #49056 (Resolved): faulty behaviour running ceph orch apply mds with missing fsname
Sebastian Wagner
10:41 AM Bug #49056: faulty behaviour running ceph orch apply mds with missing fsname
always use yaml files! ... Sebastian Wagner
10:39 AM Bug #48916 (Duplicate): "File system None does not exist in the map" in upgrade:octopus-x:paralle...
Sebastian Wagner
10:38 AM Bug #49014 (Resolved): OSD service specifications ignore "rotational: 0"
Sebastian Wagner
10:37 AM Feature #47139 (Resolved): Require a minimum version for podman/docker
Sebastian Wagner
10:37 AM Feature #47139: Require a minimum version for podman/docker
we can't backport podman >= 2.0 to octopus! Old octopus versions don#t support podman 2, thus we have to have a way f... Sebastian Wagner
10:26 AM Bug #48164 (Resolved): Orchestrator: failed deployments leave orphaned auth entries
Sebastian Wagner
10:25 AM Bug #45279: cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (21) Is a dir...
Could you please verify that /tmp/ceph-tmp6sp3jhv3 is in fact a file?
Then, could you please run the docker comman...
Sebastian Wagner
02:09 AM Bug #49228 (Resolved): qa/tasks/cephadm.py: file changed as we read it
pacific backport: https://github.com/ceph/ceph/pull/39403 Neha Ojha

02/10/2021

05:05 PM Bug #49143: rados/upgrade/pacific-x/parallel: monclient(hunting): authenticate timed out after 30...
The problem seems to occur when the first mon is restarted after upgrade.... Neha Ojha
03:23 PM Bug #49239 (Resolved): cephadm cannot deploy OSDs with selinux-policy-minimum
When the following conditions are true:
# A host has @selinux-policy-targeted@,
# We mount the host's @/sys@ int...
Ken Dreyer
12:47 PM Bug #48142 (New): rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are...
Sebastian Wagner
12:35 PM Feature #49235 (Resolved): cephadm: Log number of already upgraded daemons during upgrade process
During the upgrade process it would be helpful if cephadm displays the number of daemons which have aleady been upgra... Gunther Heinrich
11:26 AM Bug #49233 (New): cephadm shell: TLS handshake timeout
https://pulpito.ceph.com/swagner-2021-02-09_10:28:14-rados:cephadm-wip-swagner2-testing-2021-02-08-1109-pacific-distr... Sebastian Wagner
11:23 AM Bug #49232 (Can't reproduce): standard_init_linux.go:211: exec user process caused "exec format e...
https://pulpito.ceph.com/swagner-2021-02-09_10:28:14-rados:cephadm-wip-swagner2-testing-2021-02-08-1109-pacific-distr... Sebastian Wagner

02/09/2021

10:21 PM Bug #49228: qa/tasks/cephadm.py: file changed as we read it
In pacific... Neha Ojha
10:15 PM Bug #49228 (Resolved): qa/tasks/cephadm.py: file changed as we read it
... Neha Ojha
02:35 PM Bug #49223: unrecognized arguments: --container-init
If I remember correctly, the "--container-init" saga went about like so:
1. in general, there is a need for contai...
Nathan Cutler
10:09 AM Bug #49223: unrecognized arguments: --container-init
looks like https://github.com/ceph/ceph/pull/36822 was broken back then and https://github.com/ceph/ceph/pull/37648 n... Sebastian Wagner
09:51 AM Bug #49223 (Resolved): unrecognized arguments: --container-init
... Sebastian Wagner
10:15 AM Bug #49126 (Fix Under Review): rook: 'ceph orch ls' throws type error
Varsha Rao

02/08/2021

06:47 PM Bug #49143: rados/upgrade/pacific-x/parallel: monclient(hunting): authenticate timed out after 30...
... Neha Ojha
06:30 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
description: rados/cephadm/upgrade/{1-start 2-repo_digest/repo_digest 3-start-upgrade
4-wait distro$/{centos_8....
Deepika Upadhyay
03:52 PM Documentation #49214 (Resolved): Docs: howto Restore MON quorum
https://docs.ceph.com/en/latest/rados/operations/add-or-rm-mons/#removing-monitors-from-an-unhealthy-cluster
Sebastian Wagner
09:38 AM Bug #48068 (Resolved): cephadm: Various properties like 'last_refresh' do not contain timezone
Volker Theile
06:13 AM Bug #49013: cephadm: Service definition causes some container startups to fail
> 1. During upgrade, the new mgr doesn't redeploy the other mgrs (again) to ensure the unit.run file is in sync with ... Gunther Heinrich
 

Also available in: Atom