Activity
From 10/18/2020 to 11/16/2020
11/16/2020
- 08:49 PM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- Michael Fritch wrote:
> Perhaps we could start mirroring these sepia related container deps on quay.ceph.io ??
> ht... - 06:55 PM Bug #48041: mgr/cephadm: Allow customizing mgr/cephadm/lsmcli_blink_lights_cmd per host
- Juan Miguel Olmo Martínez wrote:
> To allow the user to introduce commands that we cannot test to make "in the wild... - 03:52 PM Cleanup #48251 (Won't Fix): Find a better way to allow customizing the blink_device_light_cmd cmd
- with the current approach, it is possible to run...
- 11:53 AM Feature #48247 (New): cephadm: RGW rgw_ldap_secret
- When deploying RGW via cephadm the path specified in rgw_ldap_secret should be mounted into the container.
Alterna... - 11:44 AM Bug #47513 (Fix Under Review): rook: 'ceph orch ps' does not show image and container id correctly
11/14/2020
- 12:07 AM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- Another example, but this time `prom/alertmanager:v0.20.0`, `ceph/ceph-grafana:6.6.2`, and `prom/prometheus:v2.18.1`
...
11/13/2020
- 07:14 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- Has this been fixed in octopus yet? It was seen here: a/yuriw-2020-11-13_16:25:11-rados-octopus-distro-basic-smithi/5...
- 07:14 PM Bug #48144 (Duplicate): rados/cephadm/upgrade: ceph_mons times out
- 06:43 PM Bug #48144: rados/cephadm/upgrade: ceph_mons times out
- /a/yuriw-2020-11-13_16:25:11-rados-octopus-distro-basic-smithi/5619781
- 05:09 PM Bug #48223 (Fix Under Review): host key might not exist during host refresh
- 05:06 PM Bug #48223: host key might not exist during host refresh
- Which eventually breaks the cli in a bad way:...
- 05:05 PM Bug #48223: host key might not exist during host refresh
- ...
- 05:03 PM Bug #48223 (Resolved): host key might not exist during host refresh
- ssh communication failures during a cephadm host add/remove can break the serve loop:...
- 04:41 PM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- Perhaps we could start mirroring these sepia related container deps on quay.ceph.io ??
https://wiki.sepia.ceph.com/d... - 04:38 PM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- This is becoming more frequent on various ceph images (v15.2.0, octopus, latest-devel-master etc.) and also the monit...
11/12/2020
- 11:51 PM Bug #48158: cephadm bootstrap fails with custom ssh port
- Thilo-Alexander Ginkel wrote:
> Problem solved. Thanks a lot for your support! :-)
Thanks for the bug report! :)
... - 11:48 PM Bug #48158 (Fix Under Review): cephadm bootstrap fails with custom ssh port
- 12:46 PM Bug #48158: cephadm bootstrap fails with custom ssh port
- Problem solved. Thanks a lot for your support! :-)
- 03:34 AM Bug #48158: cephadm bootstrap fails with custom ssh port
- cephadm will use a default ssh_config as follows:...
- 03:25 AM Bug #48158: cephadm bootstrap fails with custom ssh port
- finally had a chance to test this and the errors reported from the underlying remoto lib are less than helpful:
<p... - 11:51 PM Bug #48162 (Resolved): octopus: all mgr daemons do not support module 'cephadm'
- Neha Ojha wrote:
> retesting with https://github.com/ceph/ceph-build/pull/1704
https://pulpito.ceph.com/nojha-202... - 09:18 PM Bug #48162: octopus: all mgr daemons do not support module 'cephadm'
- retesting with https://github.com/ceph/ceph-build/pull/1704
- 05:42 AM Bug #48162 (Fix Under Review): octopus: all mgr daemons do not support module 'cephadm'
- - https://github.com/ceph/ceph-build/pull/1700
- https://github.com/ceph/ceph-container/pull/1753 - 05:43 PM Bug #47513: rook: 'ceph orch ps' does not show image and container id correctly
- Expected output:...
- 05:40 PM Bug #47513 (In Progress): rook: 'ceph orch ps' does not show image and container id correctly
- Related ceph and nfs-ganesha version discussion: https://github.com/rook/rook/issues/6565
- 09:53 AM Bug #47035: TypeError: _daemon_action_redeploy() missing 1 required positional argument: 'image'
- Came here to report this bug. Because the report above is a bit incomplete, here is some more information for clarifi...
- 07:53 AM Bug #48071 (In Progress): rook: 'ceph orch ls' does not list nfs-ganesha daemons
11/11/2020
- 10:23 PM Bug #48205 (Fix Under Review): cephadm: last local ceph image tag can be null
- 10:19 PM Bug #48205: cephadm: last local ceph image tag can be null
- Filtering this out to the list of image names seems more reasonable:...
- 10:18 PM Bug #48205: cephadm: last local ceph image tag can be null
- `--filter dangling=false` does not appear to trim the list when the image tag is null...
- 10:16 PM Bug #48205 (Resolved): cephadm: last local ceph image tag can be null
- ...
- 08:29 PM Bug #45808: cephadm/test_adoption.sh: Error parsing image configuration: Invalid status code retu...
- This is likely docker rate limiting as per https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH...
- 07:38 PM Bug #45808: cephadm/test_adoption.sh: Error parsing image configuration: Invalid status code retu...
- http://qa-proxy.ceph.com/teuthology/yuriw-2020-11-05_20:17:33-rados-wip-yuri2-testing-2020-11-05-1029-octopus-distro-...
- 07:08 PM Bug #48162: octopus: all mgr daemons do not support module 'cephadm'
- https://pulpito.ceph.com/yuriw-2020-11-10_20:43:58-rados-octopus-distro-basic-smithi/...
- 05:06 PM Bug #48158: cephadm bootstrap fails with custom ssh port
- Still no luck... Removed all ceph containers, removed /var/lib/ceph and /etc/ceph, then:...
- 04:02 AM Bug #48158: cephadm bootstrap fails with custom ssh port
- maybe not entirely ideal, but I think this will work:
https://github.com/ceph/ceph/commit/193a1b3fc2faee50ccc87f78e4... - 02:17 AM Bug #48158: cephadm bootstrap fails with custom ssh port
- eh sorry, `ceph orch restart mgr` happens async, so issuing a `host add` immediately after won't work without some ki...
- 04:14 PM Bug #47234 (Resolved): by default, cephadm creates 4 MONs
- 01:56 PM Cleanup #48113 (Resolved): doc/mgr/orchestrator: Add hints related to custom containers to the docs
11/10/2020
- 08:52 PM Bug #48158: cephadm bootstrap fails with custom ssh port
- I tried including the restart in @cephadm@ after setting the ssh config, but that also does not seem to have any effe...
- 08:23 PM Bug #48158: cephadm bootstrap fails with custom ssh port
- Thanks for your reply! Unfortunately, the workaround does not seem to work:...
- 07:58 PM Bug #48158 (Need More Info): cephadm bootstrap fails with custom ssh port
- These are set using the generic ceph key/value store (ceph config-key), which would require a restart of the mgr. It ...
- 01:37 PM Bug #48158: cephadm bootstrap fails with custom ssh port
- Manually attempting to apply the troubleshooting instructions does indeed work, so the root cause is unclear to me:
... - 08:12 PM Bug #48171: catatonit not available on CentOS
- https://pulpito.ceph.com/mgfritch-2020-11-10_01:16:20-rados:cephadm-wip-mgfritch-testing-2020-11-09-1644-distro-basic...
- 08:11 PM Bug #48171 (Resolved): catatonit not available on CentOS
- https://pulpito.ceph.com/mgfritch-2020-11-10_01:16:20-rados:cephadm-wip-mgfritch-testing-2020-11-09-1644-distro-basic...
- 05:29 PM Bug #48166 (Fix Under Review): cephadm should be run as root
- 05:25 PM Bug #48166 (Resolved): cephadm should be run as root
- ...
- 11:27 AM Bug #48164 (Resolved): Orchestrator: failed deployments leave orphaned auth entries
- During upgrade tests to cephadm I had troubles deploying mds nodes. In the end I managed to deploy containerized mds ...
- 09:06 AM Bug #48162 (Resolved): octopus: all mgr daemons do not support module 'cephadm'
- ...
11/09/2020
- 10:59 PM Bug #48158 (Resolved): cephadm bootstrap fails with custom ssh port
- Attempting to bootstrap a Ceph cluster using @cephadm@ fails if the host is using a non-standard SSH port instead of ...
- 10:50 PM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- /a/teuthology-2020-11-09_07:01:01-rados-master-distro-basic-smithi/5605208
- 08:28 PM Bug #48157 (Resolved): test_cephadm.sh failure You have reached your pull rate limit. You may inc...
- http://qa-proxy.ceph.com/teuthology/yuriw-2020-11-09_17:43:09-rados-wip-yuri8-testing-2020-11-09-0807-octopus-distro-...
- 10:22 PM Bug #48144: rados/cephadm/upgrade: ceph_mons times out
- I think this is a duplicate?
See: https://tracker.ceph.com/issues/48142#note-3 - 10:18 PM Bug #48144: rados/cephadm/upgrade: ceph_mons times out
- /a/teuthology-2020-11-09_07:01:01-rados-master-distro-basic-smithi/5605118
11/06/2020
- 09:55 PM Bug #48107 (New): cephadm fails to deploy iscsi gateway when selinux is enabled
- 05:33 PM Bug #48107: cephadm fails to deploy iscsi gateway when selinux is enabled
- Nathan Cutler wrote:
> How did you deploy your iscsi gateway, if not using cephadm?
iSCSI gateway was deployed usi... - 08:36 AM Bug #48107: cephadm fails to deploy iscsi gateway when selinux is enabled
- > After a closer look, it looks like iSCSI gateway deployment doesn't use ceph-daemon
cephadm is not called "ceph-... - 07:39 PM Bug #48144 (Duplicate): rados/cephadm/upgrade: ceph_mons times out
- ...
- 04:05 PM Cleanup #48140: cephadm: provide dashboard URL + credentials in an friendly way
- Also somewhat related:
https://github.com/ceph/ceph/pull/37712#issuecomment-722631095
bin/cephadm needs to provid... - 11:52 AM Cleanup #48140 (Duplicate): cephadm: provide dashboard URL + credentials in an friendly way
- Right now after bootstrap stage, cephadm returns dashboard URL and credentials within the output of the whole command...
- 03:45 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- Fixed by this PR:
https://github.com/ceph/ceph/pull/37133
But that would require us to move the upgrade start poi... - 03:44 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- It's also appearing in the nightly Octopus test runs:
https://pulpito.ceph.com/teuthology-2020-11-01_02:30:03-rados-... - 03:43 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- A few more examples of this failure:
https://pulpito.ceph.com/kchang-2020-11-04_06:38:14-rados:cephadm-wip-kiefer-te... - 03:30 PM Bug #48142 (Resolved): rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privilege...
- https://pulpito.ceph.com/mgfritch-2020-11-04_22:57:20-rados:cephadm-wip-mgfritch-testing-2020-11-04-1200-distro-basic...
11/05/2020
- 09:20 PM Bug #48107: cephadm fails to deploy iscsi gateway when selinux is enabled
- Nathan Cutler wrote:
> This looks like a mistake. https://github.com/ceph/ceph/pull/31321 contains a single commit 0... - 10:15 AM Cleanup #48113: doc/mgr/orchestrator: Add hints related to custom containers to the docs
- Backport PR for Octopus is https://github.com/ceph/ceph/pull/37962
- 09:59 AM Cleanup #48113 (Pending Backport): doc/mgr/orchestrator: Add hints related to custom containers t...
11/04/2020
- 06:52 PM Bug #48120 (Fix Under Review): JSONDecodeErrors unclear during cephadm upgrade
- 06:42 PM Bug #48120 (Resolved): JSONDecodeErrors unclear during cephadm upgrade
- a/teuthology/mgfritch-2020-11-03_14:22:58-rados:cephadm-wip-mgfritch-testing-2020-11-02-1045-distro-basic-smithi/5586...
- 06:40 PM Bug #48119 (Fix Under Review): JSONDecodeErrors break OSD deploy
- 06:24 PM Bug #48119: JSONDecodeErrors break OSD deploy
- a/teuthology/mgfritch-2020-11-03_14:22:58-rados:cephadm-wip-mgfritch-testing-2020-11-02-1045-distro-basic-smithi/5586...
- 06:23 PM Bug #48119 (Resolved): JSONDecodeErrors break OSD deploy
- a/teuthology/mgfritch-2020-11-03_14:22:58-rados:cephadm-wip-mgfritch-testing-2020-11-02-1045-distro-basic-smithi/5586...
- 06:23 PM Feature #48074 (Resolved): cephadm logs needs a -f (follow) option
- Ahh thanks! That works fine, and since it's documented in later versions of cephadm, we can probably close this.
- 06:18 PM Bug #48118 (Fix Under Review): JSONDecodeErrors break the server loop
- 06:16 PM Bug #48118 (Resolved): JSONDecodeErrors break the server loop
- a/mgfritch-2020-11-03_14:22:58-rados:cephadm-wip-mgfritch-testing-2020-11-02-1045-distro-basic-smithi/5586920/teuthol...
- 01:52 PM Feature #48114 (Duplicate): Cephadm to support Adding multiple instances of RGW in same node for ...
- Cephadm to support Adding multiple instances of RGW in same node for 5.0
- 01:41 PM Cleanup #48113 (Fix Under Review): doc/mgr/orchestrator: Add hints related to custom containers t...
- 01:38 PM Cleanup #48113 (Resolved): doc/mgr/orchestrator: Add hints related to custom containers to the docs
- The documentation how to deploy custom containers need some cleanup and additional information how to handle paths.
- 10:22 AM Bug #47924 (Closed): rook: 'ceph orch daemon add nfs' fails due to invalid field value
- Fixed by https://tracker.ceph.com/issues/47923
- 10:17 AM Bug #48105: cephadm.py: failure on interactive on error for archive file handling
- to reproduce: on teuthology: ...
- 09:20 AM Bug #48107 (Resolved): cephadm fails to deploy iscsi gateway when selinux is enabled
- This looks like a mistake. https://github.com/ceph/ceph/pull/31321 contains a single commit 0444025aaf559a662882abc49...
- 07:38 AM Bug #47922 (Closed): rook: Failed to load ceph-mgr modules: cephadm, dashboard
- This issue no longer occurs with latest master branch image.
- 12:23 AM Bug #47923 (Pending Backport): rook: 'ceph orch apply nfs' throws error if no ganesha daemons are...
11/03/2020
- 10:46 PM Feature #48074: cephadm logs needs a -f (follow) option
- arbitrary args to be can be passed to journalctl:...
- 10:46 PM Bug #48107 (Can't reproduce): cephadm fails to deploy iscsi gateway when selinux is enabled
- With Ceph 15.2.5 iSCSI Gateway containers will still fail start.
Applying the changes from the following pull requ... - 06:17 PM Bug #48105 (Can't reproduce): cephadm.py: failure on interactive on error for archive file handling
when running on interactive on error mode,iirc, observed we might not be handling archive path generation:...- 11:30 AM Feature #48102 (New): cephadm: configure HA (cluster flags) for Alertmanager
- While HA in metrics (Grafana) is less relevant (manual switchover is the basic workaround), alerting should always pe...
- 09:32 AM Bug #48068 (Fix Under Review): cephadm: Various properties like 'last_refresh' do not contain tim...
11/02/2020
- 09:56 PM Bug #47107: device-health-metrics unavailable because image ceph/ceph:latest has smartmontools 6.6
- Dimitri temporarily added the Copr repository to the container images in https://github.com/ceph/ceph-container/pull/...
- 06:36 PM Bug #46038: cephadm mon start failure: Failed to reset failed state of unit ceph-9342dcfe-afd5-11...
- observed in octopus:
http://qa-proxy.ceph.com/teuthology/yuriw-2020-10-30_15:36:14-rados-wip-yuri-testing-2020-10-2... - 05:35 PM Feature #48074 (Resolved): cephadm logs needs a -f (follow) option
- It would be nice to be able to follow logs as they are being printed from cephadm. Basically, be able to tell it to s...
- 03:57 PM Bug #48072 (Fix Under Review): ppa:projectatomic is no longer maintained
- 03:54 PM Bug #48072: ppa:projectatomic is no longer maintained
- ubuntu bionic (18.04) installs podman via ppa:projectatomic...
- 03:52 PM Bug #48072 (Resolved): ppa:projectatomic is no longer maintained
- a/mgfritch-2020-10-29_02:49:56-rados:cephadm-wip-mgfritch-testing-2020-10-28-1744-distro-basic-smithi/5569611/
<pr... - 03:51 PM Bug #48071 (Resolved): rook: 'ceph orch ls' does not list nfs-ganesha daemons
- ...
- 10:04 AM Bug #48068 (In Progress): cephadm: Various properties like 'last_refresh' do not contain timezone
- 08:19 AM Bug #48068 (Resolved): cephadm: Various properties like 'last_refresh' do not contain timezone
- The property 'last_refresh', 'created', 'started', 'last_configured' or 'last_deployed' in 'ServiceDescription' and '...
10/31/2020
- 10:48 AM Bug #48031 (Fix Under Review): Cephadm: Needs to pass cluster.listen-address to alertmanager
10/30/2020
- 11:08 AM Bug #48041: mgr/cephadm: Allow customizing mgr/cephadm/lsmcli_blink_lights_cmd per host
- Volker Theile wrote:
...
> The problem is that the template to build the command line to enable/disable the LEDs is... - 08:22 AM Bug #48041 (Fix Under Review): mgr/cephadm: Allow customizing mgr/cephadm/lsmcli_blink_lights_cmd...
- 07:11 AM Bug #48041: mgr/cephadm: Allow customizing mgr/cephadm/lsmcli_blink_lights_cmd per host
- Juan Miguel Olmo Martínez wrote:
> Could you mind to explain better the problem?
>
> The blinking light feature i...
10/29/2020
- 04:35 PM Bug #46764 (Can't reproduce): cephadm (ceph orch apply) sometimes gets "stuck" and cannot deploy ...
- no longer reproducible
- 04:13 PM Bug #48041: mgr/cephadm: Allow customizing mgr/cephadm/lsmcli_blink_lights_cmd per host
Could you mind to explain better the problem?
The blinking light feature is to use with individual storage devic...- 03:59 PM Bug #48041 (Resolved): mgr/cephadm: Allow customizing mgr/cephadm/lsmcli_blink_lights_cmd per host
- PR https://github.com/ceph/ceph/pull/36911 introduced a feature to customize the command that is internally used to s...
- 03:06 PM Bug #47360 (Resolved): cephadm: osd unit.run creates /var/run/ceph/$FSID too late, so OSD may not...
- backport to octopus via https://github.com/ceph/ceph/pull/37436
- 01:57 PM Feature #46182 (Resolved): cephadm should use the same image reference across the cluster
- backported to octopus via https://github.com/ceph/ceph/pull/37436
- 01:56 PM Bug #46814 (Resolved): cephadm: Deploying alertmanager image is broken
- backported to octopus via https://github.com/ceph/ceph/pull/36450
- 01:06 PM Bug #46665 (Pending Backport): cephadm plugin: Failure to start service stops service loop; no ot...
10/28/2020
- 05:49 PM Bug #48031: Cephadm: Needs to pass cluster.listen-address to alertmanager
- I've submitted a pull request.
https://github.com/ceph/ceph/pull/37883
- 05:12 PM Bug #48031 (Resolved): Cephadm: Needs to pass cluster.listen-address to alertmanager
- When using public IP with the ceph dashboard/monitoring, alert manager will not start with the following error messag...
- 03:41 PM Bug #47923 (Fix Under Review): rook: 'ceph orch apply nfs' throws error if no ganesha daemons are...
10/27/2020
- 04:07 PM Bug #48019 (Can't reproduce): cephadm: `ceph daemon <daemon-name> ...` is broken
- ...
- 04:06 PM Documentation #45564: cephadm: document workaround for accessing the admin socket by entering run...
- I just noticed this ticket is in the Documentation tracker. I opened a new ticket #48019 to track the actual bug (it ...
10/26/2020
- 09:47 PM Bug #47107: device-health-metrics unavailable because image ceph/ceph:latest has smartmontools 6.6
- RHEL 8.3 is going to ship smartmontools 7 (https://bugzilla.redhat.com/show_bug.cgi?id=1671154).
In the meantime, ... - 11:14 AM Bug #45628: cephadm qa: smoke should verify daemons are actually running
- The teuthology project is for tracking issues in teuthology itself, not for tracking missing test cases.
According...
10/24/2020
- 10:06 AM Bug #47387 (Resolved): rook: 'ceph orch ps' does not list daemons correctly
- The backport of this issue is done in https://github.com/ceph/ceph/pull/37436
10/23/2020
- 10:29 AM Feature #47970 (New): cephadm: enable user to retrieve configuration templates for monitoring com...
- Due to the necessity to migrate configuration files of monitoring components manually, the user should be able to con...
- 10:20 AM Bug #47969 (New): cephadm: output migration of alertmanager is confusing
- Though the migration process runs through successfully, the displayed output is probably confusing:...
- 10:05 AM Bug #47924 (Fix Under Review): rook: 'ceph orch daemon add nfs' fails due to invalid field value
- 09:56 AM Bug #47923: rook: 'ceph orch apply nfs' throws error if no ganesha daemons are deployed
- Currently, it works only if nfs-ganesha daemons are already deployed. Otherwise too, it should work. Since 'orch appl...
- 09:48 AM Bug #47968 (Resolved): rook: 'ceph orch rm' throws type error
- ...
- 08:10 AM Bug #47501 (Resolved): cephadm: Error bootstraping with '--container-init' option
10/22/2020
- 11:42 AM Bug #44990 (New): cephadm: exec: "/usr/bin/ceph-mon": stat /usr/bin/ceph-mon: no such file or dir...
- seems like this issue still exists as i see the fix PR was backported to octopus, seen this in:
/a/yuriw-2020-10-...
10/21/2020
- 03:42 PM Bug #47924 (In Progress): rook: 'ceph orch daemon add nfs' fails due to invalid field value
- 08:44 AM Bug #47924 (Closed): rook: 'ceph orch daemon add nfs' fails due to invalid field value
- ...
- 10:55 AM Bug #46031 (Resolved): Exception: Failed to validate Drive Group: block_wal_size must be of type int
- 08:39 AM Bug #47923 (Resolved): rook: 'ceph orch apply nfs' throws error if no ganesha daemons are deployed
- ...
- 08:03 AM Bug #47921: Bad auth caps for orchestrated mds daemon
- mds created by "ceph orch apply mds 1" get auth caps of
mds.test_fs.******.gpqtol
key: ***************==
caps: [... - 08:01 AM Bug #47921 (Can't reproduce): Bad auth caps for orchestrated mds daemon
- mds created by "ceph orch apply mds 1" get auth caps of...
- 08:02 AM Bug #47922 (Closed): rook: Failed to load ceph-mgr modules: cephadm, dashboard
- Mgr log...
- 07:23 AM Bug #47862 (Pending Backport): cepham no longer requires apparmor-abstractions on SUSE
10/20/2020
- 07:15 PM Bug #47916: podman containers running in a detached state do not output logs to journald
- Configuring journald as the log driver allows for the conmon logs to show the actual output from the failed daemon co...
- 07:11 PM Bug #47916 (Resolved): podman containers running in a detached state do not output logs to journald
- When a service has failed, it can be difficult to determine the actual cause:...
10/19/2020
- 07:00 PM Bug #47905 (Resolved): cephadm: cephadm bootstrap is missing structured output. (was: logging to...
- cephadm CLI is currently using stderr for logging instead of stdout for info/debug statements....
- 08:10 AM Feature #47885 (New): Add networking checks
- The networking checks could be added to the orchestrator, given its access to each of the nodes.
It could e.g.
1...
Also available in: Atom