Activity
From 09/26/2022 to 10/25/2022
10/25/2022
- 07:00 PM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- I rebooted last night, all items report a refreshed time of about 13 hours ago, when I rebooted....
- 12:14 PM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- this seems to imply the cephadm service loop just isn't running at all. Does the REFRESHED column in `ceph orch devic...
- 05:34 PM Bug #57917: cephadm: duplicate log entry for /var/log/ceph/cephadm.log
- As a workaround, I tried a `prerotate` script:...
- 04:08 AM Bug #57917: cephadm: duplicate log entry for /var/log/ceph/cephadm.log
- this is also problematic for users upgrading from Octopus based clusters to Pacific.
- 04:06 AM Bug #57917 (New): cephadm: duplicate log entry for /var/log/ceph/cephadm.log
- Configuration of logrotate for /var/lib/ceph/cephadm.log was added (and backported to Pacific) via this PR:
https://... - 12:18 PM Cleanup #57921 (Resolved): orchestrator: orch upgrade status help message is wrong
- Currently the `orch upgrade status` help message is just a copy of the `orch upgrade check` help message...
- 04:00 AM Cleanup #50168 (Resolved): cephadm: move bin/cephadm from the git tree to download.ceph.com
- 03:02 AM Backport #57638 (Resolved): pacific: applying osd service spec with size filter fails if there's ...
10/22/2022
- 03:16 AM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- Seems I haven't seen the "host address is empty" error in about 10 days now.... Not sure if that is because of DNS,...
10/21/2022
- 03:55 PM Bug #56951: rook/smoke: Updating cephclusters/rook-ceph is forbidden
- /a/yuriw-2022-10-12_16:24:50-rados-wip-yuri8-testing-2022-10-12-0718-quincy-distro-default-smithi/7063866/
- 03:36 PM Bug #52321: qa/tasks/rook times out: 'check osd count' reached maximum tries (90) after waiting f...
- /a/yuriw-2022-10-12_16:24:50-rados-wip-yuri8-testing-2022-10-12-0718-quincy-distro-default-smithi/7063706/
- 09:53 AM Bug #57910 (New): ingress: HAProxy fails to start because keepalived IP address not yet available...
- After deploying a new cluster _sometimes_ HAProxy fails to start on ingress nodes:...
- 06:03 AM Feature #55490: cephadm: allow passing grafana cert and frontend-api-url in spec
- The OP mentioned @set-grafana-frontend-api-url@ but missed mentioning setting @set-grafana-api-url@ from a spec which...
10/20/2022
- 06:11 PM Feature #57909 (Resolved): cephadm: make logging host refresh data to debug logs configurable
- The amount of data we log in the debug logs when refreshing a host is too verbose, even for debug level. It renders t...
- 03:01 PM Backport #57638: pacific: applying osd service spec with size filter fails if there's tiny (KB-si...
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/48243
merged - 02:58 PM Backport #57639: pacific: cephadm: `ceph orch ps` doesn't list container versions in some cases
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/48210
merged
10/19/2022
- 05:56 PM Bug #57341: cephadm: failures from tests comparing output strings are difficult to debug
- See attached screenshot for a better colorized example.
- 05:53 PM Bug #57341: cephadm: failures from tests comparing output strings are difficult to debug
- I did a few minutes of research and found two packages that may help:
pytest-mock (https://pytest-mock.readthedocs.i... - 12:27 PM Bug #57897 (New): ceph mgr restart causes restart of all iscsi daemons in a loop
- We have observed that since v17.2.4, a restart of the active ceph mgr appears to cause all iSCSI daemons to restart a...
- 10:30 AM Feature #57894 (Fix Under Review): Move prometheus spec check to the service_spec module
- 10:18 AM Feature #57894 (Pending Backport): Move prometheus spec check to the service_spec module
10/18/2022
- 06:04 PM Bug #57891 (Resolved): [Gibba Cluster] HEALTH_ERR: Upgrade: failed due to an unexpected exception
- - Upgrade paused due to one host not being reachable in the cluster.
- Resumed the upgrade with the resume command
... - 03:49 PM Backport #57787 (In Progress): quincy: mgr/nfs: Add a sectype field to nfs exports created by nfs...
- 08:19 AM Bug #57096: osd not restarting after upgrading to quincy due to podman args --cgroups=split
- I manually created the unit.meta, and it seems to work. thanks again.
- 06:28 AM Bug #57096: osd not restarting after upgrading to quincy due to podman args --cgroups=split
- The unit.meta file is not yet present in Octopus. I'll try to figure something out or wait for the PR release.
Tha...
10/17/2022
- 07:29 PM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- alright, looking back at the original traceback...
- 06:55 PM Bug #57884 (Resolved): cephadm: attempting a daemon redeploy of the active mgr with a specified i...
- If I run something like...
- 10:54 AM Feature #57878 (Resolved): Add typing checks for rgw module
- 09:32 AM Bug #57876 (Fix Under Review): prometheus ERROR failed to collect metrics
- 09:13 AM Bug #57876 (Resolved): prometheus ERROR failed to collect metrics
- ...
10/14/2022
- 06:50 PM Bug #57870 (Resolved): cephadm: --apply-spec is trying to do too much and failing as a result
- --apply-spec is intended to do 2 things:
1) distribute ssh keys to hosts with hosts specs in the applied spec
2) ... - 04:04 PM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- Oh, by all combinations, I mean I created DNS entries for all hosts, not just ceph02.
- 04:03 PM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- I add DNS entries for all combinations. So both ceph02.oldname.local and ceph02.domain.local are now valid names but...
- 10:45 AM Bug #57781 (Rejected): Fix prometheus dependencies calculation
- closing as the current behavior is correct. We just need to add some comments to clarify the logic.
- 10:21 AM Bug #57366 (Pending Backport): prometheus is not re-deployed when service-discovery port changes
- 10:20 AM Bug #57816 (Fix Under Review): Add support to configure protocol (http or https) for Grafana url ...
10/13/2022
- 04:08 PM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- it's odd that the hostname it reports not having an address for isn't even a hostname it has stored "ceph02.domain.lo...
10/11/2022
- 07:55 AM Backport #55991: pacific: Allow setting crush_device_class in OSD service specs
- I just tried to use the feature "crush_device_class" as it's supposed to be available, but it fails (I try to create ...
- 02:16 AM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- So, I did notice that I had set the domain name on one of the nodes to the "oldname.local" (when I was doing the find...
- 01:19 AM Bug #57800: ceph orch upgrade does not appear to work with FQNDs.
- what does `ceph orch host ls` report for this host? This error should only be raised if we can't find any IP stored f...
- 01:24 AM Documentation #57596: MON Service
- the "networks" parameter inside the service spec is a separate thing from the public/cluster network. The public_netw...
10/10/2022
- 02:25 PM Bug #57816 (Pending Backport): Add support to configure protocol (http or https) for Grafana url ...
- Right now cephadm always deploy Grafana by using https. In some testing scenarios it would be helpful to configure th...
10/09/2022
- 01:20 AM Bug #57800 (New): ceph orch upgrade does not appear to work with FQNDs.
- This is purely speculative on my part, but after attempting an upgrade to 17.2.4 from 17.2.3, it just sits there doin...
10/07/2022
- 04:46 PM Backport #57793 (New): quincy: Update monitoring doc to reflect the new location of grafana key/cert
- 04:32 PM Documentation #57769 (Pending Backport): Update monitoring doc to reflect the new location of gra...
10/06/2022
- 07:49 PM Bug #57303: rados/cephadm: Failed to fetch package version from https://shaman.ceph.com/api/searc...
- Laura Flores wrote:
> /a/yuriw-2022-09-20_17:39:55-rados-wip-yuri5-testing-2022-09-19-1007-pacific-distro-default-sm... - 06:58 PM Backport #57788 (Resolved): pacific: mgr/nfs: Add a sectype field to nfs exports created by nfs m...
- https://github.com/ceph/ceph/pull/49929
- 06:58 PM Backport #57787 (Resolved): quincy: mgr/nfs: Add a sectype field to nfs exports created by nfs mg...
- https://github.com/ceph/ceph/pull/48531
- 06:51 PM Feature #57404 (Pending Backport): mgr/nfs: Add a sectype field to nfs exports created by nfs mgr...
- 06:35 PM Feature #57786 (Resolved): cephadm: open ports in firewall when deploying iscsi
- specifically, 3260 and whatever the user provides for the api_port. We already have logic in the deploy command in th...
- 06:30 PM Bug #57750: cephadm fails to upgrade systems not running sudo
- I can confirm that this was with the root user, no custom user involved.
Upgrades prior to 17.x.x worked like a ch... - 06:24 PM Bug #57750: cephadm fails to upgrade systems not running sudo
- assuming this wasn't with a custom (not-root) ssh user anyway, in which case sudo would be required as cephadm needs ...
- 06:22 PM Bug #57750 (In Progress): cephadm fails to upgrade systems not running sudo
- this is a legit bug. However, I think this should have been fixed by https://github.com/ceph/ceph/pull/47898 which ap...
- 04:10 PM Bug #57750: cephadm fails to upgrade systems not running sudo
- The documentation does not state that one need sudo at all. It's an option. So one cannot make the assumption that ev...
- 03:52 PM Bug #57750 (Need More Info): cephadm fails to upgrade systems not running sudo
- I'd say that the expected behavior. The user you use with cephadm needs passwordless sudo access to all the hosts tha...
- 11:49 AM Bug #57695 (Resolved): cephadm: upgrade tests fail with "Upgrade: Paused due to UPGRADE_BAD_TARGE...
- 10:57 AM Bug #57781 (Rejected): Fix prometheus dependencies calculation
- https://github.com/ceph/ceph/pull/46400 introduced a new http service discovery mechanism but we are still including ...
- 12:33 AM Backport #57772 (New): quincy: cephadm: watch Grafana certificates
- 12:22 AM Feature #44461 (Pending Backport): cephadm: watch Grafana certificates
10/05/2022
- 11:07 PM Bug #51361 (New): KillMode=none is deprecated
- I was wrong about @KillMode=none@ for my use case
- 08:54 PM Bug #57771 (Pending Backport): orch/cephadm suite: 'TESTDIR=/home/ubuntu/cephtest bash -s' fails
- It seems to be failing to install some selinux package...
- 02:20 PM Documentation #57769 (In Progress): Update monitoring doc to reflect the new location of grafana ...
- 02:20 PM Documentation #57769 (Pending Backport): Update monitoring doc to reflect the new location of gra...
- As part of the PR https://github.com/ceph/ceph/pull/47098 grafane key/cert now are stored per node but doc has not be...
- 01:58 PM Bug #57173 (Resolved): cephadm: bootstrap should return non-zero exit code when applying spec fails
- 01:57 PM Backport #57379 (Resolved): pacific: cephadm: bootstrap should return non-zero exit code when app...
10/03/2022
- 08:33 PM Bug #57755 (New): task/test_orch_cli: test_cephfs_mirror times out
- /a/yuriw-2022-09-29_16:44:24-rados-wip-lflores-testing-distro-default-smithi/7048282...
- 08:31 PM Bug #54029: orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_orch_cli} test f...
- @Matan yes, let's create a new Tracker. I found the same issue on a test branch which includes the patch for this Tra...
- 12:21 PM Bug #57750 (In Progress): cephadm fails to upgrade systems not running sudo
- cephadm fails to upgrade systems not running sudo.
It appears to have started with this commit:
https://github...
09/30/2022
- 06:32 PM Bug #51361 (Fix Under Review): KillMode=none is deprecated
- I filed a "fix for this":https://github.com/ceph/ceph/pull/48317 because it was preventing me from setting up cluster...
- 01:55 AM Bug #54029: orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_orch_cli} test f...
- yuriw-2022-09-27_23:37:28-rados-wip-yuri2-testing-2022-09-27-1455-distro-default-smithi/7046362
09/29/2022
- 07:32 PM Bug #57311: rook: ensure CRDs are installed first
- /a/yuriw-2022-09-27_23:37:28-rados-wip-yuri2-testing-2022-09-27-1455-distro-default-smithi/7046305
- 02:41 PM Bug #57311: rook: ensure CRDs are installed first
- /a/yuriw-2022-09-27_23:37:28-rados-wip-yuri2-testing-2022-09-27-1455-distro-default-smithi/7046147
- 07:16 PM Bug #52321: qa/tasks/rook times out: 'check osd count' reached maximum tries (90) after waiting f...
- yuriw-2022-09-27_23:37:28-rados-wip-yuri2-testing-2022-09-27-1455-distro-default-smithi/7046227
- 06:59 PM Bug #49287: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope n...
- yuriw-2022-09-27_23:37:28-rados-wip-yuri2-testing-2022-09-27-1455-distro-default-smithi/7046226
- 01:25 PM Feature #57725 (New): cephadm: add default dest to build.py
- Was thinking that a file named "cephadm" in the directory where buld.py lives (typically ceph/src/cephadm) would make...
09/28/2022
- 01:05 PM Bug #57303: rados/cephadm: Failed to fetch package version from https://shaman.ceph.com/api/searc...
- Laura Flores wrote:
> /a/yuriw-2022-09-20_17:39:55-rados-wip-yuri5-testing-2022-09-19-1007-pacific-distro-default-sm... - 01:02 PM Bug #57695 (Resolved): cephadm: upgrade tests fail with "Upgrade: Paused due to UPGRADE_BAD_TARGE...
- it looks like this is because we now officially have v18 in main and the test still starts from Octopus (v15.2.0 spec...
- 11:08 AM Feature #51618 (Rejected): rgw_frontends configuration in config database not persistent
- 11:08 AM Feature #51618: rgw_frontends configuration in config database not persistent
- The issue sounds to be solved by the proposal, the proposed feature must be opened a separate tracker.
- 10:59 AM Bug #54235 (Closed): Filtered out host ceph03: does not belong to mon public_network
- Closing as this issue is a duplicate for an already solved BUG. Feel free to reopen if you think it's not.
- 10:57 AM Bug #54235 (Duplicate): Filtered out host ceph03: does not belong to mon public_network
- 10:32 AM Bug #49287: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope n...
- I've already provided a detailed explanation of the root cause of this issue. Please consider upgrading the needed sw...
- 10:28 AM Bug #54142 (Resolved): quincy cephadm-purge-cluster needs work
- I'm not able to reproduce these issues with the code on the main branch anymore. Please, feel free to re-open if you ...
09/27/2022
- 05:36 PM Bug #57303 (New): rados/cephadm: Failed to fetch package version from https://shaman.ceph.com/api...
- /a/yuriw-2022-09-20_17:39:55-rados-wip-yuri5-testing-2022-09-19-1007-pacific-distro-default-smithi/7039204
@Adam h... - 05:27 PM Bug #49287: podman: setting cgroup config for procHooks process caused: Unit libpod-$hash.scope n...
- /a/yuriw-2022-09-20_17:39:55-rados-wip-yuri5-testing-2022-09-19-1007-pacific-distro-default-smithi/7039249
- 05:22 PM Bug #57689 (New): cephadm/smoke-roleless: RuntimeError: dictionary changed size during iteration
- /a/yuriw-2022-09-20_17:39:55-rados-wip-yuri5-testing-2022-09-19-1007-pacific-distro-default-smithi/7039167...
- 05:10 PM Bug #57255: rados/cephadm/mds_upgrade_sequence, pacific : cephadm [ERR] Upgrade: Paused due to UP...
- /a/yuriw-2022-09-20_17:39:55-rados-wip-yuri5-testing-2022-09-19-1007-pacific-distro-default-smithi/7039447
- 04:58 PM Bug #57268: rook: The CustomResourceDefinition "installations.operator.tigera.io" is invalid
- /a/yuriw-2022-09-20_17:39:55-rados-wip-yuri5-testing-2022-09-19-1007-pacific-distro-default-smithi/7039330
- 04:10 PM Bug #56951: rook/smoke: Updating cephclusters/rook-ceph is forbidden
- /a/yuriw-2022-09-23_20:38:59-rados-wip-yuri6-testing-2022-09-23-1008-quincy-distro-default-smithi/7042736
- 04:08 PM Bug #57311: rook: ensure CRDs are installed first
- /a/yuriw-2022-09-23_20:38:59-rados-wip-yuri6-testing-2022-09-23-1008-quincy-distro-default-smithi/7042656
- 03:06 PM Bug #57311: rook: ensure CRDs are installed first
- /a/yuriw-2022-09-21_21:00:57-rados-wip-yuri3-testing-2022-09-21-0921-distro-default-smithi/7040424
09/26/2022
- 10:29 PM Bug #57651 (Fix Under Review): cephadm: serve loop just loops forever if migration_current is too...
- 06:33 PM Bug #57675 (Resolved): mgr/cephadm: upgrades with 3 or more mgr daemons can get stuck in endless ...
- When daemons are upgraded by cephadm, there are two criteria taken into
account for a daemon to be considered totall... - 02:53 AM Backport #57638 (In Progress): pacific: applying osd service spec with size filter fails if there...
- 02:52 AM Backport #57637 (In Progress): quincy: applying osd service spec with size filter fails if there'...
Also available in: Atom