Activity
From 12/27/2020 to 01/25/2021
01/25/2021
- 11:53 PM Bug #48993: cephadm: 'mgr stat' and/or 'pg dump' output truncated
- The reproducer on that other bug doesn't work for 20.04.. different bug. However, I can reproduce it on ubuntu 18.04...
- 09:32 PM Bug #48993: cephadm: 'mgr stat' and/or 'pg dump' output truncated
- Current theory: common denominator is the ubuntu 18.04 version of podman, which is currently:...
- 09:29 PM Bug #48993 (Resolved): cephadm: 'mgr stat' and/or 'pg dump' output truncated
- Two symptoms:
- cephadm bootstrap 'ceph mgr dump' output is truncated (always at 180224 bytes, oddly)
exampl... - 06:14 PM Bug #48981: cephadm exporter: manager errors out with assertion error
- relates to https://github.com/ceph/ceph/pull/39061
- 04:55 PM Bug #48981 (Resolved): cephadm exporter: manager errors out with assertion error
- ...
- 04:59 PM Bug #48982 (Resolved): cephadm: ubuntu_18_04: Error: error creating container storage: the contai...
- mon dameon stops and dies: ...
- 04:45 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- http://qa-proxy.ceph.com/teuthology/ideepika-2021-01-25_12:19:15-rados-wip-deepika-testing-2021-01-25-1527-distro-bas...
- 04:29 PM Feature #48980: orch: add image properties to monitoring spec files
- ...
- 04:28 PM Feature #48980 (Closed): orch: add image properties to monitoring spec files
- add image properties to monitoring spec files
*There is a downside*: If you use this property, users will loose th... - 04:28 PM Feature #48979 (New): bin/cephadm: add possibilty to query default monitoring images to cephadm
- possibily add this to --help????
- 04:24 PM Feature #48978 (New): cephadm: show default container images in ceph orch status
- cephadm: show default container images in ceph orch status
- 11:07 AM Bug #48068: cephadm: Various properties like 'last_refresh' do not contain timezone
- Backport to Octopus: https://github.com/ceph/ceph/pull/39059
- 10:51 AM Documentation #48974 (Rejected): document repo_digest
- > cephadm converts the image to the sha256 digest, in order to keep the cluster in a consistent state.
>
> Like, ...
01/22/2021
- 11:15 AM Bug #48870: cephadm: Several services in error status after upgrade to 15.2.8: unrecognized argum...
- I did some analysis of the node-exporter on the osd node 3 to see what might be happening.
To me it looks as if th...
01/21/2021
- 05:53 PM Feature #47139 (Fix Under Review): Require a minimum version for podman/docker
- 03:50 PM Bug #48951: teuthology only collected coredumps, not daemon logs
- This is due to the daemons not starting, these coredumps are all from podman:...
- 12:28 PM Bug #48951 (Duplicate): teuthology only collected coredumps, not daemon logs
- ...
- 11:00 AM Bug #48594 (Pending Backport): cephadm: too many osd privileges for osd caps
- 10:58 AM Bug #48947 (Pending Backport): cephadm: fix rgw osd cap tag
- 10:55 AM Bug #48947 (Resolved): cephadm: fix rgw osd cap tag
- The syntax is "allow rwx tag rgw ='.
- 10:50 AM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- @Sebastian seeing this issue still in octopus:
/ceph/teuthology-archive/yuriw-2021-01-18_19:17:40-rados-wip-yuri2-te...
01/20/2021
- 04:34 PM Bug #48939: Orchestrator removes mon daemon from wrong host when removing host from cluster
- Quick update to confirm this behavior. I have been able to reproduce this on my personal homelab ceph cluster, also r...
- 04:21 PM Bug #48939 (Can't reproduce): Orchestrator removes mon daemon from wrong host when removing host ...
- It is as shocking as the subject describes.
To summarize:
Removing host mon1 nukes mon.mon3
Removing host mon3... - 09:41 AM Bug #48925 (Fix Under Review): cephadm: iscsi missing mgr permissions
- 06:54 AM Bug #48933 (Can't reproduce): cephadm: EOFError: couldnt load message header, expected 9 bytes, g...
- Found several uncatched exceptions, if I remember correctly, one host was rebooting or updating at that time....
- 12:10 AM Bug #48930 (Resolved): when removing the iscsi service, the gateway config object remains
- When the first rbd-target-api daemon starts it creates a gateway.conf object. When the iscsi service is removed via "...
01/19/2021
- 10:50 PM Bug #45628 (Fix Under Review): cephadm qa: smoke should verify daemons are actually running
- 07:34 PM Bug #46429: cephadm fails bootstrap with new Podman Versions 2.0.1 and 2.0.2
- Current cephadm avoids combining --cap-add and --privileged, but older cephadm does not, and some distros still have ...
- 05:28 PM Bug #48826: cephadm: does not tolerate 15.2.4 upgrade state
- if you happen to have a mgr failover from .8 to .5, users might also hit this bug.
- 03:37 PM Bug #48826 (Won't Fix): cephadm: does not tolerate 15.2.4 upgrade state
- I first tried to upgraded to .8, hit a different error (the ceph-volume --filter-batch thing), then switched to .5 in...
- 05:15 PM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- https://github.com/ceph/ceph-sepia-secrets/pull/595...
- 04:18 PM Support #48630: non-LVM OSD do not start after upgrade from 15.2.4 -> 15.2.7
- I think you probably want to migrate to ceph-volume for now.
- 04:12 PM Bug #47968 (Pending Backport): rook: 'ceph orch rm' throws type error
- 04:01 PM Feature #45766: cephadm: Removal: make sure, enough daemons joined the maps
- this is related to the scheduler. not a simple addition to ok-to-stop
- 04:00 PM Feature #44875 (New): mgr/rook: PlacementSpec to K8s POD scheduling conversion
- was github pr 35542
- 03:55 PM Bug #45973: Adopted MDS daemons are removed by the orchestrator because they're orphans
- still open
- 03:53 PM Documentation #45564 (Duplicate): cephadm: document workaround for accessing the admin socket by ...
- 03:52 PM Bug #48510: CEPHADM_REFRESH_FAILED: detail item 0 not a [unicode] string
- https://github.com/ceph/ceph/pull/38935/files#r560281346
- 03:48 PM Bug #48694 (Fix Under Review): ceph-volume: unrecognized arguments: --filter-for-batch
- 02:27 PM Bug #48925 (Resolved): cephadm: iscsi missing mgr permissions
- Error after deploying iscsi daemons:...
- 09:27 AM Bug #48924 (Resolved): cephadm: upgrade process failed to pull target image: not enough values to...
- When trying to upgrade a cluster from version 15.2.7 to 15.2.8 the process fails after seeemingly switching over the ...
01/18/2021
- 09:19 PM Bug #48916: "File system None does not exist in the map" in upgrade:octopus-x:parallel-master
- Perhaps, you need to add...
- 09:02 PM Bug #48916: "File system None does not exist in the map" in upgrade:octopus-x:parallel-master
- This is testing `blogbench`(https://github.com/yuriw/ceph/tree/wip-yuriw-octopus-x-master/qa/suites/upgrade/octopus-x...
- 08:57 PM Bug #48916 (Duplicate): "File system None does not exist in the map" in upgrade:octopus-x:paralle...
- Run: https://pulpito.ceph.com/teuthology-2021-01-16_16:12:35-upgrade:octopus-x:parallel-master-distro-basic-smithi/
... - 05:19 PM Bug #48598: "ceph orch daemon redeploy" fails with [errno 13] RADOS permission denied
- bumping up priority on this one
- 04:31 PM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- https://pulpito.ceph.com/swagner-2021-01-18_11:41:36-rados:cephadm-wip-swagner-testing-2021-01-15-1448-distro-basic-s...
- 04:30 PM Bug #48894: cephadm e2e: ceph device monitoring off: Error EINVAL
- https://pulpito.ceph.com/swagner-2021-01-18_11:41:36-rados:cephadm-wip-swagner-testing-2021-01-15-1448-distro-basic-s...
- 01:13 PM Bug #48870: cephadm: Several services in error status after upgrade to 15.2.8: unrecognized argum...
- It seems that Ubuntu/Podman is not the cause for this issue because I am running into the same errors after updating ...
- 07:11 AM Bug #48870: cephadm: Several services in error status after upgrade to 15.2.8: unrecognized argum...
- I restarted all hosts but that didn't solve the problem unfortunately:...
- 11:52 AM Bug #48891: cephadm does not set required dependencies in systemd
- Sebastian Wagner wrote:
> Thank you very much for reporting this!
Oh. I'm sorry i've reported this as i did not f... - 11:13 AM Bug #44990: cephadm: exec: "/usr/bin/ceph-mon": stat /usr/bin/ceph-mon: no such file or directory
- This is no longer critical, as it only affects octopus at this point.
01/16/2021
- 08:47 PM Bug #45279: cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (21) Is a dir...
- I am still seeing the same problem:...
- 11:05 AM Bug #48826: cephadm: does not tolerate 15.2.4 upgrade state
- If you're in an intermediate upgrade step and one MGR is already upgraded and the other one is still running with the...
- 03:24 AM Bug #48826: cephadm: does not tolerate 15.2.4 upgrade state
- We're not trying to support downgrade at all, right?
- 02:32 AM Bug #48826: cephadm: does not tolerate 15.2.4 upgrade state
- What was the job config?
- 01:06 AM Bug #48891 (Duplicate): cephadm does not set required dependencies in systemd
- Thank you very much for reporting this!
- 01:02 AM Bug #44990: cephadm: exec: "/usr/bin/ceph-mon": stat /usr/bin/ceph-mon: no such file or directory
- it has nothing to do with the image. The issue went away with https://github.com/ceph/ceph/pull/35524 indicating some...
- 12:54 AM Bug #44990: cephadm: exec: "/usr/bin/ceph-mon": stat /usr/bin/ceph-mon: no such file or directory
- I've been staring at that last failure for a few days, and I can't figure out what could possibly have caused it. Th...
01/15/2021
- 05:14 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- ...
- 03:59 PM Bug #48594: cephadm: too many osd privileges for osd caps
- https://github.com/ceph/ceph/pull/38910 also needs backporting
- 03:57 PM Bug #48120 (Resolved): JSONDecodeErrors unclear during cephadm upgrade
- 03:56 PM Bug #48119 (Resolved): JSONDecodeErrors break OSD deploy
- 03:56 PM Bug #48118 (Resolved): JSONDecodeErrors break the server loop
- 03:56 PM Bug #48205 (Resolved): cephadm: last local ceph image tag can be null
- 03:56 PM Bug #48223 (Resolved): host key might not exist during host refresh
- 03:56 PM Bug #47684 (Resolved): cephadm: auth get failed: failed to find osd.27 in keyring retval: -2
- 03:55 PM Documentation #47651 (Resolved): mgr/cephadm: document custom configuration templates
- 02:56 PM Bug #45628: cephadm qa: smoke should verify daemons are actually running
- latest output looks like ...
- 11:53 AM Bug #48870: cephadm: Several services in error status after upgrade to 15.2.8: unrecognized argum...
- Ok, I see two distinct issues here:
1. --filter-for-batch which is addressed in https://github.com/ceph/ceph/pull... - 10:54 AM Bug #48870 (In Progress): cephadm: Several services in error status after upgrade to 15.2.8: unre...
- 10:54 AM Bug #48870: cephadm: Several services in error status after upgrade to 15.2.8: unrecognized argum...
- possible solution: https://github.com/ceph/ceph/pull/38926
- 11:20 AM Bug #48894 (Can't reproduce): cephadm e2e: ceph device monitoring off: Error EINVAL
- https://pulpito.ceph.com/swagner-2021-01-15_09:42:49-rados:cephadm-wip-swagner-testing-2021-01-14-1551-distro-basic-s...
- 11:13 AM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- unfortunately, it's still there: https://pulpito.ceph.com/swagner-2021-01-15_09:42:49-rados:cephadm-wip-swagner-testi...
- 05:55 AM Bug #48891 (Duplicate): cephadm does not set required dependencies in systemd
- cephadm does not set the dependency list correctly. This leads to race conditions. Example:
In our environment we ...
01/14/2021
- 10:10 PM Feature #48846 (Fix Under Review): cephadm bootstrap: add --cluster-network
- 12:16 PM Bug #48870: cephadm: Several services in error status after upgrade to 15.2.8: unrecognized argum...
- ok: everything was added in https://github.com/ceph/ceph/pull/37520
* the --filter-for-batch implementation in cep... - 06:34 AM Bug #48870 (Resolved): cephadm: Several services in error status after upgrade to 15.2.8: unrecog...
- I initiated the upgrade to the latest version of Ceph from 15.2.5 (Ubuntu 20.04.1, Podman 2.1.1). At first everything...
01/13/2021
- 08:28 PM Feature #48846: cephadm bootstrap: add --cluster-network
- please not that a major part of this issue is updating the documentation and existing error message to guide users.
- 01:28 AM Feature #48846: cephadm bootstrap: add --cluster-network
- I'm happy to work on this (it's only fair, since I raised it initially on the orch call!)
- 03:09 PM Documentation #46546 (Resolved): doc/cephadm: "For each file system" is redundant in cephadm adop...
- This issue no longer presents in the documentation. It is likely that I fixed it in the interim, and did not realize ...
- 12:27 AM Fix #48664 (Closed): attempting to add a host that doesn't have py3 install results in the wrong ...
- 12:27 AM Feature #47805 (Closed): orchestrator: add the ability to place a host into and out of maintenance
01/12/2021
- 12:34 PM Bug #48826: cephadm: does not tolerate 15.2.4 upgrade state
- looks like a downgrade to me.
https://github.com/ceph/ceph/commit/36dc59987136f921c24932b2e0a2530ce3a22152 was me... - 08:53 AM Feature #48846 (Closed): cephadm bootstrap: add --cluster-network
- ...
- 08:47 AM Cleanup #44676: cephadm: Replace execnet (and remoto)
- One main problem we have is the quality of error messages. They're extraordinary bad.
01/11/2021
- 08:51 PM Bug #48157 (Pending Backport): test_cephadm.sh failure You have reached your pull rate limit. You...
- 07:16 PM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- /ceph/teuthology-archive/yuriw-2021-01-08_16:38:07-rados-wip-yuri4-testing-2021-01-07-1041-octopus-distro-basic-smith...
- 03:37 PM Bug #48826 (Won't Fix): cephadm: does not tolerate 15.2.4 upgrade state
- ...
- 03:36 PM Bug #48597: pybind/mgr/cephadm: mds_join_fs not cleaned up
- first attempt at https://github.com/ceph/ceph/pull/38844
- 11:47 AM Feature #48822 (Closed): Add proper port management to mgr/cephadm
- When thinking about co-locating daemons, we also have to think about the ports used by daemons:
* Automatically ch... - 11:38 AM Feature #45938 (Closed): "ceph orch daemon add osd" lacks an ability to specify DB/WAL devices
- please use `ceph orch apply ...` instead for this.
- 11:35 AM Tasks #46551 (New): cephadm: Add better a better hint how to add a host
- 11:08 AM Bug #48068 (Pending Backport): cephadm: Various properties like 'last_refresh' do not contain tim...
- 09:56 AM Bug #48068 (Resolved): cephadm: Various properties like 'last_refresh' do not contain timezone
- 09:55 AM Bug #48068 (Pending Backport): cephadm: Various properties like 'last_refresh' do not contain tim...
- 10:48 AM Bug #48779: orchestrator provides no ceph-[mon,mgr,osd,mds,...].target equivalent
- https://github.com/ceph/ceph/compare/master...jan--f:cephadm-add-typed-targets#diff-d0f7acffbce59b9e36a1479d1b1f32955...
01/08/2021
- 07:38 PM Documentation #48806 (New): documentation trobleshooting: cephadm; how do i start all containers?
- cephadm; how do i start all containers?
- 04:31 PM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- https://github.com/ceph/ceph-cm-ansible/pull/602
- 04:19 PM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- Ah. Okay. So this is because you're running podman on Bionic. ceph-cm-ansible isn't expecting this.
See
https:... - 02:00 PM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- Hm. I can still the the error:
https://pulpito.ceph.com/swagner-2021-01-07_14:03:38-rados:cephadm-wip-swagner-test... - 01:53 PM Bug #48799 (Can't reproduce): test_cephadm: stderr Job for container.alertmanager.a.service faile...
- https://pulpito.ceph.com/swagner-2021-01-07_14:03:38-rados:cephadm-wip-swagner-testing-2021-01-07-1213-distro-basic-s...
01/07/2021
- 06:33 PM Bug #48788: cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (21) Is a dir...
- Tried cephadm bootstrap again after switching to normal docker package:...
- 06:31 PM Bug #48788: cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (21) Is a dir...
- For kicks I tried this on a different VM which is not using snap and it works:...
- 06:22 PM Bug #48788: cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (21) Is a dir...
- I modified cephadm so I can poke around and see during this run that the following file is created:...
- 06:03 PM Bug #48788: cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (21) Is a dir...
- ...
- 06:01 PM Bug #48788 (Duplicate): cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (...
- I am on a fresh Ubuntu 20.04 LTS install and unable to use cephadm bootstrap to install ceph.
This is related to [... - 05:24 PM Bug #45279: cephadm bootstrap: monmaptool --create: error writing to '/tmp/monmap': (21) Is a dir...
- I am on a fresh Ubuntu 20.04 LTS install and also getting the same error when trying to bootstrap....
- 01:55 PM Tasks #48780: cephadm qa: Add teuthology test for non-trivial drive groups to qa/rados/suites/cep...
- The rados/cephadm/dashboard suite partially achieve this.
https://github.com/ceph/ceph/blob/master/qa/suites/rados/c... - 09:11 AM Tasks #48780 (New): cephadm qa: Add teuthology test for non-trivial drive groups to qa/rados/suit...
- So far teuthology cephadm task only deploys OSDs "manually" via `ceph orch daemon add osd" command so it would be nic...
- 09:05 AM Bug #48779: orchestrator provides no ceph-[mon,mgr,osd,mds,...].target equivalent
- If you want to stop all all OSDs, on a particular host, you should probably call ...
- 08:31 AM Bug #48779 (Won't Fix): orchestrator provides no ceph-[mon,mgr,osd,mds,...].target equivalent
- In the olden days systemd could stop all daemons of a type on a single node via the daemon type specific targets (e.g...
01/06/2021
- 04:14 PM Bug #48715 (Fix Under Review): docker-mirror: x509: certificate relies on legacy Common Name fiel...
- I believe this is resolved. I'm not sure why but just restarting the mirror seemed to take care of it. I know the p...
- 03:26 PM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- Sebastian Wagner wrote:
> https://pulpito.ceph.com/swagner-2021-01-06_08:53:53-rados:cephadm-wip-swagner-testing-202... - 03:19 PM Bug #48715: docker-mirror: x509: certificate relies on legacy Common Name field, use SANs or temp...
- https://pulpito.ceph.com/swagner-2021-01-06_08:53:53-rados:cephadm-wip-swagner-testing-2021-01-05-1550-distro-basic-s...
- 12:45 PM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- aah, silly of me, pardon. Thanks for fixing it!
- 12:04 PM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- Deepika Upadhyay wrote:
> hey Dan,
>
> still observing this issue:
> [...]
>
> /ceph/teuthology-archive/yuri... - 09:21 AM Bug #48157: test_cephadm.sh failure You have reached your pull rate limit. You may increase the l...
- hey Dan,
still observing this issue: ... - 09:11 AM Bug #48142: rados:cephadm/upgrade/mon_election tests are failing: CapAdd and privileged are mutua...
- upgrade test: ...
- 12:16 AM Bug #48758 (Rejected): "ImportError (No module named rados) ... ERROR" in upgrade:octopus-x-master
- 12:00 AM Bug #48758: "ImportError (No module named rados) ... ERROR" in upgrade:octopus-x-master
- This is test was running on wrong workunit from `nuatilus`
Re-testing it https://pulpito.ceph.com/teuthology-2021-...
01/05/2021
- 11:27 PM Bug #48758 (Rejected): "ImportError (No module named rados) ... ERROR" in upgrade:octopus-x-master
- This is a try to run rbd test_rbd_python tests for cephadm octopus-x => master upgrade.
Run: https://pulpito.ceph.... - 07:52 PM Bug #48754: "failed xx 'sudo systemctl start ceph-None@rgw.client.1'" in upgrade:octopus-x-master
- self.fsid in
https://github.com/ceph/teuthology/blob/8e49dda43b7cf60665a020d72cd31ec6d06ae4c5/teuthology/orchestr... - 07:51 PM Bug #48754: "failed xx 'sudo systemctl start ceph-None@rgw.client.1'" in upgrade:octopus-x-master
- ...
- 06:44 PM Bug #48754 (Resolved): "failed xx 'sudo systemctl start ceph-None@rgw.client.1'" in upgrade:octop...
- This is a try to run rgw_ragweed_prepare tests for cephadm octopus-x => master upgrade.
qa/tasks/rgw.py doesn't use ... - 12:52 AM Bug #48261: cephadm ceph-volume inventory -- --format json-pretty: INFO:cephadm:/usr/bin/podman:...
- Not entirely sure how to reproduce properly but I hope this might help.
I encountered the same issue with the follow...
01/04/2021
Also available in: Atom