Activity
From 03/27/2024 to 04/25/2024
Today
- 01:33 AM CephFS Bug #65603 (Fix Under Review): mds: quiesce timeout due to a freezing directory
- 01:10 AM CephFS Bug #65647: Evicted kernel client may get stuck after reconnect
- Mykola Golub wrote:
> Our customer were observing sporadic "client isn't responding to mclientcaps(revoke)" issue so... - 01:00 AM CephFS Bug #65647: Evicted kernel client may get stuck after reconnect
- Xiubo Li wrote in #note-1:
> I think you have enabled *recover_session* in kclient ?
>
> [...]
>
> More detail... - 12:48 AM CephFS Bug #65647: Evicted kernel client may get stuck after reconnect
- I think you have enabled *recover_session* in kclient ?...
- 12:38 AM CephFS Bug #65660: mds: drop client metrics during recovery
- Is this new in the upstream master ? As I remembered we have improved this and the clients will only send the metrics...
- 12:34 AM CephFS Bug #65660 (In Progress): mds: drop client metrics during recovery
- When the rank is coming up, there's little reason to record historical metrics from the clients. We've also seen floo...
04/24/2024
- 11:16 PM Bug #65659 (New): OSD Resize Increases Used Capacity Not Available Capacity
- h1. Deviation from expected behavior
After resizing the underlying disk at the hypervisor and OS level *resizing t... - 07:49 PM rbd Bug #46875: TestLibRBD.TestPendingAio: test_librbd.cc:4539: Failure or SIGSEGV
- from https://jenkins.ceph.com/job/ceph-pull-requests/133893/consoleFull...
- 07:41 PM CephFS Bug #65658 (Fix Under Review): mds: MetricAggregator::ms_can_fast_dispatch2 acquires locks
- 07:33 PM CephFS Bug #65658 (Fix Under Review): mds: MetricAggregator::ms_can_fast_dispatch2 acquires locks
- There was a lot of discussion surrounding this in
https://github.com/ceph/ceph/pull/26004/
but circling back we... - 06:21 PM Orchestrator Bug #65657 (New): doc: lack of clarity for explicit placement analogue in yaml spec
- https://docs.ceph.com/en/latest/cephadm/services/#explicit-placements
Specifically, I'm wondering if "host:[ip]=na... - 05:45 PM CephFS Tasks #65615 (Resolved): lchown corrupts symlink entry
- The code was using parent dir ent fscrypt info/key. Using an incorrect key to decrypt, will yield incorrect plaintext...
- 04:58 PM rgw Bug #65656: Thread pool size benchmarking
- Test env:
---------
3x MON/MGR nodes
Dell R630
2x E5-2683 v3 (28 total cores, 56 threads)
128 GB RAM
8x... - 04:41 PM rgw Bug #65656 (New): Thread pool size benchmarking
- Our recent RGW thread pool size profiling (RHEL 9.2, Ceph 18.2.0-131) revealed that for both smaller (max 256KB) and ...
- 03:59 PM Orchestrator Backport #64844 (Resolved): reef: Regression: Permanent KeyError: 'TYPE' : return self.blkid_api['TYPE'] == 'part'
- 03:58 PM Orchestrator Bug #65035 (Duplicate): ERROR: required file missing from config-json: idmap.conf
- duplicate of https://tracker.ceph.com/issues/65155
- 03:54 PM Orchestrator Bug #64118 (Resolved): cephadm: RuntimeError: Failed command: apt-get update: E: The repository 'https://download.ceph.com/debian-quincy jammy Release' does not have a Release file.
- I think this should be fixed now that we have quincy jammy builds
- 03:34 PM Orchestrator Backport #65378 (Resolved): squid: cephadm: client-keyring also overwrites ceph.conf
- 03:15 PM rgw Bug #65654 (New): run-bucket-check.sh: failed assert len(json_out) == len(unlinked_keys)
- https://qa-proxy.ceph.com/teuthology/suriarte-2024-04-23_15:04:03-rgw-rgw-update-boost-redis-distro-default-smithi/76...
- 03:13 PM nvme-of Backport #65649 (In Progress): squid: Change some default values for OMAP lock parameters in nvmeof conf file
- 01:45 PM nvme-of Backport #65649 (In Progress): squid: Change some default values for OMAP lock parameters in nvmeof conf file
- https://github.com/ceph/ceph/pull/56497
- 02:56 PM nvme-of Backport #65650 (In Progress): reef: Change some default values for OMAP lock parameters in nvmeof conf file
- 01:46 PM nvme-of Backport #65650 (In Progress): reef: Change some default values for OMAP lock parameters in nvmeof conf file
- https://github.com/ceph/ceph/pull/56498
- 02:52 PM rbd Bug #65653 (New): run-rbd-unit-tests-0.sh: TestMigration.StressLive failure
- from https://jenkins.ceph.com/job/ceph-pull-requests/133815/consoleFull on a squid pr:...
- 02:52 PM Orchestrator Feature #65398: allow images from private repos in teuthology test/ceph orch/cephadm
- Sorry the last two weeks have been much busier than usual and this slipped my mind. I discussed this with Adam King a...
- 12:55 AM Orchestrator Feature #65398: allow images from private repos in teuthology test/ceph orch/cephadm
- Any thoughts on this, John? I have to install a cluster from a private repo tomorrow, and it reminded me we'd had th...
- 02:44 PM Bug #65652: vstart.sh can not start
- https://github.com/ceph/ceph/pull/57077
- 02:12 PM Bug #65652 (New): vstart.sh can not start
2024-04-24T21:26:01.158+0800 7f4ec09ffd40 -1 load dlopen(/home/ecs-assist-user/ceph/build/lib/libec_jerasure.so): /...- 02:40 PM rgw Bug #64841 (Triaged): java_s3tests: testObjectCreateBadExpectMismatch failure
- 02:32 PM rgw Bug #62136: "test pushing kafka s3 notification on master" - no events are sent
- this says resolved, but i still see failures like this on main:...
- 02:12 PM rgw Bug #65651 (New): s3select: test_true_false_in_expressions s3test failure
- from a rgw/sts job based on recent main
https://qa-proxy.ceph.com/teuthology/cbodley-2024-04-24_12:59:55-rgw-wip-cbo... - 01:40 PM nvme-of Feature #65566 (Pending Backport): Change some default values for OMAP lock parameters in nvmeof conf file
- 01:39 PM rgw Feature #18621 (Resolved): rgw: change default chunk size
- 12:28 PM rgw Bug #65648 (New): TestAMQP.MaxConnections FAILED ceph_assert(!conn->state)
- ...
- 11:56 AM Dashboard Bug #61312: The command "ceph config set mgr mgr/dashboard/redirect_resolve_ip_addr True" fails
- Nizamudeen tells me the following through Slack:
BEGIN QUOTED TEXT
this particular configuration is introduced ... - 11:54 AM Documentation #65631 (Resolved): clarify dual-stack mode
- 11:53 AM RADOS Backport #65646 (Fix Under Review): squid: osd/scrub: must disable reservation timeout for reserver-based requests
- 11:12 AM RADOS Backport #65646 (Fix Under Review): squid: osd/scrub: must disable reservation timeout for reserver-based requests
- 11:18 AM CephFS Bug #65647 (New): Evicted kernel client may get stuck after reconnect
- Our customer were observing sporadic "client isn't responding to mclientcaps(revoke)" issue so they configured auto e...
- 11:04 AM RADOS Bug #65044 (Pending Backport): osd/scrub: must disable reservation timeout for reserver-based requests
- 10:03 AM rgw Bug #65645 (New): lifecycle notifications are sent from radosgw-admin
- when "radosgw-admin lc process" is called, and there are buckets that have bucket notification events set with "Objec...
- 09:43 AM CephFS Backport #65644 (Fix Under Review): quincy: qa/cephfs: absence of e03331e causes test_nfs to fail
- @tasks.cephfs.test_nfs.TestNFS.test_non_existent_cluster@ failed on here - https://pulpito.ceph.com/vshankar-2024-03-...
- 09:33 AM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Ilya Dryomov wrote in #note-17:
> However, the "unable to connect to remote cluster" error isn't cleared and you cont... - 09:31 AM rgw Bug #64999: Slow RGW multisite sync due to "304 Not Modified" responses on primary zone
- Hi All,
I just wanted to quick follow-up on my previous query about "Slow RGW multisite sync
due to '304 Not Modi... - 08:36 AM Dashboard Bug #65643 (New): mgr/dashboard: dashboard landing page cant be seen as readonly
- As a read only user you should be able to view the landing page, but it is not possible
- 06:54 AM Dashboard Cleanup #65070 (Resolved): mgr/dashboard: use alertmanager v2 APIs mgr/dashboard: short_description
- 06:54 AM Dashboard Backport #65255 (Resolved): squid: mgr/dashboard: use alertmanager v2 APIs mgr/dashboard: short_description
- 06:22 AM crimson Bug #65585: unittest-seastore (Timeout)
- If each test's execution time was correct, timeout is caused by "stuck in one of tests".
e.g. https://jenkins.ceph... - 02:07 AM CephFS Tasks #65613: truncate failing when using path
- Greg Farnum wrote in #note-2:
> Hmm, I'm surprised you found missing Server logic here. Shouldn't that have turned u...
04/23/2024
- 11:18 PM bluestore Bug #56262: crash: BlueStore::_txc_create(BlueStore::Collection*, BlueStore::OpSequencer*, std::list<Context*, std::allocator<Context*> >*, boost::intrusive_ptr<TrackedOp>)
- There seems to be some race condition at the time of OSD shutdown. The kv db handle was destroyed and one of OSD thre...
- 10:27 PM RADOS Bug #54515: mon/health-mute.sh: TEST_mute: return 1 (HEALTH WARN 3 mgr modules have failed dependencies)
- /a/lflores-2024-04-01_18:07:25-rados-wip-yuri8-testing-2024-03-25-1419-distro-default-smithi/7634102
- 06:38 PM Dashboard Bug #62972: ERROR: test_list_enabled_module (tasks.mgr.dashboard.test_mgr_module.MgrModuleTest)
- https://jenkins.ceph.com/job/ceph-api/72895/ on main
- 06:30 PM RADOS Backport #65376 (In Progress): quincy: crash: void PaxosService::propose_pending(): assert(have_pending)
- 06:29 PM RADOS Backport #65377 (In Progress): reef: crash: void PaxosService::propose_pending(): assert(have_pending)
- 06:28 PM mgr Backport #65621 (In Progress): quincy: mgr: update cluster state for new maps from the mons before notifying modules
- 06:28 PM mgr Backport #65623 (In Progress): reef: mgr: update cluster state for new maps from the mons before notifying modules
- 06:27 PM mgr Backport #65622 (In Progress): squid: mgr: update cluster state for new maps from the mons before notifying modules
- 06:27 PM CephFS Backport #65620 (In Progress): squid: qa: test_max_items_per_obj open procs not fully cleaned up
- 06:26 PM CephFS Backport #65619 (In Progress): squid: mds: quiesce_counter decay rate initialized from wrong config
- 06:23 PM CephFS Backport #65273 (In Progress): squid: PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- 06:20 PM Bug #64095 (Resolved): ceph-exporter is not included in the deb packages
- 06:19 PM Bug #63637 (Resolved): debian packaging is missing bcrypt dependency for ceph-mgr's .requires file
- 06:18 PM Backport #63638 (Resolved): reef: debian packaging is missing bcrypt dependency for ceph-mgr's .requires file
- 06:12 PM Backport #63638: reef: debian packaging is missing bcrypt dependency for ceph-mgr's .requires file
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/54662
merged - 06:18 PM Backport #65172 (Resolved): reef: ceph-exporter is not included in the deb packages
- 06:13 PM Backport #65172: reef: ceph-exporter is not included in the deb packages
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56541
merged - 05:52 PM CephFS Bug #65603 (In Progress): mds: quiesce timeout due to a freezing directory
- 04:37 PM rgw Backport #65640 (In Progress): squid: [rgw][accounts] bucket quota management at account-level
- 04:35 PM rgw Backport #65640 (In Progress): squid: [rgw][accounts] bucket quota management at account-level
- https://github.com/ceph/ceph/pull/57058
- 04:35 PM rgw Feature #65551 (Pending Backport): [rgw][accounts] bucket quota management at account-level
- 04:23 PM bluestore Bug #65482 (Fix Under Review): bluestore/bluestore_types: check 'it' valid before using
- 04:22 PM rgw Backport #65002 (Resolved): quincy: [CVE-2023-46159] RGW crash upon misconfigured CORS rule
- 04:06 PM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Greg Farnum wrote in #note-33:
> Venky Shankar wrote in #note-30:
> > OK. I'll elaborate. Generally, clients are no... - 04:02 PM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Venky Shankar wrote in #note-32:
> Dhairya Parmar wrote in #note-28:
> > as mentioned in yesterday's standup - some... - 03:25 PM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Venky Shankar wrote in #note-30:
> OK. I'll elaborate. Generally, clients are not trustable - someone can hook up a ... - 03:59 PM RADOS Bug #61832: Restoring #61785: osd-scrub-dump.sh: ERROR: Extra scrubs after test completion...not expected
- /a/yuriw-2024-04-22_18:19:58-rados-wip-yuri2-testing-2024-04-17-0823-reef-distro-default-smithi/7668423
- 03:53 PM bluestore Bug #56788: crash: void KernelDevice::_aio_thread(): abort
- /a/yuriw-2024-04-22_18:19:58-rados-wip-yuri2-testing-2024-04-17-0823-reef-distro-default-smithi/7668449...
- 03:49 PM CephFS Tasks #65613: truncate failing when using path
- Hmm, I'm surprised you found missing Server logic here. Shouldn't that have turned up in kernel fscrypt testing? Xiub...
- 03:48 PM RADOS Bug #62992: Heartbeat crash in reset_timeout and clear_timeout
- /a/yuriw-2024-04-22_18:19:58-rados-wip-yuri2-testing-2024-04-17-0823-reef-distro-default-smithi/7668452
- 03:44 PM Orchestrator Bug #64208: test_cephadm.sh: Container version mismatch causes job to fail.
- /a/yuriw-2024-04-22_18:19:58-rados-wip-yuri2-testing-2024-04-17-0823-reef-distro-default-smithi/7668470
- 01:49 PM CephFS Feature #65637 (New): mds: continue sending heartbeats during recovery when MDS journal is large
- When the MDS reaches up:rejoin / up:resolve after spending a long time (hours) in up:replay, it often gets in an loop...
- 01:40 PM rgw Backport #65636 (In Progress): squid: release note for rgw_realm init
- 01:39 PM rgw Backport #65636 (In Progress): squid: release note for rgw_realm init
- https://github.com/ceph/ceph/pull/57055
- 01:39 PM rgw Bug #65575 (Pending Backport): release note for rgw_realm init
- 01:34 PM Dashboard Backport #65255 (In Progress): squid: mgr/dashboard: use alertmanager v2 APIs mgr/dashboard: short_description
- 01:19 PM Feature #63703: If a prefix is available, allow it be used to narrow the bounds of OMAP iterator
- Xiang Li wrote in #note-1:
> Is anyone trying out this new feature? Can I give it a try?
I don't think anyone has... - 02:12 AM Feature #63703: If a prefix is available, allow it be used to narrow the bounds of OMAP iterator
- Is anyone trying out this new feature? Can I give it a try?
- 01:05 PM crimson Bug #65635 (New): unittest-omap-manager (Failed) on AARCH64
- [ RUN ] omap_manager_test/omap_manager_test_t.force_leafnode_split_merge_fullandbalanced/0
INFO 2024-04-23 08:... - 11:58 AM Bug #65634 (New): rbd-mirror user does not have enough permissions to obtain (daemon) health status information
- We are testing rbd-mirroring. There seems to be a permission error with the rbd-mirror user. Using this user to query...
- 09:13 AM crimson Bug #65585: unittest-seastore (Timeout)
- https://jenkins.ceph.com/job/ceph-pull-requests-arm64/55512/console...
- 07:34 AM Messengers Bug #65401: msg: conneciton between mgr and osd is periodically down which leads heavy load to mgr
- the periodically connection fault can be found in log by following steps:
1. set ms_connection_idle_timeout=60; debu... - 06:41 AM Documentation #65631 (Fix Under Review): clarify dual-stack mode
- 04:42 AM Documentation #65631 (Resolved): clarify dual-stack mode
- Robert Sander asks whether Ceph supports dual-stack mode. Dual-stack mode is when both IPv4 and IPv6 networks are use...
- 06:21 AM crimson Bug #65632 (New): crimson osd crashes due to daggling pointers of operation blockers
- There are time gaps between the destruction of OSDMapBlockers and OSDMapBlockers unreferencing from BlockingEvents. I...
- 05:45 AM Bug #65629 (Fix Under Review): cephfs_mirror: display 'sync_bytes' in peer status
- 03:56 AM Bug #65629 (In Progress): cephfs_mirror: display 'sync_bytes' in peer status
- 03:55 AM Bug #65629 (Fix Under Review): cephfs_mirror: display 'sync_bytes' in peer status
- Display 'sync_bytes' for the 'last_synced_snap' in the 'peer status' command output. This is analogous with the perf ...
- 04:56 AM RADOS Feature #65583: mon store data should be available depending on the user keyring
- > My understanding is the idea is restrict the visibility of configurables' values.
Yes, that's right, but can you... - 04:32 AM Documentation #65609 (Resolved): Documentation of maximum port number is incorrect
- 04:10 AM CephFS Bug #65630 (In Progress): mds: rename request was deadlocked between two different MDSs
- This is reported by Nigel, more detail please see https://www.mail-archive.com/ceph-users@ceph.io/msg24587.html
In... - 03:04 AM crimson Bug #65628 (New): unittest-seastore (Timeout)
- There is a certain probability of timeout happened on both ARM and X86 CI.
e.g.
1. https://jenkins.ceph.com/job/cep...
04/22/2024
- 11:00 PM RADOS Bug #65235: upgrade/reef-x/stress-split: "OSDMAP_FLAGS: noscrub flag(s) set" warning in cluster log
- Unfortunately, noscrub and nodeep-scrub are not the only warnings we would need to mask for the thrashosds-health tes...
- 10:51 PM RADOS Cleanup #65521: Add expected warnings in cluster log to ignorelists
- More in this run: https://pulpito.ceph.com/lflores-2024-04-01_18:07:25-rados-wip-yuri8-testing-2024-03-25-1419-distro...
- 10:46 PM Orchestrator Bug #64374: Error ENOENT: module 'cephadm' reports that it cannot run on the active manager daemon: No module named 'mgr_module' (pass --force to force enablement)
- /a/lflores-2024-04-01_18:07:25-rados-wip-yuri8-testing-2024-03-25-1419-distro-default-smithi/7634080
- 09:33 PM mgr Bug #65627 (New): Centos 9 stream ceph container iscsi test failure
- h3. Missing k8sevents module
While waiting for the mgr to start, we get this traceback message:
teuthology.log
<... - 09:19 PM rgw Bug #65626: rgw: false assumption on vault bucket key deletion
- PR: https://github.com/ceph/ceph/pull/57046
- 09:16 PM rgw Bug #65626 (New): rgw: false assumption on vault bucket key deletion
- On bucket key deletion when the request to change the property of the key for deletion_allowed to true, it is expecte...
- 09:14 PM rgw Backport #65409: quincy: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56818
merged - 09:13 PM rgw Backport #65341: quincy: rgw: update options yaml file so LDAP uri isn't an invalid example
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56722
merged - 09:08 PM rgw Backport #63961: quincy: rgw: lack of headers in 304 response
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/55095
merged - 09:07 PM rgw Backport #63253: quincy: Add bucket versioning info to radosgw-admin bucket stats output
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/54190
merged - 08:37 PM rbd Bug #65487 (In Progress): rbd-mirror daemon in ERROR state, require manual restart
- Hi Nir,
Thanks for providing verbose logs. For now, I have all the information I need.
Due to rbd-mirror daemo... - 08:34 PM rgw Backport #65625 (In Progress): quincy: rgw/crypt/barbican: 'Namespace' object has no attribute 'admin_endpoints'
- 08:33 PM rgw Backport #65625 (In Progress): quincy: rgw/crypt/barbican: 'Namespace' object has no attribute 'admin_endpoints'
- https://github.com/ceph/ceph/pull/57045
- 08:26 PM rgw Bug #61772 (Pending Backport): rgw/crypt/barbican: 'Namespace' object has no attribute 'admin_endpoints'
- 08:12 PM rbd Bug #65624 (New): [pybind] CLONE_FORMAT and some other image options are not exposed
- C/C++ API:...
- 07:53 PM rgw Backport #64766 (Resolved): reef: SSL session id reuse speedup mechanism of the SSL_CTX_set_session_id_context is not working
- 07:50 PM rgw Bug #62063 (New): notification tests fail on 'radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph'
- happening on quincy: https://qa-proxy.ceph.com/teuthology/yuriw-2024-04-20_15:31:09-rgw-wip-yuri4-testing-2024-04-19-...
- 07:30 PM RADOS Bug #65517: rados/thrash-erasure-code-crush-4-nodes: ceph task fails at getting monitors
- Bump up./
- 07:18 PM RADOS Bug #56393: failed to complete snap trimming before timeout
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648606 was on fbfd55d0098...
- 07:12 PM mgr Backport #65623 (In Progress): reef: mgr: update cluster state for new maps from the mons before notifying modules
- https://github.com/ceph/ceph/pull/57065
- 07:12 PM mgr Backport #65622 (In Progress): squid: mgr: update cluster state for new maps from the mons before notifying modules
- https://github.com/ceph/ceph/pull/57064
- 07:12 PM mgr Backport #65621 (In Progress): quincy: mgr: update cluster state for new maps from the mons before notifying modules
- https://github.com/ceph/ceph/pull/57066
- 07:11 PM CephFS Backport #65620 (In Progress): squid: qa: test_max_items_per_obj open procs not fully cleaned up
- https://github.com/ceph/ceph/pull/57063
- 07:11 PM CephFS Backport #65619 (In Progress): squid: mds: quiesce_counter decay rate initialized from wrong config
- https://github.com/ceph/ceph/pull/57062
- 07:07 PM mgr Bug #64799 (Pending Backport): mgr: update cluster state for new maps from the mons before notifying modules
- I'll sit on the backports for a while.
- 07:06 PM CephFS Bug #65022 (Pending Backport): qa: test_max_items_per_obj open procs not fully cleaned up
- 07:04 PM CephFS Bug #65342 (Pending Backport): mds: quiesce_counter decay rate initialized from wrong config
- 06:47 PM CephFS Bug #50821: qa: untar_snap_rm failure during mds thrashing
- ...
- 06:42 PM CephFS Bug #65618 (New): qa: fsstress: cannot execute binary file: Exec format error
- ...
- 06:40 PM RADOS Bug #53768 (Closed): timed out waiting for admin_socket to appear after osd.2 restart in thrasher/defaults workload/small-objects
- 06:39 PM CephFS Fix #65617 (Fix Under Review): qa: increase debugging for snap_schedule
- 06:36 PM CephFS Fix #65617 (Fix Under Review): qa: increase debugging for snap_schedule
- 06:39 PM rgw Bug #65567: admin_socket_output: signal: Terminated from term radosgw
- note from tracker scrub: looks like a duplicate of https://tracker.ceph.com/issues/59380.
- 06:31 PM rgw Bug #65567 (Duplicate): admin_socket_output: signal: Terminated from term radosgw
- 06:33 PM CephFS Bug #65616 (New): pybind/mgr/snap_schedule: 1m scheduled snaps not reliably executed
- Check timestamps:...
- 06:28 PM RADOS Bug #61774: centos 9 testing reveals rocksdb "Leak_StillReachable" memory leak in mons
- Update: Still working to understand why my local reproducer worked with the latest fix but not in teuthology.
- 06:23 PM RADOS Bug #62839 (Closed): Teuthology failure in LibRadosTwoPoolsPP.HitSetWrite
- Cache tiering is deprecated, sorry.
- 06:19 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- In QA. Pinged.
- 06:17 PM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- Still in QA.
- 06:16 PM RADOS Bug #44510: osd/osd-recovery-space.sh TEST_recovery_test_simple failure
- Hi Nitzan, would you mind taking a look?
- 06:14 PM CephFS Tasks #65615 (Resolved): lchown corrupts symlink entry
- lchown corrupts symlink entry:...
- 06:12 PM RADOS Bug #65449: NeoRadosWatchNotify.WatchNotifyTimeout failed due to nonexistent pool
- In review.
- 06:09 PM RADOS Bug #64519: OSD/MON: No snapshot metadata keys trimming
- note from scrub: bump up.
- 06:07 PM RADOS Feature #65583: mon store data should be available depending on the user keyring
- This sounds like a feature request, not a bug.
My understanding is the idea is restrict the visibility of configurab... - 05:56 PM CephFS Bug #65614 (Fix Under Review): client: resends request to same MDS it just received a forward from if it does not have an open session with the target
- 05:46 PM CephFS Bug #65614 (Fix Under Review): client: resends request to same MDS it just received a forward from if it does not have an open session with the target
- ...
- 05:53 PM RADOS Documentation #16258: ceph audit logs are not logging to ceph.audit.log if we specify "mon cluster log file" option
- If something stays in tracker, without huge attention, for 8+ years, it's probably not a high prio...
- 08:04 AM RADOS Documentation #16258: ceph audit logs are not logging to ceph.audit.log if we specify "mon cluster log file" option
- No idea if this is still applicable. Unassigning from me because it hasn't been touched for almost a decade, and I'll...
- 05:47 PM RADOS Bug #53240: full-object read crc is mismatch, because truncate modify oi.size and forget to clear data_digest
- New changes in the PR (a unit test fix). Need to reQA.
- 05:39 PM RADOS Bug #65371: rados: PeeringState::calc_replicated_acting_stretch populate acting set before checking if < bucket_max
- In review.
- 05:38 PM RADOS Bug #52657: MOSDPGLog::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_NAUTILUS)'
- Yuri provided an update. Still in QA.
- 05:32 PM rgw Backport #64496 (Resolved): squid: keystone admin token is not invalidated on http 401 response
- 05:32 PM rgw Backport #65353 (Resolved): squid: rgwlc: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault
- 05:32 PM rgw Backport #64552 (Resolved): squid: rgw/multisite: objects named "." or ".." are not replicated
- 04:55 PM CephFS Bug #65603: mds: quiesce timeout due to a freezing directory
- Another one: https://pulpito.ceph.com/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/766829...
- 04:41 PM CephFS Bug #65603: mds: quiesce timeout due to a freezing directory
- Another case: https://pulpito.ceph.com/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-default-smithi/76682...
- 02:44 PM CephFS Bug #65603: mds: quiesce timeout due to a freezing directory
- ...
- 02:43 PM CephFS Bug #65603: mds: quiesce timeout due to a freezing directory
- Another instance of the same at https://pulpito.ceph.com/leonidus-2024-04-22_12:36:42-fs-wip-lusov-quiescer-distro-de...
- 04:54 PM CephFS Tasks #64133: Make pjd work on fscrypt
- Make pjd tests pass that are failing:...
- 04:50 PM CephFS Tasks #65613 (Resolved): truncate failing when using path
- The fix:...
- 04:44 PM CephFS Tasks #65613 (Resolved): truncate failing when using path
- Reproducer:...
- 04:19 PM Bug #65612 (New): qa: logrotate fails when state file is already locked
- ...
- 04:16 PM rgw Bug #65160: rgw/lc: A few buckets stuck in UNINITIAL state
- Can this be backported to Squid?
- 03:23 PM RADOS Bug #49158 (Resolved): doc: ceph-monstore-tools might create wrong monitor store
- 03:21 PM Documentation #57125 (Resolved): Improve wording of /doc/rados/*
- 03:21 PM Documentation #57108 (Resolved): add ".. prompt :: bash $" to /doc/rados
- 03:15 PM Bug #64446 (Resolved): Backport PR#55540 to Squid (and only Squid) when its commits are merged to main
- 03:14 PM Documentation #65161 (Resolved): Update Zabbix Documentation
- 03:12 PM Documentation #65599 (Resolved): "ceph osd crush rename bucket" command missing
- 03:06 PM Bug #65249 (Resolved): peering_graph.generated.dot renders weird
- I used these instructions to build an SVG file of the peering graph:
$ git clone https://github.com/ceph/ceph.git
... - 11:14 AM rbd Backport #65550 (In Progress): squid: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- 11:14 AM rbd Backport #65549 (In Progress): reef: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- 11:12 AM rbd Backport #65547 (In Progress): quincy: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- 10:45 AM Bug #65611 (New): Segmentation fault in upkeep_main
- ...
- 10:27 AM CephFS Bug #65606: workload fails due to slow ops, assert in logs mds/Locker.cc: 551 FAILED ceph_assert(!lock->is_waiter_for(SimpleLock::WAIT_WR) || lock->is_waiter_for(SimpleLock::WAIT_XLOCK))
- Another instance of this issue: https://pulpito.ceph.com/leonidus-2024-04-21_11:37:13-fs-wip-lusov-quiescer-distro-de...
- 09:42 AM CephFS Bug #65606: workload fails due to slow ops, assert in logs mds/Locker.cc: 551 FAILED ceph_assert(!lock->is_waiter_for(SimpleLock::WAIT_WR) || lock->is_waiter_for(SimpleLock::WAIT_XLOCK))
- fixing the @request_drop_foreign_locks@ method uncovered another crash due to the same reason, this time when droppin...
- 10:08 AM crimson Bug #65610 (New): unittest-object-data-handler crashes testing object_data_handler_test_t.overwrite_then_read_within_transaction
- ...
- 09:28 AM crimson Bug #65491: recover_missing: racing read got wrong version
- > *Hypothesis 2:*
> See: 'Version bump'. Version was bumped to 12 and then both requests were requeued (requeueing c... - 09:23 AM Documentation #65609 (Resolved): Documentation of maximum port number is incorrect
- The highest port number used by OSD or MDS daemons was increased from 7300 to 7568 in https://github.com/ceph/ceph/pu...
- 08:43 AM crimson Bug #64206: obc->is_loaded_and_valid() assertion
- https://pulpito.ceph.com/matan-2024-04-21_15:36:23-crimson-rados-wip-matanb-crimson-testing-snap-overlap-distro-crims...
- 08:06 AM RADOS Cleanup #10506 (Rejected): mon: get rid of QuorumServices
- I hope this might have been addressed at some point. If not, it probably no longer makes sense to mess with the monit...
- 08:03 AM RADOS Bug #42519: During deployment of the ceph,when the main node starts slower than the other nodes.It may lead to generate a core by assert.
- No idea if this is still applicable. Unassigning from me because it hasn't been touched for 4 years now, and I'll lik...
- 06:20 AM CephFS Bug #50260: pacific: qa: "rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty"
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648870
The *mnt.0* was... - 06:13 AM CephFS Bug #64707: suites/fsstress.sh hangs on one client - test times out
- Laura Flores wrote in #note-16:
> /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-defa... - 06:05 AM Bug #65608 (New): Mirroring mode of rbd image changes when migrated between pools
- When an rbd image is mirrored and migrated between pools (rbd migration) the mirroring mode changes from "snapshot" (...
- 05:37 AM rgw Bug #64999: Slow RGW multisite sync due to "304 Not Modified" responses on primary zone
Hi Shilpa,
We are eagerly waiting for your direction to resolve it.
I appreciate your attention to this matter....- 05:12 AM CephFS Bug #65607: mds deadlock between 'lookup' and the 'rename/create, etc' requests
- This possibly caused by the lock order issue as in https://tracker.ceph.com/issues/62123.
- 05:09 AM CephFS Bug #65607 (Need More Info): mds deadlock between 'lookup' and the 'rename/create, etc' requests
- Have suggested Erich to make *max_mds = 1* to reproduce it to get rid of the noises.
- 04:51 AM CephFS Bug #65607: mds deadlock between 'lookup' and the 'rename/create, etc' requests
- As Erich mentioned he enabled multiple active MDSs, but he only updated the block ops from on MDS. I guess maybe anot...
- 04:33 AM CephFS Bug #65607 (Need More Info): mds deadlock between 'lookup' and the 'rename/create, etc' requests
- This is reported by Eric, more detail please see https://www.mail-archive.com/ceph-users@ceph.io/msg24587.html
The... - 04:12 AM Dashboard Bug #65571 (Resolved): mgr/dashboard: run-tox-mgr-dashboard-py3 failure in make check
- 04:12 AM Dashboard Backport #65581 (Resolved): squid: mgr/dashboard: run-tox-mgr-dashboard-py3 failure in make check
04/21/2024
- 07:27 PM CephFS Bug #65606: workload fails due to slow ops, assert in logs mds/Locker.cc: 551 FAILED ceph_assert(!lock->is_waiter_for(SimpleLock::WAIT_WR) || lock->is_waiter_for(SimpleLock::WAIT_XLOCK))
- The incorrect behavior of the method that stripped the local quiesce lock from the request resulted in the crash when...
- 07:17 PM CephFS Bug #65606 (Fix Under Review): workload fails due to slow ops, assert in logs mds/Locker.cc: 551 FAILED ceph_assert(!lock->is_waiter_for(SimpleLock::WAIT_WR) || lock->is_waiter_for(SimpleLock::WAIT_XLOCK))
- 06:57 PM CephFS Bug #65606: workload fails due to slow ops, assert in logs mds/Locker.cc: 551 FAILED ceph_assert(!lock->is_waiter_for(SimpleLock::WAIT_WR) || lock->is_waiter_for(SimpleLock::WAIT_XLOCK))
- We had a successful quiesce on the mds.0 followed by the said export dir request. The export dir request has failed t...
- 06:30 PM CephFS Bug #65606 (Fix Under Review): workload fails due to slow ops, assert in logs mds/Locker.cc: 551 FAILED ceph_assert(!lock->is_waiter_for(SimpleLock::WAIT_WR) || lock->is_waiter_for(SimpleLock::WAIT_XLOCK))
https://pulpito.ceph.com/leonidus-2024-04-21_11:37:13-fs-wip-lusov-quiescer-distro-default-smithi/7666598/
The f...- 06:08 PM CephFS Bug #65605 (Duplicate): fsx.sh workload fails with status 2 due to a makefile error
- Duplicate of https://tracker.ceph.com/issues/64572
- 06:06 PM CephFS Bug #65605: fsx.sh workload fails with status 2 due to a makefile error
- another instance of the same failure https://pulpito.ceph.com/leonidus-2024-04-21_11:37:13-fs-wip-lusov-quiescer-dist...
- 06:05 PM CephFS Bug #65605 (Duplicate): fsx.sh workload fails with status 2 due to a makefile error
- https://pulpito.ceph.com/leonidus-2024-04-21_11:37:13-fs-wip-lusov-quiescer-distro-default-smithi/7666610/...
- 05:46 PM CephFS Bug #65604 (New): dbench.sh workload times out after 3h when run with-quiescer
- https://pulpito.ceph.com/leonidus-2024-04-21_11:37:13-fs-wip-lusov-quiescer-distro-default-smithi/7666604/
No quie... - 05:14 PM CephFS Bug #65603: mds: quiesce timeout due to a freezing directory
- https://pulpito.ceph.com/leonidus-2024-04-21_11:37:13-fs-wip-lusov-quiescer-distro-default-smithi/7666602/...
- 04:24 PM CephFS Bug #65603: mds: quiesce timeout due to a freezing directory
- The directory appears to be fragmenting, as we see from a few messages in the log...
- 04:09 PM CephFS Bug #65603 (Fix Under Review): mds: quiesce timeout due to a freezing directory
- Analyzing one of the ETIMEDOUT error for a quiesce, looking at
https://pulpito.ceph.com/leonidus-2024-04-21_11:37:13... - 01:21 PM crimson Bug #65532 (Fix Under Review): osd crashes due to invalid clone_range ops
- 01:17 PM crimson Bug #64782 (Resolved): test_python.sh TestIoctx.test_locator failes in cases of SeaStore
- 01:12 PM crimson Bug #65531 (In Progress): crimson-osd: dump_historic_slow_ops command not correctly run
- 01:00 PM crimson Support #65602 (New): Support RBD mirror testing
- See: qa/suites/rbd/mirror-thrash and qa/suites/rbd/mirror
- 12:57 PM crimson Bug #65601 (New): rados_python.yaml enable tests
- Currently some of rados_python tests are disabled:...
- 12:27 PM bluestore Fix #65600 (New): bluefs alloc unit should only be shrink
- The alloc unit has already forbidden changed for bluestore, what's more, it should forbidden increased in bluefs. Oth...
- 09:43 AM crimson Bug #65474 (Resolved): mgr crash due to corrupted incremental osdmap sent by crimson-osds
- 09:43 AM crimson Bug #65200 (Resolved): PeeringState::get_peer_info(pg_shard_t) const: Assertion `it != peer_info.end()' failed.
- 09:42 AM crimson Bug #59242 (Resolved): [crimson] Pool compression does not take effect
- 09:25 AM CephFS Backport #65556 (Fix Under Review): squid: mds: avoid recalling Fb when quiescing file
- 09:21 AM CephFS Backport #65556 (In Progress): squid: mds: avoid recalling Fb when quiescing file
- 09:08 AM crimson Bug #63647: SnapTrimEvent AddressSanitizer: heap-use-after-free
- https://pulpito.ceph.com/matan-2024-04-21_07:41:30-crimson-rados-wip-matanb-crimson-only-testing-april-17-distro-crim...
- 08:12 AM rgw Feature #53662: rgw: radosgw-admin can list and remove bucket notification topics; it must also be able to create them
- agree we should close.
* topic creation by an admin will mess up the topic ownership logic
* we can create notifica... - 07:58 AM Documentation #65599: "ceph osd crush rename bucket" command missing
- Eugen Block, as usual, to the rescue:
https://lists.ceph.io/hyperkitty/list/ceph-users@ceph.io/message/IQUPWQZ5ZIQ... - 07:52 AM Documentation #65599 (Resolved): "ceph osd crush rename bucket" command missing
- https://docs.ceph.com/en/latest/rados/operations/crush-map/
The "ceph osd crush rename bucket" command is not list... - 12:13 AM Bug #65598 (New): github v18.2.2 tag removed
- I have some automation that looks for git tags on github that broke recently because the v18.2.2 tag was removed from...
04/20/2024
- 03:07 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- I reproduced the issue again with debug logs.
Tested flow:
- Configure rbd mirroring on both clusters
- Wait for...
04/19/2024
- 11:30 PM CephFS Bug #65595 (Fix Under Review): mds: missing policylock acquisition for quiesce
- 11:28 PM CephFS Bug #65595 (Fix Under Review): mds: missing policylock acquisition for quiesce
- In order to check an inode's F_QUIESCE_BLOCK, the quiesce_inode op must acquire the policylock. Furthermore, to ensur...
- 11:20 PM RADOS Bug #51729: Upmap verification fails for multi-level crush rule
- Update on this bug:
We are pretty close to getting the fix out for this. Thanks all for waiting so long. In additi... - 07:21 PM RADOS Backport #65593: squid: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- https://github.com/ceph/ceph/pull/57006
- 06:55 PM RADOS Backport #65593 (New): squid: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616025/remote/smithi098/log/b1f19696-e81a-11ee...
- 06:51 PM RADOS Bug #65185 (Fix Under Review): OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- 05:57 PM rgw Feature #20094 (Resolved): RFW: make civetweb max request size configurable to allow larger s3 object metadata
- 05:56 PM rgw Feature #19917 (Closed): radosgw access log is lacking useful information
- 05:56 PM rgw Feature #19510 (Resolved): per-object storage class
- 05:55 PM rgw Feature #20398 (Resolved): rgw: Swift TempURL does not support prefix-based scope
- 05:54 PM rgw Feature #20733 (Closed): RGW bucket limits
- 05:53 PM rgw Feature #20650 (Resolved): Support webhook for authentication
- 05:52 PM rgw Feature #20795 (Resolved): rgw: the TempURL implementation should support ISO8601 in temp_url_expires
- 05:52 PM rgw Feature #20883 (Resolved): rgw: responses for HEAD/GET on Swift's container should contain Last-Modified
- 05:51 PM rgw Feature #21334 (Resolved): support log response header “x-amz-request-id ”
- 05:50 PM rgw Feature #21799 (Rejected): multisite: sync parts of multipart uploads
- 05:49 PM rgw Feature #22565 (Resolved): Multiple Data Pool Support for a Bucket
- this is supported through storage classes: https://docs.ceph.com/en/latest/radosgw/placement/
- 05:48 PM rgw Feature #24335 (Resolved): Get the user metadata of the user used to sign the request
- 05:46 PM rgw Feature #24493 (Resolved): rgw does not implement list_object_v2 in S3
- 05:43 PM rgw Feature #24507 (Resolved): [rfe] rgw: relaxed region constraint enforcement
- 05:41 PM rgw Feature #39084 (Resolved): ability to control user op mask via admin apis
- 05:40 PM rgw Feature #40241 (Rejected): radosgw: ldap groups
- 05:39 PM rgw Feature #40242 (Rejected): radosgw-admin: export & import buckets
- 05:37 PM rgw Feature #40392 (Rejected): radosgw-admin: create bucket
- 05:35 PM rgw Feature #40714 (Closed): usage log differ from civetweb and beast
- 05:35 PM rgw Feature #41062 (Resolved): Extend SSE-KMS in Rados Gateway to support HashiCorp Vault
- 05:35 PM rgw Feature #41222 (Rejected): multisite: delay sync data to non-master zone
- 05:34 PM rgw Feature #42513 (Resolved): rgw: radosgw-admin command line parsing cleanup and improvements
- 05:34 PM rgw Feature #42627 (Resolved): rgw: bucket granularity sync: bucket dependency index
- 05:33 PM rgw Feature #42626 (Resolved): rgw: bucket granularity sync: core sync changes
- 05:33 PM rgw Feature #42625 (Resolved): rgw: bucket granularity sync: sync policy
- 05:33 PM rgw Feature #42272 (Resolved): rgw set cpu affinity at startup
- 05:33 PM rgw Feature #42493 (Rejected): Simplify Login Radosgw-admin API
- ceph provides a shell script in https://github.com/ceph/ceph/blob/main/examples/rgw/rgw_admin_curl.sh that adds sigv2...
- 05:31 PM rgw Feature #45444 (Resolved): Add bucket name to bucket stats error logging
- 05:30 PM rgw Feature #45568 (Resolved): Swift Extract Archive Operation
- 05:30 PM rgw Feature #45748 (Closed): recommended max number of buckets....
- we don't intend there to be any scaling limit to the number of total buckets in the system. there are limitations on ...
- 05:27 PM rgw Feature #46028 (Resolved): RGW User Policy
- 05:25 PM rgw Feature #48402 (Resolved): multisite option to enable keepalive
- 05:25 PM rgw Feature #48513 (Rejected): uses librgw2 to directly access the rados cluster for hadoop
- 05:24 PM rgw Feature #48798 (Resolved): RGW:Multisite: Verify if the synced object is identical to source
- 05:24 PM rgw Feature #49227 (Resolved): rgw: register daemon in service map with more details
- 05:22 PM rgw Feature #50262 (Duplicate): rgw header size limit should configurable
- 05:20 PM rgw Feature #53546 (Resolved): rgw/beast: add max_header_size option with 16k default, up from 4k
- 05:09 PM rgw Feature #55016 (Resolved): radosgw-admin should allow setting user policy
- 05:07 PM rgw Bug #23264 (In Progress): Server side encryption support for s3 COPY operation
- 05:07 PM rgw Feature #55481 (Resolved): The latest version of server encryption does not support "aes256" as kms encryption method
- 05:04 PM rgw Feature #55640 (Rejected): make lua scripting optional
- the attached pull request closed a year ago
i personally don't see much benefit to disabling lua at compile time. ... - 04:56 PM rgw Feature #53662 (Need More Info): rgw: radosgw-admin can list and remove bucket notification topics; it must also be able to create them
- trying to scrub some old feature requests. is there still interest in this?
in general, i don't think radosgw-admi... - 04:36 PM rgw Feature #59593 (Closed): The capability of resetting an empty bucket to the clean-slate state in multi-site environment
- 03:46 PM rgw Feature #63930 (Duplicate): s3: implement GetObjectAttributes
- 03:06 PM rgw Feature #64190 (Resolved): support lifecycle NewerNoncurrentVersions in NoncurrentVersionExpiration
- already backported to squid with https://github.com/ceph/ceph/pull/56144
- 02:50 PM RADOS Bug #65591 (New): Pool MAX_AVAIL goes UP when an OSD is marked down+in
- Example:
* Cluster with 4 OSD nodes, 10 OSDs each
* 3x replicated pool
* `max_avail` from `ceph df detail --format... - 01:32 PM rgw Bug #65590 (New): rgw_multi.tests.test_topic_notification_sync: PutBucketNotificationConfiguration fails with ConcurrentModification
- ...
- 12:09 PM Support #65589 (New): is there any method to restore deleted rbd images
- Hi, there
We're running a very old ceph rbd cluster. Today a team deleted a bunch of (about 1.5k images and 12TiB ... - 10:39 AM rbd Backport #65588 (New): quincy: insufficient randomness for group and group snapshot IDs
- 10:38 AM rbd Backport #65587 (New): squid: insufficient randomness for group and group snapshot IDs
- 10:38 AM rbd Backport #65586 (New): reef: insufficient randomness for group and group snapshot IDs
- 10:34 AM rbd Bug #65573 (Pending Backport): insufficient randomness for group and group snapshot IDs
- 09:39 AM Bug #65176: BlueFS: _estimate_log_size_N calculates the log size incorrectly
- What is calculated here should be the total bytes occupied by the names of all files.@ Igor Fedotov
- 09:29 AM crimson Bug #65531: crimson-osd: dump_historic_slow_ops command not correctly run
- https://github.com/ceph/ceph/pull/56994
- 07:50 AM crimson Bug #65585: unittest-seastore (Timeout)
- https://github.com/ceph/ceph/pull/56979...
- 07:47 AM crimson Bug #65585: unittest-seastore (Timeout)
- https://github.com/ceph/ceph/pull/56982...
- 07:35 AM crimson Bug #65585: unittest-seastore (Timeout)
- The pasted log is from https://github.com/ceph/ceph/pull/56998#issuecomment-2065880693
- 07:33 AM crimson Bug #65585 (New): unittest-seastore (Timeout)
- ...
- 07:18 AM ceph-volume Bug #65584 (New): ceph-volume: use os.makedirs to implement mkdir_p
- ceph-volume failed if /var/lib/ceph/osd/ does not exist...
- 06:47 AM RADOS Feature #65583 (New): mon store data should be available depending on the user keyring
- For the specific ceph user data should be restricted on the mon store.
Let's say if client.user1 store data `clien... - 06:00 AM Backport #65582 (New): squid: qa/vstart_runner: increase timeout for sake of "Ceph API tests" CI job
- 05:41 AM Bug #65565 (Pending Backport): qa/vstart_runner: increase timeout for sake of "Ceph API tests" CI job
- 05:40 AM Dashboard Backport #65581 (In Progress): squid: mgr/dashboard: run-tox-mgr-dashboard-py3 failure in make check
- 05:29 AM Dashboard Backport #65581 (Resolved): squid: mgr/dashboard: run-tox-mgr-dashboard-py3 failure in make check
- https://github.com/ceph/ceph/pull/56999
- 05:23 AM Dashboard Bug #65571 (Pending Backport): mgr/dashboard: run-tox-mgr-dashboard-py3 failure in make check
- 05:17 AM CephFS Bug #65580 (New): mds/client: add dummy client feature to test client eviction
- Currently, fs:upgrade:featureful_client:old_client uses octopus client with a newer MDS. The octopus client lacks a p...
- 03:49 AM CephFS Fix #65579 (New): mds: use _exit for QA killpoints rather than SIGABRT
- Using signals to abruptly kill the MDS has a few issues:
- teuthology logs are polluted with stacktraces
- coredu... - 03:37 AM cephsqlite Bug #65494 (Fix Under Review): ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- 02:48 AM devops Backport #65578 (In Progress): reef: ccache is always miss in confusa14
- 02:38 AM devops Backport #65578 (In Progress): reef: ccache is always miss in confusa14
- https://github.com/ceph/ceph/pull/56993
- 02:47 AM devops Backport #65577 (In Progress): squid: ccache is always miss in confusa14
- 02:38 AM devops Backport #65577 (In Progress): squid: ccache is always miss in confusa14
- https://github.com/ceph/ceph/pull/56992
- 02:47 AM devops Backport #65576 (In Progress): quincy: ccache is always miss in confusa14
- 02:38 AM devops Backport #65576 (In Progress): quincy: ccache is always miss in confusa14
- https://github.com/ceph/ceph/pull/56991
- 02:30 AM devops Bug #65175 (Pending Backport): ccache is always miss in confusa14
- 12:45 AM Bug #65249: peering_graph.generated.dot renders weird
- size="7,7" in peering_graph_generated.dot causes the peering_graph_generated.svg file to look the (wrong) way that ca...
- 12:37 AM Bug #65249: peering_graph.generated.dot renders weird
- dot -Tsvg doc/dev/peering_graph.generated.dot > doc/dev/peering_graph.generated.svg
The above command as of today ... - 12:34 AM CephFS Bug #65563: WARNING: CPU: 7 PID: 40807 at mm/page_alloc.c:4545 __alloc_pages+0x1e7/0x270
- I have fix the kernel call trace in kernel space, the patch like is https://patchwork.kernel.org/project/ceph-devel/l...
04/18/2024
- 11:05 PM rbd Bug #54292: run-rbd-unit-tests-127.sh times out on Jenkins "make check" runs
- sorry to pile on, but it's hard to know which tracker issue is related to which crash. from squid pr https://jenkins....
- 10:12 PM rgw Bug #65575 (Pending Backport): release note for rgw_realm init
- 09:25 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- ...
- 08:19 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- https://github.com/rzarzynski/ceph/commit/1a4d3f01816cedb15106fe2cdb52322029482827 changed ScrubMap::object::attrs to...
- 09:24 PM rgw Bug #64841: java_s3tests: testObjectCreateBadExpectMismatch failure
- i tried running the python reproducer from https://tracker.ceph.com/issues/58286, but it doesn't reproduce the @bad m...
- 09:08 PM rgw Bug #64841: java_s3tests: testObjectCreateBadExpectMismatch failure
- thanks Ali, that's super helpful. i came across https://tracker.ceph.com/issues/58286 which looks like the exact same...
- 08:55 PM rgw Bug #64841: java_s3tests: testObjectCreateBadExpectMismatch failure
- Here is a snippet with two of those "bad method" statements from the log I referenced in the last comment.
https:/... - 08:33 PM rgw Bug #64841: java_s3tests: testObjectCreateBadExpectMismatch failure
- After having radosgw under valgrind and running the java s3tests I was able to reproduce the "failed to read header: ...
- 05:21 PM rbd Bug #65573 (Fix Under Review): insufficient randomness for group and group snapshot IDs
- 05:12 PM rbd Bug #65573 (Pending Backport): insufficient randomness for group and group snapshot IDs
- Nithya noticed that group IDs end up being very similar:...
- 05:13 PM Dashboard Feature #56429: mgr/dashboard: Remote user authentication (e.g. via apache2)
- If SSO should be the primary login method, and the local login is only needed for emergencies (Network/IdP down), the...
- 05:07 PM Dashboard Feature #56429: mgr/dashboard: Remote user authentication (e.g. via apache2)
- Hello Ernesto,
This interface seems to imply that a username and password is entered on a login page and passed to... - 05:06 PM rbd Backport #65548 (Duplicate): reef: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- The bot created two reef backport tickets for some reason.
- 04:18 PM rgw Feature #65551 (Fix Under Review): [rgw][accounts] bucket quota management at account-level
- 04:00 PM RADOS Bug #64519: OSD/MON: No snapshot metadata keys trimming
- Eugen Block wrote in #note-10:
> Thanks, Matan! It sounds very promising. I talked to the customer and they are will... - 03:34 PM RADOS Backport #65306: squid: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56814
merged - 03:33 PM RADOS Backport #65312: squid: decoding chunk_refs_by_hash_t return wrong values
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56697
merged - 03:33 PM RADOS Backport #65072: squid: rados/thrash: slow reservation response from 1 (115547ms) in cluster log
- https://github.com/ceph/ceph/pull/56482 merged
- 03:31 PM RADOS Backport #65140: squid: osd: modify PG deletion cost for mClock scheduler
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56474
merged - 03:31 PM mgr Backport #65117: squid: rados/upgrade/parallel: [WRN] TELEMETRY_CHANGED: Telemetry requires re-opt-in
- Laura Flores wrote:
> https://github.com/ceph/ceph/pull/56457
merged - 03:30 PM RADOS Backport #65097: squid: ceph osd pool rmsnap clone object leak
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56432
merged - 03:03 PM CephFS Backport #65295: squid: High cephfs MDS latency and CPU load with snapshots and unlink operations
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56671
merged - 03:03 PM CephFS Backport #65106: squid: qa: probabilistically ignore PG_AVAILABILITY/PG_DEGRADED
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56665
merged - 03:02 PM CephFS Backport #65275: squid: mds: some request errors come from errno.h rather than fs_types.h
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56663
merged - 02:21 PM CephFS Bug #65563: WARNING: CPU: 7 PID: 40807 at mm/page_alloc.c:4545 __alloc_pages+0x1e7/0x270
- Venky, IMO this also should be the same issue with this:
https://pulpito.ceph.com/vshankar-2024-03-13_13:59:32-fs... - 10:19 AM CephFS Bug #65563 (Fix Under Review): WARNING: CPU: 7 PID: 40807 at mm/page_alloc.c:4545 __alloc_pages+0x1e7/0x270
- 07:45 AM CephFS Bug #65563 (In Progress): WARNING: CPU: 7 PID: 40807 at mm/page_alloc.c:4545 __alloc_pages+0x1e7/0x270
- The mds sent out the open session reply with *cap_auths [MDSCapAuth( uid=1000 gids=1301readable=1, writeable=1),MDSCa...
- 07:26 AM CephFS Bug #65563 (Fix Under Review): WARNING: CPU: 7 PID: 40807 at mm/page_alloc.c:4545 __alloc_pages+0x1e7/0x270
- https://pulpito.ceph.com/yuriw-2024-04-05_22:36:11-fs-wip-yuri7-testing-2024-04-04-0800-distro-default-smithi/7642062...
- 02:21 PM rgw Bug #64971 (New): Rgw lifecycle skip
- 02:20 PM rgw Bug #64983 (Fix Under Review): multisite: two-zonegroup tests get stuck in redirect loops
- 02:17 PM rgw Bug #65216 (In Progress): rgw: only accept valid ipv4 from host header
- 02:16 PM rgw Bug #65369 (Fix Under Review): rgw: allow disabling bucket stats on head bucket
- 02:16 PM rgw Bug #65397 (Fix Under Review): rgw: allow disabling mdsearch APIs
- 02:15 PM rgw Bug #65436 (Need More Info): Getting Object Crashing radosgw services
- > After upgrade to 17.2.7, this bug gone
it sounds like this bug is fixed in later point release, can you please t... - 02:10 PM rgw Bug #65462 (Fix Under Review): rgw: differentiate ssl enforcement based on encryption mode
- 02:09 PM rgw Bug #65468 (Fix Under Review): rgw: set correct requestId and hostId on s3select error
- 02:03 PM rgw Bug #65337 (Fix Under Review): rgw: Segmentation fault in rgw::notify::Manager during realm reload
- 01:48 PM CephFS Bug #65572 (New): Command failed (workunit test fs/snaps/untar_snap_rm.sh) on smithi155 with status 1
- This has started to show up again (with fs/thrash). See: https://pulpito.ceph.com/yuriw-2024-04-05_22:36:11-fs-wip-yu...
- 01:28 PM CephFS Backport #65570 (Fix Under Review): squid: Quiesce may fail randomly with EBADF due to the same root submitted to the MDCache multiple times under the same quiesce request
- 12:40 PM CephFS Backport #65570 (Fix Under Review): squid: Quiesce may fail randomly with EBADF due to the same root submitted to the MDCache multiple times under the same quiesce request
- 01:22 PM Dashboard Bug #65571 (Resolved): mgr/dashboard: run-tox-mgr-dashboard-py3 failure in make check
- ...
- 01:02 PM Dashboard Bug #62972: ERROR: test_list_enabled_module (tasks.mgr.dashboard.test_mgr_module.MgrModuleTest)
- https://jenkins.ceph.com/job/ceph-api/72585/
- 01:02 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Ilya Dryomov wrote in #note-14:
> log_to_file gets set to true by Rook as part of enabling the log collector:
>
>... - 07:23 AM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- log_to_file gets set to true by Rook as part of enabling the log collector:
https://github.com/rook/rook/blob/a9fd... - 12:58 PM RADOS Bug #65449 (Fix Under Review): NeoRadosWatchNotify.WatchNotifyTimeout failed due to nonexistent pool
- 12:09 PM RADOS Bug #65449: NeoRadosWatchNotify.WatchNotifyTimeout failed due to nonexistent pool
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659537
- 12:35 PM CephFS Bug #65545 (Pending Backport): Quiesce may fail randomly with EBADF due to the same root submitted to the MDCache multiple times under the same quiesce request
- 12:10 PM RADOS Bug #44510: osd/osd-recovery-space.sh TEST_recovery_test_simple failure
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659542
- 12:09 PM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659395
/a/yuriw-2024-04-... - 12:08 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659312
/a/yuriw-2024-04-... - 12:07 PM RADOS Cleanup #65521: Add expected warnings in cluster log to ignorelists
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659305
- 12:06 PM RADOS Bug #65183: Overriding an EC pool needs the "--yes-i-really-mean-it" flag in addition to "force"
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659300
/a/yuriw-2024-04-... - 12:06 PM Orchestrator Bug #52109: test_cephadm.sh: Timeout('Port 8443 not free on 127.0.0.1.',)
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659292
- 12:05 PM RADOS Bug #62839: Teuthology failure in LibRadosTwoPoolsPP.HitSetWrite
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659285
- 12:05 PM RADOS Bug #64437: qa/standalone/scrub/osd-scrub-repair.sh: TEST_repair_stats_ec: test 26 = 13
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659280
- 12:04 PM RADOS Bug #61774: centos 9 testing reveals rocksdb "Leak_StillReachable" memory leak in mons
- /a/yuriw-2024-04-16_23:25:35-rados-wip-yuriw-testing-20240416.150233-distro-default-smithi/7659275/
/a/yuriw-2024-04... - 11:19 AM CephFS Bug #64659: mds: switch to using xlists instead of elists
- @vshankar any thoughts on this?
- 11:00 AM Dashboard Bug #65569 (New): exporter: allow all zone names pattern for sync counters
- Currently exporter only supports zone name which have `-`'s in between for rgw sync metrics. Adopt the regex to also...
- 10:28 AM crimson Bug #65568 (New): osd crashes when trimming snaps involves unrecovered objects
- The current crimson implementation doesn't recover objects when trimming snaps. So, if we are trimming a snapshot, an...
- 09:58 AM Bug #65228 (Fix Under Review): class:device-class config database mask does not work for osd_compact_on_start
- 09:52 AM rgw Bug #65567 (Duplicate): admin_socket_output: signal: Terminated from term radosgw
- ...
- 09:49 AM Dashboard Bug #65506 (Resolved): rgw roles e2e tests failure
- 09:49 AM Dashboard Backport #65542 (Resolved): squid: rgw roles e2e tests failure
- 09:44 AM Bug #65565: qa/vstart_runner: increase timeout for sake of "Ceph API tests" CI job
- The commit has been cherry-picked to a different PR for a faster merge and to avoid circular dependency for CI to be ...
- 09:24 AM Bug #65565 (Pending Backport): qa/vstart_runner: increase timeout for sake of "Ceph API tests" CI job
- 09:37 AM nvme-of Feature #65566 (Pending Backport): Change some default values for OMAP lock parameters in nvmeof conf file
- We want to change some default values in the OMAP lock parameters in the nvmeof conf file generated by cephadm:
* ... - 09:35 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Dhairya Parmar wrote in #note-28:
> as mentioned in yesterday's standup - some of the PRs (https://github.com/ceph/c... - 09:19 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- apart from the discussion about MDS verifying clients OSDs set, https://tracker.ceph.com/issues/64563#note-28 also ne...
- 09:23 AM Bug #65533 (Resolved): qa/vstart_runner.py: don't let command run after timeout
- 09:15 AM CephFS Bug #65564 (New): Test failure: test_snap_schedule_subvol_and_group_arguments_08 (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSubvolAndGroupArguments)
- /a/yuriw-2024-04-05_22:36:11-fs-wip-yuri7-testing-2024-04-04-0800-distro-default-smithi/7642196...
- 08:33 AM CephFS Bug #64977: mds spinlock due to lock contention leading to memory exaustion
- We've uploaded a new set of logs with debug_ms 1 at 20d8ba67-8bb0-4cfc-a986-b72ec250728d
- 07:03 AM CephFS Bug #54404 (Closed): snap-schedule retention not working as expected
- Closing tracker due to lack of info.
If no valid retention is found during pruning phase, then all snapshots are imm...
04/17/2024
- 09:41 PM RADOS Bug #65557 (Closed): Admin socket times out after osd restart
- This was actually related to a WIP branch that hasn't merged yet.
- 08:46 PM RADOS Bug #65557 (Closed): Admin socket times out after osd restart
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652505...
- 09:34 PM RADOS Bug #65559 (Closed): src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- Actually seems related to a WIP branch that hadn't been merged yet.
- 08:55 PM RADOS Bug #65559 (Closed): src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652491...
- 09:24 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Nir Soffer wrote in #note-12:
> Ilya Dryomov wrote in #note-11:
> > Hi Nir,
> >
> > I think the problem is the m... - 01:08 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Ilya Dryomov wrote in #note-11:
> Hi Nir,
>
> I think the problem is the method you used to set these config opti... - 01:04 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Hi Nir,
I think the problem is the method you used to set these config options. Note that the way it's done in OD... - 08:46 PM RADOS Bug #53768: timed out waiting for admin_socket to appear after osd.2 restart in thrasher/defaults workload/small-objects
- Laura Flores wrote in #note-12:
> /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-defaul... - 01:59 AM RADOS Bug #53768: timed out waiting for admin_socket to appear after osd.2 restart in thrasher/defaults workload/small-objects
- @lflores There's little chance that the above crash is related to what Joseph saw here, let's close this one and open...
- 08:08 PM CephFS Backport #65556 (Fix Under Review): squid: mds: avoid recalling Fb when quiescing file
- 08:03 PM CephFS Bug #65472 (Pending Backport): mds: avoid recalling Fb when quiescing file
- 07:58 PM devops Bug #65555 (New): old pinned mistune in admin/doc-requirements.txt is vulnerable to CVE-2022-34749
- @admin/doc-requirements.txt@ pins to an older @mistune@ library version. Security scanners treat this as a vulnerabil...
- 07:32 PM Dashboard Bug #46735: FAIL: test_all (tasks.mgr.dashboard.test_rgw.RgwBucketTest)
- from https://jenkins.ceph.com/job/ceph-api/72562/consoleFull...
- 07:18 PM Orchestrator Bug #65546: quincy|reef: qa/suites/upgrade/pacific-x: failure to pull image causes dead jobs
- https://pulpito.ceph.com/teuthology-2024-04-17_01:16:02-upgrade:quincy-x-reef-distro-default-smithi/
- 02:12 PM Orchestrator Bug #65546 (New): quincy|reef: qa/suites/upgrade/pacific-x: failure to pull image causes dead jobs
- https://pulpito.ceph.com/teuthology-2024-04-17_01:08:06-upgrade:pacific-x-reef-distro-default-smithi/
Beyond the i... - 07:06 PM mgr Bug #64799: mgr: update cluster state for new maps from the mons before notifying modules
- Per let's not hurry up with backporting this chnage. IMHO it deserves some _baking_ in `main`.:
> let's not hurry... - 07:03 PM RADOS Bug #62588: ceph config set allows WHO to be osd.*, which is misleading
- I created a pull request for this: https://github.com/ceph/ceph/pull/56971
A warning message is now generated if use... - 06:36 PM Bug #65509: osd: remove outdated, incorrect truncate asserts in ECTransaction's generate_transactions
- Per https://github.com/ceph/ceph/pull/56924#issuecomment-2061948862 a workaround exists:
> (...) we could recover ... - 06:04 PM Dashboard Bug #47612: ERROR: setUpClass (tasks.mgr.dashboard.test_health.HealthTest)
- https://jenkins.ceph.com/job/ceph-api/72561/...
- 06:00 PM RADOS Bug #53000: OSDMap/OSDMapTest.BUG_51842/2: ThreadPool::WorkQueue<ParallelPGMapper::Item>::_void_dequeue: pure virtual method called
- from https://jenkins.ceph.com/job/ceph-pull-requests/133465/consoleFull...
- 05:34 PM Orchestrator Bug #65554 (In Progress): mgr/nfs: nfs module commands do not accept json-pretty format
- ...
- 03:36 PM Orchestrator Bug #65553 (In Progress): cephadm: agent tries to json load response payload before checking for errors
- If the connection itself fails, the agent will end up hitting another exception...
- 03:33 PM rgw Backport #65353 (In Progress): squid: rgwlc: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault
- 03:30 PM rgw Backport #64496 (In Progress): squid: keystone admin token is not invalidated on http 401 response
- 03:28 PM rgw Backport #64552 (In Progress): squid: rgw/multisite: objects named "." or ".." are not replicated
- 03:26 PM rgw Feature #65551 (Pending Backport): [rgw][accounts] bucket quota management at account-level
- Account feature has been introduced by https://github.com/ceph/ceph/pull/54333 and we are planning to migrate our rad...
- 03:19 PM rbd Backport #65550 (In Progress): squid: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- https://github.com/ceph/ceph/pull/57031
- 03:18 PM rbd Backport #65549 (In Progress): reef: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- https://github.com/ceph/ceph/pull/57030
- 03:07 PM rbd Backport #65548 (Duplicate): reef: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- 03:07 PM rbd Backport #65547 (In Progress): quincy: [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- https://github.com/ceph/ceph/pull/57029
- 03:02 PM rbd Bug #65481 (Pending Backport): [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- 01:40 PM CephFS Bug #65545 (Fix Under Review): Quiesce may fail randomly with EBADF due to the same root submitted to the MDCache multiple times under the same quiesce request
- 01:34 PM CephFS Bug #65545 (Pending Backport): Quiesce may fail randomly with EBADF due to the same root submitted to the MDCache multiple times under the same quiesce request
- Reported by the QE team at https://bugzilla.redhat.com/show_bug.cgi?id=2275459...
- 01:37 PM rgw Feature #65050: Add alternative way for providing user name/password for Kafka endpoint authentication
- Needs review. Corresponding PR is here:
https://github.com/ceph/ceph/pull/56493 - 01:01 PM CephFS Backport #65325 (In Progress): reef: client: log message when unmount call is received
- 01:01 PM CephFS Backport #65326 (In Progress): quincy: client: log message when unmount call is received
- 12:52 PM CephFS Backport #65365 (In Progress): reef: qa: run TestSnapshots.test_kill_mdstable for all mount types
- 12:52 PM CephFS Backport #65366 (In Progress): squid: qa: run TestSnapshots.test_kill_mdstable for all mount types
- 12:51 PM CephFS Backport #65520 (In Progress): reef: qa: cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log
- 12:50 PM CephFS Backport #65519 (In Progress): squid: qa: cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log
- 12:38 PM rgw Backport #65543 (In Progress): squid: rgw: increase log level on abort_early
- 12:37 PM rgw Backport #65543 (In Progress): squid: rgw: increase log level on abort_early
- https://github.com/ceph/ceph/pull/56949
- 12:37 PM Backport #65540 (In Progress): reef: Add alerts to ceph monitoring stack for the nvmeof gateways
- 09:39 AM Backport #65540 (In Progress): reef: Add alerts to ceph monitoring stack for the nvmeof gateways
- https://github.com/ceph/ceph/pull/56948
- 12:37 PM Orchestrator Bug #63784: qa/standalone/mon/mkfs.sh:'mkfs/a' already exists and is not empty: monitor may already exist
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648785/
- 12:37 PM rgw Backport #65544 (New): reef: rgw: increase log level on abort_early
- 12:35 PM RADOS Bug #50245: TEST_recovery_scrub_2: Not enough recovery started simultaneously
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648573/
- 12:31 PM Backport #65539 (In Progress): squid: Add alerts to ceph monitoring stack for the nvmeof gateways
- 09:39 AM Backport #65539 (In Progress): squid: Add alerts to ceph monitoring stack for the nvmeof gateways
- https://github.com/ceph/ceph/pull/56947
- 12:27 PM rgw Bug #65469 (Pending Backport): rgw: increase log level on abort_early
- 12:11 PM Orchestrator Bug #65035: ERROR: required file missing from config-json: idmap.conf
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648585/
- 12:04 PM RADOS Bug #53544: src/test/osd/RadosModel.h: ceph_abort_msg("racing read got wrong version") in thrash_cache_writeback_proxy_none tests
- @lflores FYI seeing this one after a while in one of the main runs - /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-tes...
- 11:48 AM Dashboard Backport #64791 (In Progress): squid: mgr/dashboard: In rgw multisite, during zone creation acess/secret key should not be compulsory provide an edit option to set these keys
- 11:47 AM RADOS Bug #56393: failed to complete snap trimming before timeout
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648606
/a/yuriw-2024-04-... - 11:45 AM Dashboard Bug #61786: test_dashboard_e2e.sh: Can't run because no spec files were found; couldn't determine Mocha version
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648833
- 11:43 AM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648693/
- 10:48 AM Orchestrator Bug #63502: Regression: Permanent KeyError: 'TYPE' : return self.blkid_api['TYPE'] == 'part'
- Vadym Kukharenko wrote in #note-1:
> I got the same problem.
> Fistly tried to upgrade from 17.2.6 to 17.2.7.
> Se... - 10:14 AM Dashboard Bug #65534 (Fix Under Review): mgr/dashboard: grafana dashboad doesn't exist when anonymous_access is enabled
- 07:10 AM Dashboard Bug #65534 (Fix Under Review): mgr/dashboard: grafana dashboad doesn't exist when anonymous_access is enabled
- Overall Performance does not display the graphs
Description of problem:
# cat /var/lib/ceph/tmp/grafana.yaml
s... - 10:10 AM Dashboard Cleanup #65207 (Resolved): mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- 10:10 AM Dashboard Backport #65504 (Resolved): reef: mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- 10:09 AM Dashboard Backport #65505 (Resolved): squid: mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- 10:08 AM Dashboard Backport #65542 (In Progress): squid: rgw roles e2e tests failure
- 10:02 AM Dashboard Backport #65542 (Resolved): squid: rgw roles e2e tests failure
- https://github.com/ceph/ceph/pull/56945
- 10:07 AM RADOS Bug #62512: osd msgr-worker high cpu 300% due to throttle-osd_client_messages get_or_fail_fail (osd_client_message_cap=256)
- Please see my latest update to the PR: https://github.com/ceph/ceph/pull/53477
I can confirm the fix is good and a... - 09:55 AM Dashboard Bug #65506 (Pending Backport): rgw roles e2e tests failure
- 09:45 AM Dashboard Bug #65541 (New): Empty (string, list, object) should be blank in dashboard
- Empty (string, list, object) should be blank in dashboard
We need to see how we show empty data structures in dash... - 09:31 AM Backport #65538 (New): reef: Add alerts to ceph monitoring stack for the nvmeof gateways
- 09:28 AM Feature #64335 (Pending Backport): Add alerts to ceph monitoring stack for the nvmeof gateways
- 08:37 AM Orchestrator Bug #64868: cephadm/osds, cephadm/workunits: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) in cluster log
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648713/
/a/yuriw-2024-04... - 08:29 AM RADOS Bug #64942: rados/verify: valgrind reports "Invalid read of size 8" error.
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648773/
- 08:23 AM Orchestrator Bug #64871: rados/cephadm/workunits: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON)" in cluster log
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648786/
/a/yuriw-2024-04-... - 08:16 AM Messengers Documentation #65537 (New): RDMA support
- Hi guys,
I needed to setup Ceph over RDMA, but I faced many issues! Because there is not enough info in the docume... - 08:03 AM CephFS Bug #65536 (Fix Under Review): mds: after the unresponsive client was evicted the blocked slow requests were not successfully cleaned up
- 07:50 AM CephFS Bug #65536 (Fix Under Review): mds: after the unresponsive client was evicted the blocked slow requests were not successfully cleaned up
Firstly a *client.188978:3 lookup #0x10000000000/csi* client request came and then was added to the waiter list:
...- 07:47 AM Orchestrator Bug #64872: rados/cephadm/smoke: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) in cluster log
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648850/
- 07:33 AM Orchestrator Bug #65017: cephadm: log_channel(cephadm) log [ERR] : Failed to connect to smithi090 (10.0.0.9). Permission denied
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648574/
- 07:23 AM RADOS Bug #61774: centos 9 testing reveals rocksdb "Leak_StillReachable" memory leak in mons
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648705
/a/yuriw-2024-04-... - 07:16 AM RADOS Bug #65517: rados/thrash-erasure-code-crush-4-nodes: ceph task fails at getting monitors
- /a/yuriw-2024-04-09_14:35:50-rados-wip-yuri5-testing-2024-03-21-0833-distro-default-smithi/7648565
/a/yuriw-2024-04-... - 06:57 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Venky Shankar wrote in #note-15:
> Dhairya Parmar wrote in #note-14:
> > Venky Shankar wrote in #note-13:
> > > Ve... - 06:47 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Dhairya Parmar wrote in #note-14:
> Venky Shankar wrote in #note-13:
> > Venky Shankar wrote in #note-11:
> > > Dh... - 06:14 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Venky Shankar wrote in #note-13:
> Venky Shankar wrote in #note-11:
> > Dhairya Parmar wrote in #note-10:
> > > I ... - 05:03 AM CephFS Feature #65503 (New): mgr/stats, cephfs-top: provide per volume/sub-volume based performance metrics to monitor / troubleshoot performance issues
- 04:54 AM CephFS Feature #65503 (Rejected): mgr/stats, cephfs-top: provide per volume/sub-volume based performance metrics to monitor / troubleshoot performance issues
- 04:40 AM Bug #65533 (Resolved): qa/vstart_runner.py: don't let command run after timeout
- LocalRemote.run() accepts parameter @timeout@ but it is not passed to @subprocess@ and therefore has no effect.
- 03:16 AM crimson Bug #65532: osd crashes due to invalid clone_range ops
- It seems that this is due to incorrect clone_overlap calculations, will go into it.
- 03:15 AM crimson Bug #65532 (Fix Under Review): osd crashes due to invalid clone_range ops
- ...
- 02:51 AM crimson Bug #65531: crimson-osd: dump_historic_slow_ops command not correctly run
- I don't think it's necessary to put history_cliend_request and history_slow_cliend_request together, so I will separa...
- 02:34 AM crimson Bug #65531 (In Progress): crimson-osd: dump_historic_slow_ops command not correctly run
- right now, historic ops and historic slow ops all placed in OperationTypeCode::historic_client_request op_list, use l...
- 01:43 AM rgw Bug #65436: Getting Object Crashing radosgw services
- I have same issue. After some days, i found bug https://tracker.ceph.com/issues/61359
After upgrade to 17.2.7, this ...
04/16/2024
- 11:19 PM RADOS Cleanup #65521 (New): Add expected warnings in cluster log to ignorelists
- Relevant Slack conversation:
Hey all, as I brought up in today's RADOS call, there has been a surge of cluster war... - 11:19 PM CephFS Backport #65520 (In Progress): reef: qa: cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log
- https://github.com/ceph/ceph/pull/56951
- 11:18 PM CephFS Backport #65519 (In Progress): squid: qa: cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log
- https://github.com/ceph/ceph/pull/56950
- 11:17 PM CephFS Bug #65271 (Pending Backport): qa: cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log
- 09:06 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Ilya Dryomov wrote in #note-9:
> Nir Soffer wrote in #note-8:
> > Yes, the configuration is applied to both cluster... - 08:22 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Nir Soffer wrote in #note-8:
> Yes, the configuration is applied to both clusters. If I understand correctly,
> The... - 08:06 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Ilya Dryomov wrote in #note-7:
> Nir Soffer wrote in #note-6:
> > The other log file (e.g. 62f28287-356f-4f81-87dc-... - 07:04 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Nir Soffer wrote in #note-6:
> The other log file (e.g. 62f28287-356f-4f81-87dc-51bb05942553-client.rbd-mirror-peer.... - 01:03 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Nir Soffer wrote in #note-5:
> > https://github.com/red-hat-storage/ocs-operator/blob/4a0325d824a409e84fac21ffbf0a... - 08:55 PM RADOS Bug #53768: timed out waiting for admin_socket to appear after osd.2 restart in thrasher/defaults workload/small-objects
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652505
- 08:55 PM CephFS Bug #65518 (Fix Under Review): mds: regular file inode flags are not replicated by the policylock
- 08:53 PM CephFS Bug #65518 (Fix Under Review): mds: regular file inode flags are not replicated by the policylock
- Currently, the flags are only replicated for directory inodes.
- 08:40 PM RADOS Bug #64437: qa/standalone/scrub/osd-scrub-repair.sh: TEST_repair_stats_ec: test 26 = 13
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652514
- 08:37 PM RADOS Bug #65517: rados/thrash-erasure-code-crush-4-nodes: ceph task fails at getting monitors
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620629
/a/yuriw-2024-03... - 08:36 PM RADOS Bug #65517: rados/thrash-erasure-code-crush-4-nodes: ceph task fails at getting monitors
- Hey @nmordech can you have a look?
- 08:35 PM RADOS Bug #65517: rados/thrash-erasure-code-crush-4-nodes: ceph task fails at getting monitors
- Looks like the change was made in https://github.com/ceph/ceph/pull/53308, which did initially pass QA testing, but m...
- 08:31 PM RADOS Bug #65517 (New): rados/thrash-erasure-code-crush-4-nodes: ceph task fails at getting monitors
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652508...
- 08:16 PM Orchestrator Bug #64872: rados/cephadm/smoke: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) in cluster log
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652511
- 08:12 PM RADOS Bug #65422: upgrade/quincy-x/parallel: "1 pg degraded (PG_DEGRADED)" in cluster log
- ...
- 08:10 PM RADOS Bug #65235: upgrade/reef-x/stress-split: "OSDMAP_FLAGS: noscrub flag(s) set" warning in cluster log
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652474...
- 08:08 PM RADOS Bug #62776: rados: cluster [WRN] overall HEALTH_WARN - do not have an application enabled
- /a/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7652467
- 08:05 PM Bug #65509 (Fix Under Review): osd: remove outdated, incorrect truncate asserts in ECTransaction's generate_transactions
- https://github.com/ceph/ceph/pull/56924
- 02:47 PM Bug #65509: osd: remove outdated, incorrect truncate asserts in ECTransaction's generate_transactions
- See: https://github.com/ceph/ceph/pull/56924
- 02:03 PM Bug #65509 (Fix Under Review): osd: remove outdated, incorrect truncate asserts in ECTransaction's generate_transactions
- User hit this:...
- 08:04 PM CephFS Bug #65496 (Fix Under Review): mds: ceph.dir.subvolume and ceph.quiesce.blocked is not properly replicated
- 02:42 AM CephFS Bug #65496 (Fix Under Review): mds: ceph.dir.subvolume and ceph.quiesce.blocked is not properly replicated
- The logic for checking if an inode already had these vxattrs set has the serious defect that it will only execute xlo...
- 07:59 PM Orchestrator Backport #65415 (Resolved): squid: cephadm: test_cephadm script fails with "ERROR: required file missing from config-json: idmap.conf"
- 07:58 PM Orchestrator Backport #65382 (Resolved): squid: NLM should be enabled in NFS-Ganesha config file for locking functionality to work with v3 protocol
- 07:55 PM Orchestrator Bug #64865 (Resolved): cephadm: Health check failed: 1 osds down (OSD_DOWN) in cluster log
- 07:55 PM Orchestrator Backport #65414 (Resolved): squid: cephadm: Health check failed: 1 osds down (OSD_DOWN) in cluster log
- 07:54 PM Dashboard Bug #64870: Health check failed: 1 osds down (OSD_DOWN)" in cluster log
- More in this run:
https://pulpito.ceph.com/yuriw-2024-04-11_17:03:54-rados-wip-yuri6-testing-2024-04-02-1310-distro-... - 07:50 PM Dashboard Bug #64870: Health check failed: 1 osds down (OSD_DOWN)" in cluster log
- And in a cephadm test: /a/yuriw-2024-04-10_14:17:51-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/765...
- 07:44 PM Dashboard Bug #64870: Health check failed: 1 osds down (OSD_DOWN)" in cluster log
- Also found in an upgrade test:
description: rados/upgrade/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-task... - 07:51 PM Orchestrator Bug #52109: test_cephadm.sh: Timeout('Port 8443 not free on 127.0.0.1.',)
- /a/yuriw-2024-04-10_14:17:51-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7650646
- 07:42 PM RADOS Bug #65231: upgrade/quincy-x/parallel: "Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
- /a/yuriw-2024-04-10_14:17:51-rados-wip-yuri6-testing-2024-04-02-1310-distro-default-smithi/7650700
- 07:30 PM rgw Backport #64510 (Resolved): squid: backport rgw/lc: decorating log events with more details
- 07:29 PM rgw Backport #64949 (Resolved): squid: rgw-multisite: add x-rgw-replicated-at
- 07:29 PM rgw Backport #65292 (Resolved): squid: pubsub: validate Name in CreateTopic api
- 07:28 PM rgw Backport #65297 (Resolved): squid: allow AWS lifecycle event types to configure lifecycle notifications and Replication notifications
- 07:28 PM rgw Backport #65375 (Resolved): squid: lifecycle transition crashes since reloading bucket attrs for notification
- 07:27 PM rgw Feature #65466 (Resolved): rgw user accounts
- 07:27 PM rgw Backport #65402 (Resolved): squid: persistent topic stats test fails
- 07:27 PM rgw Feature #50078 (Resolved): [RFE] multisite: Bucket notification information should be shared between zones.
- 07:27 PM rgw Backport #64818 (Resolved): squid: [RFE] multisite: Bucket notification information should be shared between zones.
- 07:26 PM rgw Backport #65467 (Resolved): squid: rgw user accounts
- 07:25 PM rgw Backport #65411 (Resolved): squid: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- 06:14 PM Dashboard Backport #65515 (In Progress): squid: mgr/dashboard: fix duplicate grafana panels when on mgr failover
- 05:58 PM Dashboard Backport #65515 (In Progress): squid: mgr/dashboard: fix duplicate grafana panels when on mgr failover
- https://github.com/ceph/ceph/pull/56931
- 06:10 PM Dashboard Backport #65513 (In Progress): quincy: mgr/dashboard: fix duplicate grafana panels when on mgr failover
- 05:51 PM Dashboard Backport #65513 (In Progress): quincy: mgr/dashboard: fix duplicate grafana panels when on mgr failover
- https://github.com/ceph/ceph/pull/56930
- 06:01 PM Dashboard Backport #65512 (In Progress): reef: mgr/dashboard: fix duplicate grafana panels when on mgr failover
- 05:51 PM Dashboard Backport #65512 (In Progress): reef: mgr/dashboard: fix duplicate grafana panels when on mgr failover
- https://github.com/ceph/ceph/pull/56929
- 05:52 PM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Dhairya Parmar wrote in #note-27:
> Venky Shankar wrote in #note-26:
> > Dhairya Parmar wrote in #note-25:
> > > V... - 08:24 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Dhairya Parmar wrote in #note-27:
> Venky Shankar wrote in #note-26:
> > Dhairya Parmar wrote in #note-25:
> > > V... - 08:09 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- as mentioned in yesterday's standup - some of the PRs (https://github.com/ceph/ceph/pull/49971, https://github.com/ce...
- 05:51 PM Dashboard Backport #65514 (New): squid: mgr/dashboard: fix duplicate grafana panels when on mgr failover
- 05:49 PM Dashboard Bug #64970 (Pending Backport): mgr/dashboard: fix duplicate grafana panels when on mgr failover
- 05:22 PM RADOS Bug #62588: ceph config set allows WHO to be osd.*, which is misleading
- ...
- 05:11 PM rgw Backport #65351 (Resolved): squid: rgw: crash in lc while transitioning to cloud
- 04:59 PM Orchestrator Documentation #64596 (Resolved): secure monitoring stack support is not documented
- 04:59 PM Orchestrator Backport #64631 (Resolved): squid: secure monitoring stack support is not documented
- 04:19 PM Bug #65511 (In Progress): cephadm: anonymous_access: false is dropped from grafana spec after apply
- ...
- 03:17 PM Dashboard Bug #65506: rgw roles e2e tests failure
- Same issue happening on squid hence adding backport
- 11:54 AM Dashboard Bug #65506 (Fix Under Review): rgw roles e2e tests failure
- 11:09 AM Dashboard Bug #65506 (Resolved): rgw roles e2e tests failure
- *Rgw roles tests failing with 500 internal server error:*...
- 03:06 PM rgw Backport #65427 (Resolved): squid: Admin Ops socket crashes RGW
- 02:17 PM crimson Bug #65491: recover_missing: racing read got wrong version
- Not a fix yet, bug I added few missing log lines that may help here:
https://github.com/ceph/ceph/pull/56916/commits... - 10:06 AM crimson Bug #65491: recover_missing: racing read got wrong version
- WIP
- 01:57 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- Yeah, there is a change in @attrs@ processing. Already prepared a commit: https://github.com/rzarzynski/ceph/commit/c...
- 01:53 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
I am considering the following suspect(s):
PR #54930 modified ScrubMap::object::attrs (where we see a problem) from ...- 01:28 PM CephFS Bug #65508 (Fix Under Review): qa: lockup not long enough to for test_quiesce_authpin_wait
- 01:25 PM CephFS Bug #65508 (Fix Under Review): qa: lockup not long enough to for test_quiesce_authpin_wait
- https://pulpito.ceph.com/leonidus-2024-04-16_05:41:33-fs-wip-lusov-quiesce-xlock-distro-default-smithi/7657916/
- 12:08 PM Bug #65507 (New): diskprediction_local failed with python3.10
- 1. failed messages:...
- 11:56 AM Dashboard Backport #65504 (In Progress): reef: mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- 10:44 AM Dashboard Backport #65504 (Resolved): reef: mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- https://github.com/ceph/ceph/pull/56921
- 11:55 AM Dashboard Backport #65505 (In Progress): squid: mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- 10:44 AM Dashboard Backport #65505 (Resolved): squid: mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- https://github.com/ceph/ceph/pull/56920
- 11:47 AM RADOS Bug #65449 (In Progress): NeoRadosWatchNotify.WatchNotifyTimeout failed due to nonexistent pool
- 11:05 AM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- Venky Shankar wrote in #note-33:
> OK. So this bug has upgrades written all over it - it seemed obvious given that t... - 10:56 AM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- OK. So this bug has upgrades written all over it - it seemed obvious given that this is an upgrade task but we were t...
- 10:39 AM CephFS Feature #65503 (New): mgr/stats, cephfs-top: provide per volume/sub-volume based performance metrics to monitor / troubleshoot performance issues
- Reported by BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2275081
Currently the cephfs-top utility only displays... - 10:37 AM Dashboard Cleanup #65207 (Pending Backport): mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- 10:37 AM Dashboard Backport #65502 (New): squid: mgr/dashboard: provide hub Cluster HA for multi-cluster setup
- 10:32 AM Dashboard Bug #65499 (Pending Backport): mgr/dashboard: provide hub Cluster HA for multi-cluster setup
- 06:23 AM Dashboard Bug #65499 (Pending Backport): mgr/dashboard: provide hub Cluster HA for multi-cluster setup
- When adding a cluster to the multi-cluster setup, set all the mgr IP's as cross_origin_url in the connected cluster t...
- 10:08 AM Dashboard Backport #65501 (In Progress): squid: mgr/dashboard: snap schedule remove minutely from retention policy dropdown
- 09:51 AM Dashboard Backport #65501 (In Progress): squid: mgr/dashboard: snap schedule remove minutely from retention policy dropdown
- https://github.com/ceph/ceph/pull/56918
- 10:07 AM Dashboard Backport #65500 (In Progress): reef: mgr/dashboard: snap schedule remove minutely from retention policy dropdown
- 09:51 AM Dashboard Backport #65500 (In Progress): reef: mgr/dashboard: snap schedule remove minutely from retention policy dropdown
- https://github.com/ceph/ceph/pull/56917
- 09:52 AM RADOS Bug #64519: OSD/MON: No snapshot metadata keys trimming
- Thanks, Matan! It sounds very promising. I talked to the customer and they are willing to test this cleanup procedure...
- 09:45 AM Dashboard Bug #65493 (Pending Backport): mgr/dashboard: snap schedule remove minutely from retention policy dropdown
- 05:58 AM Messengers Bug #65401: msg: conneciton between mgr and osd is periodically down which leads heavy load to mgr
- Could anyone give a review on this? Thanks very much!
- 05:51 AM Dashboard Backport #65498 (New): squid: mgr/dashboard: fetch prometheus api host with ip addr
- 05:48 AM Dashboard Bug #65302 (Pending Backport): mgr/dashboard: fetch prometheus api host with ip addr
- 04:45 AM CephFS Bug #65497 (Fix Under Review): qa: enhance labelled perf counters tests in test_admin.py
- 04:28 AM CephFS Backport #65347 (In Progress): squid: qa: failed cephfs-shell test_reading_conf
- 02:50 AM crimson Bug #64680: transaction_manager_test/tm_random_block_device_test_t.scatter_allocation/0 status failed
- This is caused by prefilling rbm devices, which is used to create scatterly allocated devices and is only used in uni...
- 01:05 AM crimson Feature #65478: Support SnapMapper::Scrubber
- It will be completed in these few days
04/15/2024
- 10:55 PM RADOS Bug #64863 (Resolved): rados/thrash-old-clients: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c in cluster log
- https://github.com/ceph/ceph/pull/56619
Radoslaw Zarzynski wrote in #note-3:
> Hmm, I think I saw Laura's PR for ... - 10:30 PM rgw Backport #65339 (Resolved): squid: rgw: update options yaml file so LDAP uri isn't an invalid example
- 10:30 PM rgw Backport #65412 (Resolved): squid: multisite: test_object_sync gets wrong object body: b'<x-rgw' != b'asdasd'
- 10:29 PM rgw Backport #64954 (Resolved): squid: Notification FilterRules for S3key, S3Metadata & S3Tags spit incorrect json output
- 10:23 PM cephsqlite Bug #65494: ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- Ilya Dryomov wrote in #note-6:
> Nir Soffer wrote:
> > Restarting the ceph-mgr pod does not help, rbd-mirroring is ... - 09:36 PM cephsqlite Bug #65494: ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- Nir Soffer wrote:
> Restarting the ceph-mgr pod does not help, rbd-mirroring is broken and
> we don't have any work... - 09:03 PM cephsqlite Bug #65494 (In Progress): ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- 09:03 PM cephsqlite Bug #65494: ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- ...
- 08:58 PM cephsqlite Bug #65494: ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- Tested with:
* image: quay.io/ceph/ceph:v18
* imageID: quay.io/ceph/ceph@sha256:8c1697a0a924bbd625c9f1b33893bbc47b9... - 07:53 PM cephsqlite Bug #65494: ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- Looks like a sqlite issue. Patrick, can you take a look please?
- 07:15 PM cephsqlite Bug #65494: ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- Thread from rook slack:
https://rook-io.slack.com/archives/CK9CF5H2R/p1711467112958679 - 07:13 PM cephsqlite Bug #65494 (Fix Under Review): ceph-mgr critical error: "Module 'devicehealth' has failed: table Device already exists"
- h1. Description
We have a random error (about 1 in 200 deploys) when after creating a rook
cephcluster and cephbl... - 10:14 PM RADOS Bug #62776: rados: cluster [WRN] overall HEALTH_WARN - do not have an application enabled
- /a/yuriw-2024-04-09_01:16:20-rados-reef-release-distro-default-smithi/7647437
- 10:11 PM Dashboard Bug #64377: tasks/e2e: Modular dependency problems
- /a/yuriw-2024-04-09_01:16:20-rados-reef-release-distro-default-smithi/7647494
- 10:07 PM CephFS Bug #64946: qa: unable to locate package libcephfs1
- /a/yuriw-2024-04-09_01:16:20-rados-reef-release-distro-default-smithi/7647487
/a/yuriw-2024-04-09_01:16:20-rados-reef... - 10:01 PM RADOS Bug #58893: test_map_discontinuity: AssertionError: wait_for_clean: failed before timeout expired
- /a/yuriw-2024-04-09_01:16:20-rados-reef-release-distro-default-smithi/7647835
- 06:25 PM RADOS Bug #58893: test_map_discontinuity: AssertionError: wait_for_clean: failed before timeout expired
- Just a supplement to Nitzan's comment:
* this PG was @down@ and
* @ 'blocked_by': [2]@.
This brings the questi... - 09:59 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Ilya Dryomov wrote in #note-4:
> > This is not ODF environment, this is upstream rook environment.
> >
> > You ca... - 09:21 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Nir Soffer wrote in #note-3:
> It can be, but rbd mirror should fail (and restart) if pod networking is broken, no?
... - 08:54 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Ilya Dryomov wrote in #note-1:
> Hi Nir,
>
> rbd-mirror daemon states that it was unable to connect to the remote... - 08:44 PM rbd Bug #65487: rbd-mirror daemon in ERROR state, require manual restart
- Tested with:
* image: quay.io/ceph/ceph:v18
* imageID: quay.io/ceph/ceph@sha256:06ddc3ef5b66f2dcc6d16e41842d33a3d... - 08:43 PM rbd Bug #65487 (Need More Info): rbd-mirror daemon in ERROR state, require manual restart
- Hi Nir,
rbd-mirror daemon states that it was unable to connect to the remote cluster. Could it be some kind of po... - 01:26 PM rbd Bug #65487 (In Progress): rbd-mirror daemon in ERROR state, require manual restart
- h1. Description
We experience a random error in rbd-mirror daemon, occurring 1-2 times per 100 deployments.
Whe... - 09:59 PM RADOS Bug #62992: Heartbeat crash in reset_timeout and clear_timeout
- /a/yuriw-2024-04-09_01:16:20-rados-reef-release-distro-default-smithi/7647721
/a/yuriw-2024-04-09_01:16:20-rados-reef... - 09:55 PM Orchestrator Bug #64208: test_cephadm.sh: Container version mismatch causes job to fail.
- /a/yuriw-2024-04-09_01:16:20-rados-reef-release-distro-default-smithi/7647904
/a/yuriw-2024-04-09_01:16:20-rados-reef... - 09:51 PM RADOS Bug #65183: Overriding an EC pool needs the "--yes-i-really-mean-it" flag in addition to "force"
- /a/yuriw-2024-04-09_01:16:20-rados-reef-release-distro-default-smithi/7647523
/a/yuriw-2024-04-09_01:16:20-rados-reef... - 08:32 PM RADOS Bug #65495: 1 slow request in rgw suite causes test failure
- i see that one of the osds on the other node has a similarly large log:...
- 08:19 PM RADOS Bug #65495 (New): 1 slow request in rgw suite causes test failure
- on an integration branch based on squid, a rgw suite job failed due to 'slow request' errors: https://qa-proxy.ceph.c...
- 06:32 PM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- Bump up. In QA.
- 06:31 PM RADOS Bug #65227: noscrub cluster flag prevents deep-scrubs from starting
- IIRC Ronen is already working on start orchestration between deep- and shallow-scrubs,
- 06:28 PM RADOS Bug #50222: osd: 5.2s0 deep-scrub : stat mismatch
- Bump up.
- 06:27 PM RADOS Bug #64519: OSD/MON: No snapshot metadata keys trimming
- The PR is in QA.
- 06:16 PM RADOS Bug #65449: NeoRadosWatchNotify.WatchNotifyTimeout failed due to nonexistent pool
- Hi Nitzan! Would you mind taking a look?
- 06:11 PM RADOS Bug #59670 (Need More Info): Ceph status shows PG recovering when norecover flag is set
- The fix has been merged on 5 Jan 2024, so this could fit. It has been bacported only to Reef.
Wes Dillingham, do y... - 06:10 PM Orchestrator Backport #65383 (In Progress): reef: NLM should be enabled in NFS-Ganesha config file for locking functionality to work with v3 protocol
- 05:53 PM RADOS Bug #65422: upgrade/quincy-x/parallel: "1 pg degraded (PG_DEGRADED)" in cluster log
- Needs to be whitelisted; will bring this issue and others like it to the next RADOS meeting so we can divide up that ...
- 05:46 PM RADOS Bug #65422: upgrade/quincy-x/parallel: "1 pg degraded (PG_DEGRADED)" in cluster log
- Thanks Venky, it was a mistake that I added it there in the first place.
- 01:15 PM RADOS Bug #65422: upgrade/quincy-x/parallel: "1 pg degraded (PG_DEGRADED)" in cluster log
- Laura, handing this back to you since this isn't really cephfs related.
- 05:45 PM RADOS Bug #53472 (Need More Info): Active OSD processes do not see reduced memory target when adding more OSDs
- Pacific is EOL. Does it replicate on newer releases?
- 05:45 PM RADOS Bug #53472: Active OSD processes do not see reduced memory target when adding more OSDs
- This tracker is 2 years old. I'm not sure how the situation was back then but, at least now, BlueStore is observing t...
- 05:35 PM RADOS Bug #52657: MOSDPGLog::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_NAUTILUS)'
- Waiting for upstream QA.
- 05:34 PM RADOS Bug #65371: rados: PeeringState::calc_replicated_acting_stretch populate acting set before checking if < bucket_max
- Bump up.
- 05:02 PM rbd Bug #65481 (Fix Under Review): [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- 09:51 AM rbd Bug #65481 (Pending Backport): [test] krbd_msgr_segments and krbd_rxbounce fail on 8.stream
- https://qa-proxy.ceph.com/teuthology/yuriw-2024-04-09_15:14:48-krbd-reef-release-testing-default-smithi/7649268/teuth...
- 04:15 PM CephFS Documentation #57011: doc: 'profile cephfs-mirror' description is missing
- not sure why this got moved out of cephfs, this is our documentation bug
- 03:58 PM cleanup Tasks #65471 (Fix Under Review): rgw_sal_posix.cc printf compiler warnings
- 03:48 PM Dashboard Bug #65493 (Pending Backport): mgr/dashboard: snap schedule remove minutely from retention policy dropdown
- Remove minutely from retention policy dropdown
- 03:28 PM rgw Bug #65473 (Fix Under Review): rgw: exclude logging of request payer for 403 requests
- 03:27 PM Orchestrator Backport #65417 (In Progress): squid: cephadmin returns "1" on successful host-maintenance enter/exit - should return "0"
- 03:26 PM Orchestrator Backport #65415 (In Progress): squid: cephadm: test_cephadm script fails with "ERROR: required file missing from config-json: idmap.conf"
- 03:23 PM Orchestrator Backport #65382 (In Progress): squid: NLM should be enabled in NFS-Ganesha config file for locking functionality to work with v3 protocol
- 03:20 PM Orchestrator Backport #65381 (In Progress): squid: upgrade/quincy-x/stress-split: cephadm failed to parse grafana.ini file due to inadequate permission
- 03:14 PM Orchestrator Backport #65378 (In Progress): squid: cephadm: client-keyring also overwrites ceph.conf
- 02:59 PM crimson Bug #53661 (Closed): Creation of the cluster failed with the crimson build
- Please re-open if still relevant.
- 02:55 PM crimson Bug #53047 (Closed): cmake command not found in the standalone cluster to execute cmake -DWITH_SEASTAR=ON .. command
- Please re-open if still relevant.
- 02:54 PM crimson Bug #52623 (Closed): Cache tries to get an invalid root extent
- Please re-open if still relevant.
- 02:53 PM crimson Bug #51639 (Closed): crimson/store_nbd: crash after start
- Please re-open if still relevant.
- 02:52 PM crimson Bug #47597 (Closed): got crush when stop one osd and restart it during rados bench
- Please re-open if still relevant.
- 02:52 PM crimson Bug #47030 (Closed): segault when evicting osdmap from cache
- Please re-open if still relevant.
- 02:50 PM crimson Bug #57547 (Closed): Hang with seastore at wait_for_active stage
- Please re-open if still relevant.
- 02:49 PM crimson Bug #57548 (Closed): Hang with alienstore
- Please re-open if still relevant.
- 02:49 PM crimson Subtask #45535 (Closed): crimson: crimson-osd failure in ceph-container
- Please re-open if still relevant.
- 02:40 PM rgw Bug #64571: lifecycle transition crashes since reloading bucket attrs for notification
- The cause for this issue seems to be due to multiple LC worker threads updating the same `bucket` handle, which is no...
- 02:31 PM CephFS Backport #65489 (In Progress): squid: mds: enhance scrub to fragment/merge dirfrags
- 01:28 PM CephFS Backport #65489 (In Progress): squid: mds: enhance scrub to fragment/merge dirfrags
- https://github.com/ceph/ceph/pull/56896
- 02:30 PM CephFS Backport #65488 (In Progress): reef: mds: enhance scrub to fragment/merge dirfrags
- 01:28 PM CephFS Backport #65488 (In Progress): reef: mds: enhance scrub to fragment/merge dirfrags
- https://github.com/ceph/ceph/pull/56895
- 02:28 PM CephFS Backport #65490 (In Progress): quincy: mds: enhance scrub to fragment/merge dirfrags
- 01:28 PM CephFS Backport #65490 (In Progress): quincy: mds: enhance scrub to fragment/merge dirfrags
- https://github.com/ceph/ceph/pull/56894
- 02:11 PM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Venky Shankar wrote in #note-11:
> Dhairya Parmar wrote in #note-10:
> > I was confident of the code, I've mentione... - 01:27 PM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- this doesn't seem related to test cases at all
time when the MGR_DOWN warning was seen:... - 06:13 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Dhairya Parmar wrote in #note-10:
> I was confident of the code, I've mentioned this in https://tracker.ceph.com/iss... - 02:09 PM CephFS Bug #65423 (Need More Info): Monitor crashes down when I try to create a FS. The stacks maybe related to metadata server map decoder during the PAXOS service
- fuchen ma wrote in #note-1:
> Another information:
> I found that the version of the non-crashed is 18.2.2, and the... - 02:08 PM CephFS Bug #65455 (Need More Info): read operation hung in Client::get_caps
- tod chen wrote in #note-1:
> the ceph version is 15.2.17 and 16.2.14
ceph 15.x is EOL'd and unsupported. Could yo... - 02:06 PM rgw Bug #65463: rgw/notifications: test data path v2 persistent migration fails
- * even tough no crash is observed, it seems like a similar issue to: https://tracker.ceph.com/issues/65337. when runn...
- 01:49 PM crimson Bug #65491 (In Progress): recover_missing: racing read got wrong version
- ...
- 01:12 PM rgw Bug #65486 (Fix Under Review): valgrind error on kafka shutdown
- 01:11 PM rgw Bug #65486 (Fix Under Review): valgrind error on kafka shutdown
- see: https://tracker.ceph.com/issues/65337#note-4
may cause crash on close. - 01:06 PM CephFS Bug #62123: mds: detect out-of-order locking
- This may also caused *MDS Behind on Trimming...*: https://www.mail-archive.com/ceph-users@ceph.io/msg24587.html.
- 12:24 PM bluestore Backport #65485: squid: bluestore/bluestore_types: check 'it' valid before using
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56891
ceph-backport.sh versi... - 12:10 PM bluestore Backport #65485 (New): squid: bluestore/bluestore_types: check 'it' valid before using
- 12:18 PM CephFS Feature #61866: MDSMonitor: require --yes-i-really-mean-it when failing an MDS with MDS_HEALTH_TRIM or MDS_HEALTH_CACHE_OVERSIZED health warnings
- Patrick, should we include other health warnings too? I didn't include it in PR because it was mentioned on this tick...
- 12:17 PM bluestore Backport #65484: reef: bluestore/bluestore_types: check 'it' valid before using
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56890
ceph-backport.sh versi... - 12:09 PM bluestore Backport #65484 (New): reef: bluestore/bluestore_types: check 'it' valid before using
- 12:16 PM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Venky Shankar wrote in #note-26:
> Dhairya Parmar wrote in #note-25:
> > Venky Shankar wrote in #note-24:
> > > Dh... - 10:52 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Dhairya Parmar wrote in #note-25:
> Venky Shankar wrote in #note-24:
> > Dhariya,
> >
> > Anything blocking w.r.... - 10:10 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Venky Shankar wrote in #note-24:
> Dhariya,
>
> Anything blocking w.r.t. the design for this enhancement? The lag... - 10:00 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Dhariya,
Anything blocking w.r.t. the design for this enhancement? The laggy OSD list is obviously something that ... - 12:14 PM bluestore Backport #65483: quincy: bluestore/bluestore_types: check 'it' valid before using
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56889
ceph-backport.sh versi... - 12:09 PM bluestore Backport #65483 (New): quincy: bluestore/bluestore_types: check 'it' valid before using
- 12:10 PM CephFS Bug #65157 (Can't reproduce): cephfs-mirror: set layout.pool_name xattr of destination subvol correctly
- Can't reproduce this:...
- 12:03 PM CephFS Backport #65316 (In Progress): squid: mds: CInode::item_caps used in two different lists
- https://github.com/ceph/ceph/pull/56887
- 12:03 PM CephFS Backport #65315 (In Progress): reef: mds: CInode::item_caps used in two different lists
- https://github.com/ceph/ceph/pull/56886
- 11:50 AM bluestore Bug #65482 (Pending Backport): bluestore/bluestore_types: check 'it' valid before using
- 11:47 AM bluestore Bug #65482 (Fix Under Review): bluestore/bluestore_types: check 'it' valid before using
- When sanitizer is enabled, unittest_bluestore_types fails as following
[ RUN ] sb_info_space_efficient_map_t.... - 10:27 AM crimson Feature #65478: Support SnapMapper::Scrubber
- junxiang mu wrote in #note-1:
> I can try implement this, can i tack this issue? :)
No problem!
I noticed that y... - 09:30 AM crimson Feature #65478: Support SnapMapper::Scrubber
- I can try implement this, can i tack this issue? :)
- 08:59 AM crimson Feature #65478 (New): Support SnapMapper::Scrubber
- We need to make crimson aware about SnapMapper::Scrubber and the purged snaps flow (track record_purged_snaps() in th...
- 10:23 AM CephFS Bug #61009: crash: void interval_set<T, C>::erase(T, T, std::function<bool(T, T)>) [with T = inodeno_t; C = std::map]: assert(p->first <= start)
- Explanation of the preallocated machinery which might help in the future:
I played around a bit more with prealloc... - 10:20 AM CephFS Bug #61009: crash: void interval_set<T, C>::erase(T, T, std::function<bool(T, T)>) [with T = inodeno_t; C = std::map]: assert(p->first <= start)
- Please see https://github.com/ceph/ceph/pull/53752#issuecomment-2056469527 for the status of the change.
This issu... - 09:09 AM ceph-volume Backport #65480 (In Progress): squid: prepare/create/activate refactor
- 09:06 AM ceph-volume Backport #65480 (In Progress): squid: prepare/create/activate refactor
- https://github.com/ceph/ceph/pull/56883
- 09:07 AM crimson Bug #57739 (Need More Info): crimson: LogMissingRequest and RepRequest operator<< access possibly invalid req
- 09:06 AM crimson Bug #57758 (Need More Info): crimson: disable autoscale for crimson in teuthology
- 09:05 AM crimson Bug #57801 (Resolved): crimson: tag pool types as crimson, disallow snapshot, scrub, ec operations
- 09:05 AM crimson Bug #64975 (Resolved): crimson: Health check failed: 9 scrub errors (OSD_SCRUB_ERRORS)" in cluster log'
- 09:03 AM Dashboard Bug #65479 (Fix Under Review): mgr/dashboard: use grafana server instead of grafana-server in grafana 10.4.0
- The grafana-server command is deprecated in grafana v10.4.0. It is advised to use grafan server in place of it.
- 09:02 AM crimson Bug #57990 (Closed): Crimson OSD crashes when trying to bring it up
- Yingxin Cheng wrote in #note-1:
> Crimson is not production ready yet, and there will be no backport to Quincy.
>
... - 09:01 AM crimson Bug #58391 (Need More Info): crimson-osd can't finish "mkfs" under RelWithDebInfo build type
- @rainman
Is this still relevant? - 08:59 AM ceph-volume Cleanup #61827 (Pending Backport): prepare/create/activate refactor
- 08:58 AM ceph-volume Cleanup #61827 (Fix Under Review): prepare/create/activate refactor
- 08:54 AM crimson Bug #61227 (Resolved): [crimson] ceph df stats are twice of actual values
- 08:52 AM ceph-volume Bug #65477 (Fix Under Review): `ceph-volume lvm prepare` does not create LVs anymore when using partitions
- 08:27 AM ceph-volume Bug #65477 (Fix Under Review): `ceph-volume lvm prepare` does not create LVs anymore when using partitions
- `ceph-volume lvm prepare` used to create VGs/LVs on partitions. This has changed with commit 1e7223281fa044c9653633e3...
- 08:50 AM crimson Bug #61875 (Resolved): crimson crashes during reboot when there are snap objects
- 08:50 AM crimson Bug #62526: during recovery crimson sends OI_ATTR with MAXed soid and kills classical OSDs
- @rzarzynski,
Is https://github.com/ceph/ceph/pull/53084 still relevant? - 08:48 AM crimson Bug #62550 (Resolved): osd crashes when doing peering
- 08:48 AM crimson Bug #63307 (Resolved): crimson: SnapTrimObjSubEvent doesn't actually seem to submit delta_stats
- 08:46 AM crimson Bug #64282 (Resolved): osd crashes due to unexpected pg creation
- 08:45 AM crimson Bug #64535 (Resolved): crimson osd crashes during crimson-rados-experimental teuthology tests
- 08:11 AM crimson Bug #64782 (Fix Under Review): test_python.sh TestIoctx.test_locator failes in cases of SeaStore
- 08:09 AM crimson Bug #65113 (Fix Under Review): crimson: SnapTrimObjSubEvent num_bytes stats calculation
- 08:07 AM crimson Bug #65130: crimson: crimson-rados did not detect reintroduction of https://tracker.ceph.com/issues/61875
- Added label: crimson-replicated-recovery to track all the required fixes
https://github.com/ceph/ceph/pulls?q=+is%... - 08:06 AM crimson Bug #65247 (Need More Info): ObjectContext::drop_recovery_read(): Assertion `recovery_read_marker' failed.
- 08:05 AM Dashboard Backport #65465 (In Progress): squid: mgr/dashboard: fixed snap schedule repeat frequency validation to prevent duplicates
- 08:05 AM crimson Feature #65288 (Fix Under Review): crimson: OSD support `trim stale osdmaps` socket command
- 08:04 AM Dashboard Backport #65464 (In Progress): reef: mgr/dashboard: fixed snap schedule repeat frequency validation to prevent duplicates
- 08:03 AM crimson Bug #65399 (Fix Under Review): osd crash due to deferred recovery
- 08:03 AM crimson Bug #65451 (Fix Under Review): tri_mutex::promote_from_read(): Assertion `readers == 1' failed.
- 08:02 AM crimson Bug #65453 (Fix Under Review): osd crashes due to outdated recovery ops
- 08:02 AM crimson Bug #65474 (Fix Under Review): mgr crash due to corrupted incremental osdmap sent by crimson-osds
- 08:01 AM crimson Feature #65476 (In Progress): Support Erasure coded pools
- 07:41 AM crimson Bug #64332 (In Progress): seastar submodule: Enable SEASTAR_GATE_HOLDER_DEBUG
- 05:40 AM Dashboard Backport #65168 (In Progress): quincy: mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- 05:34 AM Dashboard Backport #65170 (In Progress): reef: mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- 04:40 AM mgr Bug #59580: memory leak (RESTful module, maybe others?)
- waitting for https://github.com/ceph/ceph/pull/54984 merge and backport
04/14/2024
- 02:43 PM mgr Bug #59580: memory leak (RESTful module, maybe others?)
- Hi,
It seems that the ceph-mgr oom issue happened again on 16.2.15. We had ceph-mgr "oom" this morning.
I have ... - 11:01 AM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- h2. Analysis (WIP)
* the following test run is a sure way to create the ‘__header’ failure in ‘main’:
@./teutholo... - 01:39 AM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- I created a test branch with some extra logging and managed to reproduce the issue with slightly more info....
- 07:17 AM crimson Bug #65474 (Resolved): mgr crash due to corrupted incremental osdmap sent by crimson-osds
- ...
04/13/2024
- 12:05 AM rgw Bug #65473: rgw: exclude logging of request payer for 403 requests
- PR: https://github.com/ceph/ceph/pull/56868
- 12:02 AM rgw Bug #65473 (Fix Under Review): rgw: exclude logging of request payer for 403 requests
- As per AWS doc (https://docs.aws.amazon.com/AmazonS3/latest/userguide/RequesterPaysBuckets.html#ChargeDetails), reque...
04/12/2024
- 10:24 PM CephFS Bug #65472 (Pending Backport): mds: avoid recalling Fb when quiescing file
- To avoid extensive flushes by the client. (We don't need to trigger an fsync to quiesce a tree.)
See also: https:/... - 08:48 PM cleanup Tasks #65471 (Fix Under Review): rgw_sal_posix.cc printf compiler warnings
- ...
- 08:40 PM rgw Feature #65470 (New): Beast lacks ssl_short_trust option to reload ssl certificate without restart
- Previously civetweb rgw had an option (ssl_short_trust) to automatically reload certs, for instance when they are sho...
- 08:01 PM rgw Bug #65469 (Fix Under Review): rgw: increase log level on abort_early
- 07:59 PM rgw Bug #65469 (Pending Backport): rgw: increase log level on abort_early
- The function is typically invoked on client errors like NoSuchBucket. Logging these errors with level 1 may initially...
- 07:29 PM rgw Bug #65337: rgw: Segmentation fault in rgw::notify::Manager during realm reload
- the crash during the realm reload is due to connection being destroyed while its in use,
we call `kafka::shutdown` d... - 05:22 PM rgw Bug #65337: rgw: Segmentation fault in rgw::notify::Manager during realm reload
- @yuvalif the crash issue with kafka is all about the conn->destroyed being called while publish_internal() might be p...
- 05:00 PM rgw Bug #65337: rgw: Segmentation fault in rgw::notify::Manager during realm reload
- In our testing we are seeing the same crash, however we do not see it during the realm upload or shutdown.
Its just ... - 06:56 PM rgw Bug #65468: rgw: set correct requestId and hostId on s3select error
- PR: https://github.com/ceph/ceph/pull/56864
- 06:51 PM rgw Bug #65468 (Fix Under Review): rgw: set correct requestId and hostId on s3select error
- Previously, these fields remained constant despite the possibility of populating them with appropriate values.
- 06:12 PM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- Venky Shankar wrote in #note-31:
> [...]
>
> And patched up the yaml to use the custom quincy build to upgrade to... - 05:11 PM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- ...
- 10:18 AM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- I have a custom quincy branch (patched with debug in ceph-fuse/fuse_ll). That should give us enough debug to see what...
- 06:04 PM rgw Backport #65467 (In Progress): squid: rgw user accounts
- 05:58 PM rgw Backport #65467 (Resolved): squid: rgw user accounts
- https://github.com/ceph/ceph/pull/56863
- 05:56 PM rgw Feature #65466 (Resolved): rgw user accounts
- 05:18 PM rgw Bug #64381 (Resolved): iam role: CreateDate can go backwards
- 05:17 PM rgw Bug #64475 (Resolved): multisite: forwarded CreateRole request generates different CreateDate
- 03:35 PM rgw Bug #61772 (Closed): rgw/crypt/barbican: 'Namespace' object has no attribute 'admin_endpoints'
- 03:32 PM rgw-testing Bug #17776 (Closed): rgw: test aws4
- 03:05 PM Dashboard Backport #65465 (In Progress): squid: mgr/dashboard: fixed snap schedule repeat frequency validation to prevent duplicates
- https://github.com/ceph/ceph/pull/56881
- 03:05 PM Dashboard Backport #65464 (In Progress): reef: mgr/dashboard: fixed snap schedule repeat frequency validation to prevent duplicates
- https://github.com/ceph/ceph/pull/56880
- 03:01 PM Dashboard Bug #64980 (Pending Backport): mgr/dashboard: fixed snap schedule repeat frequency validation to prevent duplicates
- 02:59 PM Dashboard Backport #65459 (In Progress): reef: mgr/dashboard: fix snap schedule delete retention
- 11:52 AM Dashboard Backport #65459 (In Progress): reef: mgr/dashboard: fix snap schedule delete retention
- https://github.com/ceph/ceph/pull/56862
- 02:58 PM Dashboard Backport #65458 (In Progress): squid: mgr/dashboard: fix snap schedule delete retention
- 11:52 AM Dashboard Backport #65458 (In Progress): squid: mgr/dashboard: fix snap schedule delete retention
- https://github.com/ceph/ceph/pull/56861
- 02:57 PM rgw Bug #65463 (New): rgw/notifications: test data path v2 persistent migration fails
- from https://qa-proxy.ceph.com/teuthology/cbodley-2024-04-12_12:44:47-rgw-wip-rgw-account-v3-distro-default-smithi/76...
- 02:41 PM rgw Bug #65462: rgw: differentiate ssl enforcement based on encryption mode
- PR: https://github.com/ceph/ceph/pull/56860
- 02:40 PM rgw Bug #65462 (Fix Under Review): rgw: differentiate ssl enforcement based on encryption mode
- Implement distinct SSL enforcement configurations for SSE-S3, SSE-C, and SSE-KMS encryption methods.
This can be hel... - 02:23 PM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- I was confident of the code, I've mentioned this in https://tracker.ceph.com/issues/65265#note-6. I then raised a PR ...
- 01:32 PM Bug #65228 (In Progress): class:device-class config database mask does not work for osd_compact_on_start
- 01:28 PM cleanup Tasks #65460 (New): audit rgw_get_request_metadata(), stop storing unneccessary headers as xattrs
- @rgw_get_request_metadata()@ adds object/bucket xattrs for most of the headers in @x_meta_map@ (which stores any head...
- 12:14 PM CephFS Tasks #64819 (Resolved): data corruption during rmw after lseek
- The reproducers above were simplifications of failures/errors from running the ffsb test suite on a fscrypt enabled d...
- 12:09 PM CephFS Tasks #64819: data corruption during rmw after lseek
- As seen in note 2, the case of modification happening at start of block and contained within a single block is now ad...
- 12:11 PM CephFS Bug #62246: qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- Venky Shankar wrote in #note-12:
> Rishabh, do we need this for squid too?
Answering this myself - the PR was mer... - 08:21 AM CephFS Bug #62246: qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- Rishabh, do we need this for squid too?
- 11:44 AM Dashboard Bug #65370 (Pending Backport): mgr/dashboard: fix snap schedule delete retention
- 10:21 AM Dashboard Bug #65457: mgr/dashboard: ninja fails on `src/pybind/mgr/dashboard/frontend/dist`
- Sorry for the Chinese words.
核心已转储 should be core dump. - 09:57 AM Dashboard Bug #65457: mgr/dashboard: ninja fails on `src/pybind/mgr/dashboard/frontend/dist`
- Arch is arm. I don’t know if nodejs needs to adjust to the arm architecture.
- 09:41 AM Dashboard Bug #65457 (New): mgr/dashboard: ninja fails on `src/pybind/mgr/dashboard/frontend/dist`
- After I install deps, execute `./do_cmake.sh`, cd `bulid` and `ninja`.
It fails on dashboard frontend.
There is... - 09:12 AM CephFS Backport #63834 (Resolved): reef: mon/FSCommands: support swapping file systems by name
- 09:11 AM CephFS Backport #63407 (Resolved): quincy: cephfs: print better error message when MDS caps perms are not right
- 08:48 AM rgw Backport #65003 (Resolved): reef: [CVE-2023-46159] RGW crash upon misconfigured CORS rule
- 08:14 AM CephFS Bug #62188: AttributeError: 'RemoteProcess' object has no attribute 'read'
- Rishabh Dave wrote in #note-9:
> All the recent failures are from QA runs for Reef, this is because the fix for this... - 06:56 AM CephFS Bug #65455: read operation hung in Client::get_caps
- the ceph version is 15.2.17 and 16.2.14
- 06:55 AM CephFS Bug #65455 (Need More Info): read operation hung in Client::get_caps
- How to reproduce the scene
1. I used two nfs ganesha+libcephfs as the nfs server (server1, server2), and used the sa... - 06:40 AM RADOS Bug #59831: crash: void ECBackend::continue_recovery_op(ECBackend::RecoveryOp&, RecoveryMessages*): assert(pop.data.length() == sinfo.aligned_logical_offset_to_chunk_offset( after_progress.data_recovered_to - op.recovery_progress.data_recovered_to))
- I had the same problem with version 14.2.21，is there any progress...
- 05:59 AM crimson Bug #65453 (Fix Under Review): osd crashes due to outdated recovery ops
- PGs' recovery backends don't discard old recovery ops...
- 05:53 AM CephFS Bug #65246 (Fix Under Review): qa/cephfs: test_multifs_single_path_rootsquash (tasks.cephfs.test_admin.TestFsAuthorize)
- 05:18 AM CephFS Feature #57481 (Pending Backport): mds: enhance scrub to fragment/merge dirfrags
- 05:09 AM RADOS Bug #59670: Ceph status shows PG recovering when norecover flag is set
- We saw this issue again in another setup and it has been fixed here: https://github.com/ceph/ceph/pull/54708.
The p... - 03:04 AM Bug #65452: peer pg_info_t's last_complete in primary pg cannot be updated
- !clipboard-202404121104-m5hdh.png!
- 03:03 AM Bug #65452: peer pg_info_t's last_complete in primary pg cannot be updated
- !clipboard-202404121103-bsweb.png!
- 03:02 AM Bug #65452: peer pg_info_t's last_complete in primary pg cannot be updated
- !clipboard-202404121102-9u6kt.png!
- 03:01 AM Bug #65452: peer pg_info_t's last_complete in primary pg cannot be updated
- case:primary osd executes do_osd_ops write fail and need to execute record_write_error. In record_write_error functio...
- 02:47 AM Bug #65452 (New): peer pg_info_t's last_complete in primary pg cannot be updated
- !clipboard-202404121047-ovd7r.png!
- 02:07 AM crimson Bug #65451: tri_mutex::promote_from_read(): Assertion `readers == 1' failed.
- Probably can be addressed by https://github.com/ceph/ceph/commit/3a6332fd6676da590b9ede46954b2a6a74308bd7, will split...
- 01:58 AM crimson Bug #65451 (Fix Under Review): tri_mutex::promote_from_read(): Assertion `readers == 1' failed.
- See the assert failure in osd.1 from https://pulpito.ceph.com/yingxin-2024-04-11_01:17:19-crimson-rados-ci-yingxin-cr...
04/11/2024
- 11:03 PM RADOS Bug #65450: rados/thrash-old-clients: "PG_BACKFILL: Low space hindering backfill" warning in cluster log
- Should be evaluated to see whether this should be added to the ignorelist, or if it points to a larger bug.
- 11:03 PM RADOS Bug #65450 (New): rados/thrash-old-clients: "PG_BACKFILL: Low space hindering backfill" warning in cluster log
- /a/yuriw-2024-04-09_14:58:25-rados-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7649192...
- 10:59 PM RADOS Bug #65449 (Fix Under Review): NeoRadosWatchNotify.WatchNotifyTimeout failed due to nonexistent pool
- /a/yuriw-2024-04-09_14:58:25-rados-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7649011...
- 10:49 PM Dashboard Bug #64377: tasks/e2e: Modular dependency problems
- /a/yuriw-2024-04-09_14:58:25-rados-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7649198
- 10:44 PM Orchestrator Bug #52109: test_cephadm.sh: Timeout('Port 8443 not free on 127.0.0.1.',)
- /a/yuriw-2024-04-09_14:58:25-rados-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648958
- 10:42 PM ceph-volume Bug #56620: Deploy a ceph cluster with cephadm，using ceph-volume lvm create command to create osd can not managed by cephadm
- Looks like a case of this:
/a/yuriw-2024-04-09_14:58:25-rados-wip-yuri4-testing-2024-04-08-1432-distro-default-smith... - 09:49 PM Orchestrator Bug #65233: upgrade/cephfs/mds_upgrade_sequence: 'ceph orch ps' command times out
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648940
- 09:44 PM rgw Backport #65427 (In Progress): squid: Admin Ops socket crashes RGW
- 02:37 PM rgw Backport #65427 (Resolved): squid: Admin Ops socket crashes RGW
- https://github.com/ceph/ceph/pull/56840
- 06:50 PM CephFS Bug #62188 (Duplicate): AttributeError: 'RemoteProcess' object has no attribute 'read'
- All the recent failures are from QA runs for Reef, this is because the fix for this issue (https://tracker.ceph.com/i...
- 06:24 PM CephFS Backport #65441 (New): quincy: qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- 06:24 PM CephFS Backport #65440 (New): reef: qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- 06:22 PM CephFS Bug #62246 (Pending Backport): qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- 05:33 PM CephFS Bug #62246: qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- *The PR linked here fixes multiple issues. This specific commit
from the PR fixes the issue -
https://github.com/ceph... - 05:32 PM CephFS Bug #62246 (Resolved): qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- 05:14 PM rgw Bug #65436 (Need More Info): Getting Object Crashing radosgw services
- Hello,
We are seeing crashes when users are trying to get a specific file.... - 02:48 PM rgw Feature #63915 (New): propagate kafka errors to client in case of sync notifications
- 02:37 PM rgw Backport #65426 (New): quincy: Admin Ops socket crashes RGW
- 02:36 PM rgw Backport #65425 (New): reef: Admin Ops socket crashes RGW
- 02:30 PM rgw Bug #64244 (Pending Backport): Admin Ops socket crashes RGW
- 02:20 PM rgw Cleanup #63962 (New): rgw-file: FLAG_SYMBOLIC_LINK decl aliases other flags
- 02:17 PM rgw Bug #64805 (Fix Under Review): rgw: dynamic resharding will block write op
- 02:11 PM rgw Bug #61710 (Won't Fix): quincy/pacific: PUT requests during reshard of versioned bucket fail with 404 and leave behind dark data
- 02:05 PM rgw Bug #63378 (New): rgw/multisite: Segmentation fault during full sync
- 01:55 PM CephFS Bug #65261: qa/cephfs: cephadm related failure on fs/upgrade job
- https://pulpito.ceph.com/rishabh-2024-04-08_08:23:45-fs-wip-rishabh-testing-20240407.092921-reef-testing-default-smit...
- 01:13 PM CephFS Backport #62425 (Fix Under Review): reef: nofail option in fstab not supported
- 01:12 PM CephFS Backport #62426 (Fix Under Review): quincy: nofail option in fstab not supported
- 01:12 PM CephFS Backport #63362 (Fix Under Review): quincy: mds: create an admin socket command for raising a signal
- 01:12 PM CephFS Backport #63363 (Fix Under Review): reef: mds: create an admin socket command for raising a signal
- 01:11 PM CephFS Backport #63479 (Fix Under Review): reef: src/mds/MDLog.h: 100: FAILED ceph_assert(!segments.empty())
- 01:11 PM CephFS Backport #63480 (Fix Under Review): quincy: src/mds/MDLog.h: 100: FAILED ceph_assert(!segments.empty())
- 01:11 PM CephFS Backport #63822 (Fix Under Review): reef: cephfs/fuse: renameat2 with flags has wrong semantics
- 01:10 PM CephFS Tasks #63669 (Fix Under Review): qa: add teuthology tests for quiescing a group of subvolumes
- 12:19 PM CephFS Bug #64977 (Fix Under Review): mds spinlock due to lock contention leading to memory exaustion
- 11:25 AM rbd Bug #65421 (Duplicate): upgrade/reef-x/stress-split: TestMigration.StressLive failure
- This isn't specific to upgrade/reef-x/stress-split -- no need to track separately.
- 09:07 AM CephFS Bug #65423: Monitor crashes down when I try to create a FS. The stacks maybe related to metadata server map decoder during the PAXOS service
- fuchen ma wrote in #note-1:
> Another information:
> I found that the version of the non-crashed is 18.2.2, and the... - 09:06 AM CephFS Bug #65423: Monitor crashes down when I try to create a FS. The stacks maybe related to metadata server map decoder during the PAXOS service
- Another information:
I found that the version of the non-crashed is 18.2.2, and the version of the crashed ones are ... - 08:35 AM CephFS Bug #65423 (Need More Info): Monitor crashes down when I try to create a FS. The stacks maybe related to metadata server map decoder during the PAXOS service
- I have created a ceph cluster with 5 monitors and 2 metadata servers.
After that, I want to create a fs. Thus, I use... - 09:03 AM Orchestrator Documentation #65424 (New): hardware-monitoring/#developers is broken
- https://docs.ceph.com/en/latest/hardware-monitoring/#developpers
It just contains a bunch of python-mock doc stri... - 06:11 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Dhairya mentioned that the tracebacks seems in the mgr logs are logged by object formatter and not necessarily unhand...
- 04:30 AM Orchestrator Backport #65414 (In Progress): squid: cephadm: Health check failed: 1 osds down (OSD_DOWN) in cluster log
04/10/2024
- 10:15 PM RADOS Bug #65422 (New): upgrade/quincy-x/parallel: "1 pg degraded (PG_DEGRADED)" in cluster log
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648908...
- 10:01 PM rbd Bug #65421 (Duplicate): upgrade/reef-x/stress-split: TestMigration.StressLive failure
- ...
- 09:11 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648890
/a/yuriw-2024-0... - 07:26 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648938/remote/smithi122...
- 05:13 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- Laura Flores wrote:
> /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616025/remote/smithi098... - 08:59 PM CephFS Bug #64707: suites/fsstress.sh hangs on one client - test times out
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648870
- 07:46 PM CephFS Fix #65408 (Fix Under Review): qa: under valgrind, restart valgrind/mds when MDS exits with 0
- So, the mds_valgrind_exit already exists and is turned on. The original problem in #65314 wasn't caused by a failover...
- 05:18 PM CephFS Fix #65408: qa: under valgrind, restart valgrind/mds when MDS exits with 0
- @vshankar test
- 05:18 PM CephFS Fix #65408: qa: under valgrind, restart valgrind/mds when MDS exits with 0
- @pdonnell test
- 01:33 PM CephFS Fix #65408: qa: under valgrind, restart valgrind/mds when MDS exits with 0
- (Trying to see if redmine adds Venky to the "Watchers" list)
- 01:32 PM CephFS Fix #65408: qa: under valgrind, restart valgrind/mds when MDS exits with 0
- test @vshankar
- 01:31 PM CephFS Fix #65408: qa: under valgrind, restart valgrind/mds when MDS exits with 0
- test @vshankar
- 01:27 PM CephFS Fix #65408: qa: under valgrind, restart valgrind/mds when MDS exits with 0
- cc @vshankar
- 01:27 PM CephFS Fix #65408 (Fix Under Review): qa: under valgrind, restart valgrind/mds when MDS exits with 0
- Instead of issuing a re-...
- 07:31 PM Orchestrator Feature #65398: allow images from private repos in teuthology test/ceph orch/cephadm
- it's possible that the intent was to preface the call to pull_image with something that logs into the repo on the rem...
- 05:29 PM Orchestrator Feature #65398: allow images from private repos in teuthology test/ceph orch/cephadm
- Thanks, I see that the pull_image function doesn't honor those settings currently. I have some other somewhat related...
- 04:56 PM Orchestrator Feature #65398: allow images from private repos in teuthology test/ceph orch/cephadm
- the command that was failing was cephadm.py:pull_image, which invokes sudo cephadm --image <name> pull. I'm not 100%...
- 02:11 PM Orchestrator Feature #65398: allow images from private repos in teuthology test/ceph orch/cephadm
- In theory it should work. The code in the task translates the yaml paramaters to cli parameters for bootstrap. Here's...
- 02:02 AM Orchestrator Feature #65398 (New): allow images from private repos in teuthology test/ceph orch/cephadm
- It appears as though the cephadm teuthology task supports private registries (those that require username/password lo...
- 07:06 PM rgw Backport #65351 (Fix Under Review): squid: rgw: crash in lc while transitioning to cloud
- 06:51 PM CephFS Tasks #64819: data corruption during rmw after lseek
- There's also another case that is failing rmw:
reproducer:... - 06:31 PM CephFS Tasks #64819: data corruption during rmw after lseek
- The RC for this issue is fixed by:...
- 05:39 PM Orchestrator Backport #65419 (New): quincy: cephadmin returns "1" on successful host-maintenance enter/exit - should return "0"
- 05:39 PM Orchestrator Backport #65418 (New): reef: cephadmin returns "1" on successful host-maintenance enter/exit - should return "0"
- 05:39 PM Orchestrator Backport #65417 (In Progress): squid: cephadmin returns "1" on successful host-maintenance enter/exit - should return "0"
- https://github.com/ceph/ceph/pull/56903
- 05:37 PM Orchestrator Bug #65122 (Pending Backport): cephadmin returns "1" on successful host-maintenance enter/exit - should return "0"
- 05:37 PM RADOS Bug #64460: rados/upgrade/parallel: "[WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648863
- 05:33 PM Orchestrator Bug #64868: cephadm/osds, cephadm/workunits: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) in cluster log
- Also during stress/split: yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7...
- 05:31 PM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648857
- 05:31 PM Orchestrator Backport #65416 (New): reef: cephadm: test_cephadm script fails with "ERROR: required file missing from config-json: idmap.conf"
- 05:31 PM Orchestrator Backport #65415 (Resolved): squid: cephadm: test_cephadm script fails with "ERROR: required file missing from config-json: idmap.conf"
- https://github.com/ceph/ceph/pull/56902
- 05:30 PM Orchestrator Bug #65155 (Pending Backport): cephadm: test_cephadm script fails with "ERROR: required file missing from config-json: idmap.conf"
- 05:29 PM RADOS Bug #65235: upgrade/reef-x/stress-split: "OSDMAP_FLAGS: noscrub flag(s) set" warning in cluster log
- There are many instances of this flag getting set in the test run intentionally, so it makes sense to whitelist.
<pr... - 05:27 PM nvme-of Feature #65259 (Resolved): cephadm - make changes to ceph-nvmeof.conf template
- 05:27 PM nvme-of Backport #65296 (Rejected): squid: cephadm - make changes to ceph-nvmeof.conf template
- Handling this backport as part of https://github.com/ceph/ceph/pull/56497 that includes other changes to the nvmeof c...
- 05:26 PM Orchestrator Bug #65234: upgrade/quincy-x/stress-split: cephadm failed to parse grafana.ini file due to inadequate permission
- /a/yuriw-2024-04-09_14:58:21-upgrade-wip-yuri4-testing-2024-04-08-1432-distro-default-smithi/7648854
- 05:23 PM Orchestrator Backport #65414 (Resolved): squid: cephadm: Health check failed: 1 osds down (OSD_DOWN) in cluster log
- https://github.com/ceph/ceph/pull/56826
- 05:17 PM Orchestrator Bug #64865 (Pending Backport): cephadm: Health check failed: 1 osds down (OSD_DOWN) in cluster log
- 05:10 PM CephFS Bug #50719: xattr returning from the dead (sic!)
- Those MDs logs would be everything. they are from the moment I built the MDS services until you requested the logs wh...
- 04:41 PM rgw Backport #65412 (In Progress): squid: multisite: test_object_sync gets wrong object body: b'<x-rgw' != b'asdasd'
- 04:40 PM rgw Backport #65412 (Resolved): squid: multisite: test_object_sync gets wrong object body: b'<x-rgw' != b'asdasd'
- https://github.com/ceph/ceph/pull/56822
- 04:36 PM rgw Bug #65373 (Pending Backport): multisite: test_object_sync gets wrong object body: b'<x-rgw' != b'asdasd'
- 02:51 PM rgw Bug #63791: RGW: a subuser with no permission can still list buckets and create buckets
- This commit can be backported to quincy reef ?
- 02:25 PM rgw Bug #63791 (Resolved): RGW: a subuser with no permission can still list buckets and create buckets
- 02:40 PM mgr Backport #65154: quincy: pybind/mgr/devicehealth: "rados.ObjectNotFound: [errno 2] RADOS object not found (Failed to operate read op for oid $dev"
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56480
merged - 02:38 PM CephFS Backport #63823 (Fix Under Review): quincy: cephfs/fuse: renameat2 with flags has wrong semantics
- 02:36 PM bluestore Backport #63914 (Resolved): quincy: Some of ObjectStore/*Deferred* test cases are failing with bluestore_allocator is set to bitmap
- 02:32 PM Feature #63801: verified mon backups
- Christian Rohmann wrote in #note-2:
> My thoughts would be:
> * Full restore might not always be wanted, so extra... - 02:32 PM Dashboard Backport #65026: quincy: mgr/dashboard: Develop a Chinese version for dashboard
- Rongqi Sun wrote in #note-2:
> please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/p... - 02:31 PM bluestore Bug #63795: Some of ObjectStore/*Deferred* test cases are failing with bluestore_allocator is set to bitmap
- https://github.com/ceph/ceph/pull/55779 merged
- 02:24 PM Feature #64436 (Fix Under Review): rgw: add remaining x-amz-replication-status options
- 01:38 PM rgw Backport #65411 (In Progress): squid: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- 01:36 PM rgw Backport #65411 (Resolved): squid: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- https://github.com/ceph/ceph/pull/56820
- 01:38 PM rgw Backport #65410 (In Progress): reef: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- 01:36 PM rgw Backport #65410 (In Progress): reef: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- https://github.com/ceph/ceph/pull/56819
- 01:38 PM rgw Backport #65409 (In Progress): quincy: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- 01:36 PM rgw Backport #65409 (In Progress): quincy: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- https://github.com/ceph/ceph/pull/56818
- 01:32 PM rgw Bug #65334 (Pending Backport): Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- 01:24 PM CephFS Bug #65262 (Triaged): qa/cephfs: kernel_untar_build.sh failed due to build error
- 01:17 PM rgw Backport #65402 (In Progress): squid: persistent topic stats test fails
- backport included in https://github.com/ceph/ceph/pull/56069 for https://tracker.ceph.com/issues/64818
- 10:39 AM rgw Backport #65402 (Resolved): squid: persistent topic stats test fails
- 12:56 PM CephFS Bug #65350 (Triaged): mgr/snap_schedule: restore yearly spec from uppercase Y to lowercase y
- 12:29 PM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- @ NotADirectoryError@ is probably not a valid (in-built) exception in some python version. My question is, if this ex...
- 08:40 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Venky Shankar wrote in #note-3:
> Thanks for taking a look, Laura.
>
> Dhariya, please take this one. AFAICT, thi... - 12:23 PM Orchestrator Bug #65407: sequence item 0: expected str instance, dict found
- /var/log/user.log:Apr 10 15:03:33 d3p1u01-rc9h7j020-01 ceph-mgr[4176565]: [cephadm ERROR cephadm.serve] Failed to app...
- 12:20 PM Orchestrator Bug #65407 (New): sequence item 0: expected str instance, dict found
- ceph version 17.2.4 (1353ed37dec8d74973edc3d5d5908c20ad5a7332) quincy (stable)
ceph orch apply -i osd_ssd.yaml
<pre... - 12:01 PM CephFS Backport #65406 (New): quincy: mds: Reduce log level for messages when mds is stopping
- 12:01 PM CephFS Backport #65405 (New): reef: mds: Reduce log level for messages when mds is stopping
- 12:01 PM CephFS Backport #65404 (New): squid: mds: Reduce log level for messages when mds is stopping
- 11:57 AM CephFS Bug #65260 (Pending Backport): mds: Reduce log level for messages when mds is stopping
- 11:44 AM CephFS Bug #56288: crash: Client::_readdir_cache_cb(dir_result_t*, int (*)(void*, dirent*, ceph_statx*, long, Inode*), void*, int, bool)
- Venky Shankar wrote in #note-18:
> So, for some reason this part of the code
>
> [...]
>
> especially derefere... - 11:34 AM CephFS Bug #56288: crash: Client::_readdir_cache_cb(dir_result_t*, int (*)(void*, dirent*, ceph_statx*, long, Inode*), void*, int, bool)
- So, for some reason this part of the code...
- 07:58 AM CephFS Bug #56288: crash: Client::_readdir_cache_cb(dir_result_t*, int (*)(void*, dirent*, ceph_statx*, long, Inode*), void*, int, bool)
- I haven't been unable to reproduce this with the main branch. If possible, please collect ceph-mds coredump and attac...
- 11:28 AM CephFS Bug #65317 (Fix Under Review): cephfs_mirror: update peer status for invalid metadata in remote snapshot
- 11:06 AM RADOS Backport #65307 (In Progress): quincy: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- 11:06 AM RADOS Backport #65306 (In Progress): squid: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- 11:05 AM RADOS Backport #65305 (In Progress): reef: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- 10:47 AM CephFS Bug #48680: mds: scrubbing stuck "scrub active (0 inodes in the stack)"
- This might be due to enabling of frags as seen in the job description for the job mentioned in comment#4 and probably...
- 10:39 AM rgw Backport #65403 (New): reef: persistent topic stats test fails
- 10:37 AM rgw Bug #65354 (Duplicate): rgw/notifications: topic migration test failures
- the issues above are failures due to test issues that were fixed here: https://tracker.ceph.com/issues/63909
sometim... - 10:32 AM CephFS Bug #65171 (Fix Under Review): Provide metrics support for the Replication Start/End Notifications
- 10:31 AM rgw Bug #63909 (Pending Backport): persistent topic stats test fails
- 10:19 AM RADOS Feature #54525: osd/mon: log memory usage during tick
- PR: https://github.com/ceph/ceph/pull/56812
- 10:15 AM RADOS Bug #58893: test_map_discontinuity: AssertionError: wait_for_clean: failed before timeout expired
in: /a/yuriw-2024-04-02_15:39:50-rados-wip-yuri2-testing-2024-04-01-1235-quincy-distro-default-smithi/7636676
th...- 05:34 AM RADOS Bug #58893: test_map_discontinuity: AssertionError: wait_for_clean: failed before timeout expired
- /a/yuriw-2024-04-02_15:39:50-rados-wip-yuri2-testing-2024-04-01-1235-quincy-distro-default-smithi/7636676
- 09:46 AM rgw Bug #65337: rgw: Segmentation fault in rgw::notify::Manager during realm reload
- the valgrind report indicates a crash during sutdown. when we shutdown the kafka manager, we destroy all connections,...
- 09:44 AM Messengers Bug #65401: msg: conneciton between mgr and osd is periodically down which leads heavy load to mgr
- I'm not sure this is by designed or a mistake, so I push a pr for disccussion. pr：https://github.com/ceph/ceph/pull/5...
- 09:26 AM Messengers Bug #65401 (New): msg: conneciton between mgr and osd is periodically down which leads heavy load to mgr
- I find the connection between osd and mgr are periodically mark_down due to ms_connection_idle_timeout config.
This ... - 08:56 AM Dashboard Feature #65268 (Resolved): mgr/dashboard: update NVMe-oF API "listener add" sync
- 08:56 AM Dashboard Backport #65390 (Resolved): squid: mgr/dashboard: update NVMe-oF API "listener add" sync
- 08:33 AM Bug #52604 (Closed): osd: mkfs: bluestore_stored > 235GiB from start
- The fix was merged
- 07:55 AM RADOS Bug #64519: OSD/MON: No snapshot metadata keys trimming
- Eugen Block wrote in #note-6:
> I know I'm a bit early asking this, but I helped raise this issue and Mykola picked ... - 07:45 AM Bug #65400 (New): ceph-exporter
- During the run of the ocs-ci tests (for example "test_fsgroupchangepolicy_when_depoyment_scaled") we receive the foll...
- 07:02 AM bluestore Bug #65298: Free space can be leaked in Quincy+ when bdev_async_discard is enabled
- PR https://github.com/ceph/ceph/pull/56744 should solve this issue
- 06:40 AM crimson Bug #65399 (Fix Under Review): osd crash due to deferred recovery
- Crimson OSD will fail if a recovery op is finished after a recovery/backfill is deferred:...
- 05:40 AM RADOS Bug #50222: osd: 5.2s0 deep-scrub : stat mismatch
- /a/yuriw-2024-04-02_15:39:50-rados-wip-yuri2-testing-2024-04-01-1235-quincy-distro-default-smithi/7636628
- 05:37 AM RADOS Bug #64725: rados/singleton: application not enabled on pool 'rbd'
- /a/yuriw-2024-04-02_15:39:50-rados-wip-yuri2-testing-2024-04-01-1235-quincy-distro-default-smithi/7636638
/a/yuriw-2... - 04:47 AM CephFS Bug #64977: mds spinlock due to lock contention leading to memory exaustion
- The *client.379194623:32785 lookup* request was spinning infinitely in MDS:...
- 03:05 AM CephFS Bug #62123 (Fix Under Review): mds: detect out-of-order locking
- 01:11 AM rgw Bug #64803 (Fix Under Review): ninja all on fedora 39 fails because arrow_ext requires C++14
04/09/2024
- 11:50 PM rgw Bug #65397: rgw: allow disabling mdsearch APIs
- PR: https://github.com/ceph/ceph/pull/56802
- 11:48 PM rgw Bug #65397 (Fix Under Review): rgw: allow disabling mdsearch APIs
- Since this is visible to the bucket owners, it can be presumed to be a functional feature. Providing the ability to d...
- 07:31 PM rgw Bug #65337: rgw: Segmentation fault in rgw::notify::Manager during realm reload
- i managed to reproduce under valgrind. this report of use-after-free looks relevant:...
- 06:54 PM Orchestrator Bug #64208 (In Progress): test_cephadm.sh: Container version mismatch causes job to fail.
- 06:36 PM Orchestrator Bug #65396 (New): smb service takes a very long time to delete
- Executing `ceph orch rm smb.foo` gets stuck in `<deleting>` phase.
I suspect that there may be an issue removing s... - 04:23 PM mgr Feature #64318: mgr/prometheus add support for TLS and client cert authentication
- Redouane Kachach Elhichou wrote in #note-5:
> Christian Rohmann wrote:
> > Redouane Kachach Elhichou wrote:
> > ... - 04:18 PM RADOS Bug #65227: noscrub cluster flag prevents deep-scrubs from starting
- https://github.com/ceph/ceph/blob/main/doc/dev/osd_internals/scrub.rst
https://github.com/ceph/ceph/blob/v17.2.7/src... - 03:31 PM Orchestrator Bug #65367 (Resolved): PermissionError: [Errno 13] Permission denied in the fake filesystem
- all 4 PRs are now merged. This should no longer occur in any make check runs started after this point.
- 03:16 PM Orchestrator Bug #65395 (Fix Under Review): [node-proxy] the agent shouldn't fail when RedFish returns empty data
- 03:11 PM Orchestrator Bug #65395 (Fix Under Review): [node-proxy] the agent shouldn't fail when RedFish returns empty data
- If for some reason the redfish returns empty data, node-proxy fails because it can't access non-existing keys, it bas...
- 03:06 PM Stable releases Tasks #65393: reef v18.2.3
- h3. QE VALIDATION (STARTED 4/8/23)
PRs list => https://pad.ceph.com/p/reef_v18.2.3_QE_PRs_LIST
*%{color:blue}Releas... - 03:02 PM Stable releases Tasks #65393 (New): reef v18.2.3
- h3. Workflow
* "Preparing the release":http://ceph.com/docs/master/dev/development-workflow/#preparing-a-new-relea... - 03:05 PM Orchestrator Feature #65394 (In Progress): [node-proxy] implement 'endpoints discovering'
- RFE in order to add the required logic in order to make the daemon explore the API for discovering the different endp...
- 03:03 PM Orchestrator Bug #65392 (Fix Under Review): [node-proxy] the node-proxy daemon crashes when get_logger() is passed a log level
- 02:48 PM Orchestrator Bug #65392 (Fix Under Review): [node-proxy] the node-proxy daemon crashes when get_logger() is passed a log level
- ...
- 02:52 PM Backport #65368 (Resolved): squid: install-deps: enable copr ceph/grpc
- 11:28 AM Backport #65368 (In Progress): squid: install-deps: enable copr ceph/grpc
- 02:51 PM Bug #65184 (Resolved): install-deps: enable copr ceph/grpc
- 02:41 PM rgw Bug #65334: Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- the barbican repo changed the name of this branch to @unmaintained/xena@
- 02:34 PM Backport #65391 (In Progress): squid: osd/scrub: "reservation requested while still reserved" error in cluster log
- 02:15 PM Backport #65391 (In Progress): squid: osd/scrub: "reservation requested while still reserved" error in cluster log
- 02:10 PM Bug #64827 (Pending Backport): osd/scrub: "reservation requested while still reserved" error in cluster log
- 12:39 PM rgw Bug #64308: CORS Preflight Failure After Upgrading to 17.2.7
- Will the backports make it into the next release of Quincy/Reef?
- 11:54 AM Dashboard Backport #65390 (In Progress): squid: mgr/dashboard: update NVMe-oF API "listener add" sync
- 09:36 AM Dashboard Backport #65390 (Resolved): squid: mgr/dashboard: update NVMe-oF API "listener add" sync
- https://github.com/ceph/ceph/pull/56783
- 09:55 AM CephFS Bug #64977: mds spinlock due to lock contention leading to memory exaustion
- Posted more logs at fed9e44e-a0ec-4692-ae23-6a1047fe9247
- 09:29 AM Dashboard Feature #65268 (Pending Backport): mgr/dashboard: update NVMe-oF API "listener add" sync
- 08:41 AM Bug #61598 (Duplicate): gcc-14: FTBFS "error: call to non-'constexpr' function 'virtual unsigned int DoutPrefixProvider::get_subsys() const'"
- 08:08 AM Feature #63801: verified mon backups
- *This is really a good idea to have built-in! Thanks for taking this up!*
We have been using a custom backup scrip... - 08:02 AM CephFS Bug #65389 (New): The ceph_readdir function in libcephfs returns incorrect d_reclen value
- When @struct dirent@ entries are returned by @ceph_readdir()@ function, the field @d_reclen@ is always 1.
Based on... - 07:19 AM CephFS Bug #65388 (New): The MDS_SLOW_REQUEST warning is flapping even though the slow requests don't go away
- I have caught a cluster in an unhealthy state - probably some MDS deadlock that results in requests being blocked (de...
- 07:14 AM CephFS Bug #65171 (In Progress): Provide metrics support for the Replication Start/End Notifications
- 03:14 AM Support #64378: Slow / Single backfilling on Reef (18.2.1-pve2)
- Aha, there's a new feature in Ceph that auto-resets these values:
https://docs.ceph.com/en/quincy/rados/configurat... - 01:53 AM Support #64378: Slow / Single backfilling on Reef (18.2.1-pve2)
- I observe the same problem on 18.2.1:...
- 02:57 AM Linux kernel client Bug #51279: kclient hangs on umount (testing branch)
- I have added more debug logs and will dump why the *flushsnap_ack* was dropped directly:...
- 01:48 AM Orchestrator Bug #65387 (New): cephadm: Unable to use gather-facts without podman/docker installed
- cephadm gather-facts can be used to gather inventory across the hosts to validate hardware prior to deployment. Howev...
- 12:22 AM Bug #65386 (New): rados: create test to validate replica read
- RADOS supports the ability to send reads to replicas rather than the primary. The primary use for this feature is to...
04/08/2024
- 11:23 PM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- Taking this back.
- 06:16 PM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- The fix is in QA.
- 10:35 PM CephFS Backport #65066 (Resolved): squid: make check: QuiesceDbTest.MultiRankRecovery Failed
- merged
- 09:22 PM crimson Feature #65384 (New): seastar/crimson objecter
- Create an objecter implementation usable from crimson-osd. This would mean integrating with the seastar reactor and ...
- 08:35 PM RADOS Bug #52657: MOSDPGLog::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_NAUTILUS)'
- Fix approved.
- 05:49 PM RADOS Bug #52657: MOSDPGLog::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_NAUTILUS)'
- Comments addressed, rereview requested.
- 08:21 PM Orchestrator Backport #65383 (In Progress): reef: NLM should be enabled in NFS-Ganesha config file for locking functionality to work with v3 protocol
- https://github.com/ceph/ceph/pull/56909
- 08:21 PM Orchestrator Backport #65382 (Resolved): squid: NLM should be enabled in NFS-Ganesha config file for locking functionality to work with v3 protocol
- https://github.com/ceph/ceph/pull/56901
- 08:09 PM Orchestrator Feature #65144 (Pending Backport): NLM should be enabled in NFS-Ganesha config file for locking functionality to work with v3 protocol
- 07:59 PM Orchestrator Backport #65381 (In Progress): squid: upgrade/quincy-x/stress-split: cephadm failed to parse grafana.ini file due to inadequate permission
- https://github.com/ceph/ceph/pull/56900
- 07:57 PM Orchestrator Bug #65234 (Pending Backport): upgrade/quincy-x/stress-split: cephadm failed to parse grafana.ini file due to inadequate permission
- 07:52 PM Orchestrator Backport #65380 (New): quincy: cephadm: client-keyring also overwrites ceph.conf
- 07:51 PM Orchestrator Backport #65379 (New): reef: cephadm: client-keyring also overwrites ceph.conf
- 07:51 PM Orchestrator Backport #65378 (Resolved): squid: cephadm: client-keyring also overwrites ceph.conf
- https://github.com/ceph/ceph/pull/56899
- 07:49 PM Orchestrator Bug #65335 (Pending Backport): cephadm: client-keyring also overwrites ceph.conf
- 07:09 PM Orchestrator Bug #65367 (In Progress): PermissionError: [Errno 13] Permission denied in the fake filesystem
- This is an issue on all branches. I have opened a patch to pin pyfakefs to fix this on each branch.
main PR: https... - 07:10 AM Orchestrator Bug #65367 (Resolved): PermissionError: [Errno 13] Permission denied in the fake filesystem
- e.g.
https://jenkins.ceph.com/job/ceph-pull-requests/132835/consoleFull
Shows:
if winerror is not N... - 06:39 PM RADOS Backport #65377 (In Progress): reef: crash: void PaxosService::propose_pending(): assert(have_pending)
- https://github.com/ceph/ceph/pull/57067
- 06:39 PM RADOS Backport #65376 (In Progress): quincy: crash: void PaxosService::propose_pending(): assert(have_pending)
- https://github.com/ceph/ceph/pull/57068
- 06:39 PM CephFS Bug #61749 (Resolved): mds/MDSRank: op_tracker of mds have slow op alway.
- 06:38 PM CephFS Backport #61993 (Resolved): reef: mds/MDSRank: op_tracker of mds have slow op alway.
- 06:16 PM CephFS Backport #61993: reef: mds/MDSRank: op_tracker of mds have slow op alway.
- merged
- 06:36 PM CephFS Bug #61732 (Resolved): pacific: test_cluster_info fails from "No daemons reported"
- 06:36 PM rgw Bug #61359 (Resolved): Consistency bugs with OLH objects
- 06:34 PM RADOS Bug #59813 (Pending Backport): crash: void PaxosService::propose_pending(): assert(have_pending)
- 06:32 PM ceph-volume Bug #59375 (Pending Backport): ceph-volume should support symbolic links to devices e.g. for multipath
- 06:32 PM RADOS Bug #59670: Ceph status shows PG recovering when norecover flag is set
- Bump up. IIRC there was a very similar ticket Aishwarya has poked with.
- 06:30 PM RADOS Bug #63891: mon/AuthMonitor: fix potential repeated global_id
- Bump up.
- 06:28 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- This tracker seems to be another one about the @deep_scrub@ story. If so, it should be fixed with the upcoming fix.
- 06:26 PM rgw Bug #21705 (Won't Fix): civetweb: setgid/groups/uid fails if multiple civetweb frontends declared
- Civetweb was removed since Quincy
- 06:22 PM devops Bug #15872 (Resolved): Removing ceph packages should remove 'ceph' user and 'ceph' group as well
- 06:19 PM RADOS Bug #56393: failed to complete snap trimming before timeout
- The PR has been merged on March 25th; the run is from 24th – the hope has not died (yet) ;-).
- 06:18 PM CephFS Backport #65294: reef: High cephfs MDS latency and CPU load with snapshots and unlink operations
- merged
- 06:18 PM CephFS Backport #62287: reef: ceph_test_libcephfs_reclaim crashes during test
- merged
- 06:17 PM devops Bug #15876 (Resolved): rpm installation might result in multiple "ceph" users/groups with different uids/gids existing in the system at the same time
- 06:15 PM rgw Backport #65375 (In Progress): squid: lifecycle transition crashes since reloading bucket attrs for notification
- squid backport included in https://github.com/ceph/ceph/pull/56069 for https://tracker.ceph.com/issues/64818
- 05:58 PM rgw Backport #65375 (Resolved): squid: lifecycle transition crashes since reloading bucket attrs for notification
- 06:15 PM devops Bug #15145 (Resolved): cephfs repair tools should move to ceph-common package
- 06:14 PM devops Bug #15146 (Resolved): some sub-packages depend on "ceph"
- 06:13 PM RADOS Backport #65374 (In Progress): squid: qa: "ceph tell 4.3a deep-scrub" command not found
- backport PR: 56766
- 05:43 PM RADOS Backport #65374 (In Progress): squid: qa: "ceph tell 4.3a deep-scrub" command not found
- 06:13 PM RADOS Bug #64938: Pool created with single PG splits into many on single OSD causes OSD to hit max_pgs_per_osd
- Bump up. Prashant, let's talk about it.
- 06:01 PM RADOS Bug #64437: qa/standalone/scrub/osd-scrub-repair.sh: TEST_repair_stats_ec: test 26 = 13
- bump up.
- 05:56 PM RADOS Bug #64519: OSD/MON: No snapshot metadata keys trimming
- Looks pretty backportable but let's wait for Matan's word.
- 05:51 PM rgw Bug #64571 (Pending Backport): lifecycle transition crashes since reloading bucket attrs for notification
- 05:50 PM RADOS Bug #53240: full-object read crc is mismatch, because truncate modify oi.size and forget to clear data_digest
- Bump up.
- 05:41 PM RADOS Bug #64972 (Pending Backport): qa: "ceph tell 4.3a deep-scrub" command not found
- 03:49 PM RADOS Bug #64972: qa: "ceph tell 4.3a deep-scrub" command not found
- note a new PR suggested as a fix
- 04:24 PM rgw Bug #65373 (Fix Under Review): multisite: test_object_sync gets wrong object body: b'<x-rgw' != b'asdasd'
- 04:14 PM rgw Bug #65373 (Pending Backport): multisite: test_object_sync gets wrong object body: b'<x-rgw' != b'asdasd'
- several test cases fail when comparing the object data between zones:...
- 03:56 PM CephFS Bug #65372 (New): qa: The following counters failed to be set on mds daemons: {'mds.exported', 'mds.imported'}
- /a/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/7641709...
- 03:40 PM CephFS Bug #64977: mds spinlock due to lock contention leading to memory exaustion
- We see this pattern during the incidents that both cap revocation and grants spike and then we see the node consuming...
- 03:37 PM Orchestrator Bug #64864 (Resolved): cephadm: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c in cluster log
- 03:12 PM CephFS Bug #62188: AttributeError: 'RemoteProcess' object has no attribute 'read'
- Another instance: /a/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-default-smithi/764163...
- 03:10 PM CephFS Bug #62188: AttributeError: 'RemoteProcess' object has no attribute 'read'
- Rishabh, this is seen in reef run: /a/yuriw-2024-04-05_18:13:21-fs-wip-yuri4-testing-2024-04-04-1128-reef-distro-def...
- 03:10 PM CephFS Bug #62246: qa/cephfs: test_mount_mon_and_osd_caps_present_mds_caps_absent fails
- Rishabh, ping?
- 02:49 PM RADOS Bug #65371 (Fix Under Review): rados: PeeringState::calc_replicated_acting_stretch populate acting set before checking if < bucket_max
- I noticed that in the final stage of the func PeeringState::calc_replicated_acting_stretch we are populating the acti...
- 01:41 PM Dashboard Bug #65370 (Pending Backport): mgr/dashboard: fix snap schedule delete retention
- Fix snap schedule delete retention arguments
- 01:08 PM rgw Bug #65369: rgw: allow disabling bucket stats on head bucket
- PR: https://github.com/ceph/ceph/pull/56756
- 01:05 PM rgw Bug #65369 (Fix Under Review): rgw: allow disabling bucket stats on head bucket
- This will result in calculating bucket stats on each request which can be manipulated by the user and could be a heav...
- 01:08 PM CephFS Bug #64717: MDS stuck in replay/resolve use
- Would introducing states like up:active-replay and up:active-standby-replay help.
This states would imply that the M... - 12:27 PM CephFS Bug #61407: mds: abort on CInode::verify_dirfrags
- old logs have been purged
deferring until a fresh qa report is available - 12:16 PM Feature #64952: crc32 optimized for s390x arch
- Teuthology tests at s390x nodes are passing.
- 12:05 PM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- `validate_cephfs_path()` calls `cephfs_path_is_dir()` for every path, if the path is not a dir it raises `NotADirecto...
- 11:57 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- how are we hitting this now, this code has been existent since quite sometime and it always had worked fine
- 12:04 PM CephFS Bug #63830: MDS fails to start
- I rebuilt code tagged at v17.2.7 on my Fedora 35 VM and launched gdb with the locally built ceph-mds and the core dum...
- 11:58 AM Backport #65368 (Resolved): squid: install-deps: enable copr ceph/grpc
- https://github.com/ceph/ceph/pull/56778
- 11:52 AM Bug #65184 (Pending Backport): install-deps: enable copr ceph/grpc
- 11:12 AM rgw Bug #65354: rgw/notifications: topic migration test failures
- I'm trying to address these issues in this PR: https://github.com/ceph/ceph/pull/55663
* this commit: https://github... - 10:34 AM RADOS Feature #54525: osd/mon: log memory usage during tick
- let me try implement this
- 09:15 AM Bug #65176: BlueFS: _estimate_log_size_N calculates the log size incorrectly
- linke wang wrote in #note-2:
> Does avg_dir_size or avg_file_size represent the meaning of the bytes occupied by the... - 06:25 AM Bug #65176: BlueFS: _estimate_log_size_N calculates the log size incorrectly
- Does avg_dir_size or avg_file_size represent the meaning of the bytes occupied by the name? Or does it have other mea...
- 06:27 AM CephFS Backport #65366 (In Progress): squid: qa: run TestSnapshots.test_kill_mdstable for all mount types
- https://github.com/ceph/ceph/pull/56952
- 06:27 AM CephFS Backport #65365 (In Progress): reef: qa: run TestSnapshots.test_kill_mdstable for all mount types
- https://github.com/ceph/ceph/pull/56953
- 06:19 AM CephFS Fix #63432 (Pending Backport): qa: run TestSnapshots.test_kill_mdstable for all mount types
- 05:53 AM CephFS Bug #65364 (New): Provide metrics support for the Target Cluster Disconnection status
- From BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2270654,
Currently there is no metrics supports which can ale... - 05:41 AM rgw Bug #64999: Slow RGW multisite sync due to "304 Not Modified" responses on primary zone
- Hello Shilpa,
Thanks for your response.
At the moment full data not yet synced,
Currently On DC site we have ... - 05:38 AM rgw Bug #64999: Slow RGW multisite sync due to "304 Not Modified" responses on primary zone
Hello Shilpa,
Thanks for your response.
At the moment full data not yet synced,
Currently On DC site we hav...- 02:30 AM crimson Feature #65288: crimson: OSD support `trim stale osdmaps` socket command
- PR：https://github.com/ceph/ceph/pull/56742
- 02:28 AM CephFS Backport #65322 (In Progress): quincy: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- 02:25 AM CephFS Backport #65321 (In Progress): reef: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- 02:25 AM CephFS Backport #65323 (In Progress): squid: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- 02:19 AM CephFS Backport #65323 (New): squid: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- 02:17 AM CephFS Bug #62036 (Pending Backport): src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- 02:17 AM CephFS Bug #62036 (Fix Under Review): src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- 02:15 AM CephFS Backport #65363 (New): reef: qa/cephfs: test_idem_unaffected_root_squash fails
- 02:15 AM CephFS Backport #65362 (New): quincy: qa/cephfs: test_idem_unaffected_root_squash fails
- 02:15 AM CephFS Backport #65361 (New): squid: qa/cephfs: test_idem_unaffected_root_squash fails
- 02:01 AM CephFS Bug #63141 (Pending Backport): qa/cephfs: test_idem_unaffected_root_squash fails
- 02:00 AM rgw Bug #64841: java_s3tests: testObjectCreateBadExpectMismatch failure
- After several dozen runs of java-s3tests locally I was able to see what Casey was seeing in his logs in one run. The ...
04/07/2024
- 05:48 PM Bug #64017 (Resolved): osd/scrub: 'max_osd_scrub' default of '1' is too low
- 05:46 PM Backport #64019 (Resolved): reef: osd/scrub: 'max_osd_scrub' default of '1' is too low
- 05:45 PM Backport #64018 (Resolved): quincy: osd/scrub: 'max_osd_scrub' default of '1' is too low
- Merged (PR https://github.com/ceph/ceph/pull/55174)
- 11:40 AM bluestore Backport #65358 (New): quincy: BlueFS log runway space exhausted
- 11:40 AM bluestore Backport #65357 (New): squid: BlueFS log runway space exhausted
- 11:40 AM bluestore Backport #65356 (New): reef: BlueFS log runway space exhausted
- 11:33 AM bluestore Fix #58759 (Pending Backport): BlueFS log runway space exhausted
- 07:47 AM Orchestrator Bug #65355 (New): FAILED tests/test_util_funcs.py::test_call[stdout-very-long] - AssertionError
- For example, https://jenkins.ceph.com/job/ceph-pull-requests-arm64/54751/
Shows:
==============================...
04/06/2024
- 04:06 PM rgw Bug #20279 (Resolved): rgw: the RGWBucket::link() doesn't handle some errors
- 04:03 PM rgw Bug #20210 (Can't reproduce): rgw: fix input/output err when using radosgw-admin bi list
- 03:59 PM rgw Bug #17987 (Fix Under Review): incorrect entry is removed when a part is upload more than once during multipart-upload
- 03:58 PM rgw Bug #19011 (Resolved): rgw: add radosclient finisher to perf counter
- 03:55 PM rgw Bug #16957 (Won't Fix): NO_SSL_DL is not passed to civetweb during compilation of radosgw
- Civetweb was removed in Quincy release
- 03:51 PM rgw Bug #16854 (Can't reproduce): the usage log record date is 16 hours later than the real operate time in jewel1 10.2.2 rgw
- 03:47 PM rgw Bug #15881 (Duplicate): rgw: radosgw-admin bucket rm fail to delete object that begining with double underscores
- 03:45 PM rgw Bug #16027 (Can't reproduce): Add support for swift bulk delete with POST request
- 11:39 AM rgw Bug #52900 (Resolved): segfault on FIPS enabled server as result of EVP_md5 disabled in openssl
- 11:35 AM rgw Bug #63428 (Fix Under Review): RGW: multipart get wrong storage class metadata
- 11:25 AM CephFS Backport #65323 (Rejected): squid: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- Pacific is EOL
04/05/2024
- 09:11 PM rgw Bug #65354: rgw/notifications: topic migration test failures
- https://qa-proxy.ceph.com/teuthology/cbodley-2024-04-04_17:31:11-rgw-wip-rgw-account-topic-distro-default-smithi/7640...
- 09:08 PM rgw Bug #65354 (Duplicate): rgw/notifications: topic migration test failures
- two failures on main in https://qa-proxy.ceph.com/teuthology/cbodley-2024-04-05_17:23:35-rgw-wip-64571-distro-default...
- 08:58 PM RADOS Bug #64802: rados: generalize stretch mode pg temp handling to be usable without stretch mode
- Just created final revision: https://github.com/ceph/ceph/pull/56233
waiting for review - 07:37 PM CephFS Bug #64538: cephfs-shell: hangs and then aborts
- Venky, I think this should be backported. Any thoughts?
Leaving the status of this ticket unchanged for now. - 05:28 PM Dashboard Cleanup #65110 (Resolved): mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- 05:27 PM Dashboard Backport #65208 (Resolved): squid: mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- 05:27 PM Dashboard Backport #65209 (Resolved): reef: mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- 05:26 PM Dashboard Subtask #65025 (Resolved): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 05:26 PM Dashboard Backport #65303 (Resolved): squid: mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 05:26 PM Dashboard Backport #65333 (Resolved): reef: mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 05:24 PM rgw Backport #65353 (Resolved): squid: rgwlc: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault
- https://github.com/ceph/ceph/pull/56960
- 05:24 PM rgw Backport #65352 (New): reef: rgwlc: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault
- 05:18 PM rgw Bug #65188 (Pending Backport): rgwlc: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault
- 04:47 PM rgw Backport #65351 (Resolved): squid: rgw: crash in lc while transitioning to cloud
- 04:39 PM rgw Bug #65251 (Pending Backport): rgw: crash in lc while transitioning to cloud
- 03:46 PM CephFS Bug #65350: mgr/snap_schedule: restore yearly spec from uppercase Y to lowercase y
- Introduced-by: https://github.com/ceph/ceph/pull/53070
- 03:29 PM CephFS Bug #65350 (Triaged): mgr/snap_schedule: restore yearly spec from uppercase Y to lowercase y
- I had mistakenly flipped this case when flipping the monthly spec.
- 03:04 PM CephFS Backport #65063 (Resolved): squid: qa: "cluster [WRN] Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds" in cluster log "
- 02:57 PM CephFS Backport #65063: squid: qa: "cluster [WRN] Health detail: HEALTH_WARN 1 filesystem is online with fewer MDS than max_mds" in cluster log "
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56402
merged - 03:04 PM CephFS Bug #64988 (Resolved): qa: fs:workloads mgr client evicted indicated by "cluster [WRN] evicting unresponsive client smithi042:x (15288), after 303.306 seconds"
- 03:04 PM CephFS Backport #65093 (Resolved): squid: qa: fs:workloads mgr client evicted indicated by "cluster [WRN] evicting unresponsive client smithi042:x (15288), after 303.306 seconds"
- 02:58 PM CephFS Backport #65093: squid: qa: fs:workloads mgr client evicted indicated by "cluster [WRN] evicting unresponsive client smithi042:x (15288), after 303.306 seconds"
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56423
merged - 03:03 PM CephFS Bug #65182 (Resolved): mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- 03:03 PM CephFS Backport #65214 (Resolved): squid: mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- 02:59 PM CephFS Backport #65214: squid: mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56564
merged - 02:47 PM Dashboard Bug #47612: ERROR: setUpClass (tasks.mgr.dashboard.test_health.HealthTest)
- https://jenkins.ceph.com/job/ceph-api/71882/
- 12:09 PM Linux kernel client Bug #64471: kernel: upgrades from quincy/v18.2.[01]/reef to main|squid fail with kernel oops
- Seen again in squid testing. These are backport tests by Yuri so kdb was left enabled and https://github.com/ceph/teu...
- 11:58 AM Linux kernel client Bug #57656: [testing] dbench: write failed on handle 10009 (Resource temporarily unavailable)
- This shows up on almost every run. Latest: https://pulpito.ceph.com/yuriw-2024-04-01_20:57:46-fs-wip-yuri3-testing-20...
- 11:38 AM CephFS Bug #65021 (Duplicate): qa/suites/fs/nfs: cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log
- Duplicates https://tracker.ceph.com/issues/65265
- 11:38 AM CephFS Bug #65021: qa/suites/fs/nfs: cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log
- This is ceph-mgr daemon which has mgr/nfs related backtrace...
- 10:49 AM CephFS Backport #65348 (Rejected): reef: qa: failed cephfs-shell test_reading_conf
- This issues is only seen with centos9, therefore it doesn't apply to reef release.
- 06:21 AM CephFS Backport #65348 (Rejected): reef: qa: failed cephfs-shell test_reading_conf
- 10:49 AM CephFS Backport #65346 (Rejected): quincy: qa: failed cephfs-shell test_reading_conf
- This issues is only seen with centos9, therefore it doesn't apply to quincy release.
- 06:20 AM CephFS Backport #65346 (Rejected): quincy: qa: failed cephfs-shell test_reading_conf
- 09:29 AM rgw Bug #64971: Rgw lifecycle skip
- Sorry for late responds - have been out of office for some weeks.
```
ceph config get client.rgw rgw_lifecycle_w... - 06:20 AM CephFS Backport #65347 (In Progress): squid: qa: failed cephfs-shell test_reading_conf
- https://github.com/ceph/ceph/pull/56910
- 06:17 AM CephFS Bug #63699 (Pending Backport): qa: failed cephfs-shell test_reading_conf
- 05:29 AM CephFS Bug #65345 (Fix Under Review): cephfs_mirror: increment sync_failures when sync_perms() and sync_snaps() fails
- When sync_perms() fails _inc_failed_count() is called for incrementing the failed count.
So it's better reflect tha... - 03:39 AM Feature #65343: rgw/multisite: bucket replication scrubbing
- This issue tracker is a duplicate of https://tracker.ceph.com/issues/65344.
Can somebody who has permission close th... - 03:35 AM Feature #65343 (New): rgw/multisite: bucket replication scrubbing
- Scrubbing multisite replication related data (logs etc.) to make sure the bucket level replication is in consistent s...
- 03:37 AM rgw Feature #65344 (New): rgw/multisite: bucket replication scrubbing
- Scrubbing multisite replication related data (logs etc.) to make sure the bucket level replication is in consistent s...
- 03:16 AM rgw Bug #63178 (Resolved): multisite: don't write data/bilog entries for lifecycle transitions/deletes
- 03:12 AM rgw Backport #64088 (Resolved): reef: multisite: don't write data/bilog entries for lifecycle transitions/deletes
- 01:38 AM CephFS Bug #65342 (Fix Under Review): mds: quiesce_counter decay rate initialized from wrong config
- 01:20 AM CephFS Bug #65342 (Pending Backport): mds: quiesce_counter decay rate initialized from wrong config
- 01:30 AM rgw Backport #65340 (In Progress): reef: rgw: update options yaml file so LDAP uri isn't an invalid example
- PR: https://github.com/ceph/ceph/pull/56721
- 12:03 AM rgw Backport #65341 (In Progress): quincy: rgw: update options yaml file so LDAP uri isn't an invalid example
04/04/2024
- 11:47 PM rgw Backport #65339 (In Progress): squid: rgw: update options yaml file so LDAP uri isn't an invalid example
- 11:35 PM rgw Backport #65339 (Resolved): squid: rgw: update options yaml file so LDAP uri isn't an invalid example
- https://github.com/ceph/ceph/pull/56720
- 11:35 PM rgw Backport #65341 (In Progress): quincy: rgw: update options yaml file so LDAP uri isn't an invalid example
- https://github.com/ceph/ceph/pull/56722
- 11:35 PM rgw Backport #65340 (In Progress): reef: rgw: update options yaml file so LDAP uri isn't an invalid example
- 11:32 PM rgw Bug #65277 (Pending Backport): rgw: update options yaml file so LDAP uri isn't an invalid example
- 11:29 PM rgw Bug #63995 (Resolved): lc: expiration action with Days tag specified doesn't cleanup the expired delete-marker
- 09:49 PM Orchestrator Feature #65338 (New): Add --continue-on-error for `cephadm bootstrap`
- When a cluster specification is passed like:...
- 08:15 PM Feature #64840: Add posibility to disable stretch mode
- +1
- 08:09 PM rgw Bug #65337: rgw: Segmentation fault in rgw::notify::Manager during realm reload
- the rgw log corresponding to rgw.client.1 is flooded with curl errors of the form:
> Couldn't connect to server re... - 07:51 PM rgw Bug #65337: rgw: Segmentation fault in rgw::notify::Manager during realm reload
- > Notice that the class rgw::notify::Manager::process_queue is embedded in the stack trace, which leads me to believe...
- 06:50 PM rgw Bug #65337 (Fix Under Review): rgw: Segmentation fault in rgw::notify::Manager during realm reload
- Notice that the class *rgw::notify::Manager::process_queue* is embedded in the stack trace, which leads me to believe...
- 07:47 PM Orchestrator Bug #64208: test_cephadm.sh: Container version mismatch causes job to fail.
- squid patch https://github.com/ceph/ceph/pull/56715
- 07:40 PM Orchestrator Bug #64208: test_cephadm.sh: Container version mismatch causes job to fail.
- reef patch https://github.com/ceph/ceph/pull/56714
- 06:13 PM Orchestrator Bug #64208: test_cephadm.sh: Container version mismatch causes job to fail.
- this issue I believe is currently on reef (and sort of squid, it will start appearing as soon as main gets bumped to ...
- 06:56 PM rgw Bug #64571 (Fix Under Review): lifecycle transition crashes since reloading bucket attrs for notification
- 06:38 PM rgw Bug #64571: lifecycle transition crashes since reloading bucket attrs for notification
- As discussed https://github.com/ceph/ceph/pull/55657#discussion_r1541310624, removing the logic to reload the bucket ...
- 06:26 PM Orchestrator Bug #65335 (Pending Backport): cephadm: client-keyring also overwrites ceph.conf
- If you have something like...
- 06:08 PM rgw Bug #65334 (Pending Backport): Command failed with status 128: 'git clone -b stable/xena https://github.com/openstack/barbican.git /home/ubuntu/cephtest/barbican'
- ...
- 05:03 PM Dashboard Backport #65333 (In Progress): reef: mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 05:01 PM Dashboard Backport #65333 (Resolved): reef: mgr/dashboard: Make advanced fields consistent for forms in dashboard
- https://github.com/ceph/ceph/pull/56710
- 04:59 PM Dashboard Subtask #65025 (Pending Backport): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 04:58 PM Dashboard Subtask #65025 (Fix Under Review): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 04:54 PM Dashboard Subtask #65025 (Pending Backport): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 04:54 PM Dashboard Subtask #65025 (Fix Under Review): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 06:54 AM Dashboard Subtask #65025 (Pending Backport): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 04:56 PM Dashboard Backport #65303 (In Progress): squid: mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 06:56 AM Dashboard Backport #65303 (Resolved): squid: mgr/dashboard: Make advanced fields consistent for forms in dashboard
- https://github.com/ceph/ceph/pull/56708
- 04:54 PM Dashboard Backport #65209 (In Progress): reef: mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- 04:53 PM Dashboard Backport #65208 (In Progress): squid: mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- 04:35 PM Dashboard Feature #65024 (Resolved): mgr/dashboard: Add advanced fieldset component for forms
- 07:03 AM Dashboard Feature #65024 (Pending Backport): mgr/dashboard: Add advanced fieldset component for forms
- 07:03 AM Dashboard Feature #65024 (Fix Under Review): mgr/dashboard: Add advanced fieldset component for forms
- 04:35 PM Dashboard Backport #65156 (Resolved): squid: mgr/dashboard: Add advanced fieldset component for forms
- 07:12 AM Dashboard Backport #65156 (In Progress): squid: mgr/dashboard: Add advanced fieldset component for forms
- 04:35 PM Dashboard Backport #65304 (Resolved): reef: mgr/dashboard: Add advanced fieldset component for forms
- 07:14 AM Dashboard Backport #65304 (In Progress): reef: mgr/dashboard: Add advanced fieldset component for forms
- 07:05 AM Dashboard Backport #65304 (Resolved): reef: mgr/dashboard: Add advanced fieldset component for forms
- https://github.com/ceph/ceph/pull/56692
- 04:30 PM Dashboard Bug #62972: ERROR: test_list_enabled_module (tasks.mgr.dashboard.test_mgr_module.MgrModuleTest)
- https://jenkins.ceph.com/job/ceph-api/71790/
- 03:57 PM rgw Backport #64465: reef: Uploads by AWS Go SDK v2 fail with XAmzContentSHA256Mismatch when Checksum is requested
- Matt, any chance this could come with the next point release to Reef (which we currently run)?
- 03:52 PM cleanup Tasks #65332 (New): remove Bucket::merge_and_store_attrs()
- from https://github.com/ceph/ceph/pull/56583#issuecomment-2037285274:
> i have a general distaste for Bucket::merge_... - 03:31 PM Orchestrator Backport #64844: reef: Regression: Permanent KeyError: 'TYPE' : return self.blkid_api['TYPE'] == 'part'
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56323
merged - 03:20 PM Bug #65331 (New): Issue with crushrule datacenter
- We created stretch cluster with 6 nodes , 4 disks in each
Rule is working only on 3 or 4 replicas
rule Stretch... - 02:43 PM RADOS Bug #64978: from rgw suite: HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering
- Hi Casey, I'll take a look.
- 02:32 PM rgw Bug #64999: Slow RGW multisite sync due to "304 Not Modified" responses on primary zone
- 304 Not Modified means that there is no change in the object since the time it was last synced.
you mention that all... - 02:28 PM rgw Backport #65328 (New): quincy: notifications: notification will be sent even if op has failed
- 02:28 PM rgw Backport #65327 (New): reef: notifications: notification will be sent even if op has failed
- 02:25 PM rgw Bug #63791 (Fix Under Review): RGW: a subuser with no permission can still list buckets and create buckets
- 02:21 PM rgw Bug #63855 (Pending Backport): notifications: notification will be sent even if op has failed
- 02:21 PM rgw Bug #63859 (Resolved): notifications/lifecycle: failure to commit a notification should not be considered an error
- 02:16 PM rgw Bug #64431 (In Progress): metadata sync does not replicate iam OpenIDConnectProvider metadata
- 02:09 PM rgw Bug #65216 (Fix Under Review): rgw: only accept valid ipv4 from host header
- 01:44 PM CephFS Backport #65326 (In Progress): quincy: client: log message when unmount call is received
- https://github.com/ceph/ceph/pull/56954
- 01:43 PM CephFS Backport #65325 (In Progress): reef: client: log message when unmount call is received
- https://github.com/ceph/ceph/pull/56955
- 01:37 PM CephFS Bug #64503 (Pending Backport): client: log message when unmount call is received
- 01:06 PM CephFS Backport #65320 (In Progress): quincy: cephfs-mirror: use monotonic clocks in cephfs mirror daemon
- 12:30 PM CephFS Backport #65320 (In Progress): quincy: cephfs-mirror: use monotonic clocks in cephfs mirror daemon
- https://github.com/ceph/ceph/pull/56702
- 12:59 PM CephFS Backport #65319 (In Progress): reef: cephfs-mirror: use monotonic clocks in cephfs mirror daemon
- 12:30 PM CephFS Backport #65319 (In Progress): reef: cephfs-mirror: use monotonic clocks in cephfs mirror daemon
- https://github.com/ceph/ceph/pull/56701
- 12:57 PM CephFS Backport #65318 (In Progress): squid: cephfs-mirror: use monotonic clocks in cephfs mirror daemon
- 12:30 PM CephFS Backport #65318 (In Progress): squid: cephfs-mirror: use monotonic clocks in cephfs mirror daemon
- https://github.com/ceph/ceph/pull/56700
- 12:35 PM CephFS Bug #63141: qa/cephfs: test_idem_unaffected_root_squash fails
- Not changing status of this ticket since it probably needs to be backported.
- 12:30 PM CephFS Backport #65323 (In Progress): squid: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- https://github.com/ceph/ceph/pull/56746
- 12:30 PM CephFS Backport #65322 (In Progress): quincy: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- https://github.com/ceph/ceph/pull/56748
- 12:30 PM CephFS Backport #65321 (In Progress): reef: src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- https://github.com/ceph/ceph/pull/56747
- 12:24 PM CephFS Bug #62265 (Pending Backport): cephfs-mirror: use monotonic clocks in cephfs mirror daemon
- 12:23 PM CephFS Bug #62036 (Pending Backport): src/mds/MDCache.cc: 5131: FAILED ceph_assert(isolated_inodes.empty())
- 12:20 PM CephFS Feature #58072 (Resolved): enable 'ceph fs new' use 'ceph fs set' options
- 12:18 PM CephFS Bug #48562 (Fix Under Review): qa: scrub - object missing on disk; some files may be lost
- Milind, I'm taking this one.
- 11:50 AM CephFS Bug #65317 (Fix Under Review): cephfs_mirror: update peer status for invalid metadata in remote snapshot
- 11:38 AM CephFS Backport #65316 (In Progress): squid: mds: CInode::item_caps used in two different lists
- 11:38 AM CephFS Backport #65315 (In Progress): reef: mds: CInode::item_caps used in two different lists
- 11:35 AM CephFS Bug #64008 (Pending Backport): mds: CInode::item_caps used in two different lists
- Let's hold off the backport till we get a cleaner run for fs suite in main branch.
- 11:28 AM CephFS Bug #65314 (New): valgrind error: Leak_PossiblyLost posix_memalign UnknownInlinedFun ceph::buffer::v15_2_0::list::refill_append_space(unsigned int)
- /a/vshankar-2024-03-31_13:02:52-fs-wip-vshankar-testing-20240330.172700-testing-default-smithi/7632454
Teuthology ... - 11:26 AM RADOS Backport #65311 (Rejected): squid: decoding chunk_refs_by_hash_t return wrong values
- backport script already created another tracker, rejecting that one
- 11:17 AM RADOS Backport #65311 (Rejected): squid: decoding chunk_refs_by_hash_t return wrong values
- When running ceph dencoder test on clang-14 compiled JSON dump of chunk_refs_by_hash_t will show:...
- 11:23 AM RADOS Backport #65312 (In Progress): squid: decoding chunk_refs_by_hash_t return wrong values
- 11:18 AM RADOS Backport #65312 (In Progress): squid: decoding chunk_refs_by_hash_t return wrong values
- https://github.com/ceph/ceph/pull/56697
- 11:22 AM RADOS Backport #65313 (New): squid: decoding chunk_refs_by_hash_t return wrong values
- 11:19 AM RADOS Backport #65313 (New): squid: decoding chunk_refs_by_hash_t return wrong values
- When running ceph dencoder test on clang-14 compiled JSON dump of chunk_refs_by_hash_t will show:...
- 11:14 AM RADOS Bug #64854 (Pending Backport): decoding chunk_refs_by_hash_t return wrong values
- 10:39 AM Dashboard Bug #65310 (In Progress): mgr/dashboard: fix NFS routes for Object and File navigation
- h3. fix NFS routes for Object and File navigation
Currently, the same NFS nav link gets activated for both Object ... - 10:06 AM RADOS Bug #64519: OSD/MON: No snapshot metadata keys trimming
- I know I'm a bit early asking this, but I helped raise this issue and Mykola picked it up in the devel mailing list. ...
- 10:04 AM CephFS Bug #65309 (New): qa: dbench.sh failed with "ERROR: handle 10318 was not found"
- Link to the job - https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing...
- 10:00 AM RADOS Bug #64735 (Fix Under Review): OSD/MON: rollback_to snap the latest overlap is not right
- 09:51 AM CephFS Bug #65308 (New): qa: fs was offline but also unexpectedly degraded
- Link to the failure - https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-tes...
- 08:51 AM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Thanks for taking a look, Laura.
Dhariya, please take this one. AFAICT, this exception should have been handle in ... - 08:35 AM CephFS Bug #63931: qa: test_mirroring_init_failure_with_recovery failure
- Venky Shankar wrote:
> Jos Collin wrote:
> > Venky Shankar wrote:
> > > Jos Collin wrote:
> > > > Venky Shankar w... - 06:38 AM CephFS Bug #63931: qa: test_mirroring_init_failure_with_recovery failure
- Jos Collin wrote:
> Venky Shankar wrote:
> > Jos Collin wrote:
> > > Venky Shankar wrote:
> > > > Jos Collin wrot... - 06:34 AM CephFS Bug #63931: qa: test_mirroring_init_failure_with_recovery failure
- Venky Shankar wrote:
> Jos Collin wrote:
> > Venky Shankar wrote:
> > > Jos Collin wrote:
> > > > @Venky,
> > > ... - 06:31 AM CephFS Bug #63931: qa: test_mirroring_init_failure_with_recovery failure
- Jos Collin wrote:
> Venky Shankar wrote:
> > Jos Collin wrote:
> > > @Venky,
> > >
> > > When we disable mirror... - 06:22 AM CephFS Bug #63931: qa: test_mirroring_init_failure_with_recovery failure
- Venky Shankar wrote:
> Jos Collin wrote:
> > @Venky,
> >
> > When we disable mirroring using 'ceph mgr module di... - 08:26 AM RADOS Backport #65307 (In Progress): quincy: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- https://github.com/ceph/ceph/pull/56815
- 08:26 AM RADOS Backport #65306 (In Progress): squid: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- https://github.com/ceph/ceph/pull/56814
- 08:26 AM RADOS Backport #65305 (In Progress): reef: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- https://github.com/ceph/ceph/pull/56813
- 08:22 AM RADOS Bug #64347 (Pending Backport): src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- 08:02 AM CephFS Backport #65066: squid: make check: QuiesceDbTest.MultiRankRecovery Failed
- Due to a mistake, the commit was not included in https://github.com/ceph/ceph/pull/56202, so here is a dedicated PR f...
- 07:56 AM CephFS Backport #65066 (In Progress): squid: make check: QuiesceDbTest.MultiRankRecovery Failed
- 07:57 AM Orchestrator Bug #65263: upgrade stalls after upgrading one ceph-mgr daemon
- Another instance: https://pulpito.ceph.com/vshankar-2024-03-31_13:02:52-fs-wip-vshankar-testing-20240330.172700-testi...
- 07:56 AM Dashboard Cleanup #65207 (Fix Under Review): mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- 07:50 AM CephFS Backport #64941 (In Progress): quincy: qa: Add multifs root_squash testcase
- 07:32 AM bluestore Bug #63121 (Resolved): KeyValueDB/KVTest.RocksDB_estimate_size tests failing
- 07:31 AM bluestore Backport #64928 (Resolved): reef: KeyValueDB/KVTest.RocksDB_estimate_size tests failing
- 07:31 AM bluestore Backport #64091 (Resolved): reef: ceph-bluestore-tool bluefs-bdev-expand doesn't adjust OSD free space when NCB mode is in use
- 07:21 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Dhairya Parmar wrote:
> Venky Shankar wrote:
> > Dhairya Parmar wrote:
> > > Me and greg had a discussion on this ... - 06:58 AM CephFS Backport #64218 (Resolved): reef: fs/cephadm/renamevolume: volume rename failure
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56171
Merged. - 06:57 AM CephFS Backport #64047 (Resolved): reef: qa: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) failure
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56169
Merged. - 06:56 AM CephFS Backport #64250 (Resolved): reef: smoke test fails from "NameError: name 'DEBUGFS_META_DIR' is not defined"
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56167
Merged. - 06:50 AM CephFS Backport #64759 (Resolved): reef: osdc/Journaler: better handle ENOENT during replay as up:standby-replay
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56016
Merged. - 06:39 AM CephFS Backport #64940 (In Progress): reef: qa: Add multifs root_squash testcase
- 06:37 AM CephFS Backport #64939 (In Progress): squid: qa: Add multifs root_squash testcase
- 06:11 AM CephFS Bug #65301 (Fix Under Review): fs:upgrade still uses centos_8* distro
- 05:45 AM CephFS Bug #65301 (Fix Under Review): fs:upgrade still uses centos_8* distro
- Ceph is switching over to centos_9[.stream] for squid+.
- 05:58 AM Dashboard Bug #65302 (Pending Backport): mgr/dashboard: fetch prometheus api host with ip addr
- Check if the PROMETHEUS_API_HOST value has the ip addr of the host if not replace hostnmame with ip addr
- 05:49 AM Dashboard Bug #64734 (Resolved): mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 05:49 AM Dashboard Backport #65205 (Resolved): reef: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 05:49 AM Dashboard Backport #65206 (Resolved): quincy: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 05:48 AM Dashboard Cleanup #64708 (Resolved): mgr/dashboard: Mark placement targets as non-required
- 05:48 AM Dashboard Backport #65210 (Resolved): squid: mgr/dashboard: Mark placement targets as non-required
- 05:48 AM Dashboard Backport #65211 (Resolved): reef: mgr/dashboard: Mark placement targets as non-required
- 04:58 AM RADOS Bug #64236 (Resolved): mon: health store size growing infinitely
- 04:58 AM RADOS Backport #64397 (Resolved): reef: mon: health store size growing infinitely
- 04:30 AM CephFS Bug #64064 (Fix Under Review): mds config `mds_log_max_segments` throws error for value -1
- 04:29 AM CephFS Backport #64584 (Resolved): reef: crash: void Locker::handle_file_lock(ScatterLock*, ceph::cref_t<MLock>&): assert(lock->get_state() == LOCK_LOCK || lock->get_state() == LOCK_MIX || lock->get_state() == LOCK_MIX_SYNC2)
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56049
Merged. - 04:20 AM CephFS Backport #64518 (Resolved): reef: mgr/volumes: Support to reject CephFS clones if cloner threads are not available
- 04:20 AM CephFS Backport #64701 (Resolved): squid: mgr/volumes: Support to reject CephFS clones if cloner threads are not available
- 02:39 AM Dashboard Backport #65285 (In Progress): reef: mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- 02:34 AM Backport #65258: reef: The way to know the data format of each OSD and MON was created
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56681
ceph-backport.sh versi... - 02:33 AM Dashboard Backport #65286 (In Progress): quincy: mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- 02:32 AM Backport #65257: quincy: The way to know the data format of each OSD and MON was created
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56679
ceph-backport.sh versi... - 12:52 AM Backport #65257: quincy: The way to know the data format of each OSD and MON was created
- -Ryotaro Banno wrote:-
-> please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/5... - 12:48 AM Backport #65257: quincy: The way to know the data format of each OSD and MON was created
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56678
ceph-backport.sh versi... - 02:28 AM Dashboard Backport #65284 (Resolved): squid: mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- 01:55 AM crimson Bug #65299 (New): crimson: check for missing object on replica read
- 01:09 AM CephFS Bug #64486 (Resolved): qa: enhance labeled perf counters test for cephfs-mirror
04/03/2024
- 10:04 PM rgw Backport #65003: reef: [CVE-2023-46159] RGW crash upon misconfigured CORS rule
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56347
merged - 10:03 PM bluestore Bug #63121: KeyValueDB/KVTest.RocksDB_estimate_size tests failing
- https://github.com/ceph/ceph/pull/56197 merged
- 10:02 PM RADOS Backport #64650: reef: min_last_epoch_clean is not updated, causing osdmap to be unable to be trimmed, and monitor db keeps growing.
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/55867
merged - 10:01 PM bluestore Bug #63858: ceph-bluestore-tool bluefs-bdev-expand doesn't adjust OSD free space when NCB mode is in use
- https://github.com/ceph/ceph/pull/55777 merged
- 09:58 PM RADOS Backport #64397: reef: mon: health store size growing infinitely
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/55548
merged - 09:05 PM rgw Backport #65297 (In Progress): squid: allow AWS lifecycle event types to configure lifecycle notifications and Replication notifications
- i included the 3 commits from https://github.com/ceph/ceph/pull/55795 in a batch backport pr https://github.com/ceph/...
- 06:20 PM rgw Backport #65297 (Resolved): squid: allow AWS lifecycle event types to configure lifecycle notifications and Replication notifications
- 07:12 PM rgw Bug #64571: lifecycle transition crashes since reloading bucket attrs for notification
- the lifecycle changes in https://github.com/ceph/ceph/pull/55657 seem to be responsible for the crash, but it's not c...
- 07:01 PM bluestore Bug #65298 (New): Free space can be leaked in Quincy+ when bdev_async_discard is enabled
- Starting in Quincy, we no longer maintain a free space map in rocksdb in bluestore (https://github.com/ceph/ceph/pull...
- 06:41 PM Orchestrator Bug #64374: Error ENOENT: module 'cephadm' reports that it cannot run on the active manager daemon: No module named 'mgr_module' (pass --force to force enablement)
- Possible fix: https://github.com/ceph/ceph/pull/56617
Although this reverted a commit that was only merged a few d... - 06:34 PM Orchestrator Bug #64374: Error ENOENT: module 'cephadm' reports that it cannot run on the active manager daemon: No module named 'mgr_module' (pass --force to force enablement)
- /a/yuriw-2024-04-01_20:57:46-rados-wip-yuri3-testing-2024-04-01-0837-squid-distro-default-smithi/7634416
- 06:36 PM Orchestrator Bug #65017: cephadm: log_channel(cephadm) log [ERR] : Failed to connect to smithi090 (10.0.0.9). Permission denied
- /a/yuriw-2024-04-01_20:57:46-rados-wip-yuri3-testing-2024-04-01-0837-squid-distro-default-smithi/7634394
- 06:32 PM RADOS Bug #64437: qa/standalone/scrub/osd-scrub-repair.sh: TEST_repair_stats_ec: test 26 = 13
- /a/yuriw-2024-04-01_20:57:46-rados-wip-yuri3-testing-2024-04-01-0837-squid-distro-default-smithi/7634392
- 06:29 PM RADOS Bug #65183: Overriding an EC pool needs the "--yes-i-really-mean-it" flag in addition to "force"
- Also needs to be fixed for qa/standalone/mon/osd-erasure-code-profile.sh:
/a/yuriw-2024-04-01_20:57:46-rados-wip-y... - 06:22 PM RADOS Bug #64347: src/osd/PG.cc: FAILED ceph_assert(!bad || !cct->_conf->osd_debug_verify_cached_snaps)
- /a/yuriw-2024-04-01_20:57:46-rados-wip-yuri3-testing-2024-04-01-0837-squid-distro-default-smithi/7634716
- 06:11 PM rgw Feature #64251 (Pending Backport): allow AWS lifecycle event types to configure lifecycle notifications and Replication notifications
- 06:01 PM rgw Backport #64954 (In Progress): squid: Notification FilterRules for S3key, S3Metadata & S3Tags spit incorrect json output
- 05:52 PM Orchestrator Bug #65234 (In Progress): upgrade/quincy-x/stress-split: cephadm failed to parse grafana.ini file due to inadequate permission
- 05:08 PM nvme-of Backport #65296 (Rejected): squid: cephadm - make changes to ceph-nvmeof.conf template
- 05:07 PM nvme-of Feature #65259 (Pending Backport): cephadm - make changes to ceph-nvmeof.conf template
- 05:02 PM CephFS Backport #65293 (In Progress): quincy: High cephfs MDS latency and CPU load with snapshots and unlink operations
- 04:43 PM CephFS Backport #65293 (In Progress): quincy: High cephfs MDS latency and CPU load with snapshots and unlink operations
- https://github.com/ceph/ceph/pull/56673
- 05:01 PM CephFS Backport #65294 (In Progress): reef: High cephfs MDS latency and CPU load with snapshots and unlink operations
- 04:44 PM CephFS Backport #65294 (In Progress): reef: High cephfs MDS latency and CPU load with snapshots and unlink operations
- https://github.com/ceph/ceph/pull/56672
- 05:01 PM CephFS Backport #65295 (In Progress): squid: High cephfs MDS latency and CPU load with snapshots and unlink operations
- 04:44 PM CephFS Backport #65295 (In Progress): squid: High cephfs MDS latency and CPU load with snapshots and unlink operations
- https://github.com/ceph/ceph/pull/56671
- 04:41 PM CephFS Bug #53192 (Pending Backport): High cephfs MDS latency and CPU load with snapshots and unlink operations
- PR https://github.com/ceph/ceph/pull/55768 has been merged in main. We would be backporting that to supported release...
- 04:28 PM CephFS Backport #65092 (Resolved): reef: qa: fs:workloads mgr client evicted indicated by "cluster [WRN] evicting unresponsive client smithi042:x (15288), after 303.306 seconds"
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56424
Merged. - 04:27 PM CephFS Backport #64922 (Resolved): reef: qa: Command failed (workunit test fs/snaps/untar_snap_rm.sh)
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56398
Merged. - 04:26 PM CephFS Backport #65065 (Resolved): reef: qa: fix continued use of log-whitelist
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56396
Merged. - 04:25 PM CephFS Backport #65058 (Resolved): reef: qa: "cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log "
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56392
Merged. - 04:24 PM CephFS Backport #64919 (Resolved): reef: qa: enhance labeled perf counters test for cephfs-mirror
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56211
Merged. - 04:22 PM CephFS Backport #64565 (Resolved): reef: Difference in error code returned while removing system xattrs using removexattr()
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/55803
Merged. - 03:01 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- Lowering the priority as it's not a real regression – it's a problem with the recently introduced warning.
- 02:56 PM Dashboard Bug #61720 (Resolved): mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- 07:36 AM Dashboard Bug #61720 (Pending Backport): mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- 02:56 PM Dashboard Backport #65282 (Resolved): quincy: mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- 09:08 AM Dashboard Backport #65282 (In Progress): quincy: mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- 07:36 AM Dashboard Backport #65282 (Resolved): quincy: mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- https://github.com/ceph/ceph/pull/56653
- 02:55 PM Dashboard Backport #65283 (Resolved): reef: mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- 08:48 AM Dashboard Backport #65283 (In Progress): reef: mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- 07:36 AM Dashboard Backport #65283 (Resolved): reef: mgr/dashboard: embedded grafana dashboards are still using the old 'graph' panel type.
- https://github.com/ceph/ceph/pull/56652
- 02:23 PM rgw Backport #65292 (In Progress): squid: pubsub: validate Name in CreateTopic api
- 02:16 PM rgw Backport #65292 (Resolved): squid: pubsub: validate Name in CreateTopic api
- https://github.com/ceph/ceph/pull/56670
- 02:16 PM rgw Backport #65291 (New): reef: pubsub: validate Name in CreateTopic api
- 02:16 PM rgw Backport #65290 (New): quincy: pubsub: validate Name in CreateTopic api
- 02:15 PM rgw Bug #65212 (Pending Backport): pubsub: validate Name in CreateTopic api
- 01:46 PM CephFS Backport #65289 (In Progress): squid: MDS daemon is using 50% CPU when idle
- 12:36 PM CephFS Backport #65289 (In Progress): squid: MDS daemon is using 50% CPU when idle
- https://github.com/ceph/ceph/pull/56668
- 01:38 PM Dashboard Bug #47612: ERROR: setUpClass (tasks.mgr.dashboard.test_health.HealthTest)
- again on squid https://jenkins.ceph.com/job/ceph-api/71686/
- 01:37 PM Orchestrator Bug #65187 (Resolved): upgrade/quincy-x/stress-split: upgrade test fails to install quincy packages
- should be fixed by the quincy jammy packages being added, which I believe was completed not long after this test run....
- 01:36 PM rgw Backport #65007 (Resolved): squid: config option to disable s3 presigned urls
- 01:01 PM CephFS Bug #57676: qa: error during scrub thrashing: rank damage found: {'backtrace'}
- Venky,
Yes, the /volumes inode 0x10000000000 is a replica on the mds where the scrub error is reported.... - 12:59 PM CephFS Bug #57676: qa: error during scrub thrashing: rank damage found: {'backtrace'}
> In tracker #64730, I am doubting the in-memory versions mismatches to be related to subtree exporting b/w active ...- 09:34 AM CephFS Bug #57676: qa: error during scrub thrashing: rank damage found: {'backtrace'}
- In tracker #64730, I am doubting the in-memory versions mismatches to be related to subtree exporting b/w active MDSs...
- 04:57 AM CephFS Bug #57676: qa: error during scrub thrashing: rank damage found: {'backtrace'}
- Kotresh Hiremath Ravishankar wrote:
> I looked into the logs from the failure https://pulpito.ceph.com/teuthology-20... - 03:50 AM CephFS Bug #57676: qa: error during scrub thrashing: rank damage found: {'backtrace'}
- I looked into the logs from the failure https://pulpito.ceph.com/teuthology-2024-03-31_21:24:02-fs-squid-distro-defau...
- 01:00 PM CephFS Backport #65107 (In Progress): quincy: qa: probabilistically ignore PG_AVAILABILITY/PG_DEGRADED
- 12:55 PM CephFS Backport #65105 (In Progress): reef: qa: probabilistically ignore PG_AVAILABILITY/PG_DEGRADED
- 12:54 PM CephFS Backport #65106 (In Progress): squid: qa: probabilistically ignore PG_AVAILABILITY/PG_DEGRADED
- 12:53 PM CephFS Backport #65274 (In Progress): reef: mds: some request errors come from errno.h rather than fs_types.h
- 12:53 PM CephFS Backport #65275 (In Progress): squid: mds: some request errors come from errno.h rather than fs_types.h
- 12:53 PM CephFS Backport #65223: squid: cephfs-mirror: use snapdiff api for efficient tree traversal
- Holding back backport for now since we are seeing some qs related failures in fs:mirror. We need to clear those out f...
- 12:43 PM RADOS Bug #64978: from rgw suite: HEALTH_WARN Reduced data availability: 1 pg inactive, 1 pg peering
- again in https://pulpito.ceph.com/cbodley-2024-04-03_02:50:19-rgw-wip-cbodley-testing-distro-default-smithi/7637874/
- 12:33 PM CephFS Bug #65261: qa/cephfs: cephadm related failure on fs/upgrade job
- Copying insight Adam shared on this failure on Slack -...
- 12:29 PM CephFS Bug #65276 (Pending Backport): MDS daemon is using 50% CPU when idle
- 12:18 PM rgw Bug #63486: reef: test_lifecycle_cloud_transition_large_obj FAILED
- Soumya Koduri wrote:
> @cbodley,
>
> Any idea if these failures are specific to only reef branch?
i think i've... - 07:05 AM rgw Bug #63486 (In Progress): reef: test_lifecycle_cloud_transition_large_obj FAILED
- @cbodley,
Any idea if these failures are specific to only reef branch? - 11:10 AM rgw Bug #65251 (Fix Under Review): rgw: crash in lc while transitioning to cloud
- 07:04 AM rgw Bug #65251 (In Progress): rgw: crash in lc while transitioning to cloud
- 10:23 AM rbd Bug #63422: librbd crash in journal discard wait_event
- Raimund Sacherer wrote:
> Joshua Baergen wrote:
> > As a consequence for the above, a very simple workaround for th... - 10:15 AM Dashboard Backport #65284 (In Progress): squid: mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- 07:44 AM Dashboard Backport #65284 (Resolved): squid: mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- https://github.com/ceph/ceph/pull/56656
- 10:11 AM Dashboard Backport #65287 (In Progress): quincy: mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- 07:44 AM Dashboard Backport #65287 (In Progress): quincy: mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- https://github.com/ceph/ceph/pull/56655
- 10:04 AM Dashboard Backport #65142 (In Progress): reef: mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- 07:37 AM Dashboard Backport #65142 (New): reef: mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- 10:04 AM CephFS Bug #64730: fs/misc/multiple_rsync.sh workunit times out
- "balancer/automatic" with multimds and scrub is likely what's causes the divergent backtraces.
That's also the cas... - 10:02 AM Dashboard Bug #65091: exporter: fix regex for rgw sync metrics
- Ceph Grafana dashboard doesn't show the Graphs in RGW sync overview page as it complains about "No Data". Ceph export...
- 08:50 AM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- Venky Shankar wrote:
> centos8.stream run w/ main branch: https://pulpito.ceph.com/vshankar-2024-04-01_04:53:22-fs:u... - 08:25 AM crimson Feature #65288 (Fix Under Review): crimson: OSD support `trim stale osdmaps` socket command
- I plan implement `trim stale osdmaps` socket command for crimson-osd, just like classic OSD
- 07:44 AM Dashboard Backport #65286 (In Progress): quincy: mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- https://github.com/ceph/ceph/pull/56680
- 07:44 AM Dashboard Backport #65285 (In Progress): reef: mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- https://github.com/ceph/ceph/pull/56682
- 07:40 AM Dashboard Bug #65174 (Pending Backport): mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- 07:37 AM Dashboard Cleanup #64579 (Pending Backport): mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- 07:13 AM Bug #65281 (New): Ceph with SPDK driver can not write data to NVMe-oF(TCP) device.
- 1.In the host, we use ceph (v18.2.2) with SPDK driver connect a NVMe-oF over TCP device, and then strat a cluster thr...
- 06:13 AM CephFS Bug #63931: qa: test_mirroring_init_failure_with_recovery failure
- Jos Collin wrote:
> @Venky,
>
> When we disable mirroring using 'ceph mgr module disable mirroring' and then enab... - 05:46 AM CephFS Backport #65280 (In Progress): quincy: doc: fix configurables section in doc/cephfs/fs-volumes.rst
- 05:30 AM CephFS Backport #65280 (In Progress): quincy: doc: fix configurables section in doc/cephfs/fs-volumes.rst
- https://github.com/ceph/ceph/pull/55690
- 05:42 AM CephFS Backport #65279 (In Progress): reef: doc: fix configurables section in doc/cephfs/fs-volumes.rst
- 05:30 AM CephFS Backport #65279 (In Progress): reef: doc: fix configurables section in doc/cephfs/fs-volumes.rst
- https://github.com/ceph/ceph/pull/56648
- 05:42 AM CephFS Backport #65278 (In Progress): squid: doc: fix configurables section in doc/cephfs/fs-volumes.rst
- 05:30 AM CephFS Backport #65278 (In Progress): squid: doc: fix configurables section in doc/cephfs/fs-volumes.rst
- https://github.com/ceph/ceph/pull/56647
- 05:29 AM CephFS Documentation #65267 (Pending Backport): doc: fix configurables section in doc/cephfs/fs-volumes.rst
- Neeraj, please include the additional commit with the original backport (and add that tracker under related issues he...
- 04:58 AM bluestore Fix #58759: BlueFS log runway space exhausted
- https://github.com/ceph/ceph/pull/42750
https://github.com/ceph/ceph/pull/48854
this two patch already backport to ... - 03:16 AM rgw Bug #65277 (Pending Backport): rgw: update options yaml file so LDAP uri isn't an invalid example
- LDAP tries to bind the URI configuration option when RGW starts. The default value is an example used to show the for...
- 12:33 AM Linux kernel client Bug #51279: kclient hangs on umount (testing branch)
- Milind Changire wrote:
> main: (Data Uninlining PR)
> https://pulpito.ceph.com/mchangir-2024-03-30_16:00:44-fs:func... - 12:31 AM mgr Cleanup #65266: mgr: unit testing for mgr modules starting up
- at least the existing dashboard exercises this case. but the missing tests is the ones which use python 3.6.8. and th...
- 12:19 AM CephFS Bug #65256 (Duplicate): umount: /home/ubuntu/cephtest/mnt.0: target is busy.
04/02/2024
- 11:32 PM CephFS Bug #65276 (Fix Under Review): MDS daemon is using 50% CPU when idle
- 10:52 PM CephFS Bug #65276 (Pending Backport): MDS daemon is using 50% CPU when idle
- MDS daemons on linux machines take ~50% CPU, with the quiesce db mgr thread at the top.
Analysis of the issue show... - 10:34 PM CephFS Backport #65275 (In Progress): squid: mds: some request errors come from errno.h rather than fs_types.h
- https://github.com/ceph/ceph/pull/56663
- 10:34 PM CephFS Backport #65274 (In Progress): reef: mds: some request errors come from errno.h rather than fs_types.h
- https://github.com/ceph/ceph/pull/56664
- 10:34 PM CephFS Backport #65273 (In Progress): squid: PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- https://github.com/ceph/ceph/pull/57061
- 10:34 PM CephFS Backport #65272 (New): reef: PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- 10:31 PM CephFS Bug #51282 (Resolved): pybind/mgr/mgr_util: .mgr pool may be created too early causing spurious PG_DEGRADED warnings
- 10:30 PM CephFS Bug #65018 (Pending Backport): PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- 10:26 PM CephFS Bug #64490 (Pending Backport): mds: some request errors come from errno.h rather than fs_types.h
- 10:16 PM CephFS Bug #65271 (Fix Under Review): qa: cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log
- 10:14 PM CephFS Bug #65271 (Pending Backport): qa: cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled" in cluster log
- 10:11 PM CephFS Bug #65136: QA failure: test_fscrypt_dummy_encryption_with_quick_group
- https://pulpito.ceph.com/pdonnell-2024-04-02_11:52:43-fs-wip-batrick-testing-20240402.004512-distro-default-smithi/76...
- 08:29 PM rgw Backport #65004 (Resolved): squid: [CVE-2023-46159] RGW crash upon misconfigured CORS rule
- 08:29 PM rgw Backport #65009 (Resolved): squid: rgw-nfs: various file mv (rename) operations fail
- 08:17 PM CephFS Bug #65265: qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Looks like the MGR went down because of:
/a/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-tes... - 02:58 PM CephFS Bug #65265 (New): qa: health warning "no active mgr (MGR_DOWN)" occurs before and after test_nfs runs
- Link to the job - https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing...
- 04:14 PM Dashboard Feature #65268 (Resolved): mgr/dashboard: update NVMe-oF API "listener add" sync
- Let's sync to the "latest" proto file.
- 04:02 PM RADOS Bug #65186 (Fix Under Review): OSDs unreachable in upgrade test
- 12:34 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- mon.a logs shows:...
- 03:44 PM CephFS Documentation #65267 (Pending Backport): doc: fix configurables section in doc/cephfs/fs-volumes.rst
- 03:25 PM Orchestrator Bug #64536 (Resolved): cephadm/nvmeof: scrape nvmeof prometheus endpoint
- 03:25 PM Orchestrator Backport #64636 (Resolved): squid: cephadm/nvmeof: scrape nvmeof prometheus endpoint
- 03:24 PM Orchestrator Bug #64491 (Resolved): cephadm: ceph-exporter fails to deploy when placed first
- 03:24 PM Orchestrator Backport #64626 (Resolved): squid: cephadm: ceph-exporter fails to deploy when placed first
- 03:21 PM mgr Cleanup #65266 (New): mgr: unit testing for mgr modules starting up
- Inspired by https://github.com/ceph/ceph/pull/56617#issue-2218534302, we saw that issues with the mgr modules caused ...
- 03:15 PM mgr Backport #65153: reef: pybind/mgr/devicehealth: "rados.ObjectNotFound: [errno 2] RADOS object not found (Failed to operate read op for oid $dev"
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56479
merged - 03:15 PM mgr Backport #65153 (Resolved): reef: pybind/mgr/devicehealth: "rados.ObjectNotFound: [errno 2] RADOS object not found (Failed to operate read op for oid $dev"
- 02:59 PM rbd Bug #63422: librbd crash in journal discard wait_event
- Hey Raimund, as far as I can tell, the default is true in all recent releases. Which release do you see this in? What...
- 07:02 AM rbd Bug #63422: librbd crash in journal discard wait_event
- Joshua Baergen wrote:
> As a consequence for the above, a very simple workaround for this crash appears to be to set... - 02:54 PM Dashboard Backport #64775 (Resolved): squid: mgr/dashboard: fix nvmeof documentation and traddr issue
- 02:54 PM Dashboard Backport #64515 (Resolved): squid: mgr/dashboard: nvmeof api broken for v1.0.0
- 02:53 PM Dashboard Backport #65100 (Resolved): squid: mgr/dashboard: update NVMe-oF API
- 02:51 PM devops Backport #65032: reef: add support for openEuler OS
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56361
merged - 02:49 PM Dashboard Backport #65028: reef: mgr/dashboard: Develop a Chinese version for dashboard
- https://github.com/ceph/ceph/pull/56359 merged
- 02:29 PM Bug #58956: Error while deploying Ceph Quincy via ceph-ansible-stable-7 on Rocky 9
- Hi,
we deployed a second cluster v17.2.7 a few weeks ago with docker + Rocky 9 and the Problem did not occur again... - 02:19 PM CephFS Bug #65022 (Fix Under Review): qa: test_max_items_per_obj open procs not fully cleaned up
- 02:16 PM CephFS Bug #65022 (In Progress): qa: test_max_items_per_obj open procs not fully cleaned up
- 01:57 PM Backport #64754 (Resolved): squid: No matching package to install: 'qatlib-devel'
- 01:00 PM Orchestrator Bug #65263 (New): upgrade stalls after upgrading one ceph-mgr daemon
- Seen here: https://pulpito.ceph.com/yuriw-2024-03-29_19:10:12-fs-wip-yuri11-testing-2024-03-28-0753-reef-distro-defau...
- 01:00 PM CephFS Bug #65262 (Triaged): qa/cephfs: kernel_untar_build.sh failed due to build error
- https://pulpito.ceph.com/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smit...
- 12:48 PM CephFS Tasks #64413: File size is not correct after rmw
- > is that somehow tricking the client to believe that the file size (which is stored as an xattr in fscrypt) can be u...
- 12:46 PM RADOS Bug #64972 (Fix Under Review): qa: "ceph tell 4.3a deep-scrub" command not found
- 12:43 PM CephFS Bug #65261 (New): qa/cephfs: cephadm related failure on fs/upgrade job
- https://pulpito.ceph.com/rishabh-2024-03-29_18:05:24-fs-wip-rishabh-testing-20240327.051042-reef-testing-default-smit...
- 12:02 PM CephFS Bug #63931 (In Progress): qa: test_mirroring_init_failure_with_recovery failure
- @Venky,
When we disable mirroring using 'ceph mgr module disable mirroring' and then enable using 'ceph fs mirror ... - 10:32 AM CephFS Bug #65260 (In Progress): mds: Reduce log level for messages when mds is stopping
- 10:31 AM CephFS Bug #65260 (Pending Backport): mds: Reduce log level for messages when mds is stopping
- Reducing the mds log level while the
mds is stopping to track down issues like mds
becoming laggy while stopping e.... - 08:18 AM nvme-of Feature #65259 (Resolved): cephadm - make changes to ceph-nvmeof.conf template
- Add these fields to the conf file under the "gateway" section and allow to configure them:
* allowed_consecutive_s... - 07:55 AM Backport #65258 (New): reef: The way to know the data format of each OSD and MON was created
- 07:55 AM Backport #65257 (New): quincy: The way to know the data format of each OSD and MON was created
- 07:48 AM Feature #57515 (Pending Backport): The way to know the data format of each OSD and MON was created
- 07:27 AM CephFS Backport #63590 (Resolved): reef: qa: fs:mixed-clients kernel_untar_build failure
- Marking this resolved since backport has been merged - https://github.com/ceph/ceph/pull/54711#event-12186859893
- 07:25 AM CephFS Bug #64685 (Resolved): mds: disable defer_client_eviction_on_laggy_osds by default
- 07:25 AM CephFS Backport #64924 (Resolved): squid: mds: disable defer_client_eviction_on_laggy_osds by default
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56194
Merged. - 07:24 AM CephFS Backport #64925 (Resolved): quincy: mds: disable defer_client_eviction_on_laggy_osds by default
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56195
Merged. - 07:24 AM CephFS Backport #64926 (Resolved): reef: mds: disable defer_client_eviction_on_laggy_osds by default
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56196
Merged. - 07:23 AM CephFS Backport #64617 (Resolved): squid: mds: check the layout in Server::handle_client_mknod
- 07:23 AM CephFS Backport #64618 (Resolved): reef: mds: check the layout in Server::handle_client_mknod
- 07:23 AM CephFS Bug #65256 (Duplicate): umount: /home/ubuntu/cephtest/mnt.0: target is busy.
- https://pulpito.ceph.com/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633108/...
- 07:11 AM RADOS Bug #64938: Pool created with single PG splits into many on single OSD causes OSD to hit max_pgs_per_osd
- The pg_autoscaler adjusts the pg_num through _maybe_adjust based on pg_num_min in this scenario. The DaemonServer the...
- 07:09 AM CephFS Backport #64762 (Resolved): squid: qa/cephfs: add MON_DOWN and `deprecated feature inline_data' to health ignorelist.
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56021
Merged. - 06:47 AM CephFS Backport #63688 (Resolved): reef: qa/cephfs: improvements for name generators in test_volumes.py
- Rishabh Dave wrote:
> https://github.com/ceph/ceph/pull/54729
Merged. - 06:10 AM RADOS Bug #64917 (Resolved): SnapMapperTest.CheckObjectKeyFormat object key changed
- 05:59 AM crimson Bug #62162: local_shared_foreign_ptr: Assertion `ptr && *ptr' failed
- https://pulpito.ceph.com/yingxin-2024-04-02_02:43:05-crimson-rados-ci-yingxin-crimson-load-obc-3-base-distro-default-...
- 05:56 AM crimson Bug #63647: SnapTrimEvent AddressSanitizer: heap-use-after-free
- https://pulpito.ceph.com/yingxin-2024-04-02_02:43:05-crimson-rados-ci-yingxin-crimson-load-obc-3-base-distro-default-...
- 05:43 AM CephFS Bug #57206 (Resolved): ceph_test_libcephfs_reclaim crashes during test
- This ticket was incorrectly marked as rejected. The patch for this ticket was merged - https://github.com/ceph/ceph/p...
- 05:00 AM CephFS Backport #63782: reef: qa: add support/qa for cephfs-shell on CentOS 9 / RHEL9
- Patrick Donnelly wrote:
> Venky Shankar wrote:
> > Also need to include changes from https://github.com/ceph/ceph/p... - 04:23 AM CephFS Backport #64737 (Resolved): reef: Memory leak detected when accessing a CephFS volume from Samba using libcephfs
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56122
Merged. - 04:23 AM Dashboard Backport #65255 (Resolved): squid: mgr/dashboard: use alertmanager v2 APIs mgr/dashboard: short_description
- https://github.com/ceph/ceph/pull/57054
- 04:22 AM Dashboard Cleanup #65070 (Pending Backport): mgr/dashboard: use alertmanager v2 APIs mgr/dashboard: short_description
- 04:21 AM Dashboard Bug #65254 (Duplicate): mgr/dashboard: mgr/dashboard: start using alertmanager v2
- 04:19 AM Dashboard Bug #65254 (Duplicate): mgr/dashboard: mgr/dashboard: start using alertmanager v2
- there is an api v2 for alertmanager which also has an endpoint like alerts/groups which might be something that is us...
- 04:18 AM Dashboard Backport #65248: squid: mgr/dashboard: Export rgw multisite status via the API
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56623
- 04:15 AM Dashboard Backport #65248 (New): squid: mgr/dashboard: Export rgw multisite status via the API
- 04:13 AM Dashboard Backport #65248 (In Progress): squid: mgr/dashboard: Export rgw multisite status via the API
- 01:58 AM Dashboard Backport #65211 (In Progress): reef: mgr/dashboard: Mark placement targets as non-required
- 01:53 AM rgw Bug #53029 (Resolved): radosgw-admin fails on "sync status" if a single RGW process is down
- Fixed in the solution for https://tracker.ceph.com/issues/62710
- 01:48 AM Dashboard Backport #65210 (In Progress): squid: mgr/dashboard: Mark placement targets as non-required
- 01:47 AM rgw Backport #64448 (Resolved): reef: invalid olh attributes on the target object after copy_object in a versioning suspended bucket
- 12:58 AM Linux kernel client Bug #51279: kclient hangs on umount (testing branch)
- main: (Data Uninlining PR)
https://pulpito.ceph.com/mchangir-2024-03-30_16:00:44-fs:functional-wip-mchangir-testing1...
04/01/2024
- 11:29 PM crimson Backport #65253 (New): squid: client io requests hang when issued before the creation of the related pgs
- 11:26 PM crimson Bug #64546 (Pending Backport): client io requests hang when issued before the creation of the related pgs
- 10:41 PM Orchestrator Bug #64208: test_cephadm.sh: Container version mismatch causes job to fail.
- /a/yuriw-2024-03-29_19:10:12-rados-wip-yuri11-testing-2024-03-28-0753-reef-distro-default-smithi/7629818
- 10:40 PM bluestore Bug #63121: KeyValueDB/KVTest.RocksDB_estimate_size tests failing
- /a/yuriw-2024-03-29_19:10:12-rados-wip-yuri11-testing-2024-03-28-0753-reef-distro-default-smithi/7629731
- 10:28 PM RADOS Bug #64437: qa/standalone/scrub/osd-scrub-repair.sh: TEST_repair_stats_ec: test 26 = 13
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620501
- 10:26 PM RADOS Bug #50371: Segmentation fault (core dumped) ceph_test_rados_api_watch_notify_pp
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620506
- 10:20 PM Orchestrator Bug #65017: cephadm: log_channel(cephadm) log [ERR] : Failed to connect to smithi090 (10.0.0.9). Permission denied
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620502
- 10:16 PM Dashboard Bug #65129 (Duplicate): Unhandled Dashboard exception related to CherryPy
- 10:15 PM Orchestrator Bug #52109: test_cephadm.sh: Timeout('Port 8443 not free on 127.0.0.1.',)
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620513
- 09:59 PM RADOS Bug #64917: SnapMapperTest.CheckObjectKeyFormat object key changed
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620572
- 09:55 PM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620621
- 09:54 PM RADOS Bug #56393: failed to complete snap trimming before timeout
- /a/yuriw-2024-03-24_22:19:24-rados-wip-yuri10-testing-2024-03-24-1159-distro-default-smithi/7620687
- 08:04 PM CephFS Tasks #64307 (Closed): Buffered write failure
- Fixed in: https://github.com/ceph/ceph/commit/9a083b0935509744234082832d12ed2734bcb6e0 and commit in wip-fscrypt.
... - 07:59 PM CephFS Tasks #64413 (Resolved): File size is not correct after rmw
- 07:21 PM CephFS Tasks #64413: File size is not correct after rmw
- See patch below for fix:...
- 07:42 PM CephFS Tasks #64723 (Closed): ffsb configure issues (gcc fails)
- Fixed by: https://tracker.ceph.com/issues/64413#note-8. The commit is in wip-fscrypt.
- 07:41 PM rgw Bug #64571: lifecycle transition crashes since reloading bucket attrs for notification
- Soumya Koduri wrote:
> >>>
> #0 __pthread_kill_implementation (
> threadid=<optimized out>, signo=signo@entry... - 07:37 PM rgw Bug #62710: multisite replication is super slow when some of the rgws configured in zonegroup are down
- @Jane,
The below changes - https://github.com/ceph/ceph/pull/53320/commits/e200499bb3c5703862b92a4d7fb534d98601f1b... - 07:26 PM CephFS Tasks #64691: Symlink target not set correctly in unencrypted dir
- Dhairya Parmar wrote:
> Christopher Hoffman wrote:
> > in->symlink_plain wasn't being set in case of non-fscrypt.
... - 07:23 PM CephFS Backport #64618: reef: mds: check the layout in Server::handle_client_mknod
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56031
merged - 07:04 PM rgw Bug #65251 (Pending Backport): rgw: crash in lc while transitioning to cloud
- This bug is track one of the issues reported with LC process which is crash in cloud-transition code path (in https:/...
- 05:29 PM Orchestrator Bug #64864 (Fix Under Review): cephadm: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c in cluster log
- 04:52 PM Orchestrator Bug #64864: cephadm: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c in cluster log
- /a/yuriw-2024-03-26_14:32:05-rados-wip-yuri8-testing-2024-03-25-1419-distro-default-smithi/7623410
/a/yuriw-2024-03-... - 04:02 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- Laura Flores wrote:
> /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616011/remote/smithi087... - 03:12 PM CephFS Bug #65230 (Resolved): upgrade/cephfs/mds_upgrade_sequence: Insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)
- This is resolved by some commit by Patrick that were merged in squid branch last week: https://github.com/ceph/ceph/p...
- 12:42 PM CephFS Bug #65230 (Triaged): upgrade/cephfs/mds_upgrade_sequence: Insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)
- 03:03 PM CephFS Bug #61265 (Duplicate): qa: tasks.cephfs.fuse_mount:process failed to terminate after unmount
- Duplicate of https://tracker.ceph.com/issues/64502
- 02:59 PM CephFS Bug #57676: qa: error during scrub thrashing: rank damage found: {'backtrace'}
- 12/54 failures in https://pulpito.ceph.com/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/ is this iss...
- 01:10 PM CephFS Bug #57676: qa: error during scrub thrashing: rank damage found: {'backtrace'}
- Kotersh, please take this one. It would be good to RCA this since this is showing up a lot in our fs suite runs.
- 02:49 PM Bug #65249 (Resolved): peering_graph.generated.dot renders weird
- The Sample State model graph (peering_graph.generated.dot) in doc/dev/peering.rst is not rendering properly. This was...
- 02:32 PM CephFS Bug #65022: qa: test_max_items_per_obj open procs not fully cleaned up
- https://pulpito.ceph.com/teuthology-2024-03-31_21:24:02-fs-squid-distro-default-smithi/7633108
Patrick, do we have... - 02:31 PM Dashboard Backport #65248 (New): squid: mgr/dashboard: Export rgw multisite status via the API
- https://github.com/ceph/ceph/pull/56623
- 02:21 PM crimson Bug #65201 (Fix Under Review): ReplicatedRecoveryBackend::prep_push_to_replica(const hobject_t&, eversion_t, pg_shard_t) Assertion ssc
- 02:20 PM crimson Bug #65200 (Fix Under Review): PeeringState::get_peer_info(pg_shard_t) const: Assertion `it != peer_info.end()' failed.
- 02:20 PM crimson Bug #65203 (Fix Under Review): ReplicatedRecoveryBackend::recalc_subsets(ObjectRecoveryInfo&, crimson::osd::SnapSetContextRef): Assertion `ssc' failed.
- 02:19 PM crimson Bug #65247 (Need More Info): ObjectContext::drop_recovery_read(): Assertion `recovery_read_marker' failed.
- * After adding a restart OSDs to the thrash tests: https://github.com/ceph/ceph/pull/56511
* Results are not from ma... - 01:36 PM Linux kernel client Bug #51279: kclient hangs on umount (testing branch)
- Xiubo Li wrote:
> https://pulpito.ceph.com/mchangir-2024-03-30_16:00:44-fs:functional-wip-mchangir-testing1-main-202... - 01:25 PM Linux kernel client Bug #51279: kclient hangs on umount (testing branch)
- https://pulpito.ceph.com/mchangir-2024-03-30_16:00:44-fs:functional-wip-mchangir-testing1-main-20240328-072412-distro...
- 01:25 PM Linux kernel client Bug #51279 (In Progress): kclient hangs on umount (testing branch)
- Reopen it since we see this again.
- 01:00 PM CephFS Bug #65116: squid: kclient: "ld: final link failed: Resource temporarily unavailable"
- Patrick Donnelly wrote:
> [...]
>
> From: /teuthology/pdonnell-2024-03-24_04:56:01-fs-wip-batrick-testing-2024032... - 12:53 PM CephFS Bug #65116 (Triaged): squid: kclient: "ld: final link failed: Resource temporarily unavailable"
- 12:56 PM CephFS Bug #65246 (Triaged): qa/cephfs: test_multifs_single_path_rootsquash (tasks.cephfs.test_admin.TestFsAuthorize)
- 12:42 PM CephFS Bug #65246: qa/cephfs: test_multifs_single_path_rootsquash (tasks.cephfs.test_admin.TestFsAuthorize)
- The issue can't be reproduced with kclient - https://pulpito.ceph.com/rishabh-2024-04-01_06:24:56-fs:functional-risha...
- 12:39 PM CephFS Bug #65246: qa/cephfs: test_multifs_single_path_rootsquash (tasks.cephfs.test_admin.TestFsAuthorize)
- I was successful in reproducing this test failure with a recent version main branch - https://pulpito.ceph.com/rishab...
- 10:44 AM CephFS Bug #65246 (Fix Under Review): qa/cephfs: test_multifs_single_path_rootsquash (tasks.cephfs.test_admin.TestFsAuthorize)
- The failure was found here - https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131...
- 12:54 PM CephFS Bug #65021 (Triaged): qa/suites/fs/nfs: cluster [WRN] Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log
- Dhairya, please have a look.
- 12:46 PM CephFS Bug #65171 (Triaged): Provide metrics support for the Replication Start/End Notifications
- 12:46 PM CephFS Bug #65171: Provide metrics support for the Replication Start/End Notifications
- Jos, please take this one.
- 12:45 PM CephFS Bug #65224: mds: fs subvolume rm fails
- Milind, please link the debug PR here for completeness.
- 12:45 PM CephFS Bug #65224 (Triaged): mds: fs subvolume rm fails
- 12:44 PM CephFS Bug #65225 (Triaged): ceph_assert on dn->get_projected_linkage()->is_remote
- 12:37 PM Orchestrator Bug #64865 (Fix Under Review): cephadm: Health check failed: 1 osds down (OSD_DOWN) in cluster log
- 11:41 AM Linux kernel client Bug #64471: kernel: upgrades from quincy/v18.2.[01]/reef to main|squid fail with kernel oops
- Xiubo Li wrote:
> Patrick and Venky,
>
> I raised a PR in *teuthology* to add a *kdb* option to allow us disable ... - 09:23 AM rgw Backport #65245 (New): reef: RGW/s3select : several issues, s3select related, some caused a crash.
- 09:23 AM rgw Backport #65244 (New): squid: RGW/s3select : several issues, s3select related, some caused a crash.
- 09:21 AM CephFS Bug #48562: qa: scrub - object missing on disk; some files may be lost
- Milind Changire wrote:
> According to qa/tasks/cephfs/test_forward_scrub.py the test that causes 'stat testdir/hardl... - 09:19 AM rgw Bug #65243 (Pending Backport): RGW/s3select : several issues, s3select related, some caused a crash.
BZ's
2254121 2254122 2254125 2252396 2242924
https://github.com/ceph/ceph/pull/56279
https://github.com/ceph/c...- 07:50 AM CephFS Bug #65073: pybind/mgr/stats/fs: log exceptions to cluster log
- This _can_ happen when @FSPerfStats.re_register_queries@ is called before mgr/stats can process a single mds report.
- 07:39 AM rgw Bug #63428: RGW: multipart get wrong storage class metadata
- https://github.com/ceph/ceph/pull/56608
- 07:36 AM RADOS Bug #64972: qa: "ceph tell 4.3a deep-scrub" command not found
- Patrick's analysis is correct. The command change did create a problem for all 'upgrade' tests. I am considering the ...
- 04:55 AM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- centos8.stream run w/ main branch: https://pulpito.ceph.com/vshankar-2024-04-01_04:53:22-fs:upgrade:mds_upgrade_seque...
- 04:40 AM CephFS Bug #65043: Unable to set timestamp to value > UINT32_MAX
- Sachin Prabhu wrote:
> Greg Farnum wrote:
> > We use unsigned int, which takes us to year 2106 — we'll have to fix ... - 02:45 AM devops Bug #65242: vstart start test cluster failed
- I install rockylinx 9 on macos visa Parallels Desktop.macos and rockylinux share ceph source code.After building ceph...
- 02:34 AM devops Bug #65242 (New): vstart start test cluster failed
- [root@node16 build]# MON=1 OSD=2 RGW=1 MGR=1 ../src/vstart.sh -d -n -X --without-dashboard
** going verbose **
..... - 01:44 AM CephFS Bug #64977 (Need More Info): mds spinlock due to lock contention leading to memory exaustion
- 01:44 AM CephFS Bug #64977: mds spinlock due to lock contention leading to memory exaustion
- There have two cases will send the *lookup* request and in the dentry revalidation one it may retry it infinitely, I ...
- 01:41 AM CephFS Bug #63906 (Fix Under Review): Inconsistent file mode across two clients
- 01:30 AM devops Bug #65175 (Fix Under Review): ccache is always miss in confusa14
- 01:28 AM Bug #65241 (In Progress): valgrind issue due to "use after free"
- /ceph/teuthology-archive/yuriw-2024-03-26_14:32:05-rados-wip-yuri8-testing-2024-03-25-1419-distro-default-smithi/7623...
- 12:37 AM CephFS Bug #64707: suites/fsstress.sh hangs on one client - test times out
- Venky Shankar wrote:
> Laura Flores wrote:
> > Looks like a case of this tracker:
> >
> > /a/teuthology-2024-03-...
03/31/2024
- 08:23 AM RADOS Bug #57845: MOSDRepOp::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_OCTOPUS)' failed
- /a/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621805
- 08:11 AM Orchestrator Bug #64868: cephadm/osds, cephadm/workunits: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) in cluster log
- /a/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621676
/a/yuriw-2024-03-... - 08:06 AM RADOS Bug #64917: SnapMapperTest.CheckObjectKeyFormat object key changed
- /a/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621668
/a/yuriw-2024-03-... - 07:51 AM Orchestrator Bug #65017: cephadm: log_channel(cephadm) log [ERR] : Failed to connect to smithi090 (10.0.0.9). Permission denied
- /a/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621636
- 07:47 AM RADOS Bug #64437: qa/standalone/scrub/osd-scrub-repair.sh: TEST_repair_stats_ec: test 26 = 13
- /a/yuriw-2024-03-25_20:36:27-rados-wip-yuri3-testing-2024-03-25-0744-distro-default-smithi/7621635
- 06:58 AM Documentation #65239 (In Progress): Update SHEC Documentation
- Original report, from https://pad.ceph.com/p/Report_Documentation_Bugs
BEGIN QUOTED TEXT
https://docs.ceph.com/en/q...
03/30/2024
- 06:49 PM RADOS Bug #65185 (In Progress): OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- There is a problem with most 'upgrade' tests, which may be the culprit here:
The command to initiate a deep scrub wa... - 08:18 AM RADOS Bug #63891: mon/AuthMonitor: fix potential repeated global_id
- cephfs client depend on global_id, and repeated global_id would result some inconsistent problem
03/29/2024
- 08:29 PM mgr Bug #65189 (In Progress): Telemetry pacific-x upgrade test pauses when upgrading to squid
- 04:46 PM mgr Bug #65189: Telemetry pacific-x upgrade test pauses when upgrading to squid
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629093
- 06:31 PM Dashboard Bug #47612: ERROR: setUpClass (tasks.mgr.dashboard.test_health.HealthTest)
- https://jenkins.ceph.com/job/ceph-api/71512/...
- 05:03 PM CephFS Bug #64707: suites/fsstress.sh hangs on one client - test times out
- Laura Flores wrote:
> Looks like a case of this tracker:
>
> /a/teuthology-2024-03-22_02:08:13-upgrade-squid-dist... - 04:49 PM CephFS Bug #64707: suites/fsstress.sh hangs on one client - test times out
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629115
- 04:56 PM CephFS Bug #61265: qa: tasks.cephfs.fuse_mount:process failed to terminate after unmount
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629107
/a/teuthology-2024-03-29_02:08:11-upgr... - 04:54 PM Orchestrator Bug #65236 (New): upgrade/cephfs/mds_upgrade_sequence: cephadm refresh fails; podman times out
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629119...
- 04:50 PM Orchestrator Bug #64868: cephadm/osds, cephadm/workunits: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) in cluster log
- Same for this upgrade test.
description: upgrade/quincy-x/parallel/{0-random-distro$/{ubuntu_22.04} 0-start 1-task... - 04:46 PM CephFS Bug #64502: pacific/quincy/v18.2.0: client: ceph-fuse fails to unmount after upgrade to main
- This issue is likely related to libfuse library in centos.9. To confirm this, I'll run the tests with centos.8 to ver...
- 04:45 PM RADOS Bug #65235 (New): upgrade/reef-x/stress-split: "OSDMAP_FLAGS: noscrub flag(s) set" warning in cluster log
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629127...
- 04:41 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629109
- 04:13 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629092
- 04:41 PM Orchestrator Bug #65234: upgrade/quincy-x/stress-split: cephadm failed to parse grafana.ini file due to inadequate permission
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629121
/a/teuthology-2024-03-29_02:08:11-upgr... - 04:32 PM Orchestrator Bug #65234 (Pending Backport): upgrade/quincy-x/stress-split: cephadm failed to parse grafana.ini file due to inadequate permission
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629110...
- 04:38 PM Dashboard Bug #65146 (Resolved): mgr/dashboard: rgw display name with dot gives validation error
- 04:38 PM Dashboard Backport #65193 (Resolved): squid: mgr/dashboard: rgw display name with dot gives validation error
- 04:38 PM Dashboard Backport #65204 (Resolved): squid: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 04:37 PM Dashboard Cleanup #64658 (Resolved): mgr/dashboard: Locking improvements in bucket create form
- 04:37 PM Dashboard Backport #64829 (Resolved): squid: mgr/dashboard: Locking improvements in bucket create form
- 04:37 PM Dashboard Backport #64830 (Resolved): reef: mgr/dashboard: Locking improvements in bucket create form
- 04:17 PM Orchestrator Bug #65233: upgrade/cephfs/mds_upgrade_sequence: 'ceph orch ps' command times out
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629102
- 04:16 PM Orchestrator Bug #65233 (New): upgrade/cephfs/mds_upgrade_sequence: 'ceph orch ps' command times out
- /a/teuthology-2024-03-29_02:08:11-upgrade-squid-distro-default-smithi/7629126...
- 04:05 PM rgw Bug #65232 (New): warning: ‘newDBStore’ violates the C++ One Definition Rule
- compiler warning from centos9 build, ex https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x8...
- 04:04 PM RADOS Bug #64972: qa: "ceph tell 4.3a deep-scrub" command not found
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7615997
- 04:03 PM RADOS Bug #64972: qa: "ceph tell 4.3a deep-scrub" command not found
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616010...
- 04:00 PM RADOS Bug #65231: upgrade/quincy-x/parallel: "Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616008
/a/teuthology-2024-03-22_02:08:13-upgr... - 03:59 PM RADOS Bug #65231: upgrade/quincy-x/parallel: "Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
- Probably normal for upgrade tests to have these, but should be investigated.
- 03:58 PM RADOS Bug #65231 (New): upgrade/quincy-x/parallel: "Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7615990...
- 03:55 PM CephFS Bug #65230: upgrade/cephfs/mds_upgrade_sequence: Insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616012
- 03:53 PM CephFS Bug #65230 (Resolved): upgrade/cephfs/mds_upgrade_sequence: Insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY)
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616023...
- 03:48 PM CephFS Bug #50279: qa: "Replacing daemon mds.b as rank 0 with standby daemon mds.c"
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616031...
- 03:09 PM Bug #65228 (Fix Under Review): class:device-class config database mask does not work for osd_compact_on_start
- I have a cluster which needs osd_compact_on_start = true
In this cluster this only applicable to ssd OSDs (where rgw... - 01:31 PM CephFS Bug #48562: qa: scrub - object missing on disk; some files may be lost
- https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/76...
- 01:02 PM RADOS Bug #59670: Ceph status shows PG recovering when norecover flag is set
- I think its more than just a cosmetic issue of the PG showing recovering as its state. It does in fact "recover" obje...
- 12:54 PM RADOS Bug #65227 (New): noscrub cluster flag prevents deep-scrubs from starting
- Observed on a 17.2.7 cluster and confirmed on an additional 17.2.7 cluster.
Reproduction steps:
- On a cluster th... - 12:25 PM bluestore Bug #53899: bluefs _allocate allocation failed - BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")
- You're using DB volume collocated with main one. So BlueFS shares the same space as user data. And your OSD ran out o...
- 12:20 PM bluestore Bug #53899: bluefs _allocate allocation failed - BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")
- Thanks Igor for investigating.
I worked around it now by re-provisioning the OSD, and letting redundancy fix it.
... - 10:47 AM bluestore Bug #53899: bluefs _allocate allocation failed - BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")
- Hi Niklas,
looks like your BlueFS is almost full. Given the following output
-5> 2024-03-28T23:55:26.039+0000... - 12:24 AM bluestore Bug #53899: bluefs _allocate allocation failed - BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")
- Small addition: `ceph osd status` shows the following:...
- 12:16 AM bluestore Bug #53899: bluefs _allocate allocation failed - BlueFS.cc: 2768: ceph_abort_msg("bluefs enospc")
- @Igor
It seems I hit this bug with a cluster initially that had initially deployed 16.2.7 and then upgraded to 18.... - 11:58 AM bluestore Bug #59811 (Duplicate): crash: RocksDBStore::estimate_prefix_size(std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)
- 11:57 AM bluestore Bug #59808 (Duplicate): crash: rocksdb::DBImpl::GetApproximateSizes(rocksdb::SizeApproximationOptions const&, rocksdb::ColumnFamilyHandle*, rocksdb::Range const*, int, unsigned long*)
- 11:56 AM bluestore Bug #60114 (Duplicate): crash: rocksdb::DBImpl::GetApproximateSizes(rocksdb::SizeApproximationOptions const&, rocksdb::ColumnFamilyHandle*, rocksdb::Range const*, int, unsigned long*)
- 10:53 AM Bug #65226 (Fix Under Review): qa: add a test - peer status show "failed" status for making changes to the remote .snap directory
- 10:45 AM Bug #65226 (Fix Under Review): qa: add a test - peer status show "failed" status for making changes to the remote .snap directory
- 10:16 AM CephFS Bug #65225 (Triaged): ceph_assert on dn->get_projected_linkage()->is_remote
- In a workload that is heavily hardlinking and moving files, we see ceph-mds assert like the following ...
- 09:52 AM CephFS Bug #65224 (Triaged): mds: fs subvolume rm fails
- `fs subvolume rm` fails when subvolume dir attempted to move to a different dir where the following code fails in src...
- 08:38 AM CephFS Feature #61334: cephfs-mirror: use snapdiff api for efficient tree traversal
- Venky Shankar wrote:
> Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and l... - 08:16 AM CephFS Feature #61334 (Pending Backport): cephfs-mirror: use snapdiff api for efficient tree traversal
- Jos, let's hold off the backport till we get fs:mirror green (fixing unrelated failures and letting the code bake a b...
- 08:13 AM CephFS Feature #61334: cephfs-mirror: use snapdiff api for efficient tree traversal
- Jos Collin wrote:
> Venky Shankar wrote:
> > Not a candidate for backport right now to allow more bake time in main... - 06:17 AM CephFS Feature #61334: cephfs-mirror: use snapdiff api for efficient tree traversal
- Venky Shankar wrote:
> Not a candidate for backport right now to allow more bake time in main branch.
What about ... - 05:38 AM CephFS Feature #61334 (Resolved): cephfs-mirror: use snapdiff api for efficient tree traversal
- Not a candidate for backport right now to allow more bake time in main branch.
- 08:23 AM CephFS Backport #65223 (New): squid: cephfs-mirror: use snapdiff api for efficient tree traversal
- 08:23 AM CephFS Backport #65222 (New): reef: cephfs-mirror: use snapdiff api for efficient tree traversal
- 08:01 AM Dashboard Backport #65221 (New): reef: ceph-mixin: Add RBD Mirror monitoring alerts
- 08:01 AM Dashboard Backport #65220 (New): squid: ceph-mixin: Add RBD Mirror monitoring alerts
- 07:57 AM Dashboard Bug #65219 (Pending Backport): ceph-mixin: Add RBD Mirror monitoring alerts
- 07:56 AM Dashboard Bug #65218 (New): mgr/dashboard: Grafana ceph-cluster.json doesn't support cluster label
- h3. Description of problem
Grafana ceph-cluster.json doesn't support cluster label.
We don't have jsonnet for ... - 07:53 AM Dashboard Backport #65194 (Resolved): reef: mgr/dashboard: rgw display name with dot gives validation error
- 07:13 AM Dashboard Backport #64790 (Resolved): reef: mgr/dashboard: In rgw multisite, during zone creation acess/secret key should not be compulsory provide an edit option to set these keys
- 07:12 AM Dashboard Backport #65169 (Resolved): squid: mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- 06:49 AM Dashboard Bug #65173 (Resolved): mgr/dashboard: s/active_mds/active_nfs in fs attach form
- 06:49 AM Dashboard Backport #65191 (Resolved): squid: mgr/dashboard: s/active_mds/active_nfs in fs attach form
- 06:49 AM Dashboard Backport #65192 (Resolved): reef: mgr/dashboard: s/active_mds/active_nfs in fs attach form
- 06:44 AM Dashboard Cleanup #65030 (Resolved): mgr/dashboard: sanitize dashboard user input field
- 06:44 AM Dashboard Backport #65197 (Resolved): quincy: mgr/dashboard: sanitize dashboard user input field
- 03:36 AM Bug #65199: autoscaler: Scale PGs based on number of objects
- Aside:
It also feels wrong that the inode information shows up as "0 B". It looks like storage is gone missing ine... - 03:12 AM CephFS Bug #65217 (Fix Under Review): cephfs: add fscrypt protection support from non-fscrypt client
- Clients that do not support fscrypt can execute operations that may cause unrecoverable data loss. Add protection on ...
- 03:07 AM Feature #64845: Support read_from_replica everywhere
- Here's a proposal for option (2) from Yehuda https://github.com/ceph/ceph/pull/56180
- 03:05 AM RADOS Bug #59196: ceph_test_lazy_omap_stats segfault while waiting for active+clean
- Closing https://github.com/ceph/ceph/pull/55596 in favour of https://github.com/ceph/ceph/pull/56574
- 12:38 AM rgw Bug #65216: rgw: only accept valid ipv4 from host header
- PR: https://github.com/ceph/ceph/pull/56573
- 12:30 AM rgw Bug #65216 (In Progress): rgw: only accept valid ipv4 from host header
- Right now the validation for ipv4 from the host header is based on the number of periods - this leads to accepting in...
03/28/2024
- 11:06 PM CephFS Bug #65018: PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- Sam and I did some digging around this bug.
Here is the timestamp of the DEGRADED warning in the cluster log. The ... - 08:34 PM CephFS Bug #65018 (Fix Under Review): PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- 06:58 PM CephFS Bug #65018: PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- Linking discussion for posterity: https://github.com/ceph/ceph/pull/56483#issuecomment-2021794648
- 02:57 PM CephFS Bug #65018 (In Progress): PG_DEGRADED warnings during cluster creation via cephadm: "Health check failed: Degraded data redundancy: 2/192 objects degraded (1.042%), 1 pg degraded (PG_DEGRADED)"
- 08:50 PM rgw Bug #65212 (Fix Under Review): pubsub: validate Name in CreateTopic api
- 04:24 PM rgw Bug #65212 (Pending Backport): pubsub: validate Name in CreateTopic api
- prevent topic names that would confuse things like ARN parsing and rados object namespacing
from https://docs.aws.... - 08:33 PM CephFS Bug #51282 (Fix Under Review): pybind/mgr/mgr_util: .mgr pool may be created too early causing spurious PG_DEGRADED warnings
- backport will be tracked by #65018
- 08:32 PM CephFS Bug #51282 (In Progress): pybind/mgr/mgr_util: .mgr pool may be created too early causing spurious PG_DEGRADED warnings
- 07:10 PM CephFS Bug #64198 (Resolved): mds: Fcb caps issued to clients when filelock is xlocked
- 06:46 PM CephFS Backport #65214 (In Progress): squid: mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- 06:45 PM CephFS Backport #65214 (Resolved): squid: mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- https://github.com/ceph/ceph/pull/56564
- 06:41 PM CephFS Bug #65182 (Pending Backport): mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- 05:46 PM rgw Backport #64792: reef: Notification kafka: Persistent messages are removed even when the broker is down and no ack is received
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56140
merged - 04:05 PM Dashboard Backport #65211 (Resolved): reef: mgr/dashboard: Mark placement targets as non-required
- https://github.com/ceph/ceph/pull/56621
- 04:05 PM Dashboard Backport #65210 (Resolved): squid: mgr/dashboard: Mark placement targets as non-required
- https://github.com/ceph/ceph/pull/56620
- 04:05 PM Dashboard Backport #65209 (Resolved): reef: mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- https://github.com/ceph/ceph/pull/56707
- 04:05 PM Dashboard Backport #65208 (Resolved): squid: mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- https://github.com/ceph/ceph/pull/56706
- 04:04 PM Dashboard Cleanup #65110 (Pending Backport): mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- 04:04 PM Dashboard Cleanup #64708 (Pending Backport): mgr/dashboard: Mark placement targets as non-required
- 04:01 PM Dashboard Backport #64830 (In Progress): reef: mgr/dashboard: Locking improvements in bucket create form
- 03:54 PM Dashboard Backport #64830 (New): reef: mgr/dashboard: Locking improvements in bucket create form
- 03:58 PM Dashboard Backport #64829 (In Progress): squid: mgr/dashboard: Locking improvements in bucket create form
- 03:53 PM Dashboard Backport #64829 (New): squid: mgr/dashboard: Locking improvements in bucket create form
- 03:52 PM Dashboard Cleanup #64658 (Pending Backport): mgr/dashboard: Locking improvements in bucket create form
- 03:51 PM Dashboard Backport #65206 (In Progress): quincy: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 03:25 PM Dashboard Backport #65206 (Resolved): quincy: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- https://github.com/ceph/ceph/pull/56558
- 03:50 PM Dashboard Backport #65205 (In Progress): reef: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 03:25 PM Dashboard Backport #65205 (Resolved): reef: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- https://github.com/ceph/ceph/pull/56557
- 03:48 PM Dashboard Backport #65204 (In Progress): squid: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 03:25 PM Dashboard Backport #65204 (Resolved): squid: mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- https://github.com/ceph/ceph/pull/56556
- 03:43 PM Dashboard Cleanup #65207 (Resolved): mgr/dashboard: Move features to advanced section in create image form and expand by default rbd config section
- h3. Move features to advanced section in create image form
A followup from the comment https://github.com/ceph/cep... - 03:23 PM Dashboard Bug #64734 (Pending Backport): mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 03:09 PM crimson Bug #65203: ReplicatedRecoveryBackend::recalc_subsets(ObjectRecoveryInfo&, crimson::osd::SnapSetContextRef): Assertion `ssc' failed.
- osd.2 osd.0 https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626298/
- 03:00 PM crimson Bug #65203 (Fix Under Review): ReplicatedRecoveryBackend::recalc_subsets(ObjectRecoveryInfo&, crimson::osd::SnapSetContextRef): Assertion `ssc' failed.
- osd.3: https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626294
After a... - 03:08 PM crimson Bug #65201: ReplicatedRecoveryBackend::prep_push_to_replica(const hobject_t&, eversion_t, pg_shard_t) Assertion ssc
- osd.0 osd.3 https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626298/
- 03:04 PM crimson Bug #65201: ReplicatedRecoveryBackend::prep_push_to_replica(const hobject_t&, eversion_t, pg_shard_t) Assertion ssc
- osd.0 osd.3 https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626297
- 02:57 PM crimson Bug #65201: ReplicatedRecoveryBackend::prep_push_to_replica(const hobject_t&, eversion_t, pg_shard_t) Assertion ssc
- osd.1 https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626294
- 02:55 PM crimson Bug #65201 (Fix Under Review): ReplicatedRecoveryBackend::prep_push_to_replica(const hobject_t&, eversion_t, pg_shard_t) Assertion ssc
- osd.3: https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626293
After a... - 03:02 PM crimson Bug #65200: PeeringState::get_peer_info(pg_shard_t) const: Assertion `it != peer_info.end()' failed.
- osd.0 https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626295/
- 02:54 PM crimson Bug #65200 (Resolved): PeeringState::get_peer_info(pg_shard_t) const: Assertion `it != peer_info.end()' failed.
- osd.1: https://pulpito.ceph.com/matan-2024-03-27_13:02:57-crimson-rados-main-distro-crimson-smithi/7626293
After a... - 02:19 PM rgw Bug #64450 (Duplicate): rgw: bucket listing failed to make forward progress w/ versioned objects
- 02:17 PM rgw Bug #64971 (Need More Info): Rgw lifecycle skip
- 02:15 PM rgw Feature #65131: perf counters for CreateMultipartUpload, AbortMultipartUpload, CompleteMultipartUpload
- thanks Matt and Paul. let's treat this as a feature request to add counters for the Create/Abort/CompleteMultipartUpl...
- 02:11 PM rgw Bug #65160 (Fix Under Review): rgw/lc: A few buckets stuck in UNINITIAL state
- 12:42 PM Bug #65199 (New): autoscaler: Scale PGs based on number of objects
- Ceph's autoscaler scales PGs based on Bytes stored. It seemingly ignores number of objects. This creates problems for...
- 11:46 AM rgw Backport #64326 (In Progress): reef: RGW: Cloud sync module fails to sync folders
- 11:40 AM CephFS Bug #65043: Unable to set timestamp to value > UINT32_MAX
- Adding additional information of the specifications for time representation in the SMB protocol.
https://learn.mic... - 11:36 AM CephFS Bug #65043: Unable to set timestamp to value > UINT32_MAX
- Greg Farnum wrote:
> We use unsigned int, which takes us to year 2106 — we'll have to fix it eventually, but why is ... - 11:14 AM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- The OSDMap CRC issue is clearly there but I'm not sure / I doubt it can explain the scrub error.
Let's ask Ronen for... - 11:03 AM RADOS Bug #65186: OSDs unreachable in upgrade test
- ...
- 11:02 AM RADOS Backport #65198: squid: Failed to encode map X with expected CRC
- https://github.com/ceph/ceph/pull/56553
- 10:31 AM RADOS Backport #65198 (In Progress): squid: Failed to encode map X with expected CRC
- 10:29 AM RADOS Backport #65198 (In Progress): squid: Failed to encode map X with expected CRC
- 10:30 AM rgw Backport #64500: reef: multisite: Deadlock in RGWDeleteMultiObj with default rgw_multi_obj_del_max_aio > 1
- Does it make sense to also add something to the release notes for folks (like me) who set
@rgw_multi_obj_del_max_a... - 10:28 AM rgw Bug #63373: multisite: Deadlock in RGWDeleteMultiObj with default rgw_multi_obj_del_max_aio > 1
- Does it make sense to also add something to the release notes for folks (like me) who set
@rgw_multi_obj_del_max_... - 09:46 AM ceph-volume Bug #64898 (Resolved): ceph-volume lvm zap fails with "undefined name 'List'"
- 09:46 AM ceph-volume Backport #64944 (Resolved): quincy: ceph-volume lvm zap fails with "undefined name 'List'"
- 09:44 AM ceph-volume Bug #64248 (Resolved): ceph-volume fails to zap encrypted journal device on partitions
- 09:44 AM ceph-volume Backport #64336 (Resolved): quincy: ceph-volume fails to zap encrypted journal device on partitions
- 08:31 AM Dashboard Backport #65197 (In Progress): quincy: mgr/dashboard: sanitize dashboard user input field
- 08:29 AM Dashboard Backport #65197 (Resolved): quincy: mgr/dashboard: sanitize dashboard user input field
- https://github.com/ceph/ceph/pull/56551
- 08:30 AM CephFS Bug #65136: QA failure: test_fscrypt_dummy_encryption_with_quick_group
- https://pulpito.ceph.com/rishabh-2024-03-27_05:27:11-fs-wip-rishabh-testing-20240326.131558-testing-default-smithi/76...
- 08:27 AM Dashboard Cleanup #65030 (Pending Backport): mgr/dashboard: sanitize dashboard user input field
- 08:25 AM Dashboard Backport #65196 (In Progress): reef: mgr/dashboard: Cloning of subvolume with the same name is not giving proper error
- 08:22 AM Dashboard Backport #65196 (In Progress): reef: mgr/dashboard: Cloning of subvolume with the same name is not giving proper error
- https://github.com/ceph/ceph/pull/56550
- 08:25 AM Dashboard Backport #65195 (In Progress): squid: mgr/dashboard: Cloning of subvolume with the same name is not giving proper error
- 08:21 AM Dashboard Backport #65195 (In Progress): squid: mgr/dashboard: Cloning of subvolume with the same name is not giving proper error
- https://github.com/ceph/ceph/pull/56549
- 08:24 AM Dashboard Backport #65194 (In Progress): reef: mgr/dashboard: rgw display name with dot gives validation error
- 08:21 AM Dashboard Backport #65194 (Resolved): reef: mgr/dashboard: rgw display name with dot gives validation error
- https://github.com/ceph/ceph/pull/56548
- 08:24 AM Dashboard Backport #65193 (In Progress): squid: mgr/dashboard: rgw display name with dot gives validation error
- 08:21 AM Dashboard Backport #65193 (Resolved): squid: mgr/dashboard: rgw display name with dot gives validation error
- https://github.com/ceph/ceph/pull/56547
- 08:23 AM Dashboard Backport #65192 (In Progress): reef: mgr/dashboard: s/active_mds/active_nfs in fs attach form
- 08:21 AM Dashboard Backport #65192 (Resolved): reef: mgr/dashboard: s/active_mds/active_nfs in fs attach form
- https://github.com/ceph/ceph/pull/56546
- 08:23 AM Dashboard Backport #65191 (In Progress): squid: mgr/dashboard: s/active_mds/active_nfs in fs attach form
- 08:21 AM Dashboard Backport #65191 (Resolved): squid: mgr/dashboard: s/active_mds/active_nfs in fs attach form
- https://github.com/ceph/ceph/pull/56545
- 08:20 AM Dashboard Bug #65173 (Pending Backport): mgr/dashboard: s/active_mds/active_nfs in fs attach form
- 08:20 AM Dashboard Bug #65145 (Pending Backport): mgr/dashboard: Cloning of subvolume with the same name is not giving proper error
- 08:20 AM Dashboard Bug #65146 (Pending Backport): mgr/dashboard: rgw display name with dot gives validation error
- 08:19 AM Dashboard Bug #65143 (Resolved): mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- 08:19 AM Dashboard Backport #65166 (Resolved): squid: mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- 08:19 AM Dashboard Backport #65165 (Resolved): reef: mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- 07:03 AM RADOS Bug #64824: mon: ceph-16.2.14/src/mon/Monitor.cc: 5661: FAILED ceph_assert(err == 0)
- ...
- 05:08 AM Backport #65172 (In Progress): reef: ceph-exporter is not included in the deb packages
- 04:51 AM Backport #65172: reef: ceph-exporter is not included in the deb packages
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56541
ceph-backport.sh versi... - 03:28 AM bluestore Support #65190 (New): Can ceph with SPDK driver has to be connected with local NVMe device?
- Hi everyone,
I am using ceph 18.2.2 to create a cluster with NVMe device under SPDK driver.
When the device is conn... - 03:04 AM rgw Bug #47866: Object not found on healthy cluster
- I have similar problem in ceph version 16.2.5. Has this problem been completely resolved in Pacific version? It only ...
- 01:40 AM crimson Feature #64375 (Resolved): crimson: introduce support for C++ coroutines
- Support has merged.
- 01:39 AM crimson Bug #64935 (Resolved): crimson: heap use after free during ~OSD()
03/27/2024
- 10:49 PM CephFS Bug #65182 (Fix Under Review): mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- 04:01 PM CephFS Bug #65182 (Resolved): mds: quiesce_inode op waiting on remote auth pins is not killed correctly during quiesce timeout/expiration
- ...
- 10:36 PM rgw Bug #65188 (Fix Under Review): rgwlc: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault
- 10:30 PM rgw Bug #65188 (Pending Backport): rgwlc: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault
- [LC-Process]: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation ...
- 10:36 PM mgr Bug #65189 (In Progress): Telemetry pacific-x upgrade test pauses when upgrading to squid
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7615987...
- 10:35 PM RADOS Bug #64972: qa: "ceph tell 4.3a deep-scrub" command not found
- Laura Flores wrote:
> Strange, the syntax in the text snippet works in a vstart cluster:
> [...]
The issue, I be... - 10:09 PM RADOS Bug #64972: qa: "ceph tell 4.3a deep-scrub" command not found
- Strange, the syntax in the text snippet works in a vstart cluster:...
- 10:18 PM Orchestrator Bug #65187 (Resolved): upgrade/quincy-x/stress-split: upgrade test fails to install quincy packages
- ...
- 08:39 PM CephFS Bug #64707: suites/fsstress.sh hangs on one client - test times out
- Looks like a case of this tracker:
/a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616007
... - 08:33 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7615991
- 08:29 PM RADOS Bug #65186: OSDs unreachable in upgrade test
- Possibly a dupe of the related tracker (crc encoding issues)
- 08:28 PM RADOS Bug #65186 (Fix Under Review): OSDs unreachable in upgrade test
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616011/remote/smithi087/log/a8e8c570-e819-11ee...
- 08:31 PM RADOS Bug #65185: OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- Laura Flores wrote:
> /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616025/remote/smithi098... - 08:21 PM RADOS Bug #65185 (Fix Under Review): OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
- /a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616025/remote/smithi098/log/b1f19696-e81a-11ee...
- 08:25 PM Orchestrator Feature #64577 (Resolved): allow idmap overrides in nfs-ganesha configuration
- 08:25 PM Orchestrator Backport #64697 (Resolved): squid: allow idmap overrides in nfs-ganesha configuration
- 08:25 PM Orchestrator Bug #64020 (Resolved): cephadm is not accounting for the memory required nvme gateways are used
- 08:24 PM Orchestrator Backport #64621 (Resolved): squid: cephadm is not accounting for the memory required nvme gateways are used
- 08:23 PM Orchestrator Backport #64993 (Resolved): squid: cephadm: public_network config check does not pick up changes in public_network unless the module restarts
- 08:22 PM Orchestrator Bug #64428 (Resolved): cephadm: host filtering with label and host pattern only uses the label
- 08:22 PM Orchestrator Backport #64687 (Resolved): squid: cephadm: host filtering with label and host pattern only uses the label
- 08:21 PM Orchestrator Bug #64433 (Resolved): cephadm: cephadm does not clean up /etc/ceph/podman-auth.json in rm-cluster
- 08:21 PM Orchestrator Backport #64633 (Resolved): squid: cephadm: cephadm does not clean up /etc/ceph/podman-auth.json in rm-cluster
- 08:20 PM Orchestrator Bug #64473 (Resolved): cephadm: asyncio timeout handler can't handle conccurent.futures.CancelledError causing the module to crash
- 08:19 PM Orchestrator Backport #64628 (Resolved): squid: cephadm: asyncio timeout handler can't handle conccurent.futures.CancelledError causing the module to crash
- 07:50 PM bluestore Bug #63858: ceph-bluestore-tool bluefs-bdev-expand doesn't adjust OSD free space when NCB mode is in use
- Gary Ritzer wrote:
> Hi. I have not seen a release for Quincy that contains this fix, is there an ETA for that?
H... - 05:27 PM bluestore Bug #63858: ceph-bluestore-tool bluefs-bdev-expand doesn't adjust OSD free space when NCB mode is in use
- Hi. I have not seen a release for Quincy that contains this fix, is there an ETA for that?
- 07:45 PM Orchestrator Bug #62638 (Resolved): mgr/cephadm is not defining haproxy tcp healthchecks for Ganesha
- 07:45 PM Orchestrator Backport #64623 (Resolved): squid: mgr/cephadm is not defining haproxy tcp healthchecks for Ganesha
- 04:43 PM RADOS Bug #65183 (Fix Under Review): Overriding an EC pool needs the "--yes-i-really-mean-it" flag in addition to "force"
- 04:23 PM RADOS Bug #65183: Overriding an EC pool needs the "--yes-i-really-mean-it" flag in addition to "force"
- Likely coming from this change:
https://github.com/ceph/ceph/pull/56287 - 04:23 PM RADOS Bug #65183 (Fix Under Review): Overriding an EC pool needs the "--yes-i-really-mean-it" flag in addition to "force"
- /a/yuriw-2024-03-26_14:32:05-rados-wip-yuri8-testing-2024-03-25-1419-distro-default-smithi/7623454...
- 04:41 PM Bug #65184 (Duplicate): install-deps: enable copr ceph/grpc
- 04:32 PM Bug #65184 (Resolved): install-deps: enable copr ceph/grpc
- needed for nvmeof management apis
- 02:58 PM rgw Backport #64766: reef: SSL session id reuse speedup mechanism of the SSL_CTX_set_session_id_context is not working
- Backport Bot wrote:
> https://github.com/ceph/ceph/pull/56120
merged - 02:45 PM rgw Backport #63856 (Resolved): reef: notification: etag is missing in CompleteMultipartUpload event
- 02:28 PM Dashboard Bug #65180 (Fix Under Review): mgr/dashboard: e2e for snapshot and clone mgmnt
- 02:25 PM rgw Bug #65179 (New): rgw incorrectly uses `Range` header in `X-Amz-Cache`
- As noted in RGW Data caching and CDN (https://docs.ceph.com/en/latest/radosgw/rgw-cache/ commited in https://github.c...
- 02:04 PM Orchestrator Feature #63781: Change default stirp size to 5MB to match multipart uploads
- Duplicates #65178
- 02:03 PM rgw Feature #65178 (New): Change default stirp size to 5MB to match multipart uploads
- The current default setting for `rgw_multipart_min_part_size` is 5MiB/5242880 (which is same as AWS minimum value). C...
- 01:52 PM rgw Bug #63486: reef: test_lifecycle_cloud_transition_large_obj FAILED
- again on reef, a failure from @test_lifecycle_noncur_cloud_transition@ in https://qa-proxy.ceph.com/teuthology/cbodle...
- 01:51 PM CephFS Bug #65073 (Fix Under Review): pybind/mgr/stats/fs: log exceptions to cluster log
- 01:49 PM rgw Bug #63786: rados_cls_all: TestCls2PCQueue.MultiProducer hangs
- again on reef in https://qa-proxy.ceph.com/teuthology/cbodley-2024-03-26_12:30:03-rgw-wip-63856-reef-distro-default-s...
- 01:46 PM rgw Bug #65177: reef: Syscall param write(buf) points to uninitialised byte(s)
- the failures only showed up on "centos 9.stream" jobs
- 01:42 PM rgw Bug #65177 (New): reef: Syscall param write(buf) points to uninitialised byte(s)
- saw on several jobs in https://pulpito.ceph.com/cbodley-2024-03-26_12:30:03-rgw-wip-63856-reef-distro-default-smithi/...
- 01:03 PM CephFS Backport #62952 (Resolved): reef: kernel/fuse client using ceph ID with uid restricted MDS caps cannot update caps
- 12:20 PM Dashboard Bug #54513 (Resolved): mgr/dashboard: Pool overall performance shows multiple entries of same pool in pool overview
- 12:20 PM Dashboard Bug #54068 (Resolved): mgr/dashboard: fix columns in host table with NaN Undefined
- 12:20 PM Dashboard Bug #50491 (Resolved): mgr/dashboard: centralized logging
- 12:19 PM Dashboard Bug #44851 (Resolved): mgr/dashboard: report bugs/improvements/feedback
- 12:19 PM Dashboard Bug #52914 (Resolved): mgr/dashboard: follow-up tasks cephadm/box
- 12:19 PM Dashboard Bug #51211 (Resolved): mgr/dashboard: grafana-piechart-plugin doesn't work with Grafana 7.5.x anymore
- 12:16 PM Bug #64360 (Resolved): BuildRocksDB.cmake doesn't pass optimization flags
- 12:16 PM Backport #64363 (Resolved): quincy: BuildRocksDB.cmake doesn't pass optimization flags
- 12:16 PM Backport #64361 (Resolved): reef: BuildRocksDB.cmake doesn't pass optimization flags
- 12:14 PM Dashboard Bug #42306 (Resolved): mgr/dashboard: editing silence: page width too small - increase responsiveness for views under 1920
- 12:14 PM Dashboard Bug #56077 (Resolved): mgr/dashboard: validate mds service id
- 12:14 PM Dashboard Backport #57246 (Rejected): pacific: mgr/dashboard: JSON-driven UI tables
- Pacific is EOL
- 12:14 PM Dashboard Backport #57086 (Rejected): pacific: mgr/dashboard: validate mds service id
- Pacific is EOL
- 12:13 PM Dashboard Backport #57013 (Rejected): pacific: mgr/dashboard: editing silence: page width too small - increase responsiveness for views under 1920
- Pacific is EOL
- 12:13 PM Dashboard Backport #56562 (Rejected): pacific: mgr/dashboard: rbd paginate image list
- Pacific is EOL
- 12:13 PM Dashboard Backport #56161 (Rejected): pacific: mgr/dashboard: rbd promotion/demotion UI
- Pacific is EOL
- 12:13 PM Dashboard Backport #56123 (Rejected): pacific: mgr/dashboard: rbd force resync from front-end
- Pacific is EOL
- 12:13 PM Dashboard Backport #56095 (Rejected): pacific: mgr/dashboard: rbd-mirror move replaying state images to syncing tab
- Pacific is EOL
- 12:13 PM Dashboard Backport #55913 (Rejected): pacific: mgr/dashboard: rbd mirroring add primary info backend side
- Pacific is EOL
- 12:13 PM Dashboard Backport #55899 (Rejected): pacific: mgr/dashboard: Add daemon logs tab to Logs component
- Pacific is EOL
- 12:13 PM Dashboard Backport #55371 (Rejected): pacific: mgr/dashboard: update grafana piechart and vonage status panel versions
- Pacific is EOL
- 12:12 PM Dashboard Backport #55127 (Rejected): pacific: mgr/dashboard: Pool overall performance shows multiple entries of same pool in pool overview
- Pacific is EOL
- 12:12 PM Dashboard Backport #54584 (Rejected): pacific: mgr/dashboard: fix columns in host table with NaN Undefined
- Pacific is EOL
- 12:12 PM Dashboard Backport #54582 (Rejected): pacific: mgr/dashboard: centralized logging
- Pacific is EOL
- 12:12 PM Dashboard Backport #53920 (Rejected): pacific: mgr/dashboard: report bugs/improvements/feedback
- Pacific is EOL
- 12:12 PM Dashboard Backport #53080 (Rejected): pacific: mgr/dashboard: follow-up tasks cephadm/box
- Pacific is EOL
- 12:12 PM Dashboard Backport #52750 (Rejected): pacific: mgr/dashboard: grafana-piechart-plugin doesn't work with Grafana 7.5.x anymore
- Pacific is EOL
- 12:11 PM mgr Bug #62165 (Resolved): [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs
- 12:11 PM mgr Backport #62886 (Resolved): quincy: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs
- 12:10 PM mgr Backport #62887 (Rejected): pacific: [pg-autoscaler] Peformance issue with the autoscaler when we have ~32768 PGs
- Pacific is EOL
- 12:10 PM mgr Bug #61922 (Resolved): [pg_autoscaler] PG auto-scaler configs on individual pools is changed after set & unset of "noautoscale" flag
- 12:10 PM mgr Backport #62978 (Resolved): quincy: [pg_autoscaler] PG auto-scaler configs on individual pools is changed after set & unset of "noautoscale" flag
- 12:09 PM mgr Backport #62977 (Resolved): reef: [pg_autoscaler] PG auto-scaler configs on individual pools is changed after set & unset of "noautoscale" flag
- 12:09 PM mgr Bug #61570 (Resolved): pg_autoscaler warns that a pool has too many pgs when it has the exact right amount
- 12:09 PM mgr Backport #62986 (Rejected): pacific: pg_autoscaler warns that a pool has too many pgs when it has the exact right amount
- 12:08 PM mgr Backport #62986: pacific: pg_autoscaler warns that a pool has too many pgs when it has the exact right amount
- Pacific is EOL
- 12:08 PM mgr Backport #62976 (Rejected): pacific: [pg_autoscaler] PG auto-scaler configs on individual pools is changed after set & unset of "noautoscale" flag
- Pacific is EOL
- 12:08 PM rgw Bug #52716 (Resolved): incorrect multipart upload owner, access denied when listing parts of multipart upload by bucket owner
- 12:07 PM rgw Backport #53147 (Rejected): pacific: incorrect multipart upload owner, access denied when listing parts of multipart upload by bucket owner
- Pacific is EOL
- 12:07 PM rgw Backport #59279 (Rejected): pacific: Copying an object to itself crashes de RGW if executed as admin user.
- Pacific is EOL
- 12:07 PM RADOS Bug #51725 (Resolved): make bufferlist::c_str() skip rebuild when it isn't necessary
- 12:06 PM RADOS Backport #52595 (Rejected): pacific: make bufferlist::c_str() skip rebuild when it isn't necessary
- Pacific is EOL
- 12:06 PM RADOS Bug #51843 (Resolved): osd/scrub: OSD crashes at PG removal
- 12:06 PM RADOS Backport #53340 (Rejected): pacific: osd/scrub: OSD crashes at PG removal
- Pacific is EOL
- 12:05 PM RADOS Bug #53294 (Resolved): rados/test.sh hangs while running LibRadosTwoPoolsPP.TierFlushDuringFlush
- 12:05 PM RADOS Bug #49525 (Resolved): found snap mapper error on pg 3.2s1 oid 3:4abe9991:::smithi10121515-14:e4 snaps missing in mapper, should be: dc was r -2...repaired
- 12:04 PM RADOS Backport #55973 (Rejected): pacific: found snap mapper error on pg 3.2s1 oid 3:4abe9991:::smithi10121515-14:e4 snaps missing in mapper, should be: dc was r -2...repaired
- Pacific is EOL
- 12:04 PM RADOS Backport #56656 (Rejected): pacific: rados/test.sh hangs while running LibRadosTwoPoolsPP.TierFlushDuringFlush
- Pacific is EOL
- 12:03 PM rgw Bug #46062 (Resolved): File Corruption in Multisite Replication with Encryption
- 12:02 PM rgw Backport #62322 (Rejected): pacific: File Corruption in Multisite Replication with Encryption
- Pacific is EOL
- 12:02 PM rgw Backport #62283 (Rejected): pacific: [ FAILED ] TestAMQP.IdleConnection (30132 ms)
- Pacific is EOL
- 12:02 PM RADOS Backport #64672 (Rejected): pacific: test_pool_min_size: AssertionError: wait_for_clean: failed before timeout expired due to down PGs
- Pacific is EOL
- 12:01 PM RADOS Backport #64410 (In Progress): quincy: map eXX had wrong heartbeat addr
- 12:00 PM RADOS Backport #64412 (In Progress): reef: map eXX had wrong heartbeat addr
- 11:58 AM RADOS Backport #64411 (Rejected): pacific: map eXX had wrong heartbeat addr
- Pacific is EOL
- 11:56 AM RADOS Backport #64407 (Rejected): pacific: Expected warnings that need to be whitelisted cause rados/cephadm tests to fail
- Pacific is EOL
- 11:56 AM RADOS Backport #64157 (Rejected): pacific: CommandFailedError (rados/test_python.sh): "RADOS object not found" causes test_rados.TestWatchNotify.test_aio_notify to fail
- Pacific is EOL
- 11:55 AM RADOS Backport #59675 (Rejected): pacific: osd:tick checking mon for new map
- Pacific is EOL
- 11:54 AM RADOS Backport #58870 (Rejected): pacific: ClsLock.TestExclusiveEphemeralStealEphemeral failed
- Pacific is EOL
- 11:54 AM Backport #57925 (Rejected): pacific: common: use fmt::print for stderr logging
- Pacific is EOL
- 11:52 AM Backport #55404 (Rejected): pacific: qa/cephfs: don't exclamation mark on test_cephfs_shell.py
- Pacific is EOL
- 11:52 AM Bug #55091 (Resolved): debian: use build-profile to manage build deps
- 11:52 AM Backport #55093 (Rejected): pacific: debian: use build-profile to manage build deps
- Pacific is EOL
- 11:52 AM Bug #51226 (Resolved): qa: import CommandFailedError from teuthology.exceptions
- 11:51 AM Backport #52955 (Rejected): pacific: qa: import CommandFailedError from teuthology.exceptions
- Pacific is EOL
- 11:51 AM Dashboard Bug #57178 (Resolved): mgr/dashboard: ensure rbd limit=0 returns 0 images
- 11:51 AM Dashboard Backport #57357 (Rejected): pacific: mgr/dashboard: ensure rbd limit=0 returns 0 images
- Pacific is EOL
- 11:50 AM Dashboard Backport #57818 (Rejected): pacific: mgr/dashboard: update legal links
- Pacific is EOL
- 11:50 AM Dashboard Bug #57511 (Resolved): mgr/dashboard: cephadm dashboard e2e failure "being covered by another element"
- 11:50 AM Dashboard Backport #57832 (Rejected): pacific: mgr/dashboard: cephadm dashboard e2e failure "being covered by another element"
- Pacific is EOL
- 11:49 AM Dashboard Bug #57805 (Resolved): mgr/dashboard: Unable to change subuser permission
- 11:48 AM Dashboard Backport #57842 (Rejected): pacific: mgr/dashboard: Unable to change subuser permission
- Pacific is EOL
- 11:48 AM Dashboard Backport #57994 (Rejected): pacific: mgr/dashboard: paginate services
- Pacific is EOL
- 11:48 AM Dashboard Bug #58504 (Resolved): mgr/dashboard: constant flickering of Service Instances tab
- 11:47 AM Dashboard Backport #58719 (Rejected): pacific: mgr/dashboard: constant flickering of Service Instances tab
- Pacific is EOL
- 11:47 AM Dashboard Bug #58297 (Resolved): mgr/dashboard: Rbd Mirroring Improvements
- 11:47 AM Dashboard Backport #59449 (Rejected): pacific: mgr/dashboard: Rbd Mirroring Improvements
- Pacific is EOL
- 11:45 AM Dashboard Bug #59500 (Resolved): mgr/dashboard: error message popping up along with force promote modal
- 11:45 AM Dashboard Backport #59502 (Rejected): pacific: mgr/dashboard: error message popping up along with force promote modal
- Pacific is EOL
- 11:35 AM CephFS Bug #64563: mds: enhance laggy clients detections due to laggy OSDs
- Venky Shankar wrote:
> Dhairya Parmar wrote:
> > Me and greg had a discussion on this today, I think the first step... - 11:16 AM RADOS Bug #57061: Use single cluster log level (mon_cluster_log_level) config to control verbosity of cluster logs while logging to external entities
- In QA.
- 11:15 AM RADOS Bug #64258: osd/PrimaryLogPG.cc: FAILED ceph_assert(inserted)
- Sent to QA.
- 11:14 AM CephFS Bug #64659: mds: switch to using xlists instead of elists
- Venky Shankar wrote:
> Dhairya Parmar wrote:
> > Venky Shankar wrote:
> > > Dhairya Parmar wrote:
> > > > Patrick... - 11:00 AM Dashboard Bug #64980 (In Progress): mgr/dashboard: fixed snap schedule repeat frequency validation to prevent duplicates
- 10:56 AM Dashboard Backport #65133 (In Progress): squid: mgr/dashboard: add support for NFSv3 exports
- 10:55 AM Dashboard Backport #65132 (In Progress): reef: mgr/dashboard: add support for NFSv3 exports
- 10:38 AM Bug #65176 (In Progress): BlueFS: _estimate_log_size_N calculates the log size incorrectly
- 10:04 AM Bug #65176 (In Progress): BlueFS: _estimate_log_size_N calculates the log size incorrectly
- In BlueFS::_estimate_log_size_N, the total size of the dir was calculated incorrectly.
It should be the number of di... - 10:19 AM Linux kernel client Bug #64471: kernel: upgrades from quincy/v18.2.[01]/reef to main|squid fail with kernel oops
- Patrick and Venky,
I raised a PR in *teuthology* to add a *kdb* option to allow us disable it when triggering the ... - 10:17 AM Dashboard Bug #64734 (Fix Under Review): mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 10:03 AM Dashboard Bug #64734 (In Progress): mgr/dashboard: Clicking on Ceph logo do not takes to the dashboard
- 09:53 AM Dashboard Subtask #65025 (Fix Under Review): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 08:56 AM Dashboard Subtask #65025 (In Progress): mgr/dashboard: Make advanced fields consistent for forms in dashboard
- 09:19 AM RADOS Bug #54744: crash: void MonMap::add(const mon_info_t&): assert(addr_mons.count(a) == 0)
- The priority level is set to "minor" ... when the time comes that messenger v1 is deprecated ... operators will disab...
- 09:17 AM RADOS Bug #54744: crash: void MonMap::add(const mon_info_t&): assert(addr_mons.count(a) == 0)
- This should be fixed indeed. I wanted to disable msgv1 on this cluster. I already had set the flag "ceph config set m...
- 09:08 AM Dashboard Bug #58838 (Resolved): mgr/dashboard: POD CPU usage is incorrect
- 09:08 AM Dashboard Backport #65102 (Resolved): squid: mgr/dashboard: POD CPU usage is incorrect
- 09:02 AM rgw Bug #62000: rgw crashed on latest ceph version 17.2.6 quincy
I found some important logs.
It not contain object ...- 08:56 AM devops Bug #65175 (Pending Backport): ccache is always miss in confusa14
- From: https://jenkins.ceph.com/job/ceph-pull-requests-arm64/54223/consoleFull
ccache -sz shows:... - 08:50 AM crimson Bug #65130 (Fix Under Review): crimson: crimson-rados did not detect reintroduction of https://tracker.ceph.com/issues/61875
- 08:18 AM Dashboard Bug #65174 (Pending Backport): mgr/dashboard: replace deprectaed grafana panel with a newer table panel
- Since the grafana table panel is deprectaed in the latest grafana versions, the existing panels should be replaced wi...
- 08:07 AM Backport #65164 (In Progress): squid: Make check (arm64) succeeded despite unit test failures in the log
- 02:23 AM Backport #65164 (In Progress): squid: Make check (arm64) succeeded despite unit test failures in the log
- https://github.com/ceph/ceph/pull/56508
- 07:59 AM rgw Bug #63428: RGW: multipart get wrong storage class metadata
As we can see here, *default.rgw.cold.data* has 0.94% used that mean the multipart object is located on cold pool. ...- 07:28 AM Dashboard Cleanup #65110 (Resolved): mgr/dashboard: Align security fieldset and tag fieldset with the rest of the bucket form
- 05:48 AM CephFS Bug #65115: cephfs_mirror: failed test test_cephfs_mirror_cancel_mirroring_and_readd showing "Bad file descriptor"
- Venky Shankar wrote:
> Jos, how is this bad file descriptor fixed by PR #56118? Did you add it by mistake?
No. Th... - 05:20 AM Dashboard Bug #61690 (Resolved): mgr/dashboard: install_deps.sh fails on vanilla CentOS 8 Stream
- 05:20 AM Dashboard Backport #61715 (Resolved): quincy: mgr/dashboard: install_deps.sh fails on vanilla CentOS 8 Stream
- 05:20 AM Dashboard Bug #61618 (Resolved): mgr/dashboard: grafana compact url deprecation notice
- 05:20 AM Dashboard Backport #61659 (Resolved): quincy: mgr/dashboard: grafana compact url deprecation notice
- 05:16 AM crimson Bug #64975 (In Progress): crimson: Health check failed: 9 scrub errors (OSD_SCRUB_ERRORS)" in cluster log'
- 05:16 AM crimson Bug #64975: crimson: Health check failed: 9 scrub errors (OSD_SCRUB_ERRORS)" in cluster log'
- Testing a different fix, the above would have reintroduced a crash on OSD startup while reading the pg log.
- 05:15 AM Dashboard Backport #65169 (In Progress): squid: mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- 04:49 AM Dashboard Backport #65169 (Resolved): squid: mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- https://github.com/ceph/ceph/pull/56505
- 05:14 AM Dashboard Bug #65173 (Fix Under Review): mgr/dashboard: s/active_mds/active_nfs in fs attach form
- 05:09 AM Dashboard Bug #65173 (Resolved): mgr/dashboard: s/active_mds/active_nfs in fs attach form
- Current:
sudo mount -t nfs -o port=<PORT> <IP of active_mds daemon>:<export_name> <MOUNT_DIRECTORY>
Expected:
... - 05:11 AM Bug #54059 (Resolved): [crypto/qat][compressor] QAT driver cannot work with encryption and compression for RGW
- 05:11 AM Backport #54061 (Rejected): pacific: latest QAT driver cannot work with encryption and compression for RGW
- Pacific is EOL
- 05:10 AM Backport #61614 (Rejected): pacific: windows clients unable to perform IO to clusters with over 200+ OSDs
- Pacific is EOL
- 05:10 AM bluestore Backport #61464 (Rejected): pacific: Fragmentation score rising by seemingly stuck thread
- Pacific is EOL
- 05:06 AM Backport #65172 (Resolved): reef: ceph-exporter is not included in the deb packages
- https://github.com/ceph/ceph/pull/56541
- 05:04 AM CephFS Bug #65171 (Fix Under Review): Provide metrics support for the Replication Start/End Notifications
- BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2270946
At present, metrics counters for the average/sum of time t... - 04:59 AM Bug #64095 (Pending Backport): ceph-exporter is not included in the deb packages
- 12:58 AM Bug #64095: ceph-exporter is not included in the deb packages
- I want to backport the patch for this issue to Reef.
Can someone update the status? (The current status is set to "F... - 04:49 AM Dashboard Backport #65170 (In Progress): reef: mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- https://github.com/ceph/ceph/pull/56877
- 04:49 AM Dashboard Backport #65168 (In Progress): quincy: mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- https://github.com/ceph/ceph/pull/56878
- 04:43 AM Dashboard Bug #65167 (Pending Backport): mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- 04:42 AM Dashboard Bug #65167 (Pending Backport): mgr/dashboard: CVE-2023-26159, CVE-2024-28849 follow-redirects package
- https://github.com/ceph/ceph/security/dependabot/120
https://github.com/ceph/ceph/security/dependabot/129 - 04:38 AM Dashboard Cleanup #64579 (Resolved): mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- 04:38 AM Dashboard Backport #65142 (Rejected): reef: mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- 04:37 AM Dashboard Backport #64608 (Resolved): squid: mgr/dashboard: replace grafana piechart panel plugin with native grafana piechart panel
- 04:36 AM Dashboard Backport #65166 (In Progress): squid: mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- 04:34 AM Dashboard Backport #65166 (Resolved): squid: mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- https://github.com/ceph/ceph/pull/56502
- 04:36 AM Dashboard Backport #65165 (In Progress): reef: mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- 04:33 AM Dashboard Backport #65165 (Resolved): reef: mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- https://github.com/ceph/ceph/pull/56501
- 04:25 AM Dashboard Bug #65143 (Pending Backport): mgr/dashboard: naming a volume starts with ".", shows it is invalid.
- 03:09 AM Backport #65127: squid: Packaging files need to note libprotobuf as a runtime dependency
- please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/56500
ceph-backport.sh versi... - 03:07 AM Backport #65127: squid: Packaging files need to note libprotobuf as a runtime dependency
- https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8...
- 03:06 AM rgw Feature #65131: perf counters for CreateMultipartUpload, AbortMultipartUpload, CompleteMultipartUpload
- Yep - so the put_ops value is incrementing at each successful 'part' being uploaded. Which means this is not a bug, b...
- 02:13 AM rgw Feature #65131: perf counters for CreateMultipartUpload, AbortMultipartUpload, CompleteMultipartUpload
- Oh, duh, this is doing multipart upload....
- 01:51 AM rgw Feature #65131: perf counters for CreateMultipartUpload, AbortMultipartUpload, CompleteMultipartUpload
- It's not just warp - boto is seeing the same thing and there is no indication of failed uploads.
- 01:42 AM rgw Feature #65131: perf counters for CreateMultipartUpload, AbortMultipartUpload, CompleteMultipartUpload
- The rgw counter we should be sampling, I think, is l_rgw put. The logic to increment the counter, as I recalled in s...
- 01:13 AM rgw Feature #65131: perf counters for CreateMultipartUpload, AbortMultipartUpload, CompleteMultipartUpload
- Also tried python's boto3 with a 64MB objects and saw a similar thing where the put count from rgw is way higher than...
- 02:23 AM Bug #64117 (Pending Backport): Make check (arm64) succeeded despite unit test failures in the log
- 02:17 AM RADOS Feature #65163 (New): Rados：Provide options for data compression levels, specified with -l, to enable writing data compressed at various levels.
- 12:40 AM Feature #57515: The way to know the data format of each OSD and MON was created
- > I'd like to backport this patch to quincy and reef, so could you add the backport tags to this issue?
Any updates?
Also available in: Atom