Project

General

Profile

Activity

From 11/06/2022 to 12/05/2022

12/05/2022

11:55 PM Orchestrator Bug #58096: test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
/a/lflores-2022-12-02_20:40:02-rados-wip-yuri6-testing-2022-11-23-1348-distro-default-smithi/7101847 Laura Flores
11:53 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
imirc tw wrote:
> I think this is container related. I suspect the container has a default timeout of 10 seconds for...
Igor Fedotov
03:56 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
I think this is container related. I suspect the container has a default timeout of 10 seconds for commands, after th... imirc tw
02:38 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
for the record, the shutdown log before the gap:... imirc tw
02:25 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
again, a gap:
2022-12-05T14:23:33.437+0000 7efcbdb19700 10 bluestore.OnodeSpace(0x562debe3f9a0 in 0x562deb647000) ...
imirc tw
02:21 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
I noticed the same, not sure why though. Will create a new log with debug to 10. imirc tw
01:57 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
I can see ~17 seconds missing in the log:
2022-12-05T09:15:31.918+0000 7f820508e700 20 bluestore.BufferSpace(0x555cf...
Igor Fedotov
01:12 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
Needed to split it in several parts due to upload limit, took -100k + 100k to be sure. imirc tw
12:42 PM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
well, let it be -50000 and +50000 lines from OSD restart point, below is a sample log line which indicates such a poi... Igor Fedotov
09:04 AM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
Igor Fedotov wrote:
> Could you please set fast osd shutdown to false and debug-bluestore to 20. Then restart OSD an...
imirc tw
07:55 AM Orchestrator Feature #58158: OSD container's shutdown timeout should be raised (and configurable)
Could you please set fast osd shutdown to false and debug-bluestore to 20. Then restart OSD and share shutdown/startu... Igor Fedotov
11:51 PM bluestore Backport #58181 (Resolved): pacific: KernelDevice::read doesn't translate error codes correctly
Backport Bot
11:51 PM bluestore Backport #58180 (Resolved): quincy: KernelDevice::read doesn't translate error codes correctly
Backport Bot
11:48 PM rgw Backport #58119: pacific: check-generated.sh failures for rgw_log_entry
Casey Bodley wrote:
> https://github.com/ceph/ceph/pull/49142
merged
Yuri Weinstein
11:46 PM bluestore Bug #57857 (Pending Backport): KernelDevice::read doesn't translate error codes correctly
Igor Fedotov
11:39 PM rbd Bug #57726 (In Progress): [rbd_support] set_localized_module_option(..., None) is spamming the au...
Ramana Raja
11:37 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
https://shaman.ceph.com/builds/ceph/wip-revert-pr-48713/2b583578473c82604cfdab2faef9f161dc2fb0b9/ Laura Flores
11:20 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
The bug reproduced on Yuri's test branch. The difference between the test branch and the main SHA is that the test br... Laura Flores
07:23 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
Laura Flores wrote:
> Scheduled 50x tests to run here: http://pulpito.front.sepia.ceph.com/lflores-2022-12-05_17:05:...
Laura Flores
07:22 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
I have a feeling that the tests I scheduled earlier on the main branch all passed since the SHA it picked up is older... Laura Flores
07:14 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
Wondering if there could have been a regression caused by https://github.com/ceph/ceph/pull/48713. Laura Flores
06:38 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
/a/yuriw-2022-11-28_21:26:12-rados-wip-yuri7-testing-2022-11-18-1548-distro-default-smithi/7095988
/a/lflores-2022-1...
Laura Flores
04:17 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
Scheduled 50x tests to run here: http://pulpito.front.sepia.ceph.com/lflores-2022-12-05_17:05:59-rados-wip-yuri10-tes... Laura Flores
04:10 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
Three recent instances of this bug in the main branch point to a regression. My next steps here will be to schedule m... Laura Flores
10:46 PM RADOS Bug #58052: Empty Pool (zero objects) shows usage.
That is every log file from every node. There are no ceph-mgr* logs. :/
Even from inside the docker on the adm n...
Brian Woods
06:33 PM RADOS Bug #58052: Empty Pool (zero objects) shows usage.
Hello. Thanks for response and the files.... Radoslaw Zarzynski
09:11 PM RADOS Bug #58173: api_aio_pp: failure on LibRadosAio.SimplePoolEIOFlag and LibRadosAio.PoolEIOFlag
Building a branch here with https://github.com/ceph/ceph/pull/49029 reverted, which can be used to verify whether it ... Laura Flores
09:03 PM RADOS Bug #58173: api_aio_pp: failure on LibRadosAio.SimplePoolEIOFlag and LibRadosAio.PoolEIOFlag
Excuse my update Sam, I see you already added it as a duplicate. Laura Flores
08:55 PM RADOS Bug #58173: api_aio_pp: failure on LibRadosAio.SimplePoolEIOFlag and LibRadosAio.PoolEIOFlag
Matan added that test within the last two weeks: https://github.com/ceph/ceph/pull/49029 Samuel Just
07:10 PM RADOS Bug #58173 (Resolved): api_aio_pp: failure on LibRadosAio.SimplePoolEIOFlag and LibRadosAio.PoolE...
The workunits/rados/test.sh script is run in the orch suite on some tests. In a few of them, these two tests were fai... Adam King
09:02 PM crimson Bug #58179: crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag
Excuse my update Sam, I see you already added it as a duplicate. Laura Flores
08:49 PM crimson Bug #58179 (Duplicate): crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag
Samuel Just
08:43 PM crimson Bug #58179 (Duplicate): crimson: api_aio_pp: [ FAILED ] LibRadosAio.SimplePoolEIOFlag
Note, this is with https://github.com/ceph/ceph/pull/49029 merged.
https://pulpito.ceph.com/sjust-2022-12-01_22:28...
Samuel Just
08:52 PM rgw Bug #57562: multisite replication issue on Quincy
Hey Adam,
Quick update on the testing that was done on both the latest PR commits (multimap and vector one), we did ...
Krunal Chheda
08:06 PM RADOS Bug #58178: FAILED ceph_assert(last_e.version.version < e.version.version)
Noticed an osd, doing this, on a cluster over the weekend. Its been crashing consistently since. Kevin Fox
08:05 PM RADOS Bug #58178 (Need More Info): FAILED ceph_assert(last_e.version.version < e.version.version)
debug -4> 2022-12-05T19:14:03.556+0000 7fe51028a200 5 osd.57 pg_epoch: 261349 pg[1.573( v 261349'617978754 (2613... Kevin Fox
07:41 PM Orchestrator Backport #58177 (Resolved): pacific: cephadm/ingress: support TLS RGW backend
https://github.com/ceph/ceph/pull/49917 Backport Bot
07:41 PM Orchestrator Backport #58176 (Resolved): quincy: cephadm/ingress: support TLS RGW backend
https://github.com/ceph/ceph/pull/49865 Backport Bot
07:40 PM Orchestrator Backport #58175 (Resolved): pacific: cephadm: cephadm always reports new or changed devices even ...
https://github.com/ceph/ceph/pull/49916 Backport Bot
07:40 PM Orchestrator Backport #58174 (Resolved): quincy: cephadm: cephadm always reports new or changed devices even i...
https://github.com/ceph/ceph/pull/49864 Backport Bot
07:39 PM Orchestrator Bug #57999 (Pending Backport): cephadm: cephadm always reports new or changed devices even if dev...
Adam King
07:35 PM Orchestrator Feature #51972 (Pending Backport): cephadm/ingress: support TLS RGW backend
Adam King
07:07 PM RADOS Bug #56733: Since Pacific upgrade, sporadic latencies plateau on random OSD/disks
I've just let Mark and Ronen know about this issue. Radoslaw Zarzynski
07:06 PM rbd Bug #58172 (Duplicate): get_rollback_snap_id throws bad variant access
The function get_rollback_snap_id generates core dump when force promoting image... Christopher Hoffman
07:05 PM RADOS Bug #58156: Monitors do not permit OSD to join after upgrading to Quincy
Radoslaw Zarzynski wrote:
> Hi Igor! What was the intermediary version during the upgrade? We merged https://github....
Igor Fedotov
06:40 PM RADOS Bug #58156: Monitors do not permit OSD to join after upgrading to Quincy
Hi Igor! What was the intermediary version during the upgrade? We merged https://github.com/ceph/ceph/pull/44090 but ... Radoslaw Zarzynski
07:00 PM RADOS Bug #58142 (In Progress): rbd-python snaps-many-objects: deep-scrub : stat mismatch
Moving to @In progress@ basing the core standup 1 Dec. Radoslaw Zarzynski
06:58 PM rgw Backport #58171 (Resolved): quincy: RGW misplaces index entries after dynamically resharding bucket
https://github.com/ceph/ceph/pull/49795 Backport Bot
06:58 PM rgw Backport #58170 (Duplicate): pacific: RGW misplaces index entries after dynamically resharding bu...
Backport Bot
06:56 PM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
Hello!
what is on disk is actually serialized from the the in-memory representation. We don't see huge numbers of ...
Radoslaw Zarzynski
06:51 PM rgw Bug #58034 (Pending Backport): RGW misplaces index entries after dynamically resharding bucket
J. Eric Ivancich
06:24 PM RADOS Bug #58166 (Need More Info): mon:DAEMON_OLD_VERSION newer versions is considered older than earlier
If your cluster is in the same state, can you please share mon logs with debug_mon=20? The following code snippet in ... Neha Ojha
02:53 PM RADOS Bug #58166: mon:DAEMON_OLD_VERSION newer versions is considered older than earlier
This was probably introduced in https://github.com/ceph/ceph/pull/36759 Tobias Urdin
02:52 PM RADOS Bug #58166 (Need More Info): mon:DAEMON_OLD_VERSION newer versions is considered older than earlier
We have a cluster with most mon/mgr/osd are running 16.2.10 and some OSDs are running 16.2.9
The healthcheck does ...
Tobias Urdin
06:24 PM RADOS Backport #58169 (Resolved): quincy: extra debugs for: [mon] high cpu usage by fn_monstore thread
https://github.com/ceph/ceph/pull/50406 Backport Bot
06:16 PM RADOS Feature #58168 (Pending Backport): extra debugs for: [mon] high cpu usage by fn_monstore thread
Radoslaw Zarzynski
06:16 PM RADOS Feature #58168 (Pending Backport): extra debugs for: [mon] high cpu usage by fn_monstore thread
Radoslaw Zarzynski
06:10 PM RADOS Bug #53806: unessesarily long laggy PG state
> I think as long as `acting` does not have duplicate entries, the logic is exactly the same as before.
Yeah. I'm ...
Radoslaw Zarzynski
05:51 PM RADOS Backport #55768: pacific: rados_api_tests: LibRadosWatchNotify.AioWatchNotify2 fails
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/46499
merged
Yuri Weinstein
05:34 PM RADOS Backport #56648: quincy: [Progress] Do not show NEW PG_NUM value for pool if autoscaler is set to...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/47925
merged
Yuri Weinstein
05:15 PM RADOS Fix #57963: osd: Misleading information displayed for the running configuration of osd_mclock_max...
https://github.com/ceph/ceph/pull/48708 merged Yuri Weinstein
05:12 PM RADOS Bug #57782: [mon] high cpu usage by fn_monstore thread
Radoslaw Zarzynski wrote:
> NOT A FIX (extra debugs): https://github.com/ceph/ceph/pull/48513
merged
Yuri Weinstein
05:04 PM rgw Bug #58167: No Authentication/Authorization for creating topics on RGW
In my example in the original comment the curl was run on a node inside the Ceph test cluster (of Apple M1 Max VMs).
...
Ulrich Klein
04:34 PM rgw Bug #58167: No Authentication/Authorization for creating topics on RGW
* creating a topic by using curl without any user credential is a critical securuty issue.
* since topics are global...
Yuval Lifshitz
04:09 PM rgw Bug #58167 (Pending Backport): No Authentication/Authorization for creating topics on RGW
I'm on a containerized Ceph 17.2.5 serving only RGW/S3 clients.
I'm experimenting with notifications for S3 bucket...
Ulrich Klein
04:56 PM CephFS Backport #57875: pacific: Permissions of the .snap directory do not inherit ACLs
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/48553
merged
Yuri Weinstein
04:54 PM CephFS Backport #57880: pacific: NFS client unable to see newly created files when listing directory con...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/48521
merged
Yuri Weinstein
04:53 PM bluestore Backport #57687 (Resolved): pacific: unable to read osd superblock on AArch64 with page size 64K
Igor Fedotov
04:52 PM bluestore Backport #57687: pacific: unable to read osd superblock on AArch64 with page size 64K
Igor Fedotov wrote:
> https://github.com/ceph/ceph/pull/48278
merged
Yuri Weinstein
04:53 PM CephFS Backport #57848: pacific: mgr/volumes: addition of human-readable flag to volume info command
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/48468
merged
Yuri Weinstein
04:53 PM Backport #57593 (Resolved): pacific: STORE==USED in ceph df
Igor Fedotov
04:51 PM Backport #57593: pacific: STORE==USED in ceph df
Igor Fedotov wrote:
> https://github.com/ceph/ceph/pull/48250
merged
Yuri Weinstein
04:50 PM bluestore Backport #57027 (Resolved): pacific: Bluefs might put an orpan op_update record in the log
Igor Fedotov
04:45 PM bluestore Backport #57027: pacific: Bluefs might put an orpan op_update record in the log
Igor Fedotov wrote:
> https://github.com/ceph/ceph/pull/48168
merged
Yuri Weinstein
04:49 PM bluestore Backport #55300 (Resolved): pacific: Hybrid allocator might return duplicate extents when perform...
Igor Fedotov
04:44 PM bluestore Backport #55300: pacific: Hybrid allocator might return duplicate extents when performing online ...
Igor Fedotov wrote:
> https://github.com/ceph/ceph/pull/48167
merged
Yuri Weinstein
04:48 PM bluestore Bug #54554 (Resolved): Bluestore volume selector improperly tracks bluefs log size
Igor Fedotov
04:42 PM bluestore Bug #54554: Bluestore volume selector improperly tracks bluefs log size
https://github.com/ceph/ceph/pull/45408 merged Yuri Weinstein
04:02 PM RADOS Bug #58165 (Fix Under Review): rados: fix extra tabs on warning for pool copy
Laura Flores
12:57 PM RADOS Bug #58165 (Resolved): rados: fix extra tabs on warning for pool copy
BZ link: https://bugzilla.redhat.com/show_bug.cgi?id=2148242 Shreyansh Sancheti
04:00 PM rbd Backport #58076: pacific: [pwl] inconsistent "rbd status" output (clean = true but dirty_bytes = ...
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/49054
merged
Yuri Weinstein
03:52 PM RADOS Bug #57632 (Fix Under Review): test_envlibrados_for_rocksdb: free(): invalid pointer
Laura Flores
09:42 AM rbd Bug #56724 (New): [rbd_support] recover from RADOS instance blocklisting
Ilya Dryomov
07:52 AM mgr Feature #58164 (In Progress): mgr/prometheus: introduce fetch_perf_counters_metrics module option
Avan Thakkar
07:43 AM mgr Feature #58164 (Pending Backport): mgr/prometheus: introduce fetch_perf_counters_metrics module o...
By default, starting from ceph version `17.2.6`, prometheus module no more export perf counters as prometheus metrics... Avan Thakkar
07:37 AM RADOS Bug #57940: ceph osd crashes with FAILED ceph_assert(clone_overlap.count(clone)) when nobackfill ...
Thomas Le Gentil wrote:
> I could avoid this crash by removing all pg for which ceph could not get the clone_bytes, ...
Thomas Le Gentil
07:34 AM Tasks #58163 (In Progress): doc/monitoring: add documentation for ceph-exporter daemon
Avan Thakkar
03:16 AM crimson Bug #58162 (New): seastar sharded object destructor assert when crimosd handle error
The backtrace :... Rixin Luo
02:31 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Now it's an occasional problem, and I'm watching to see if it happens again. I'm not sure what the trigger condition ... xianpao chen

12/04/2022

04:57 PM rbd Tasks #58161 (New): [luks] exercise layered encryption with ceph_test_librbd_fsx
fsx has proven very useful in weeding out clone and flatten-related bugs in layering code, both in librbd and krbd. Ilya Dryomov
04:40 PM rbd Bug #58160 (New): [luks] encryption-aware flatten/resize can be proxied to a peer that doesn't ha...
Currently, if this happens, all encryption awareness is stripped and the operation is carried out as if the user didn... Ilya Dryomov
02:41 PM Dashboard Feature #58159: mgr/dashboard: add option to set frame ancestors in CSP header mgr/dashboard: sh...
Avan Thakkar wrote:
> h3. Description
Introduce a dashboard module option to allow to set frame ancestors host sour...
Avan Thakkar
02:40 PM Dashboard Feature #58159 (Fix Under Review): mgr/dashboard: add option to set frame ancestors in CSP heade...
h3. Description
Introduce a dashboard module option to allow to set frame ancestors host sources so the page can be ...
Avan Thakkar
12:11 PM Orchestrator Bug #57771: orch/cephadm suite: 'TESTDIR=/home/ubuntu/cephtest bash -s' fails
/a/yuriw-2022-11-28_21:13:47-rados-wip-yuri11-testing-2022-11-18-1506-distro-default-smithi/7095058
/a/yuriw-2022-12...
Matan Breizman
12:06 PM Orchestrator Bug #52321: qa/tasks/rook times out: 'check osd count' reached maximum tries (90) after waiting f...
/a/yuriw-2022-11-28_21:13:47-rados-wip-yuri11-testing-2022-11-18-1506-distro-default-smithi/7095046/ Matan Breizman
12:00 PM Orchestrator Bug #58096: test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
/a/yuriw-2022-11-28_21:13:47-rados-wip-yuri11-testing-2022-11-18-1506-distro-default-smithi/7095033/ Matan Breizman
11:56 AM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
/a/yuriw-2022-11-28_21:13:47-rados-wip-yuri11-testing-2022-11-18-1506-distro-default-smithi/7095031/ Matan Breizman
11:46 AM RADOS Bug #53789: CommandFailedError (rados/test_python.sh): "RADOS object not found" causes test_rados...
/a/yuriw-2022-11-23_21:36:17-rados-wip-yuri11-testing-2022-11-18-1506-distro-default-smithi/7089814/ Matan Breizman
09:41 AM RADOS Backport #58144 (In Progress): pacific: mon/MonCommands: Support dump_historic_slow_ops
Matan Breizman
09:37 AM Orchestrator Feature #58158 (Resolved): OSD container's shutdown timeout should be raised (and configurable)
Currently OSD starts are very slow in quincy v.17.2.5. Disabling osd fast shutdown has no impact.
2022-12-04T09:22...
imirc tw
09:37 AM RADOS Backport #58143 (In Progress): quincy: mon/MonCommands: Support dump_historic_slow_ops
Matan Breizman

12/03/2022

06:18 AM rbd Bug #57765 (Resolved): rbd CLI inconsistencies affecting "--namespace" arg
Konstantin Shalygin
06:17 AM rbd Backport #57844 (Resolved): pacific: rbd CLI inconsistencies affecting "--namespace" arg
Konstantin Shalygin

12/02/2022

09:49 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
In a passed job, the crashes are posted:... Laura Flores
09:33 PM RADOS Bug #58098 (In Progress): qa/workunits/rados/test_crash.sh: crashes are never posted
In the job that passed, the mgr.server reports a recent crash:
/a/lflores-2022-11-30_22:53:49-rados-main-distro-de...
Laura Flores
09:06 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
In one of the jobs that passed, the OSDs were also failed for 31 seconds, but this time, the crashes were detected. S... Laura Flores
09:02 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
Didn't reproduce in the 20x run above, but it did reproduce a second time here:
/a/yuriw-2022-11-28_21:09:37-rados...
Laura Flores
09:26 PM Bug #58157 (New): Ceph Orchestrator fails to recognize partition.
I have a system that has an unmounted XFS volume (it was failing and was unmounted) that was created using the root d... Brian Woods
08:57 PM Orchestrator Bug #58096: test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
/a/yuriw-2022-11-28_21:09:37-rados-wip-yuri4-testing-2022-11-10-1051-distro-default-smithi/7094882 Laura Flores
08:56 PM Orchestrator Bug #58096: test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
@John might be. I'll mark it as related. Laura Flores
02:57 PM Orchestrator Bug #58096: test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
Duplicate of https://tracker.ceph.com/issues/58145 perhaps? John Mulligan
08:53 PM Orchestrator Bug #56000: task/test_nfs: ERROR: Daemon not found: mds.a.smithi060.ujwxef. See `cephadm ls`
/a/yuriw-2022-11-28_21:09:37-rados-wip-yuri4-testing-2022-11-10-1051-distro-default-smithi/7094866 Laura Flores
08:29 PM rbd Backport #57844: pacific: rbd CLI inconsistencies affecting "--namespace" arg
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/48459
merged
Yuri Weinstein
06:09 PM RADOS Bug #58052: Empty Pool (zero objects) shows usage.
Attaching server2 to this message.
Brian Woods
06:09 PM RADOS Bug #58052: Empty Pool (zero objects) shows usage.
I am realizing those logs are from a single host (server4).
server3 got removed today.
Attaching server1 to this me...
Brian Woods
05:42 PM RADOS Bug #58052: Empty Pool (zero objects) shows usage.
Radoslaw Zarzynski wrote:
> Well, I think the command you mentioned did effect for RGW, not MGR. I'm providing the c...
Brian Woods
04:22 PM rgw Bug #57562: multisite replication issue on Quincy
Thanks Adam, we looked at the new PR and see that you are using a vector instead of multi-map. And then do a find on ... Krunal Chheda
03:28 PM RADOS Bug #58156 (In Progress): Monitors do not permit OSD to join after upgrading to Quincy
Igor Fedotov
03:28 PM RADOS Bug #58156 (Resolved): Monitors do not permit OSD to join after upgrading to Quincy
The Nautilus cluster has been eventually upgraded to Quincy and at the end OSDs stopped joining the cluster.
The i...
Igor Fedotov
03:24 PM RADOS Bug #58155 (Resolved): mon:ceph_assert(m < ranks.size()) `different code path than tracker 50089`
Same problem with https://tracker.ceph.com/issues/50089, but it is a different code path.
We opened a new tracker ...
Kamoltat (Junior) Sirivadhna
02:51 PM rbd Backport #58084 (Resolved): pacific: [rbd_support] pool removal can wedge the task queue
Ilya Dryomov
02:49 PM rbd Backport #58084: pacific: [rbd_support] pool removal can wedge the task queue
Backport Bot wrote:
> https://github.com/ceph/ceph/pull/49056
merged
Yuri Weinstein
02:39 PM rbd Backport #58151 (In Progress): quincy: [rbd-nbd] add --snap-id option to "rbd device map" to allo...
Ilya Dryomov
12:24 PM rbd Backport #58151 (Resolved): quincy: [rbd-nbd] add --snap-id option to "rbd device map" to allow m...
https://github.com/ceph/ceph/pull/49197 Backport Bot
02:30 PM rbd Backport #58152 (In Progress): quincy: rbd snap list not change the last read when more than 64 g...
Ilya Dryomov
12:24 PM rbd Backport #58152 (Resolved): quincy: rbd snap list not change the last read when more than 64 grou...
https://github.com/ceph/ceph/pull/49196 Backport Bot
02:28 PM rbd Backport #58153 (In Progress): pacific: rbd snap list not change the last read when more than 64 ...
Ilya Dryomov
12:24 PM rbd Backport #58153 (Resolved): pacific: rbd snap list not change the last read when more than 64 gro...
https://github.com/ceph/ceph/pull/49195 Backport Bot
02:12 PM CephFS Feature #58154 (Resolved): mds: add minor segment boundaries
See PR/commits. Patrick Donnelly
12:18 PM rbd Bug #57066 (Pending Backport): rbd snap list not change the last read when more than 64 group snaps
Ilya Dryomov
12:12 PM rbd Bug #57902 (Pending Backport): [rbd-nbd] add --snap-id option to "rbd device map" to allow mappin...
Ilya Dryomov
10:58 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> Is there a good way to monitor the read/write speed of the fuse and kernel client?
Is this ...
Venky Shankar
09:48 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Is there a good way to monitor the read/write speed of the fuse and kernel client?
xianpao chen
03:52 AM Orchestrator Feature #58150 (Resolved): Addhigh level host related information to the orch host ls command
This feature provides a --detail flag on the 'orch host ls' command to show high level information that cephadm colle... Paul Cuzner
02:59 AM Bug #58120: watcher remains after "rados watch" is interrupted
Ilya Dryomov wrote:
> Hi Wang,
>
> When the client is taken down gracefully, it closes its watch(es). When the c...
王子敬 wang
01:31 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
Nitzan Mordechai wrote:
> 王子敬 wang wrote:
> > Nitzan Mordechai wrote:
> > > Since you attached part of the pglog, ...
王子敬 wang
01:30 AM bluestore Feature #57785: fragmentation score in metrics
I didn't know it was a problem until I tripped across it. The warning I think does more help then harm. Having a docu... Kevin Fox
01:06 AM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
Linked a possible solution for skipping ubuntu with this test. I scheduled a teuthology test for it, which I will use... Laura Flores
12:43 AM bluestore Bug #58022: Fragmentation score rising by seemingly stuck thread
These run away osds are not on the heavy delete workload cluster. Its a relatively lightly loaded cluster. though I c... Kevin Fox

12/01/2022

09:44 PM RADOS Bug #58130: LibRadosAio.SimpleWrite hang and pkill
Thanks for your observations, Brad! I'm going to dedicate this Tracker to `LibRadosAio.SimpleWrite` and mark it as re... Laura Flores
09:20 PM RADOS Bug #58130: LibRadosAio.SimpleWrite hang and pkill
The issue appears to be in the api_aio test as it gets started but doesn't complete.... Brad Hubbard
08:04 PM RADOS Bug #58130: LibRadosAio.SimpleWrite hang and pkill
Ran into another instance of this here:
/a/yuriw-2022-11-30_23:13:27-rados-wip-yuri2-testing-2022-11-30-0724-pacif...
Laura Flores
09:43 PM RADOS Bug #57618: rados/test.sh hang and pkilled (LibRadosWatchNotifyEC.WatchNotify)
/a/yuriw-2022-11-29_22:29:58-rados-wip-yuri10-testing-2022-11-29-1005-pacific-distro-default-smithi/7097464/ Laura Flores
09:23 PM RADOS Bug #57751: LibRadosAio.SimpleWritePP hang and pkill
possibly 58130 is related Brad Hubbard
08:44 PM bluestore Bug #58022: Fragmentation score rising by seemingly stuck thread
Igor/Adam - "But the behavior stops immediately on restart. So feels like some thread in the osd is doing something u... Vikhyat Umrao
07:30 PM RADOS Cleanup #58149 (Resolved): Clarify pool creation failure message due to exceeding max_pgs_per_osd
This was inspired by the Re: [ceph-users] proxmox hyperconverged pg calculations in ceph pacific, pve 7.2 thread.
Anthony D'Atri
07:30 PM RADOS Bug #50089 (Resolved): mon/MonMap.h: FAILED ceph_assert(m < ranks.size()) when reducing number of...
Kamoltat (Junior) Sirivadhna
06:59 PM RADOS Bug #50089 (New): mon/MonMap.h: FAILED ceph_assert(m < ranks.size()) when reducing number of moni...
Kamoltat (Junior) Sirivadhna
07:05 PM Orchestrator Bug #57303: rados/cephadm: Failed to fetch package version from https://shaman.ceph.com/api/searc...
@Adam in the rados suite, this test seems to have additional distros: https://github.com/ceph/ceph/tree/pacific/qa/su... Laura Flores
05:54 PM rgw Bug #57562: multisite replication issue on Quincy
I have a more thoroughly cleaned up and refactored fix at.
Apart from other changes, it collapses identical journa...
Adam Emerson
05:30 PM Orchestrator Backport #58148 (Resolved): pacific: test_cephadm.sh: Error: Error initializing source docker://q...
https://github.com/ceph/ceph/pull/49285 Backport Bot
05:30 PM Orchestrator Bug #58145: orch/cephadm: nfs tests failing to mount exports (mount -t nfs 10.0.31.120:/fake /mnt...
I attempted to debug this situation locally on a 3-node VM cluster. I am able to reproduce the case where mount.nfs f... John Mulligan
04:33 PM Orchestrator Bug #58145 (Pending Backport): orch/cephadm: nfs tests failing to mount exports (mount -t nfs 10....
Currently, since the sepia lab has recovered, all tests that attempt to mount NFS exports are no longer passing. All ... Adam King
05:30 PM Orchestrator Backport #58147 (Resolved): quincy: test_cephadm.sh: Error: Error initializing source docker://qu...
https://github.com/ceph/ceph/pull/49284 Backport Bot
05:27 PM Fix #58135 (Fix Under Review): radosgw:Fix Statement repetition
Daniel Gryniewicz
03:18 AM Fix #58135 (Fix Under Review): radosgw:Fix Statement repetition
There are two statements "using rgw::IAM::s3ListBucket;" in “ceph/src/test/rgw/test_rgw_iam_policy.cc”, it's repeated. weinan Gao
05:21 PM Orchestrator Bug #58146 (Pending Backport): test_cephadm.sh: Error: Error initializing source docker://quay.ce...
Adam King
05:02 PM Orchestrator Bug #58146 (Resolved): test_cephadm.sh: Error: Error initializing source docker://quay.ceph.io/ce...
/a/yuriw-2022-11-28_21:21:22-rados-wip-yuri5-testing-2022-11-18-1554-quincy-distro-default-smithi/7095613... Laura Flores
05:09 PM Orchestrator Bug #56000: task/test_nfs: ERROR: Daemon not found: mds.a.smithi060.ujwxef. See `cephadm ls`
/a/yuriw-2022-11-28_21:21:22-rados-wip-yuri5-testing-2022-11-18-1554-quincy-distro-default-smithi/7095434 Laura Flores
04:20 PM Orchestrator Bug #57287 (Closed): orch/cephadm: 'mount -t nfs 10.0.31.109:/foouser /mnt/foo' failing in nfs-rg...
closing as this stopped happening a long time ago. There does happen to be another similar looking issue right now (y... Adam King
04:12 PM RADOS Backport #58144 (Resolved): pacific: mon/MonCommands: Support dump_historic_slow_ops
https://github.com/ceph/ceph/pull/49233 Backport Bot
04:12 PM RADOS Backport #58143 (Resolved): quincy: mon/MonCommands: Support dump_historic_slow_ops
https://github.com/ceph/ceph/pull/49232 Backport Bot
04:02 PM RADOS Bug #58141 (Pending Backport): mon/MonCommands: Support dump_historic_slow_ops
Matan Breizman
12:42 PM RADOS Bug #58141 (Resolved): mon/MonCommands: Support dump_historic_slow_ops
Slow ops are being tracked in the mon while `dump_historic_slow_ops` command is not registered:
```
$ ceph daemon ....
Matan Breizman
03:56 PM RADOS Bug #58142 (In Progress): rbd-python snaps-many-objects: deep-scrub : stat mismatch
... Matan Breizman
03:47 PM rgw Bug #58127 (Fix Under Review): multisite: test_zg_master_zone_delete fails
Casey Bodley
03:45 PM RADOS Bug #56733: Since Pacific upgrade, sporadic latencies plateau on random OSD/disks
It seems more like generic RADOS issue. Adam Kupczyk
03:14 PM bluestore Bug #58099 (Need More Info): ObjectStore/StoreTestSpecificAUSize.SyntheticMatrixPreferDeferred/2 ...
Adam Kupczyk
03:12 PM rgw Bug #58104 (Won't Fix - EOL): `putlc` failed in slave zonegroup
Casey Bodley
03:12 PM rgw Bug #58105 (Won't Fix - EOL): `DeleteBucketPolicy` can not delete policy in slave zonegroup
the nautilus release is no longer supported. this was fixed in pacific Casey Bodley
03:08 PM rgw Bug #58125 (Won't Fix - EOL): In the nautilus version ceph, the notification message "awsRegion" ...
the nautilus release is no longer supported so won't receive any more backports Casey Bodley
06:14 AM rgw Bug #58125: In the nautilus version ceph, the notification message "awsRegion" parameter is null
https://tracker.ceph.com/issues/53186, can be backport to N. lei cao
03:07 PM rgw Bug #58136 (Fix Under Review): usage trim has infinite loop problem
Casey Bodley
09:04 AM rgw Bug #58136: usage trim has infinite loop problem

https://github.com/ceph/ceph/pull/49168
lei cao
06:35 AM rgw Bug #58136 (Fix Under Review): usage trim has infinite loop problem
try usage trim only specifying "--bucket", when first MAX_USAGE_TRIM_ENTRIES entries in cls method RGW_USER_USAGE_LOG... lei cao
03:06 PM Orchestrator Bug #58140 (Need More Info): quay.ceph.io/ceph-ci/ceph: manifest unknown
hi Aishwarya, this isn't an rgw bug - could you please move it to the appropriate project? Casey Bodley
12:01 PM Orchestrator Bug #58140: quay.ceph.io/ceph-ci/ceph: manifest unknown
/a/yuriw-2022-11-28_16:28:53-rados-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/7094314/
/a/yuriw-...
Matan Breizman
11:20 AM Orchestrator Bug #58140 (Need More Info): quay.ceph.io/ceph-ci/ceph: manifest unknown
Seeing the following error in cephadm/test_cephadm.sh failure:
2022-11-30T02:06:35.681 INFO:tasks.workunit.client...
Aishwarya Mathuria
01:37 PM Bug #58120 (Need More Info): watcher remains after "rados watch" is interrupted
Hi Wang,
When the client is taken down gracefully, it closes its watch(es). When the client is shut down ungracef...
Ilya Dryomov
01:33 PM CephFS Feature #58129: mon/FSCommands: support swapping file systems by name
Venky Shankar wrote:
> The operation also needs to swap the fsid and no clients should we interfering when the swap ...
Patrick Donnelly
04:10 AM CephFS Feature #58129: mon/FSCommands: support swapping file systems by name
The operation also needs to swap the fsid and no clients should we interfering when the swap is under execution. Venky Shankar
01:23 PM Linux kernel client Bug #58126 (Fix Under Review): kclient: WARNING: fs/crypto/keyring.c:242 fscrypt_destroy_keyring+...
The patchwork link: https://patchwork.kernel.org/project/ceph-devel/patch/20221201065800.18149-1-xiubli@redhat.com/ Xiubo Li
12:27 PM RADOS Bug #57757 (Fix Under Review): ECUtil: terminate called after throwing an instance of 'ceph::buff...
Nitzan Mordechai
11:04 AM CephFS Bug #58138 (In Progress): "ceph nfs cluster info" shows junk data for non-existent cluster
Dhairya Parmar
09:18 AM CephFS Bug #58138 (Resolved): "ceph nfs cluster info" shows junk data for non-existent cluster
BZ: https://bugzilla.redhat.com/show_bug.cgi?id=2149415
Steps to Reproduce(we will use a non-existent cluster name...
Dhairya Parmar
09:56 AM Bug #58139 (New): MON spams with Refuse to add /dev/sdq due to limit policy
Hi,
Ceph version 17.2.5, freshly installed with cephadm
All OSDs are fully installed and functional but the MON...
Adrien Georget
09:39 AM Orchestrator Bug #57311: rook: ensure CRDs are installed first
/a/yuriw-2022-11-28_16:28:53-rados-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/7094422/
/a/yuriw-...
Matan Breizman
09:32 AM Dashboard Backport #57829: pacific: cephadm/test_dashboard_e2e.sh: Expected to find content: '/^foo$/' with...
/a/yuriw-2022-11-30_23:13:27-rados-wip-yuri2-testing-2022-11-30-0724-pacific-distro-default-smithi/7099298 Nitzan Mordechai
09:25 AM Orchestrator Bug #53501: Exception when running 'rook' task.
/a/yuriw-2022-11-28_16:28:53-rados-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/7094123/ Matan Breizman
09:23 AM Orchestrator Bug #58096: test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
/a/yuriw-2022-11-28_21:26:12-rados-wip-yuri7-testing-2022-11-18-1548-distro-default-smithi/7096011 Nitzan Mordechai
09:08 AM ceph-volume Bug #58137 (Fix Under Review): OSD prepare job fails with KeyError: 'KNAME'
Guillaume Abrioux
08:46 AM ceph-volume Bug #58137 (Resolved): OSD prepare job fails with KeyError: 'KNAME'
... Guillaume Abrioux
08:18 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
王子敬 wang wrote:
> Nitzan Mordechai wrote:
> > Since you attached part of the pglog, i can't see how many entries yo...
Nitzan Mordechai
01:50 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
Nitzan Mordechai wrote:
> Since you attached part of the pglog, i can't see how many entries you have for log and ho...
王子敬 wang
07:55 AM bluestore Backport #58102 (In Progress): pacific: BlueStore doesn't defer small writes for pre-pacific hdd ...
Adam Kupczyk
03:41 AM RADOS Bug #53806: unessesarily long laggy PG state
Radoslaw Zarzynski wrote:
> OK, Aishwarya has found in testing that the @break@-related commit (https://github.com/c...
玮文 胡
02:15 AM Documentation #58134 (New): doc/radosgw:index error
There is a sentence in the note of chapter "Pool Placement and storage classes": "If you have not done any previous M... weinan Gao
01:28 AM CephFS Feature #58133 (Fix Under Review): qa: add test cases for fscrypt feature in kernel CephFS client
Xiubo Li
01:22 AM CephFS Feature #58133 (Resolved): qa: add test cases for fscrypt feature in kernel CephFS client
As per the documentation fscrypt is a (kernel) "library which filesystems can hook into to support transparent encryp... Xiubo Li
12:51 AM RADOS Backport #58040: quincy: osd: add created_at and ceph_version_when_created metadata
please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/49159
ceph-backport.sh versi...
Kaoru Esashika

11/30/2022

11:15 PM RADOS Bug #58132 (In Progress): qa/standalone/mon: --mon-initial-members setting causes us to populate ...
Kamoltat (Junior) Sirivadhna
11:08 PM RADOS Bug #58132 (Resolved): qa/standalone/mon: --mon-initial-members setting causes us to populate rem...
Problem:
--mon-initial-members does nothing but cause monmap
to populate ``removed_ranks`` because the way we sta...
Kamoltat (Junior) Sirivadhna
10:57 PM RADOS Bug #58098: qa/workunits/rados/test_crash.sh: crashes are never posted
Neha suggested we see how reproducible this is, so as not to mask any underlying problems by sleeping longer. I sched... Laura Flores
10:56 PM Orchestrator Cleanup #58131 (New): orchestrator: infrastructure for tracking dependencies
We have a few dependencies that affect the CI testing. Occasionally problems will pop up where a new version of that ... Adam King
10:34 PM RADOS Bug #58130 (In Progress): LibRadosAio.SimpleWrite hang and pkill
A rados api test experienced a failure after the last global tests had successfully run.
/a/yuriw-2022-11-29_22:29...
Laura Flores
09:48 PM CephFS Bug #54643 (Duplicate): crash: void Server::_unlink_local(MDRequestRef&, CDentry*, CDentry*): ass...
Patrick Donnelly
09:48 PM CephFS Bug #53179 (Duplicate): Crash when unlink in corrupted cephfs
Patrick Donnelly
09:47 PM CephFS Bug #38452 (Need More Info): mds: assert crash loop while unlinking file
Patrick Donnelly
07:31 PM RADOS Bug #58052: Empty Pool (zero objects) shows usage.
Well, I think the command you mentioned did effect for RGW, not MGR. I'm providing the commands increasing log verbos... Radoslaw Zarzynski
07:25 PM RADOS Bug #57977: osd:tick checking mon for new map
The issue during the upgrade looks awfully similar to a downstream Prashant has working on.
Prashant, would find som...
Radoslaw Zarzynski
07:14 PM Orchestrator Bug #57374: orch/cephadm: nfs-ingress-rgw tests failing with "Command failed on smithi158 with st...
/a/yuriw-2022-11-29_22:29:58-rados-wip-yuri10-testing-2022-11-29-1005-pacific-distro-default-smithi/7097649 Aishwarya Mathuria
07:09 PM RADOS Bug #58106 (Need More Info): when a large number of error ops appear in the OSDs,pglog does not t...
Radoslaw Zarzynski
10:43 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
Since you attached part of the pglog, i can't see how many entries you have for log and how many for dups
can you pl...
Nitzan Mordechai
08:38 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
王子敬 wang wrote:
> Nitzan Mordechai wrote:
> > @王子敬 wang, can you please send us the output for one of the pgs from ...
王子敬 wang
08:32 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
Nitzan Mordechai wrote:
> @王子敬 wang, can you please send us the output for one of the pgs from ceph-objectstore-tool...
王子敬 wang
07:30 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
@王子敬 wang, can you please send us the output for one of the pgs from ceph-objectstore-tool?... Nitzan Mordechai
02:16 AM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
Nitzan Mordechai wrote:
> @王子敬 wang can you please provide the output of 'ceph pg dump' ?
ok, the output in the pg_...
王子敬 wang
07:07 PM RADOS Bug #57546: rados/thrash-erasure-code: wait_for_recovery timeout due to "active+clean+remapped+la...
I think the invariant here is that the @acting@ container should not have duplicates. If it is broken, we have a more... Radoslaw Zarzynski
01:55 PM RADOS Bug #57546: rados/thrash-erasure-code: wait_for_recovery timeout due to "active+clean+remapped+la...
If there are indeed duplicated entries in the acting set, should there be a 'break' at all in this loop? It seems lik... Joshua Baergen
07:00 PM RADOS Bug #53806: unessesarily long laggy PG state
OK, Aishwarya has found in testing that the @break@-related commit (https://github.com/ceph/ceph/pull/44499/commits/9... Radoslaw Zarzynski
02:02 PM RADOS Bug #53806: unessesarily long laggy PG state
FWIW, we've seen this happen very frequently during Nautilus->{Octopus,Pacific} upgrades. I had just tracked down the... Joshua Baergen
05:05 PM CephFS Feature #58129 (Pending Backport): mon/FSCommands: support swapping file systems by name
Storage operators like Rook constantly do "reconciliation" to ensure that the desired state of the system (e.g. file ... Patrick Donnelly
04:44 PM Orchestrator Bug #57311: rook: ensure CRDs are installed first
/a/yuriw-2022-11-29_22:29:58-rados-wip-yuri10-testing-2022-11-29-1005-pacific-distro-default-smithi/7097553 Aishwarya Mathuria
04:03 PM Bug #58128 (New): FTBFS with fmtlib 9.1.0
two recent commits:
01f6bef6e5250bf568d9d5f6acf37de64236159a and
3f67d67ede11605b78f9e3abeacd9c46c37d3794
(con...
Kaleb KEITHLEY
04:02 PM rgw Bug #57562: multisite replication issue on Quincy
Adam Emerson wrote:
> Krunal Chheda wrote:
> > Thanks once again for explanation, i was trying to figure out how t...
Krunal Chheda
02:48 AM rgw Bug #57562: multisite replication issue on Quincy
Krunal Chheda wrote:
> Thanks once again for explanation, i was trying to figure out how the assert_exists() works ...
Adam Emerson
02:15 AM rgw Bug #57562: multisite replication issue on Quincy
Adam Emerson wrote:
> Krunal Chheda wrote:
> > Regarding the ENOENT, with assert_exists in place now, will the rea...
Krunal Chheda
01:38 AM rgw Bug #57562: multisite replication issue on Quincy
Specifically in cls_fifo_legacy.cc/push_part() Adam Emerson
01:37 AM rgw Bug #57562: multisite replication issue on Quincy
Krunal Chheda wrote:
> Regarding the ENOENT, with assert_exists in place now, will the read_part_header on the trim...
Adam Emerson
01:19 AM rgw Bug #57562: multisite replication issue on Quincy
Adam Emerson wrote:
> I've pushed a commit that uses assert_exists, then fetches metadata on -ENOENT.
Thanks for ...
Krunal Chheda
01:07 AM rgw Bug #57562: multisite replication issue on Quincy
I've pushed a commit that uses assert_exists, then fetches metadata on -ENOENT. Adam Emerson
03:47 PM rgw Bug #58127 (Resolved): multisite: test_zg_master_zone_delete fails
ex. http://qa-proxy.ceph.com/teuthology/cbodley-2022-09-29_01:41:10-rgw-wip-rgw-sal-bootstrap-distro-default-smithi/7... Casey Bodley
03:38 PM CephFS Bug #24403: mon failed to return metadata for mds
Was discussion about this tracker with Patrick - there are separate paxos proposals for fsmap update and the metadata... Venky Shankar
02:27 PM CephFS Bug #24403: mon failed to return metadata for mds
The MDS is identified using a nonce as well as an IP in the map, right? After the containerized OSDs managed to clobb... Greg Farnum
02:23 PM CephFS Bug #24403: mon failed to return metadata for mds
Venky Shankar wrote:
> It seems the MDS can miss sending beacon in up:boot state. This state encodes the MDS metadat...
Patrick Donnelly
12:52 PM CephFS Bug #24403: mon failed to return metadata for mds
It seems the MDS can miss sending beacon in up:boot state. This state encodes the MDS metadata and includes that in t... Venky Shankar
03:36 PM RADOS Bug #58114 (Closed): mon: FAILED ceph_assert(rank == new_rank)
Close due to this issue is found pre-merge testing from PR: https://github.com/ceph/ceph/pull/48698/ Kamoltat (Junior) Sirivadhna
01:57 PM rgw Backport #58119 (In Progress): pacific: check-generated.sh failures for rgw_log_entry
https://github.com/ceph/ceph/pull/49142 Casey Bodley
01:22 PM Dashboard Bug #54992: pacific: rados/dashboard: tasks/dashboard: cannot stat '/etc/containers/registries.co...
/a/yuriw-2022-11-29_22:29:58-rados-wip-yuri10-testing-2022-11-29-1005-pacific-distro-default-smithi/7097412 Aishwarya Mathuria
11:53 AM bluestore Bug #58113 (In Progress): BLK/Kernel: Improve protection against running one OSD twice
Igor Fedotov
09:53 AM Linux kernel client Bug #58126: kclient: WARNING: fs/crypto/keyring.c:242 fscrypt_destroy_keyring+0x7e/0xd0
Xiubo Li wrote:
> By reading the code it should be a bug in *fs/crypto/* code. When closing the file it will be dela...
Xiubo Li
09:52 AM Linux kernel client Bug #58126: kclient: WARNING: fs/crypto/keyring.c:242 fscrypt_destroy_keyring+0x7e/0xd0
By reading the code it should be a bug in *fs/crypto/* code. When closing the file it will be delayed in kernel space... Xiubo Li
09:46 AM Linux kernel client Bug #58126 (Fix Under Review): kclient: WARNING: fs/crypto/keyring.c:242 fscrypt_destroy_keyring+...
In https://pulpito.ceph.com/xiubli-2022-11-30_03:04:20-fs:fscrypt-wip-lxb-fscrypt-20221122-0933-distro-default-smithi... Xiubo Li
09:25 AM CephFS Bug #57014 (Fix Under Review): cephfs-top: add an option to dump the computed values to stdout
Jos Collin
09:09 AM rbd Bug #58110: Live migration from a QCOW2 source
There are several issues:
1.
librbd does not detect that a qcow2 is actually zlib compressed. There's no eviden...
Or Ozeri
08:26 AM rgw Bug #58125 (Won't Fix - EOL): In the nautilus version ceph, the notification message "awsRegion" ...
The content of the message is as follows:
{
"Records": [{
"eventVersion": "2.2",
"eventSource...
wang kevin
07:43 AM Documentation #58124 (Resolved): doc:Word repetition
Word repetition in swift API weinan Gao
06:38 AM Documentation #58123 (Resolved): ceph:correct help message of "--admin-daemon ADMIN_SOCKET"
The description ‘("help" for help’ after the command "--admin-daemon ADMIN_SOCKET" in the ceph help document is redun... weinan Gao
06:23 AM Dashboard Backport #58121 (In Progress): pacific: mgr/dashboard: relax the cors policies
Nizamudeen A
06:18 AM Dashboard Backport #58121 (Resolved): pacific: mgr/dashboard: relax the cors policies
https://github.com/ceph/ceph/pull/49151 Backport Bot
06:22 AM Dashboard Backport #58122 (In Progress): quincy: mgr/dashboard: relax the cors policies
Nizamudeen A
06:18 AM Dashboard Backport #58122 (Resolved): quincy: mgr/dashboard: relax the cors policies
https://github.com/ceph/ceph/pull/49150 Backport Bot
06:16 AM Dashboard Bug #58086 (Pending Backport): mgr/dashboard: relax the cors policies
Nizamudeen A
06:16 AM Dashboard Bug #58086 (Fix Under Review): mgr/dashboard: relax the cors policies
Nizamudeen A
06:02 AM Dashboard Bug #58086 (Pending Backport): mgr/dashboard: relax the cors policies
Nizamudeen A
06:02 AM Dashboard Bug #58086 (Fix Under Review): mgr/dashboard: relax the cors policies
Nizamudeen A
04:26 AM Dashboard Bug #58086 (Pending Backport): mgr/dashboard: relax the cors policies
Nizamudeen A
04:14 AM RADOS Backport #58039: pacific: osd: add created_at and ceph_version_when_created metadata
please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/49144
ceph-backport.sh versi...
Kaoru Esashika
02:48 AM Bug #58120 (Fix Under Review): watcher remains after "rados watch" is interrupted
when client is down, rbd image still has watchers. I take the down client into osd blacklist ,but still has watchers.... 王子敬 wang

11/29/2022

11:40 PM rgw Bug #57562: multisite replication issue on Quincy
This is very useful, thank you, and might explain why it's happening.
So from your logs, does it seem like the par...
Adam Emerson
11:18 PM rgw Bug #57562: multisite replication issue on Quincy
So coming back to EIO analysis, what we have found so far is this issue happens for more than 1 RGW instances running... Krunal Chheda
09:08 PM rgw Bug #57562: multisite replication issue on Quincy
Adam Emerson wrote:
> Can you point me to the PR for retrying on EIO?
https://github.com/adamemerson/ceph/pull/4/...
Krunal Chheda
04:39 PM rgw Bug #57562: multisite replication issue on Quincy
Can you point me to the PR for retrying on EIO? Adam Emerson
11:18 PM RADOS Bug #54438: test/objectstore/store_test.cc: FAILED ceph_assert(bl_eq(state->contents[noid].data, ...
/a/yuriw-2022-11-28_16:28:53-rados-wip-yuri-testing-2022-11-18-1500-pacific-distro-default-smithi/7094026 Laura Flores
09:35 PM rgw Backport #58119 (Resolved): pacific: check-generated.sh failures for rgw_log_entry
Backport Bot
09:35 PM rgw Backport #58118 (In Progress): quincy: check-generated.sh failures for rgw_log_entry
Backport Bot
09:28 PM rgw Bug #58115 (Pending Backport): check-generated.sh failures for rgw_log_entry
Casey Bodley
06:36 PM rgw Bug #58115 (Fix Under Review): check-generated.sh failures for rgw_log_entry
Casey Bodley
06:35 PM rgw Bug #58115 (Pending Backport): check-generated.sh failures for rgw_log_entry
... Casey Bodley
07:44 PM rgw Bug #57770: RGW (pacific) misplaces index entries after dynamically resharding bucket
J. Eric Ivancich wrote:
> The code on the PR seems to address the issue. My colleague Mark Kogan ran it through a te...
Nick Janus
07:21 PM rgw Bug #58111 (Fix Under Review): crash: verify_bucket_owner_or_policy
Casey Bodley
09:42 AM rgw Bug #58111: crash: verify_bucket_owner_or_policy
... Ilsoo Byun
09:28 AM rgw Bug #58111 (Resolved): crash: verify_bucket_owner_or_policy
When executing 's3cmd ls s3://a:', rgw was terminated. ... Ilsoo Byun
07:14 PM RADOS Backport #58117 (In Progress): quincy: qa/workunits/rados/test_librados_build.sh: specify redirec...
https://github.com/ceph/ceph/pull/49140 Laura Flores
06:58 PM RADOS Backport #58117 (In Progress): quincy: qa/workunits/rados/test_librados_build.sh: specify redirec...
Backport Bot
07:11 PM RADOS Backport #58116 (In Progress): pacific: qa/workunits/rados/test_librados_build.sh: specify redire...
https://github.com/ceph/ceph/pull/49139 Laura Flores
06:58 PM RADOS Backport #58116 (Resolved): pacific: qa/workunits/rados/test_librados_build.sh: specify redirect ...
Backport Bot
06:52 PM RADOS Bug #58046 (Pending Backport): qa/workunits/rados/test_librados_build.sh: specify redirect in cur...
Laura Flores
05:37 PM RADOS Bug #58046: qa/workunits/rados/test_librados_build.sh: specify redirect in curl command
Seen in Pacific run: /a/yuriw-2022-11-28_21:10:48-rados-wip-yuri10-testing-2022-11-28-1042-pacific-distro-default-smi... Aishwarya Mathuria
06:33 PM Orchestrator Bug #58066 (Fix Under Review): config key for cephadm hosts doesn't have values for all network i...
Casey Bodley
06:12 PM Feature #58112: Add OpenMPI and ROMIO modules to support "striping_factor" from MPI standard
People at OpenMPI created skeleton for CephFS support:
https://github.com/open-mpi/ompi/pull/11122
If anyone wa...
Eric Chamberland
12:46 PM Feature #58112 (New): Add OpenMPI and ROMIO modules to support "striping_factor" from MPI standard
Hi,
I am creating this feature request but maybe there is already an ongoing work about adding CephFS support for ...
Eric Chamberland
05:52 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
We discussed this tracker in the RADOS meeting. Sam pointed out that this set of tests doesn't have any actual users,... Laura Flores
05:52 PM Orchestrator Bug #54071: rados/cephadm/osds: Invalid command: missing required parameter hostname(<string>)
/a/yuriw-2022-11-28_21:10:48-rados-wip-yuri10-testing-2022-11-28-1042-pacific-distro-default-smithi/7095174 Aishwarya Mathuria
05:47 PM cephsqlite Bug #51372: pacific: libcephsqlite: segmentation fault
/a/yuriw-2022-11-28_21:10:48-rados-wip-yuri10-testing-2022-11-28-1042-pacific-distro-default-smithi/7095357
Coredump...
Aishwarya Mathuria
05:24 PM RADOS Bug #58114 (Closed): mon: FAILED ceph_assert(rank == new_rank)
/a/yuriw-2022-11-28_21:10:48-rados-wip-yuri10-testing-2022-11-28-1042-pacific-distro-default-smithi/7095280/remote/sm... Aishwarya Mathuria
05:07 PM bluestore Bug #58113 (Resolved): BLK/Kernel: Improve protection against running one OSD twice
Vikhyat Umrao
04:59 PM RADOS Bug #44595: cache tiering: Error: oid 48 copy_from 493 returned error code -2
... Aishwarya Mathuria
03:05 PM RADOS Bug #58107: mon-stretch: old stretch_marked_down_mons leads to ceph unresponsive
Therefore, there is nothing we can do but wait for the other site to come back up, so pgs can complete peering and th... Kamoltat (Junior) Sirivadhna
03:04 PM RADOS Bug #58107 (Closed): mon-stretch: old stretch_marked_down_mons leads to ceph unresponsive
Closed due to this is not a corner case but quote from Greg Farnum:
``it’s that electing those two monitors means ...
Kamoltat (Junior) Sirivadhna
04:15 AM RADOS Bug #58107 (In Progress): mon-stretch: old stretch_marked_down_mons leads to ceph unresponsive
Kamoltat (Junior) Sirivadhna
04:14 AM RADOS Bug #58107 (Closed): mon-stretch: old stretch_marked_down_mons leads to ceph unresponsive
h1. How to reproduce the issue
h2. Set up:
mon.a (zone 1) rank=0
mon.b (zone 1) rank=1
mon.c (zone 2) rank=2
...
Kamoltat (Junior) Sirivadhna
01:07 PM RADOS Bug #58106: when a large number of error ops appear in the OSDs,pglog does not trim.
@王子敬 wang can you please provide the output of 'ceph pg dump' ? Nitzan Mordechai
01:42 AM RADOS Bug #58106 (Need More Info): when a large number of error ops appear in the OSDs,pglog does not t...
When We use the s3 interface append and copy of the object gateway, a large number of error ops appear in the OSDs wh... 王子敬 wang
11:44 AM Dashboard Bug #48686 (Resolved): mgr/dashboard: permission denied when creating a NFS export
Nizamudeen A
11:44 AM Dashboard Backport #57692 (Resolved): quincy: mgr/dashboard: permission denied when creating a NFS export
Nizamudeen A
11:23 AM crimson Bug #58089 (In Progress): get_acting_recovery_backfill().empty() assertion
Matan Breizman
11:12 AM RADOS Bug #57940: ceph osd crashes with FAILED ceph_assert(clone_overlap.count(clone)) when nobackfill ...
I could avoid this crash by removing all pg for which ceph could not get the clone_bytes, except the one I was sure t... Thomas Le Gentil
10:40 AM rgw Bug #44660: Multipart re-uploads cause orphan data
It is very big problem for us.
We have a lot of big buckets with orphaned parts which use hundreds TBs of space.
...
Aleksandr Rudenko
10:39 AM rgw Bug #16767: RadosGW Multipart Cleanup Failure
It is very big problem for us.
We have a lot of big buckets with orphaned parts which use hundreds TBs of space.
...
Aleksandr Rudenko
09:02 AM RADOS Backport #57496 (Resolved): quincy: Invalid read of size 8 in handle_recovery_delete()
Nitzan Mordechai
08:16 AM rbd Bug #58110 (New): Live migration from a QCOW2 source
When using certain QCOW2 images as the source format for live migration, I encounter errors. With some QCOW2 images, ... Effi Ofer
07:13 AM CephFS Feature #56489: qa: test mgr plugins with standby mgr failover
New pull request with mgr thrasher. Milind Changire
07:05 AM RADOS Bug #50042 (Fix Under Review): rados/test.sh: api_watch_notify failures
Nitzan Mordechai
05:10 AM CephFS Bug #58109 (Pending Backport): ceph-fuse: doesn't work properly when the version of libfuse is 3....
I want to use ceph-fuse with libfuse which version is 3.6 or later, because it supports for fuse kernel feature `max_... Zhansong Gao
04:53 AM Dashboard Cleanup #58108 (Fix Under Review): mgr/dashboard: add an env variable to chose custom build folder
Nizamudeen A
04:51 AM Dashboard Cleanup #58108 (Fix Under Review): mgr/dashboard: add an env variable to chose custom build folder
as title says
Nizamudeen A
04:50 AM CephFS Bug #58095: snap-schedule: handle non-existent path gracefully during snapshot creation
Venky Shankar wrote:
> Milind Changire wrote:
> > The most common mistake that users tend to do is include the moun...
Milind Changire
02:13 AM rgw Bug #54908: crash: double const md_config_t::get_val<double>(ConfigValues const&, std::basic_stri...
Similar problem.
(gdb) bt full ...
chao wang
02:04 AM CephFS Bug #58090: Non-existent pending clone shows up in snapshot info
The `/volumes/_index/clone/` directory is empty, by the way. But that's after the snapshot was deleted successfully. ... Sebastian Hasler
02:01 AM CephFS Bug #58090: Non-existent pending clone shows up in snapshot info
Now the snapshot is deleted (finally). From the logs of our CSI provisioner, it seems that the snapshot was deleted s... Sebastian Hasler
01:35 AM rgw Bug #58105 (Won't Fix - EOL): `DeleteBucketPolicy` can not delete policy in slave zonegroup
Huber ming
01:26 AM CephFS Feature #58070: qa: add test suite to test old kernels
Patrick Donnelly wrote:
> This is certainly a good thing to add. Where do we want to put it? fs:workload?
I was ...
Xiubo Li
01:16 AM CephFS Feature #58070: qa: add test suite to test old kernels
This is certainly a good thing to add. Where do we want to put it? fs:workload? We need to be careful to avoid testin... Patrick Donnelly
01:18 AM CephFS Feature #58072: enable 'ceph fs new' use 'ceph fs set' options
I think at this point we should consider making it possible to set arbitrary settings on a fs during creation. i.e. a... Patrick Donnelly
01:11 AM rgw Bug #58104 (Won't Fix - EOL): `putlc` failed in slave zonegroup
radosgw version: nautilus(14.2.15) Huber ming

11/28/2022

10:24 PM RADOS Bug #58098 (Fix Under Review): qa/workunits/rados/test_crash.sh: crashes are never posted
Laura Flores
05:34 PM RADOS Bug #58098 (Resolved): qa/workunits/rados/test_crash.sh: crashes are never posted
/a/yuriw-2022-11-23_15:09:06-rados-wip-yuri10-testing-2022-11-22-1711-distro-default-smithi/7087281... Laura Flores
09:57 PM bluestore Backport #58103 (Resolved): quincy: BlueStore doesn't defer small writes for pre-pacific hdd osds
https://github.com/ceph/ceph/pull/49333 Backport Bot
09:56 PM bluestore Backport #58102 (Resolved): pacific: BlueStore doesn't defer small writes for pre-pacific hdd osds
https://github.com/ceph/ceph/pull/49170 Backport Bot
09:56 PM bluestore Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
Konstantin Shalygin wrote:
> Igor, PR should be replaced to 48490?
yep! done.
Igor Fedotov
09:56 PM bluestore Bug #56488 (Pending Backport): BlueStore doesn't defer small writes for pre-pacific hdd osds
Igor Fedotov
07:38 PM bluestore Bug #56488: BlueStore doesn't defer small writes for pre-pacific hdd osds
Igor, PR should be replaced to 48490? Konstantin Shalygin
09:43 PM RADOS Bug #56733: Since Pacific upgrade, sporadic latencies plateau on random OSD/disks
Just a follow-up.
Finally, what's helping us the best is increasing osd_scrub_sleep to 0.4.
Gilles Mocellin
08:38 PM Orchestrator Feature #58101 (Resolved): cephadm: allow setting mon crush locations
Useful for integrating with stretch mode, particularly for stretch mode, and even more specifically, is necessary for... Adam King
07:29 PM rgw Bug #57562: multisite replication issue on Quincy
Adam Emerson wrote:
> Yes, I just pushed it. We're still testing for regression, but you're welcome to try it.
Than...
Jane Zhu
05:31 PM rgw Bug #57562: multisite replication issue on Quincy
Yes, I just pushed it. We're still testing for regression, but you're welcome to try it. Adam Emerson
05:13 PM rgw Bug #57562: multisite replication issue on Quincy
Adam Emerson wrote:
> Also the goal is to remove tags as part of the fix.
>
> We had some confusion over where a ...
Jane Zhu
03:36 PM rgw Bug #57562: multisite replication issue on Quincy
So currently without the tag changes we see a issue as mentioned here in "comment":https://tracker.ceph.com/issues/57... Krunal Chheda
03:17 PM rgw Bug #57562: multisite replication issue on Quincy
Also the goal is to remove tags as part of the fix.
We had some confusion over where a regression is and it's conf...
Adam Emerson
03:17 PM rgw Bug #57562: multisite replication issue on Quincy
> Also based on our previous test analysis, we think removal of tags will still not completely solve the race conditi... Adam Emerson
07:17 PM bluestore Bug #58022: Fragmentation score rising by seemingly stuck thread
Newer picture, after I had just restarted the current batch of runaways. Kevin Fox
05:24 PM bluestore Bug #58022: Fragmentation score rising by seemingly stuck thread
11 more osds started doing this over the holiday weekend. Kevin Fox
07:02 PM Orchestrator Feature #58100 (Resolved): cephadm: redo service level configuration when service spec is changed
The specific case in mind writing this is that if you have an rgw frontend ssl cert specified in your spec, and then ... Adam King
06:23 PM bluestore Bug #58099 (Need More Info): ObjectStore/StoreTestSpecificAUSize.SyntheticMatrixPreferDeferred/2 ...
/a/yuriw-2022-10-19_18:35:19-rados-wip-yuri10-testing-2022-10-19-0810-distro-default-smithi/7074995/... Laura Flores
06:13 PM Orchestrator Bug #57998 (Resolved): cephadm stuck trying to download "mon"
Shawn Iverson wrote:
> I executed the following:
>
> [...]
>
> and now cephadmn is working!
was away the la...
Adam King
05:19 PM CephFS Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
Milind Changire wrote:
> Due to unavailability of debug logs, there has been some speculation about the issue during...
Venky Shankar
04:58 PM CephFS Bug #58095: snap-schedule: handle non-existent path gracefully during snapshot creation
Milind Changire wrote:
> The most common mistake that users tend to do is include the mount point path along with th...
Venky Shankar
03:39 PM CephFS Bug #58095 (Resolved): snap-schedule: handle non-existent path gracefully during snapshot creation
The most common mistake that users tend to do is include the mount point path along with the file-system path when us... Milind Changire
04:29 PM Orchestrator Bug #58096: test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
Laura Flores wrote:
> [...]
/a/yuriw-2022-11-23_15:09:06-rados-wip-yuri10-testing-2022-11-22-1711-distro-default-...
Laura Flores
04:28 PM Orchestrator Bug #58096 (New): test_cluster_set_reset_user_config: NFS mount fails due to missing ceph directory
... Laura Flores
04:01 PM Bug #58048: «EPERM: error calling ceph_mount» when trying to use subvolume commands
The project should be CephFS on this issue but I can't seem to be able to change it myself. Jérôme Poulin
03:26 PM CephFS Bug #54017: Problem with ceph fs snapshot mirror and read-only folders
Milind, this was discussed here - https://www.mail-archive.com/ceph-users@ceph.io/msg14364.html
Related bz - https...
Venky Shankar
03:06 PM CephFS Bug #58090: Non-existent pending clone shows up in snapshot info
Hi Sebastian,
There is a stray index causing this issue. Could you list the contents of `/volumes/_index/clone/` (...
Venky Shankar
02:47 PM RADOS Bug #52657: MOSDPGLog::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_NAUTILUS)'
Aishwarya Mathuria wrote:
> We suspect that this assert failure is hit in cases when we try to encode a message befo...
Ben Gao
02:44 PM CephFS Bug #58058: CephFS Snapshot Mirroring slow due to repeating attribute sync
Mathias Kuhring wrote:
> We might have found a major performance bug in the cephfs snapshot mirroring.
> We already...
Venky Shankar
01:41 PM CephFS Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
Venky Shankar
02:37 PM Dashboard Bug #58094 (Resolved): mgr/dashboard: expose slow ops per osd
h3. Description of problem
Expose the number of slow operations per osd in prometheus and create a grafana panel.
...
Pere Díaz Bou
02:20 PM Support #58093: recover deleted bucket during replication
Shafiq M wrote:
> recover deleted bucket during replication
Shafiq M
12:01 PM Support #58093 (New): recover deleted bucket during replication
recover deleted bucket during replication Shafiq M
11:42 AM rgw Documentation #58092 (New): rgw_enable_gc_threads / lc_threads not documented on web
Options rgw_enable_gc_threads and rgw_enable_lc_threads are not rendered for docs.ceph.com.
I would expect those t...
Dan van der Ster
10:07 AM ceph-volume Bug #57767: ceph-volume should check if device is locked prior to zapping it
A recent thread [1] addresses this issue again. A simple test with Nautilus shows that in previous versions ceph-volu... Eugen Block
09:11 AM rgw Backport #57238 (In Progress): pacific: crash: RGWCoroutinesStack::wakeup()
Cory Snyder
09:11 AM rgw Backport #57237 (In Progress): quincy: crash: RGWCoroutinesStack::wakeup()
Cory Snyder
09:08 AM rgw Backport #55228 (In Progress): pacific: crash: RGWGC::send_chain(cls_rgw_obj_chain&, std::basic_s...
Cory Snyder
09:06 AM rgw Backport #55227 (In Progress): quincy: crash: RGWGC::send_chain(cls_rgw_obj_chain&, std::basic_st...
Cory Snyder
09:05 AM rgw Backport #54497 (In Progress): pacific: bucket index completions may not retry after reshard
Cory Snyder
09:04 AM rgw Backport #54496 (In Progress): quincy: bucket index completions may not retry after reshard
Cory Snyder
09:03 AM rgw Backport #54155 (In Progress): pacific: rgw: "reshard cancel" errors with "invalid argument"
Cory Snyder
09:03 AM rgw Backport #54157 (In Progress): quincy: rgw: "reshard cancel" errors with "invalid argument"
Cory Snyder
09:00 AM rgw Backport #55505 (In Progress): pacific: radosgw rejects some requests without Content-MD5 Header
Cory Snyder
08:59 AM rgw Backport #55506 (In Progress): quincy: radosgw rejects some requests without Content-MD5 Header
Cory Snyder
08:44 AM Backport #57463 (In Progress): quincy: pacific: Could NOT find Python3
Cory Snyder
08:37 AM rgw Backport #57409 (In Progress): pacific: rgw: bucket list operation slow down in special scenario
Cory Snyder
08:32 AM rgw Backport #57410 (In Progress): quincy: rgw: bucket list operation slow down in special scenario
Cory Snyder
08:20 AM rgw Backport #57752 (In Progress): quincy: Log status of individual object deletions for multi-object...
Cory Snyder
07:55 AM rgw Backport #54493 (In Progress): quincy: segmentation fault in UserAsyncRefreshHandler::init_fetch
Cory Snyder
05:32 AM CephFS Bug #58082 (Fix Under Review): cephfs:filesystem became read only after Quincy upgrade
Konstantin Shalygin
05:05 AM RADOS Support #58091 (New): osd: reduce default value of osd_heartbeat_grace
Client io hang 20s when peer osd ping failure, 20s is too long. In case of network jitter, it generally does not exce... yite gu

11/27/2022

10:10 PM bluestore Feature #57785: fragmentation score in metrics
I think having the metric available opens the door for monitoring escalation for prometheus and less frequently used ... Paul Cuzner
02:08 PM CephFS Bug #58090 (New): Non-existent pending clone shows up in snapshot info
Ceph version: v17.2.5
My CephFS somehow got in a state where a snapshot has a pending clone, but the pending clone...
Sebastian Hasler
11:16 AM crimson Bug #58089: get_acting_recovery_backfill().empty() assertion
https://pulpito.ceph.com/matan-2022-11-22_15:04:25-crimson-rados-main-distro-crimson-smithi/7086464/ Matan Breizman
11:13 AM crimson Bug #58089 (Resolved): get_acting_recovery_backfill().empty() assertion
... Matan Breizman
07:46 AM Documentation #58080 (Resolved): Update RBD documentation - Ceph iSCSI Gateway does not support r...
Zac Dover

11/26/2022

05:23 PM CephFS Bug #58088 (Fix Under Review): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd()...
Ramana Raja
05:08 PM CephFS Bug #58088 (Resolved): qa/tasks/vstart_runner: TypeError: LocalFuseMount._run_mount_cmd() takes 3...
Hit this error,... Ramana Raja

11/25/2022

03:57 PM rgw Backport #58087 (In Progress): quincy: rgw/cloud-tranistion: Issues with MCG cloud endpoint
Soumya Koduri
03:56 PM rgw Backport #58087: quincy: rgw/cloud-tranistion: Issues with MCG cloud endpoint
https://github.com/ceph/ceph/pull/49061 Soumya Koduri
03:23 PM rgw Backport #58087 (In Progress): quincy: rgw/cloud-tranistion: Issues with MCG cloud endpoint
Backport Bot
03:17 PM rgw Bug #57979 (Pending Backport): rgw/cloud-tranistion: Issues with MCG cloud endpoint
Soumya Koduri
03:07 PM Dashboard Bug #58086 (Fix Under Review): mgr/dashboard: relax the cors policies
Nizamudeen A
02:55 PM Dashboard Bug #58086 (Resolved): mgr/dashboard: relax the cors policies
cors policy is strictly disabled now in the api. Relax it when needed. Nizamudeen A
12:05 PM Bug #58069: flake8 fails since Nov 23 2022
... and quincy too:
https://jenkins.ceph.com/job/ceph-pull-requests/107099/consoleText
pacific backport PR: htt...
Ilya Dryomov
11:44 AM Bug #58069: flake8 fails since Nov 23 2022
Also seen on pacific:... Ilya Dryomov
11:40 AM rbd Backport #58085 (In Progress): quincy: [rbd_support] pool removal can wedge the task queue
Ilya Dryomov
10:53 AM rbd Backport #58085 (Resolved): quincy: [rbd_support] pool removal can wedge the task queue
https://github.com/ceph/ceph/pull/49057 Backport Bot
11:37 AM rbd Backport #58084 (In Progress): pacific: [rbd_support] pool removal can wedge the task queue
Ilya Dryomov
10:53 AM rbd Backport #58084 (Resolved): pacific: [rbd_support] pool removal can wedge the task queue
https://github.com/ceph/ceph/pull/49056 Backport Bot
11:17 AM rbd Backport #58077 (In Progress): quincy: [pwl] inconsistent "rbd status" output (clean = true but d...
Ilya Dryomov
11:15 AM rbd Backport #58076 (In Progress): pacific: [pwl] inconsistent "rbd status" output (clean = true but ...
Ilya Dryomov
10:49 AM rbd Bug #52932 (Pending Backport): [rbd_support] pool removal can wedge the task queue
Ilya Dryomov
10:12 AM CephFS Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
Due to unavailability of debug logs, there has been some speculation about the issue during discussion with Venky.
T...
Milind Changire
06:07 AM Dashboard Feature #58083 (Resolved): mgr/dashboard: Add metric relative to osd blocklist
It would be good to have a metric with number of clients "blocklisted" of command: "ceph osd blocklist ls". The reaso... Aashish Sharma
05:07 AM CephFS Bug #58082: cephfs:filesystem became read only after Quincy upgrade
From the logs, the *dir(0x1)* will submit the *volumes* Dentry to metadata pool: ... Xiubo Li
04:51 AM CephFS Bug #58082 (Resolved): cephfs:filesystem became read only after Quincy upgrade
Copy the info from ceph-user mail list by Adrien:... Xiubo Li
04:54 AM CephFS Bug #52260 (Duplicate): 1 MDSs are read only | pacific 16.2.5
Will tracker and fix it in https://tracker.ceph.com/issues/58082. Xiubo Li

11/24/2022

06:00 PM Documentation #58080 (Resolved): Update RBD documentation - Ceph iSCSI Gateway does not support r...
From https://pad.ceph.com/p/Report_Documentation_Bugs:
"Applications that use SCSI 7 perstistent group reserva...
Zac Dover
05:27 PM CephFS Backport #58079 (Resolved): quincy: cephfs-top: Sorting doesn't work when the filesystems are rem...
https://github.com/ceph/ceph/pull/50151 Backport Bot
05:27 PM CephFS Backport #58078 (Resolved): pacific: cephfs-top: Sorting doesn't work when the filesystems are re...
https://github.com/ceph/ceph/pull/49303 Backport Bot
05:26 PM CephFS Bug #58028 (Pending Backport): cephfs-top: Sorting doesn't work when the filesystems are removed ...
Venky Shankar
03:28 PM Orchestrator Bug #58063 (In Progress): osd encryption is failing on ubuntu 22.04
Guillaume Abrioux
03:19 PM Orchestrator Bug #58063: osd encryption is failing on ubuntu 22.04
Ubuntu 22.04 minimal** Alex Karl
03:17 PM Orchestrator Bug #58063: osd encryption is failing on ubuntu 22.04
This issue is only reproducible on ubuntu 22.04 mini. Alex Karl
03:15 PM rbd Backport #58077 (Resolved): quincy: [pwl] inconsistent "rbd status" output (clean = true but dirt...
https://github.com/ceph/ceph/pull/49055 Backport Bot
03:15 PM rbd Backport #58076 (Resolved): pacific: [pwl] inconsistent "rbd status" output (clean = true but dir...
https://github.com/ceph/ceph/pull/49054 Backport Bot
03:13 PM rbd Bug #57872 (Pending Backport): [pwl] inconsistent "rbd status" output (clean = true but dirty_byt...
Ilya Dryomov
02:44 PM Orchestrator Bug #58066: config key for cephadm hosts doesn't have values for all network interfaces present i...
Raised following PR - https://github.com/ceph/ceph/pull/49043 Prajwal Kabbinale
02:21 PM Support #58075 (New): bucket not able to read from s3 says no bucket and replication not working ...
bucket not able to read from s3 says no bucket and replication not working for such bucket Shafiq M
01:39 PM CephFS Backport #58074 (Resolved): quincy: cephfs-top: sorting/limit excepts when the filesystems are re...
https://github.com/ceph/ceph/pull/50151 Backport Bot
01:39 PM CephFS Backport #58073 (Resolved): pacific: cephfs-top: sorting/limit excepts when the filesystems are r...
https://github.com/ceph/ceph/pull/49303 Backport Bot
01:36 PM CephFS Bug #58031 (Pending Backport): cephfs-top: sorting/limit excepts when the filesystems are removed...
Venky Shankar
01:05 PM CephFS Feature #58072 (Fix Under Review): enable 'ceph fs new' use 'ceph fs set' options
As discussed in PR [1], this flag would come handy in situations like 'ceph fs new --recover'. Need to push this enha... Dhairya Parmar
12:51 PM Bug #58069 (Resolved): flake8 fails since Nov 23 2022
Kefu Chai
06:06 AM Bug #58069 (In Progress): flake8 fails since Nov 23 2022
Kefu Chai
05:42 AM Bug #58069 (Resolved): flake8 fails since Nov 23 2022
... Kefu Chai
08:41 AM Bug #58071 (New): Bucket keys do not exist in omap
In a cluster one bucket(not found more yet) cannot list the owner because key do not exist in omap. Here: rados -p de... Jonas Nemeikšis
07:33 AM CephFS Feature #58070 (New): qa: add test suite to test old kernels
Currently there is test case will test old ceph-fuse clients with new ceph, but we also need to test the old kclient ... Xiubo Li
05:36 AM CephFS Feature #55940 (Fix Under Review): quota: accept values in human readable format as well
Dhairya Parmar
03:54 AM RADOS Bug #57977: osd:tick checking mon for new map
The more I dig, the more I'm thinking that this might be some race to do with noup, and probably has nothing to do wi... Joshua Baergen
03:42 AM RADOS Bug #57977: osd:tick checking mon for new map
Something that's probably worth mentioning - we had noup set in the cluster for each upgrade, and we wait until all O... Joshua Baergen
03:12 AM RADOS Bug #57977: osd:tick checking mon for new map
We saw this happen to roughly a dozen OSDs (1-2 per host for some hosts) during a recent upgrade from Nautilus to Pac... Joshua Baergen

11/23/2022

06:08 PM rbd Bug #52932 (Fix Under Review): [rbd_support] pool removal can wedge the task queue
Ilya Dryomov
06:06 PM rgw Bug #58059 (Resolved): s3tests v2 SignatureDoesNotMatch failures on ubuntu
Casey Bodley
05:31 PM CephFS Bug #24403: mon failed to return metadata for mds
FYI - restarting the MDS fixes the issue. Venky Shankar
05:30 PM CephFS Bug #24403: mon failed to return metadata for mds
This was seen in pacific installation. MDS entries in FSMap are fine - that serves `fs dump` and `fs status` commands... Venky Shankar
04:32 PM Documentation #57001: RBD Rewrite Project
https://github.com/ceph/ceph/pull/49035
This PR swaps the order of the first two paragraphs in the front matter.
...
Zac Dover
04:26 PM Documentation #57001: RBD Rewrite Project
https://medium.com/system-design-blog/eventual-consistency-vs-strong-consistency-b4de1f92534d - A medium.com article ... Zac Dover
04:20 PM bluestore Bug #48216: Spanning blobs list might have zombie blobs that aren't of use any more
Gilles Mocellin wrote:
> Hello,
>
> No news on that ?
> Does someone knows if the problem also happens on Quincy...
Igor Fedotov
09:46 AM bluestore Bug #48216: Spanning blobs list might have zombie blobs that aren't of use any more
Hello,
No news on that ?
Does someone knows if the problem also happens on Quincy ?
Gilles Mocellin
04:19 PM rgw Bug #57853 (Fix Under Review): multisite sync process block after long time running
Casey Bodley
04:12 PM bluestore Bug #54019: OSD::mkfs: ObjectStore::mkfs failed with error (5) Input/output error
Just made a topic for potential fix discussion at https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/CHVBMPENHO... Igor Fedotov
01:19 PM Orchestrator Bug #58067 (Fix Under Review): Service discovery are not reported consistently in ceph orch ps
Redouane Kachach Elhichou
09:09 AM Orchestrator Bug #58067 (Resolved): Service discovery are not reported consistently in ceph orch ps
Ceph orch ps lists all the daemons and open ports for each one of them. Right now cephadm service discovery port is o... Redouane Kachach Elhichou
01:18 PM rbd Bug #58062 (Duplicate): RBD tasks will stop if a pool is deleted, blocking further queue
Hi Miodrag,
Thanks for the report! This is a known issue, I'll bump the priority on the older ticket and we will ...
Ilya Dryomov
12:05 PM CephFS Bug #58031 (Fix Under Review): cephfs-top: sorting/limit excepts when the filesystems are removed...
Neeraj Pratap Singh
10:24 AM Orchestrator Bug #58066: config key for cephadm hosts doesn't have values for all network interfaces present i...
same problem is observed in v17.2.5 as well Prajwal Kabbinale
08:17 AM Orchestrator Bug #58066: config key for cephadm hosts doesn't have values for all network interfaces present i...
Affected version: v16.2.10 Prajwal Kabbinale
08:17 AM Orchestrator Bug #58066 (Fix Under Review): config key for cephadm hosts doesn't have values for all network i...
with cephadm, it stores key-value for each host that are part of the cluster.
For ex:
ceph config-key ls | grep m...
Prajwal Kabbinale
09:23 AM ceph-ansible Bug #53684: The ingress daemon uses "*" as frontend network even though a virtual_ip is specified...
Did anyone think about the fact that HAProxy configuration is created by keepalived as per https://tracker.ceph.com/i... Voja Molani
09:20 AM Orchestrator Bug #57910: ingress: HAProxy fails to start because keepalived IP address not yet available on ne...
The HAProxy service does not start until it is manually started or the server is restarted.
The problem root is th...
Voja Molani
08:05 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory...
Venky Shankar
07:40 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I've heard that too large mds_cache_memory_limit may cause problems, so I use mds_cache_memory_limit = 16GB, no speci... xianpao chen
07:25 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Any reason you are using mds_cache_memory_limit = 16GB when you have memory to spare? Venky Shankar
06:32 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
the "free -h" of the mds node(after restart the mds): ... xianpao chen
06:06 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
BTW, do you see any performance degradation on clients in general over the course and/or when the MDS is about to get... Venky Shankar
05:27 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I checked the session info, there is just one client which is holding ~1M caps. But that should not bother the MDS th... Venky Shankar
05:57 AM Dashboard Tasks #58065 (Resolved): mgr/dashboard: inventory card
Nizamudeen A
12:57 AM mgr Feature #58064 (New): mgr/restful: support for http requests
Support ceph config set mgr mgr/restful/https_mode false/true to control HTTP/HTTPS access.
yuanchun huang

11/22/2022

11:10 PM rgw Bug #58053: bucket is list in s3cmd but can not be queried after deleting
*ceph-qa-suite* should be RGW Max Gao
06:20 AM rgw Bug #58053: bucket is list in s3cmd but can not be queried after deleting
Prepare a policy.json file that contains the bucket policy for testing.... Max Gao
06:13 AM rgw Bug #58053: bucket is list in s3cmd but can not be queried after deleting
The bug can be reproduced with the following script:
Max Gao
07:48 PM rgw Bug #58059 (Fix Under Review): s3tests v2 SignatureDoesNotMatch failures on ubuntu
https://github.com/ceph/s3-tests/pull/476 Casey Bodley
05:03 PM rgw Bug #58059: s3tests v2 SignatureDoesNotMatch failures on ubuntu
bisected botocore versions down to good=botocore-1.27.96 bad=botocore-1.28.0
botocore debug log output from good=b...
Casey Bodley
03:11 PM rgw Bug #58059: s3tests v2 SignatureDoesNotMatch failures on ubuntu
boto versions from a failing run on ubuntu:... Casey Bodley
07:01 PM bluestore Feature #57785: fragmentation score in metrics
❤️ Kevin Fox
06:52 PM bluestore Feature #57785: fragmentation score in metrics
After syncing with Adam Kupczyk today: 
In the shorter term we will make the fragmentation score, both for bluefs ...
Yaarit Hatuka
06:28 PM Orchestrator Bug #58063 (In Progress): osd encryption is failing on ubuntu 22.04
Hallo folks,
i am deploying a quincy ceph cluster 17.2.0 on openstack vm with ubuntu
22.04 minimal with cephadm.
...
Alex Karl
06:17 PM RADOS Bug #57977: osd:tick checking mon for new map
I already restart osd daemon, but have no reproduct. If it happens again, I will collect more logs yite gu
03:54 PM RADOS Bug #58052: Empty Pool (zero objects) shows usage.
Radoslaw Zarzynski wrote:
> Could you please provide a log from an active mgr with @debug_ms=1@ and @debug_mgr=20@?
...
Brian Woods
03:42 PM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
I happened to have a memory problem today, then I changed mds_session_cache_liveness_decay_rate to 150s, tried "ceph ... xianpao chen
11:19 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Did you get to applying the suggested config?
Thanks for your suggestion, I will try it to...
xianpao chen
11:07 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Did you get to applying the suggested config? Venky Shankar
03:26 PM rbd Bug #58062 (Duplicate): RBD tasks will stop if a pool is deleted, blocking further queue
Hi.
I believe we found a bug with RBD tasks in Ceph MGR, dating somewhere between versions 16.2.0 and 17.2.1 (thes...
Miodrag Prelec
03:00 PM Orchestrator Feature #58061 (New): Add support for HA for ceph mgr
In the current design ceph can run multiple instances of mgr. The mgr is a core component of ceph that runs important... Redouane Kachach Elhichou
10:19 AM CephFS Bug #57523: CephFS performance degredation in mountpoint
Guys this can't be only a thing in our setup. Every time a connection puts more than a few GB into cephfs the perform... Vincent Hermes
04:42 AM rgw Bug #57562: multisite replication issue on Quincy
Hi Adam, another question regarding your changes to remove the use of "tags". I'd like to understand your opinion/pla... Jane Zhu
04:34 AM rgw Bug #57562: multisite replication issue on Quincy
Did some investigation on the latest failure of the tests on `lastest - 1` "PR":https://github.com/ceph/ceph/pull/486... Jane Zhu
03:32 AM rbd Bug #58060 (Resolved): avoid EUCLEAN error after "rbd rm" is interrupted
If rbd rm process is interrupted (for example, rbd client killed, or ceph_assert fault injection in rbd code), rbd st... xinwei wei
01:19 AM CephFS Bug #58056: ceph-fuse - fuse failed to start on CentOS 7 host machine
It seems passing invalidate flag ?... Xiubo Li

11/21/2022

09:03 PM rgw Bug #58059 (Resolved): s3tests v2 SignatureDoesNotMatch failures on ubuntu
from main branch results: https://pulpito.ceph.com/cbodley-2022-11-21_18:00:47-rgw-main-distro-default-smithi/
s3t...
Casey Bodley
08:41 PM rgw Bug #57562: multisite replication issue on Quincy
Hi Adam,
Wanted to provide you with an update about the testing that we did over the weekend,
We took 2 PR's, one w...
Krunal Chheda
08:17 PM CephFS Bug #58058 (Triaged): CephFS Snapshot Mirroring slow due to repeating attribute sync
We might have found a major performance bug in the cephfs snapshot mirroring.
We already reported it to the mailing ...
Mathias Kuhring
08:17 PM bluestore Bug #58022: Fragmentation score rising by seemingly stuck thread
Saw this on 8 more osds over the weekend. Kevin Fox
06:35 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
@Radek I have been trying to reproduce this locally with no luck. I'll try your suggestion and update if I'm successful. Laura Flores
06:34 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
Thanks for the link, Matan! I'm a bit worried the experiment there involved changing 2 parameters the same: compiler ... Radoslaw Zarzynski
06:29 PM RADOS Bug #58044 (Need More Info): ceph-osd: osd numa affinity setting doesn't work
How do you check the affinity?
Have you rebooted the OSD after the injecting the setting?
Could you please provide ...
Radoslaw Zarzynski
06:22 PM RADOS Bug #58046 (Resolved): qa/workunits/rados/test_librados_build.sh: specify redirect in curl command
Radoslaw Zarzynski
06:21 PM RADOS Bug #58052 (Need More Info): Empty Pool (zero objects) shows usage.
Could you please provide a log from an active mgr with @debug_ms=1@ and @debug_mgr=20@? We would like to see which OS... Radoslaw Zarzynski
06:11 PM CephFS Support #38374: Crash when using cephfs as /var/lib/docker in devicemapper mode
We're not using this kind of setup anymore and won't be troubleshooting further. We can close this for now, probably ... Jérôme Poulin
03:56 PM rgw Bug #55310 (Duplicate): [pacific] RadosGW instance of Cloud Sync zone crashes when objects are up...
Casey Bodley
01:53 PM CephFS Feature #58057: cephfs-top: enhance fstop tests to cover testing displayed data
The Dashboard folks could point us to tools for testing console UI based apps. Milind Changire
11:30 AM CephFS Feature #58057 (Resolved): cephfs-top: enhance fstop tests to cover testing displayed data
Right now the tests are pretty rudimentary. cephfs-top is a UI tool and writing tests can be a bit hard. Due to this ... Venky Shankar
10:42 AM Orchestrator Bug #57897: ceph mgr restart causes restart of all iscsi daemons in a loop
Hi Mykola and Redouane
Thanks for looking into this - we initially thought the same regarding the commit in the [2...
David Heap
09:52 AM rbd Bug #57941 (Rejected): Severe performance drop after writing 100 GB of data to RBD volume, depend...
Thanks for following up, Guillaume! Ilya Dryomov
09:05 AM CephFS Bug #58056 (New): ceph-fuse - fuse failed to start on CentOS 7 host machine
Hello,
We were previously using ceph v16.2.10 (docker container) with rook under kubernetes which was installed on...
Razvan Ghitescu
08:42 AM Linux kernel client Feature #58055 (New): feature to block kclient from establishing session
We need to implement https://github.com/ceph/ceph/pull/48720 for kclient too. Dhairya Parmar
07:18 AM RADOS Bug #58027: op slow from throttled to header_read
Radoslaw Zarzynski wrote:
> Hello! The most important thing is Octopus is EOL. Second, I'm also not sure whether thi...
yite gu
06:31 AM CephFS Bug #57014 (In Progress): cephfs-top: add an option to dump the computed values to stdout
Jos Collin
02:02 AM Linux kernel client Bug #58054 (Resolved): kclient: xfstests-dev generic/684 fails
... Xiubo Li
01:47 AM Linux kernel client Bug #57986 (Fix Under Review): ceph: ceph_fl_release_lock cause "unable to handle kernel paging r...
The patchwork links:
Jeff's VFS locks patch:
https://patchwork.kernel.org/project/ceph-devel/list/?series=695950
...
Xiubo Li
01:43 AM Bug #57613 (Resolved): Kernel Oops, kernel NULL pointer dereference
David Howells has fixed it in netfs layer:
https://git.kernel.org/torvalds/c/59d0d52c30d4991ac4b329f049cc37118e00f5b0
Xiubo Li
12:00 AM rgw Bug #58053 (Need More Info): bucket is list in s3cmd but can not be queried after deleting
ceph version 16.2.10
There will be a race between s3.DeleteBucket and s3.DeleteBucketPolicy. When the race happens...
Max Gao

11/20/2022

10:59 PM rbd Bug #57941: Severe performance drop after writing 100 GB of data to RBD volume, dependent on RAM ...
:facepalm: the issue has nothing to do with Ceph and everything to do with smaller EC2 instances having burstable net... Guillaume Pothier
05:23 PM RADOS Bug #58052 (Need More Info): Empty Pool (zero objects) shows usage.
I have a pool that was/is being used in a CephFS. I have migrated all of the files off of the pool and was preparing... Brian Woods
01:24 PM Orchestrator Bug #57897: ceph mgr restart causes restart of all iscsi daemons in a loop
David mentioned [1] as a potential cause when the issue was introduced. But actually I think it is [2]. And apart tha... Mykola Golub
01:10 PM Orchestrator Bug #58051 (Resolved): mgr/cephadm: try to avoid pull when getting container image info
Currently, when getting a container image info the cephadm module always runs pull command.
There may be environme...
Mykola Golub

11/18/2022

08:49 PM mgr Bug #58050 (Fix Under Review): mgr/telemetry: perf histograms are not formatted in `all` commands
Laura Flores
08:47 PM mgr Bug #58050 (Resolved): mgr/telemetry: perf histograms are not formatted in `all` commands
I noticed that perf histograms weren't getting formatted in `preview-all` and `show-all`. Laura Flores
04:16 PM rbd Bug #57941: Severe performance drop after writing 100 GB of data to RBD volume, dependent on RAM ...
In order to rule out if client side caching is a factor can you do a few tests?
Run tests without client side cach...
Christopher Hoffman
03:29 PM RADOS Bug #58049 (Resolved): mon:stretch-cluster: mishandled removed_ranks -> inconsistent peer_tracker...
First encountered in the downstream: https://bugzilla.redhat.com/show_bug.cgi?id=2142674
When we failover monitors...
Kamoltat (Junior) Sirivadhna
02:27 PM Bug #58048: «EPERM: error calling ceph_mount» when trying to use subvolume commands
Here is a log of the issue when running ceph-mds manually with the following command:
ceph-mds --setuser ceph --setg...
Jérôme Poulin
02:21 PM Bug #58048 (New): «EPERM: error calling ceph_mount» when trying to use subvolume commands
When trying to issue commands such as «ceph fs subvolume ls cephfsv2», we get a return of «Error EPERM: error calling... Jérôme Poulin
11:38 AM CephFS Bug #58028 (Fix Under Review): cephfs-top: Sorting doesn't work when the filesystems are removed ...
Jos Collin
03:23 AM Linux kernel client Bug #57686: general protection fault and CephFS kernel client hangs after MDS failover
Greg Farnum wrote:
> Xiubo, I know the patch addresses the immediate fault, but do we have any idea how or why the s...
Xiubo Li
02:54 AM Linux kernel client Bug #57686: general protection fault and CephFS kernel client hangs after MDS failover
Xiubo, I know the patch addresses the immediate fault, but do we have any idea how or why the snaprealm was corrupt i... Greg Farnum
03:02 AM Bug #58047 (New): ceph-osd: osd numa affinity setting doesn't work
After setting osd_numa_node parameter, the osd numa is not as expected.
* cpu numa node...
xu wang
12:40 AM RADOS Bug #58046 (Fix Under Review): qa/workunits/rados/test_librados_build.sh: specify redirect in cur...
Laura Flores
12:36 AM RADOS Bug #58046 (Pending Backport): qa/workunits/rados/test_librados_build.sh: specify redirect in cur...
The workunit currently grabs files with:... Laura Flores

11/17/2022

11:46 PM Orchestrator Bug #57287: orch/cephadm: 'mount -t nfs 10.0.31.109:/foouser /mnt/foo' failing in nfs-rgw-ingress...
Tracker #57163 was resolved by compiling with c++20. Perhaps that could also help in this case? Laura Flores
10:37 PM bluestore Feature #57785: fragmentation score in metrics
We have a meeting scheduled for next week to discuss this topic. Laura Flores
06:30 PM bluestore Feature #57785: fragmentation score in metrics
❤️ Kevin Fox
06:28 PM bluestore Feature #57785: fragmentation score in metrics
Thanks, Kevin. Let me talk this over with Adam and Paul, and we will decide a course of action. Laura Flores
06:15 PM bluestore Feature #57785: fragmentation score in metrics
A ceph warning for it would also be quite useful I think.
https://access.redhat.com/documentation/fr-fr/red_hat_ceph...
Kevin Fox
06:09 PM bluestore Feature #57785: fragmentation score in metrics
Thanks for sharing this, Kevin. We discussed this Tracker more in the Telemetry huddle, and we are curious if you wou... Laura Flores
05:11 PM bluestore Feature #57785: fragmentation score in metrics
We've had to hack a script together to monitor one of our clusters, and it has been useful to catch an issue:
https:...
Kevin Fox
04:25 PM bluestore Feature #57785: fragmentation score in metrics
@Kevin I have asked Paul Cuzner to take a look at this tracker and offer his opinion, as he has done a lot of work fo... Laura Flores
06:14 PM Orchestrator Bug #58045 (Resolved): orchestrator: incorrect size listed for devices in device ls
currently when printing out the size in `ceph orch device ls` the orchestrator is calling a function that continually... Adam King
05:07 PM RADOS Bug #52657: MOSDPGLog::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_NAUTILUS)'
We suspect that this assert failure is hit in cases when we try to encode a message before the connection is in a sta... Aishwarya Mathuria
03:30 PM RADOS Bug #56147: snapshots will not be deleted after upgrade from nautilus to pacific
> For already-converted clusters: Separate PR will be issued to remove/update the malformed SnapMapper keys.
https...
Matan Breizman
03:10 PM rgw Bug #58034 (In Progress): RGW misplaces index entries after dynamically resharding bucket
Casey Bodley
02:15 PM Dashboard Backport #58043 (In Progress): quincy: mgr/dashboard: missing data on hosts Grafana dashboard
Tatjana Dehler
01:50 PM Dashboard Backport #58043 (Resolved): quincy: mgr/dashboard: missing data on hosts Grafana dashboard
https://github.com/ceph/ceph/pull/48934 Backport Bot
02:12 PM Dashboard Backport #58042 (In Progress): pacific: mgr/dashboard: missing data on hosts Grafana dashboard
Tatjana Dehler
01:50 PM Dashboard Backport #58042 (Resolved): pacific: mgr/dashboard: missing data on hosts Grafana dashboard
https://github.com/ceph/ceph/pull/48933 Backport Bot
02:09 PM RADOS Bug #58044 (Need More Info): ceph-osd: osd numa affinity setting doesn't work
After setting osd_numa_node parameter, the osd numa is not as expected.

xu wang
01:42 PM Dashboard Bug #57987 (Pending Backport): mgr/dashboard: missing data on hosts Grafana dashboard
Nizamudeen A
01:20 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
Radoslaw Zarzynski wrote:
> Do we know the reason why switching g++11 helps? Is it a known compiler's bug?
See Br...
Matan Breizman
12:15 PM RADOS Bug #57940: ceph osd crashes with FAILED ceph_assert(clone_overlap.count(clone)) when nobackfill ...
Thomas Le Gentil wrote:
> the osd process does not crash if it is marked 'out'
Sorry, this is false. The OSD cras...
Thomas Le Gentil
12:05 PM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Hey,
Thanks for the update. You should try adjusting `mds_session_cache_liveness_decay_rate` to a lower value (def...
Venky Shankar
10:17 AM CephFS Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
and another side note, the crash was seen when a directory pin was removed from rank-0 mds. Pinning it back again cea... Venky Shankar
10:16 AM CephFS Bug #58041: mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == straydname)
oh, and btw this was seen in ceph-16.2.8. Venky Shankar
10:15 AM CephFS Bug #58041 (Duplicate): mds: src/mds/Server.cc: 3231: FAILED ceph_assert(straydn->get_name() == s...
... Venky Shankar
09:42 AM RADOS Backport #58040 (Resolved): quincy: osd: add created_at and ceph_version_when_created metadata
Backport Bot
09:42 AM RADOS Backport #58039 (Resolved): pacific: osd: add created_at and ceph_version_when_created metadata
Backport Bot
09:34 AM RADOS Feature #58038 (Pending Backport): osd: add created_at and ceph_version_when_created metadata
Igor Fedotov
07:24 AM RADOS Feature #58038: osd: add created_at and ceph_version_when_created metadata
PR#48298 has already been merged. Could you change the status of this issue to "Pending Backport"?
I'll create backp...
Kaoru Esashika
07:15 AM RADOS Feature #58038 (Resolved): osd: add created_at and ceph_version_when_created metadata
Add the following two OSD metadata.
- created_at: the timestamp when OSD was created. It's useful when getting som...
Kaoru Esashika
09:21 AM CephFS Feature #55215 (Fix Under Review): mds: fragment directory snapshots
Venky Shankar
08:30 AM rgw Bug #55498 (Duplicate): "AssertionError: rgw multisite test failures" in upgrade:octopus
will be fixed as part of: https://tracker.ceph.com/issues/58036 Yuval Lifshitz
08:25 AM rgw Backport #58036 (In Progress): pacific: pubsub test failures
Yuval Lifshitz
06:18 AM mgr Bug #58037: mgr crash
this crash happen after cmd "rbd perf image stats"... yite gu
05:10 AM mgr Bug #58037 (Need More Info): mgr crash
... yite gu

11/16/2022

07:30 PM rgw Backport #58036 (Resolved): pacific: pubsub test failures
https://github.com/ceph/ceph/pull/48928 Backport Bot
07:11 PM RADOS Bug #57977: osd:tick checking mon for new map
Thanks for the update! Yeah, it might stuck there. To confirm we would logs with increased debugs (maybe @debug_mon =... Radoslaw Zarzynski
07:09 PM rgw Bug #56572: pubsub test failures
this should also be backported to pacific, in order to fix the upgrade issues. see: https://tracker.ceph.com/issues/5... Yuval Lifshitz
07:06 PM RADOS Bug #51729: Upmap verification fails for multi-level crush rule
Thanks for formulating the hypothesis!
Just updating to keep this ticket in the front of the tracker.
Radoslaw Zarzynski
07:02 PM RADOS Bug #57546: rados/thrash-erasure-code: wait_for_recovery timeout due to "active+clean+remapped+la...
Yeah, worth looking the msgr encode issue has the priority. Radoslaw Zarzynski
07:00 PM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
Discussed during the RADOS Team Meeting on 15 Nov.
Linking the Nitzan's gist: https://gist.github.com/NitzanMordhai/...
Radoslaw Zarzynski
06:58 PM RADOS Bug #57989: test-erasure-eio.sh fails since pg is not in unfound
Definitely a low priority. Radoslaw Zarzynski
06:52 PM RADOS Bug #58027 (Closed): op slow from throttled to header_read
Hello! The most important thing is Octopus is EOL. Second, I'm also not sure whether this is really a bug. Seeing 0,5... Radoslaw Zarzynski
06:48 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
Do we know the reason why switching g++11 helps? Is it a known compiler's bug? Radoslaw Zarzynski
05:47 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
I was able to schedule a teuthology run: http://pulpito.front.sepia.ceph.com/lflores-2022-11-16_15:49:13-rados:single... Laura Flores
06:28 PM rgw Bug #57562: multisite replication issue on Quincy
Hey Adam,
So after all of our current testing and debugging of issue, the current race condition is that the same pa...
Krunal Chheda
04:43 PM rgw Bug #57562: multisite replication issue on Quincy
Adam Emerson wrote:
> Hold off for now, I've introduced one problem I need to debug.
Ack.
We are currently tes...
Oguzhan Ozmen
11:02 AM rgw Bug #57562: multisite replication issue on Quincy
Hold off for now, I've introduced one problem I need to debug. Adam Emerson
10:47 AM rgw Bug #57562: multisite replication issue on Quincy
I have pushed a new commit. It disables all use of the part tags. I believe this should eliminate any remaining diffi... Adam Emerson
06:08 PM rgw Bug #58020 (Fix Under Review): notifications: zero timestamp in complete multipart upload event
Yuval Lifshitz
06:06 PM rgw Backport #57561 (In Progress): quincy: pubsub test failures
Yuval Lifshitz
03:26 PM rgw Bug #58035 (Fix Under Review): Copying an object to itself crashes de RGW if executed as admin user.
Casey Bodley
10:44 AM rgw Bug #58035 (Pending Backport): Copying an object to itself crashes de RGW if executed as admin user.
This was observed after executing the test *s3tests_boto3.functional.test_s3:test_object_copy_to_itself* with a user ... Xavi Garcia
01:11 PM RADOS Bug #57940: ceph osd crashes with FAILED ceph_assert(clone_overlap.count(clone)) when nobackfill ...
the osd process does not crash if it is marked 'out' Thomas Le Gentil
12:24 PM Orchestrator Bug #58021 (Fix Under Review): Promteheus haproxy targets are down
Redouane Kachach Elhichou
10:09 AM Dashboard Bug #56970 (Resolved): mgr/dashboard: dashboard connects via ssl to an ip address instead of a dn...
Volker Theile
04:56 AM Documentation #57001: RBD Rewrite Project
As of 16 Nov 2022 AEST, the front matter of the RBD book reads as follows:
> A block is a sequence of bytes (often...
Cole Mitchell
04:53 AM Documentation #57001: RBD Rewrite Project
https://docs.ceph.com/en/quincy/rbd/ Zac Dover
04:50 AM Documentation #57108: add ".. prompt :: bash $" to /doc/rados
I slit the sheet, the sheet I slit, and on the slitted sheet I sit. Cole Mitchell
03:28 AM rgw Bug #57980 (Fix Under Review): rgw/cloud-transition: transition fails when using MCG Azure Namesp...
>>> From the http packets -> MCG is returning 403 for HEAD request and 400 for PUT request (failed). Maybe the header... Soumya Koduri

11/15/2022

09:56 PM rgw Bug #58034 (Resolved): RGW misplaces index entries after dynamically resharding bucket
When RGW reshards buckets with ~250k index entries*, I've noticed some s3:PutObject requests that return 200 end up w... J. Eric Ivancich
09:54 PM rgw Bug #57770: RGW (pacific) misplaces index entries after dynamically resharding bucket
The code on the PR seems to address the issue. My colleague Mark Kogan ran it through a test at scale and it behaved ... J. Eric Ivancich
07:34 PM rgw Bug #58033 (New): multipart copy part: use refcount optimization when possible
rgw only supports CopyObject[1] for object sizes up to rgw_max_put_size=5GB, and requires multipart with UploadPartCo... Casey Bodley
07:07 PM rgw Bug #50076 (Fix Under Review): route librdkafka log messages to rgw log
Yuval Lifshitz
05:45 PM rgw Bug #50076 (In Progress): route librdkafka log messages to rgw log
Yuval Lifshitz
06:27 PM Bug #58032 (New): ceph-base: main(): ceph-crash:109:main:TypeError: handler() takes 1 positional ...
https://bugzilla.redhat.com/show_bug.cgi?id=2141246
Description of problem:
crash happened in the background
V...
Kaleb KEITHLEY
01:49 PM CephFS Bug #58031 (Resolved): cephfs-top: sorting/limit excepts when the filesystems are removed and cre...
This happens in the main branch. Please check.
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms...
Jos Collin
01:42 PM CephFS Bug #58000 (Fix Under Review): mds: switch submit_mutex to fair mutex for MDLog
Venky Shankar
01:41 PM CephFS Bug #58008 (Fix Under Review): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate...
Venky Shankar
01:41 PM CephFS Bug #58028 (Triaged): cephfs-top: Sorting doesn't work when the filesystems are removed and created
Venky Shankar
10:12 AM CephFS Bug #58028 (Resolved): cephfs-top: Sorting doesn't work when the filesystems are removed and created
Sorting doesn't work in the following scenario
1. cephfs-top is launched and the clients are sorted by 'mlatavg(ms...
Jos Collin
11:08 AM CephFS Bug #58030 (Resolved): mds: avoid ~mdsdir's scrubbing and reporting damage health status
We are supposed to handle the case of mdsdir, where we
are not having any backtrace actually.We should prevent the
...
Neeraj Pratap Singh
10:49 AM CephFS Bug #58029 (Fix Under Review): cephfs-data-scan: multiple data pools are not supported
Mykola Golub
10:46 AM CephFS Bug #58029 (Resolved): cephfs-data-scan: multiple data pools are not supported
The tool cannot properly recover if a fs has extra data pools. We need access to all data pools on `scan_extents` ste... Mykola Golub
09:56 AM bluestore Bug #48827: Ceph Bluestore OSDs fail to start on WAL corruption
In the customer case running luminous when OSD process was run twice (let's skip how), the assert
'file->fnode.ino' ...
Adam Kupczyk
08:44 AM RADOS Bug #56772: crash: uint64_t SnapSet::get_clone_bytes(snapid_t) const: assert(clone_overlap.count(...
This bug is present in v17.2.5 Thomas Le Gentil
07:32 AM RADOS Bug #58027 (Closed): op slow from throttled to header_read
ceph version 15.2.7
Op spend 500ms from throttled to header_read...
yite gu
06:47 AM Dashboard Feature #58016 (Fix Under Review): mgr/dashboard: expose grafana port and initial_admin_password ...
Nizamudeen A
04:40 AM Dashboard Backport #58026 (Resolved): quincy: mgr/dashboard: update to angular 13
https://github.com/ceph/ceph/pull/50124 Backport Bot
04:40 AM Dashboard Backport #58025 (Resolved): quincy: mgr/dashboard: migrate bootstrap 4 to 5
https://github.com/ceph/ceph/pull/50124 Backport Bot
04:40 AM Dashboard Backport #58024 (Resolved): quincy: mgr/dashboard: update jest to 28
https://github.com/ceph/ceph/pull/50124 Backport Bot
04:28 AM Dashboard Bug #48258 (Pending Backport): mgr/dashboard: Switch from tslint to eslint
Nizamudeen A
04:27 AM Dashboard Cleanup #57972 (Pending Backport): mgr/dashboard: update jest to 28
Nizamudeen A
04:27 AM Dashboard Cleanup #57867 (Pending Backport): mgr/dashboard: migrate bootstrap 4 to 5
Nizamudeen A
04:26 AM Dashboard Cleanup #57866 (Pending Backport): mgr/dashboard: update to angular 13
Nizamudeen A
12:24 AM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
There is also a coredump located at `/a/matan-2022-09-08_11:12:20-rados:singleton-main-distro-default-smithi/7020422/... Laura Flores
12:01 AM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
Some relevant frames:... Laura Flores

11/14/2022

11:39 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
I followed Brad's ubuntu 20.04 coredump tutorial: https://source.redhat.com/personal_blogs/debugging_a_ceph_osd_cored... Laura Flores
08:20 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
The original build is by now expired, so I'm rebuilding it here: https://shaman.ceph.com/builds/ceph/wip-kefu-testing... Laura Flores
08:14 PM RADOS Bug #57632: test_envlibrados_for_rocksdb: free(): invalid pointer
Ran the test locally in an ubuntu 20.04 environment, and the test ran fine.
There is a coredump located under /a/k...
Laura Flores
10:51 PM Bug #58019: Public Access URL returns "NoSuchBucket" when rgw_swift_account_in_url is True
Found this https://github.com/ceph/ceph/pull/47341
Would love to know when we'll be getting a 16.2.11 release with...
Josh Beaman
03:50 PM Bug #58019 (New): Public Access URL returns "NoSuchBucket" when rgw_swift_account_in_url is True
The public access URL:
https://spvip-ch2-h01.ece.comcast.net/swift/v1/AUTH_{UUID_REDACTED}/{bucket-name}
Returns "N...
Josh Beaman
09:32 PM CephFS Fix #58023 (Pending Backport): mds: do not evict clients if OSDs are laggy
Monitoring perf dumps from the MDS can sometimes show that OSDs are laggy, "objecter.op_laggy" and "objecter.osd_lagg... Patrick Donnelly
05:06 PM bluestore Bug #58022 (Pending Backport): Fragmentation score rising by seemingly stuck thread
Due to issue https://tracker.ceph.com/issues/57672 we've been monitoring our clusters closely ensure it doesn't run i... Kevin Fox
04:43 PM Orchestrator Bug #58021 (Resolved): Promteheus haproxy targets are down
Using the latest code from main (v18.0) prometheus is not able to scrap the metrics of haproxy. This happens because ... Redouane Kachach Elhichou
04:18 PM Orchestrator Bug #57998: cephadm stuck trying to download "mon"
I executed the following:... Shawn Iverson
04:16 PM Orchestrator Bug #57998: cephadm stuck trying to download "mon"
Here's the dump, there is definitely something fishy here, how do I remove it?... Shawn Iverson
11:04 AM Orchestrator Bug #57998 (Need More Info): cephadm stuck trying to download "mon"
Redouane Kachach Elhichou
03:57 PM rgw Bug #58020 (Pending Backport): notifications: zero timestamp in complete multipart upload event
this is a regression due to: https://github.com/ceph/ceph/pull/42266
(original fix was: https://github.com/ceph/ceph...
Yuval Lifshitz
03:51 PM Orchestrator Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initialize
I just ran into this issue and would like to propose reopening this.
radosgw should clearly log something when sta...
Christian Rohmann
03:13 PM rgw Bug #58014 (Fix Under Review): notifications: metadata does not work for COPY events
Yuval Lifshitz
01:42 PM rgw Bug #57562: multisite replication issue on Quincy
Adam, I submitted a PR on top of yours. My 8h test has passed with this PR. The same test usually failed on earlier v... Jane Zhu
01:37 PM rgw Bug #57562: multisite replication issue on Quincy
Two more racing conditions found. These two are all on the journal entries.
h3. *Race condition 1:*...
Jane Zhu
01:27 PM CephFS Bug #58018 (Fix Under Review): mount.ceph: will fail with old kernels
Xiubo Li
10:09 AM CephFS Bug #58018 (Pending Backport): mount.ceph: will fail with old kernels
... Xiubo Li
12:10 PM bluestore Bug #53466 (Fix Under Review): OSD is unable to allocate free space for BlueFS
Igor Fedotov
12:09 PM Orchestrator Bug #57910: ingress: HAProxy fails to start because keepalived IP address not yet available on ne...
As of the current design of the ingress service, the keepalived starts before haproxy because the daemon depends on t... Redouane Kachach Elhichou
11:37 AM RADOS Bug #55750: mon: slow request of very long time
{
"description": "osd_failure(failed timeout osd.6 [v2:10.172.98.151:6800/39,v1:10.172.98.151:68...
yite gu
11:08 AM Orchestrator Bug #57897: ceph mgr restart causes restart of all iscsi daemons in a loop
I tried to reproduce the issue with the same setup but wasn't successful so far. I started from a cluster running v17... Redouane Kachach Elhichou
10:09 AM Dashboard Bug #58017 (Pending Backport): mgr/dashboard: Error "Evaluating rule failed" for CephPoolGrowthWa...
Prometheus reports an error - many-to-many matching not allowed: matching labels must be unique on one side for CephP... Aashish Sharma
05:20 AM Linux kernel client Bug #57898 (In Progress): ceph client extremely slow kernel version between 5.15 and 6.0
Xiubo Li
05:20 AM Linux kernel client Bug #57898: ceph client extremely slow kernel version between 5.15 and 6.0
I saw the same issue when testing the `6.1.0-rc1` upstream code:... Xiubo Li
02:30 AM Dashboard Feature #58016 (Resolved): mgr/dashboard: expose grafana port and initial_admin_password in servi...
as title says Nizamudeen A

11/13/2022

09:37 AM rgw Bug #58014 (Pending Backport): notifications: metadata does not work for COPY events
this is a regression due to: https://github.com/ceph/ceph/pull/39192/commits/35a4eb4410394a0014648dda7df92642f3b536d3... Yuval Lifshitz

11/12/2022

04:18 PM Bug #58013 (New): Osdmap too big lead osd crash
My cluster has failed, a large number of osds cannot be started,
and the troubleshooting found that the size of osd...
伟杰 谭
03:40 PM Bug #58012: OpTracker event duration calculation errror
with https://github.com/ceph/ceph/pull/48860 applied:
v2:...
Honggang Yang
03:05 PM Bug #58012 (Duplicate): OpTracker event duration calculation errror
h1. ceph version... Honggang Yang

11/11/2022

08:31 PM RADOS Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer
Also to note: We set `ceph config set mgr mgr_stats_period 1` on the gibba cluster to reproduce this bug. (This occur... Laura Flores
07:00 PM devops Bug #56411 (Closed): Workaround for ceph-mgr breaks Cython builds
Adam Emerson
06:27 PM RADOS Bug #49689: osd/PeeringState.cc: ceph_abort_msg("past_interval start interval mismatch") start
I think https://tracker.ceph.com/issues/49689#note-31 makes sense and the following logs also show what max_oldest_ma... Neha Ojha
05:50 PM Feature #58010 (New): Add the OpenSSF Scorecard Action
Hey, I'm Pedro and I'm working for Google and the "OpenSSF":https://openssf.org/ to improve the supply-chain security... Pedro Nacht
03:09 PM Dashboard Bug #57987 (In Progress): mgr/dashboard: missing data on hosts Grafana dashboard
Tatjana Dehler
02:11 PM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> xianpao chen wrote:
> > Venky Shankar wrote:
> > > Could you share the output of
> > >
>...
xianpao chen
01:02 PM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
xianpao chen wrote:
> Venky Shankar wrote:
> > Could you share the output of
> >
> > [...]
> >
> > Also, does...
Venky Shankar
10:52 AM Dashboard Tasks #58009 (Resolved): mgr/dashboard: style cards on the page
h3. Description
Currently on the dashboard revamp we are placing the cards within a Bootstrap grid of two rows w...
Pedro González Gómez
10:08 AM RADOS Backport #58007: pacific: bail from handle_command() if _generate_command_map() fails
please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/48846
ceph-backport.sh versi...
nikhil kshirsagar
09:07 AM RADOS Backport #58007 (Resolved): pacific: bail from handle_command() if _generate_command_map() fails
https://github.com/ceph/ceph/pull/48846 Backport Bot
10:03 AM RADOS Backport #58006: quincy: bail from handle_command() if _generate_command_map() fails
please link this Backport tracker issue with GitHub PR https://github.com/ceph/ceph/pull/48845
ceph-backport.sh versi...
nikhil kshirsagar
09:07 AM RADOS Backport #58006 (Resolved): quincy: bail from handle_command() if _generate_command_map() fails
https://github.com/ceph/ceph/pull/48845 Backport Bot
09:14 AM CephFS Bug #58008: mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
When increasing filer_max_purge_ops on a pacific version mds, pq_executing_ops/pq_executing_ops_high_water of purge_q... yixing hao
09:13 AM CephFS Bug #58008 (Resolved): mds/PurgeQueue: don't consider filer_max_purge_ops when _calculate_ops
_calculate_ops relying on a config which can be modified on the fly will cause a bug. e.g.
# A file has 20 objects...
yixing hao
09:01 AM RADOS Bug #57859 (Pending Backport): bail from handle_command() if _generate_command_map() fails
PR https://github.com/ceph/ceph/pull/48044 has been merged in main. Ponnuvel P
06:46 AM Bug #57973: rook:rook module failed to connect k8s api server because of self-signed cert with se...
It seems due to bad k8s cert trust chain. ceph is fine. This bug could be closed. Ben Gao
01:01 AM rgw Bug #57562: multisite replication issue on Quincy
It should, thank you. I don't think it's the underlying cause, but it's a good catch. Adam Emerson

11/10/2022

11:57 PM crimson Bug #58005 (Resolved): release-built osd failed to mkfs
It seems that when "seastar::need_preempt" is true, "crimson::do_for_each" will turn into a long recursive function. ... Xuehan Xu
11:37 PM RADOS Bug #56101 (Fix Under Review): Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function s...
Laura Flores
11:21 PM RADOS Bug #56101 (In Progress): Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_t...
Laura Flores
04:52 AM RADOS Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer
Thanks for your work in capturing the core Laura.
I had a look at the coredump and it shows exactly what we had sp...
Brad Hubbard
09:06 PM rgw Bug #57562: multisite replication issue on Quincy
A potential bug?
https://github.com/ceph/ceph/blob/main/src/cls/fifo/cls_fifo_types.h#L66
Should it be the follow...
Jane Zhu
07:14 PM RADOS Bug #52657: MOSDPGLog::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_NAUTILUS)'
/a/yuriw-2022-10-17_17:31:25-rados-wip-yuri7-testing-2022-10-17-0814-distro-default-smithi/7071031 Laura Flores
07:09 PM Orchestrator Bug #57311: rook: ensure CRDs are installed first
/a/yuriw-2022-10-17_17:31:25-rados-wip-yuri7-testing-2022-10-17-0814-distro-default-smithi/7070926 Laura Flores
05:11 PM Orchestrator Backport #58004 (In Progress): quincy: rook/k8s: nfs cluster creation ends up with no daemons dep...
Juan Miguel Olmo Martínez
05:00 PM Orchestrator Backport #58004 (Resolved): quincy: rook/k8s: nfs cluster creation ends up with no daemons deploy...
https://github.com/ceph/ceph/pull/48830 Backport Bot
04:54 PM Orchestrator Bug #57954 (Pending Backport): rook/k8s: nfs cluster creation ends up with no daemons deployment
Juan Miguel Olmo Martínez
03:44 PM Fix #58003 (Pending Backport): mon: add exception handling to ceph health mute
Running ceph health mute with an invalid TTL causes the mon to crash, because the exception thrown by parse_timespan(... Daniel R
03:25 PM rgw Bug #57706 (Need More Info): When creating a new user, if the 'uid' is not provided, error report...
Hi Kevin Wang,
Could I get what version of Ceph this issue occurred on? The issue does seem to be resolved in the ...
Ali Maredia
03:07 PM rgw Bug #57724 (Fix Under Review): Keys returned by Admin API during user creation on secondary zone ...
Casey Bodley
01:47 PM Orchestrator Bug #58001 (Fix Under Review): haproxy targets are not updated correctly in prometheus.yaml file
Redouane Kachach Elhichou
09:57 AM Orchestrator Bug #58001 (Resolved): haproxy targets are not updated correctly in prometheus.yaml file
steps to reproduce the issue:
1) Bootstrap a new cluster (with monitoring enabled)
2) Wait until Prometheus is up...
Redouane Kachach Elhichou
01:14 PM Bug #58002 (New): mon_max_pg_per_osd is not checked per OSD
The warning for exceeding mon_max_pg_per_osd seems to be triggered only when the average PG count over all OSDs excee... Frank Schilder
11:50 AM RADOS Bug #57989: test-erasure-eio.sh fails since pg is not in unfound
For some reason, the pool already exist... Nitzan Mordechai
08:44 AM RADOS Bug #57757 (In Progress): ECUtil: terminate called after throwing an instance of 'ceph::buffer::v...
Nitzan Mordechai
08:42 AM RADOS Bug #57618 (Fix Under Review): rados/test.sh hang and pkilled (LibRadosWatchNotifyEC.WatchNotify)
Nitzan Mordechai
08:34 AM RADOS Bug #57618: rados/test.sh hang and pkilled (LibRadosWatchNotifyEC.WatchNotify)
Some of the OSDs stopped due to valgrind errors. This is duplicate of other bug Nitzan Mordechai
08:39 AM RADOS Bug #57751 (Fix Under Review): LibRadosAio.SimpleWritePP hang and pkill
Nitzan Mordechai
08:18 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> BTW, are you *not* seeing any "oversized cache" warning for the MDS?
there is no "oversize...
xianpao chen
04:06 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
BTW, are you *not* seeing any "oversized cache" warning for the MDS? Venky Shankar
02:42 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Do you have lots of small files and frequently scan them? Venky Shankar
01:12 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Have you tried running `heap release`?
yes,but it didn't seem to work.
xianpao chen
07:38 AM RADOS Bug #57546: rados/thrash-erasure-code: wait_for_recovery timeout due to "active+clean+remapped+la...
Thanks for taking a look Radek! That's a good point since we are seeing this issue with rados/thrash-erasure-code tes... Aishwarya Mathuria
01:45 AM CephFS Bug #58000: mds: switch submit_mutex to fair mutex for MDLog
From Patrick's comment in https://github.com/ceph/ceph/pull/44180#pullrequestreview-1174516711. Xiubo Li
01:44 AM CephFS Bug #58000 (Resolved): mds: switch submit_mutex to fair mutex for MDLog
The implementations of the Mutex (e.g. std::mutex in C++) do not
guarantee fairness, they do not guarantee that the ...
Xiubo Li

11/09/2022

10:56 PM RADOS Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer
Managed to reproduce this on the Gibba cluster and produce a coredump!
The core file is located on gibba001 under ...
Laura Flores
10:11 PM rgw Bug #57706: When creating a new user, if the 'uid' is not provided, error reported as 'Permission...
On a branch close to the master branch from a vstart cluster when I try this same scenario I see:
[ali@acadia buil...
Ali Maredia
09:51 PM rgw Bug #57562: multisite replication issue on Quincy
We also found a place that might potentially cause issues.
Rgw locks the mutex and gets some data from "info" befo...
Jane Zhu
09:22 PM rgw Bug #57562: multisite replication issue on Quincy
Here is some more detailed explanation on how the -EINVAL(-22) error (hence datalog writing failure) happens based on... Jane Zhu
08:18 PM RADOS Backport #57704 (Resolved): quincy: mon/MonMap.h: FAILED ceph_assert(m < ranks.size()) when reduc...
https://github.com/ceph/ceph/pull/48321 Kamoltat (Junior) Sirivadhna
08:17 PM RADOS Backport #57705 (Resolved): pacific: mon/MonMap.h: FAILED ceph_assert(m < ranks.size()) when redu...
https://github.com/ceph/ceph/pull/48320 Kamoltat (Junior) Sirivadhna
08:17 PM RADOS Bug #50089 (Resolved): mon/MonMap.h: FAILED ceph_assert(m < ranks.size()) when reducing number of...
Kamoltat (Junior) Sirivadhna
07:08 PM CephFS Feature #57090 (Fix Under Review): MDSMonitor,mds: add MDSMap flag to prevent clients from connec...
Dhairya Parmar
06:23 PM Orchestrator Bug #57999 (Resolved): cephadm: cephadm always reports new or changed devices even if devices are...
This appears to be an issue with the "created" field changing, which should not affect equality in this case... Adam King
06:19 PM Orchestrator Bug #57998: cephadm stuck trying to download "mon"
hmm, can I see what "ceph config dump" spits out (feel free to remove anything sensitive if if necessary)? All the im... Adam King
10:51 AM Orchestrator Bug #57998 (Resolved): cephadm stuck trying to download "mon"
Entire cluster cephadm management is stuck and repeatedly tries to download an unqualified "mon" instead of the ceph ... Shawn Iverson
04:34 PM RADOS Bug #51729: Upmap verification fails for multi-level crush rule
Thanks again for looking at this.
I haven't looked further, but I suspect the issue will come down to the variable...
Chris Durham
01:22 PM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Have you tried running `heap release`? Venky Shankar
09:35 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu...
xianpao chen
09:23 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Venky Shankar wrote:
> Could you share the output of
>
> [...]
>
> Also, does running
>
> [...]
>
> redu...
xianpao chen
08:56 AM CephFS Support #57952: Pacific: the buffer_anon_bytes of ceph-mds is too large
Could you share the output of... Venky Shankar
08:32 AM crimson Bug #57990: Crimson OSD crashes when trying to bring it up
Crimson is not production ready yet, and there will be no backport to Quincy.
It is expected that there were bugs ...
Yingxin Cheng
07:46 AM Dashboard Backport #57983 (In Progress): quincy: mgr/dashboard: error message displaying when editing journ...
Pedro González Gómez
07:43 AM Dashboard Backport #57982 (In Progress): pacific: mgr/dashboard: error message displaying when editing jour...
Pedro González Gómez
03:01 AM Backport #57997 (In Progress): quincy: ceph-crash service should run as unprivileged user, not ro...
Tim Serong
02:18 AM Backport #57997 (Resolved): quincy: ceph-crash service should run as unprivileged user, not root ...
https://github.com/ceph/ceph/pull/48805 Backport Bot
02:58 AM Backport #57996 (In Progress): pacific: ceph-crash service should run as unprivileged user, not r...
Tim Serong
02:18 AM Backport #57996 (Resolved): pacific: ceph-crash service should run as unprivileged user, not root...
https://github.com/ceph/ceph/pull/48804 Backport Bot
01:54 AM Bug #57967 (Pending Backport): ceph-crash service should run as unprivileged user, not root (CVE-...
Tim Serong

11/08/2022

09:23 PM RADOS Bug #57017: mon-stretched_cluster: degraded stretched mode lead to Monitor crash
pacific backport: https://github.com/ceph/ceph/pull/48803 Kamoltat (Junior) Sirivadhna
08:59 PM RADOS Bug #57017: mon-stretched_cluster: degraded stretched mode lead to Monitor crash
quincy backport: https://github.com/ceph/ceph/pull/48802 Kamoltat (Junior) Sirivadhna
09:20 PM bluestore Feature #57785: fragmentation score in metrics
@Vikhyat, no worries. Based on Kevin's comment, I think this metric might be better suited for Prometheus than Teleme... Laura Flores
06:37 PM bluestore Feature #57785: fragmentation score in metrics
Laura - sorry I missed the update. Can you please ping Adam and Igor? Vikhyat Umrao
08:59 PM rbd Bug #57941: Severe performance drop after writing 100 GB of data to RBD volume, dependent on RAM ...
Thanks for looking into this Christopher. You are right, this is a 100% sequential workload, just filling a volume wi... Guillaume Pothier
08:23 PM rbd Bug #57941 (In Progress): Severe performance drop after writing 100 GB of data to RBD volume, dep...
Christopher Hoffman
08:23 PM rbd Bug #57941: Severe performance drop after writing 100 GB of data to RBD volume, dependent on RAM ...
I'm not familiar with PVE and how it sets up Ceph. I took a look at your testcase and it appears to be a sequential w... Christopher Hoffman
07:37 PM bluestore Fix #54299 (Need More Info): osd error restart
Igor Fedotov
07:34 PM bluestore Bug #57672 (Duplicate): SSD OSD won't start after high framentation score!
Igor Fedotov
07:27 PM bluestore Bug #53466 (In Progress): OSD is unable to allocate free space for BlueFS
Igor Fedotov
07:23 PM RADOS Bug #51729: Upmap verification fails for multi-level crush rule
I believe I've reproduced the issue using the osdmaps that Chris provided.
First, I used the osdmaptool to run the...
Laura Flores
02:49 PM rgw Bug #57911 (Fix Under Review): Segmentation fault when uploading file with bucket policy on Quincy
Daniel Gryniewicz
02:08 PM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
after rechecking the logs it looks like we are taking 2 different versions of smithi01231941-9:head
All chunks with ...
Nitzan Mordechai
05:44 AM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
@Laura, thanks for confirm that in the coredump, yes, shard0 also showing that when it get the chunk from bluestore:
...
Nitzan Mordechai
12:07 AM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
Brad and I did some more debugging today.
Here is the end of the log associated with the coredump:...
Laura Flores
12:03 PM Orchestrator Bug #57897: ceph mgr restart causes restart of all iscsi daemons in a loop
Adam King wrote:
> this is a painful one. @David at least until we have a fix for this, I will mention that setting ...
David Heap
09:42 AM Bug #57956: Ceph monitors in crash loop
liu jun wrote:
> Creating a pool causes mon to restart
>
> This is the detailed question:https://github.com/rook/...
liu jun
09:32 AM Dashboard Backport #57995 (New): quincy: mgr/dashboard: paginate services
Backport Bot
09:31 AM Dashboard Backport #57994 (Rejected): pacific: mgr/dashboard: paginate services
Backport Bot
09:21 AM Dashboard Feature #56512 (Pending Backport): mgr/dashboard: paginate services
Pere Díaz Bou
08:53 AM Support #57992: Stuck in linking when I comiple the CEPH
And when I ctrl+c to stop it and restart it with ninja, it will stuck in "dashboard nodeenv is being installed". Wenyu Huang
02:14 AM Support #57992 (New): Stuck in linking when I comiple the CEPH
I follow the README to compile the CEPH on Github(https://github.com/ceph/ceph/tree/v17.2.5#readme). When I ninja the... Wenyu Huang
08:39 AM Orchestrator Feature #51971 (New): cephadm/ingress: update keepalived container image
Reopning to check/address some of the concerns about the current keepalived image Redouane Kachach Elhichou
05:00 AM Dashboard Backport #57993 (New): quincy: mgr/dashboard: Improve level AA color contrast accessibility for d...
Backport Bot
04:51 AM Dashboard Bug #56023 (Pending Backport): mgr/dashboard: Improve level AA color contrast accessibility for d...
Nizamudeen A
01:20 AM Orchestrator Documentation #57991 (New): Migration documentation about osd service
The documentation doesn't mention how to make the osds in the cluster managed, nor how to add new osds. A new cephadm... Kevin Fox
12:24 AM crimson Bug #57990 (New): Crimson OSD crashes when trying to bring it up
Hello,
Using the `crimson-osd` Ubuntu package for Quincy, we're seeing somewhat recurrent crashes when trying to b...
Luciano Lo Giudice

11/07/2022

09:45 PM rgw Bug #57562: multisite replication issue on Quincy
> I think if the create_part is made exclusive, one of them would fail at part creation and let the other complete pa... Oguzhan Ozmen
09:27 PM RADOS Bug #57977: osd:tick checking mon for new map
Radoslaw Zarzynski wrote:
> Octopus is EOL. Does it happen on a supported release?
>
> Regardless of that, could ...
yite gu
06:13 PM RADOS Bug #57977 (Need More Info): osd:tick checking mon for new map
Octopus is EOL. Does it happen on a supported release?
Regardless of that, could you please provide logs from this...
Radoslaw Zarzynski
07:30 PM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
Also to note, we can see information about argument `to_read` here:... Laura Flores
07:27 PM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
@Nitzan, what do you think about this analysis? Or are there any other frames/locals you'd like me to check? Laura Flores
07:12 PM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
Looking at frame 12, I can see that the incorrect length (262144) for shard 0 is evident in the local variable "from"... Laura Flores
06:02 PM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
Got it to detect the right symbols with the new build!
I will attempt to analyze this coredump at a deeper level, ...
Laura Flores
03:16 PM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
According to Brad, the build needs to be as close to the test branch that originally experienced the crash as possibl... Laura Flores
07:18 PM RADOS Bug #51729: Upmap verification fails for multi-level crush rule
Thanks Chris! @Radek I have been taking some time to analyze this scenario, and will post updates soon. Laura Flores
06:36 PM RADOS Bug #51729: Upmap verification fails for multi-level crush rule
Thanks for the info! Laura, would you mind retaking a look? Radoslaw Zarzynski
06:36 PM RADOS Bug #51729 (New): Upmap verification fails for multi-level crush rule
Radoslaw Zarzynski
06:43 PM RADOS Bug #50219 (Closed): qa/standalone/erasure-code/test-erasure-eio.sh fails since pg is not in reco...
The original issue was caused by a commit in a wip branch being tested, so it's highly unprobable it's a reoccurence.... Radoslaw Zarzynski
06:42 PM RADOS Bug #57989 (New): test-erasure-eio.sh fails since pg is not in unfound
/a/lflores-2022-10-17_18:19:55-rados:standalone-main-distro-default-smithi/7071287... Radoslaw Zarzynski
06:35 PM RADOS Bug #57845: MOSDRepOp::encode_payload(uint64_t): Assertion `HAVE_FEATURE(features, SERVER_OCTOPUS...
Likely it's even a duplicate of https://tracker.ceph.com/issues/52657. Radoslaw Zarzynski
06:28 PM RADOS Bug #52136 (Fix Under Review): Valgrind reports memory "Leak_DefinitelyLost" errors.
Neha Ojha
06:26 PM RADOS Bug #57940 (Duplicate): ceph osd crashes with FAILED ceph_assert(clone_overlap.count(clone)) when...
Looks like a duplicate of 56772. Radoslaw Zarzynski
06:24 PM RADOS Bug #55141: thrashers/fastread: assertion failure: rollback_info_trimmed_to == head
Nitzan Mordechai wrote:
> Radoslaw Zarzynski wrote:
> > Well, just found a new occurance.
> Where can i find it?
...
Radoslaw Zarzynski
06:12 PM RADOS Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer
Brad and I ran a reproducer on the gibba cluster (restarting OSDs with `for osd in $(systemctl -l |grep osd|gawk '{pr... Laura Flores
06:01 PM RADOS Bug #56101: Gibba Cluster: 17.2.0 to 17.2.1 RC upgrade OSD crash in function safe_timer
Is there any news on that? Radoslaw Zarzynski
05:59 PM RADOS Bug #49689: osd/PeeringState.cc: ceph_abort_msg("past_interval start interval mismatch") start
Updated the PR link. Radoslaw Zarzynski
05:37 PM Dashboard Cleanup #57984 (In Progress): mgr/dashboard: Add tooltip
Pedro González Gómez
07:27 AM Dashboard Cleanup #57984 (Resolved): mgr/dashboard: Add tooltip
h3. Description of problem
Add tooltip for '# Local' and '# Remote' columns in rbd mirroring page.
Pedro González Gómez
03:40 PM rbd Feature #57988 (In Progress): [rbd-mirror] checksumming for snapshot-based mirroring
This is similar to an older feature request for journal-based mirroring but for snapshot-based mirroring the hope is ... Ilya Dryomov
01:48 PM CephFS Bug #57985 (Triaged): mds: warning `clients failing to advance oldest client/flush tid` seen with...
Venky Shankar
09:06 AM CephFS Bug #57985 (Pending Backport): mds: warning `clients failing to advance oldest client/flush tid` ...
https://bugzilla.redhat.com/show_bug.cgi?id=2134709
Generally seen when the MDS is heavily loaded with I/Os. Inter...
Venky Shankar
01:16 PM Dashboard Bug #57987 (Resolved): mgr/dashboard: missing data on hosts Grafana dashboard
h3. Description of problem
A lot of data is missing on the hosts Grafana dashboard (host-detail) and an error mess...
Tatjana Dehler
09:51 AM Linux kernel client Bug #57986: ceph: ceph_fl_release_lock cause "unable to handle kernel paging request at fffffffff...
There should be a race in 'filp_close()`, for example in a single process a file is opened twice with two different f... Xiubo Li
09:48 AM Linux kernel client Bug #57986 (Resolved): ceph: ceph_fl_release_lock cause "unable to handle kernel paging request a...
... Xiubo Li
09:37 AM Linux kernel client Bug #57686 (Fix Under Review): general protection fault and CephFS kernel client hangs after MDS ...
The patchwork: https://patchwork.kernel.org/project/ceph-devel/patch/20221107071759.32000-1-xiubli@redhat.com/
<pr...
Xiubo Li
06:54 AM Linux kernel client Bug #57686 (In Progress): general protection fault and CephFS kernel client hangs after MDS failover
Xiubo Li
09:23 AM Bug #57976: ceph-volume lvm activate removes /var/lib/ceph/osd/ceph-XXX folder and then chokes on...
Looks like the problem is gone after a full reboot. No idea what was going on, but it was reproducible on all nodes. Janek Bevendorff
07:22 AM Linux kernel client Bug #57898: ceph client extremely slow kernel version between 5.15 and 6.0
Minjong Kim wrote:
> https://gist.github.com/caffeinism/dbfd974374d620911a6c0c3dd1daadfb
>
> I am not good at wri...
Xiubo Li
06:54 AM Linux kernel client Bug #57817 (Duplicate): general protection fault and CephFS kernel client hangs after MDS failover
This is exactly the same issue with tracker#57686. Xiubo Li
06:37 AM Dashboard Backport #57983 (Resolved): quincy: mgr/dashboard: error message displaying when editing journam ...
https://github.com/ceph/ceph/pull/48807 Backport Bot
06:37 AM Dashboard Backport #57982 (Resolved): pacific: mgr/dashboard: error message displaying when editing journam...
https://github.com/ceph/ceph/pull/48806 Backport Bot
06:25 AM Dashboard Bug #57922 (Pending Backport): mgr/dashboard: error message displaying when editing journam mirro...
Nizamudeen A
06:23 AM Backport #57981 (New): quincy: ceph-mixin: Add Prometheus Alert for Degraded Bond
Backport Bot
06:23 AM rgw Bug #57980: rgw/cloud-transition: transition fails when using MCG Azure Namespacestore with a pre...
Few observations:
- 2022-11-03T08:42:29.718+0000 7fa1bf7e6640 0 lifecycle: ERROR: failed to check object on the ...
Soumya Koduri
06:21 AM rgw Bug #57980 (Pending Backport): rgw/cloud-transition: transition fails when using MCG Azure Namesp...
Reported by - dparkes@redhat.com
>>>>
Found Errors during cloud transition when using MCG Azure Namespacestore wit...
Soumya Koduri
06:18 AM Feature #57962 (Pending Backport): ceph-mixin: Add Prometheus Alert for Degraded Bond
Nizamudeen A
06:07 AM rgw Bug #57979 (Pending Backport): rgw/cloud-tranistion: Issues with MCG cloud endpoint
Below issues were observed while testing cloud-transition feature using MCG (Noobaa) endpoint
1) Creation of targe...
Soumya Koduri
04:35 AM Bug #57966: Ceph cluster osds failed when ms_cluster_type=async+rdma is used
the same problem on ceph 17.2.5:
root@ceph01:~# ceph crash info 2022-11-07T03:29:36.731174Z_bb6f8fea-ea87-4f83-a28a-...
guoguo jie
01:08 AM RADOS Bug #57937: pg autoscaler of rgw pools doesn't work after creating otp pool
Is there any updates? Please let me know if I can do something. Satoru Takeuchi

11/06/2022

02:27 PM Dashboard Feature #57978 (Fix Under Review): mgr/dashboard: allow to get/update RBD image metadata via REST...
h3. Description of problem
Currently we are missing an ability to get/update RBD image metadata via REST API. We c...
Mykola Golub
05:47 AM RADOS Bug #57757: ECUtil: terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of...
@brad, maybe it's a good candidate for another blog for upstream core dump analysis that you talked about (ubuntu 20.04) Nitzan Mordechai
 

Also available in: Atom