Ceph : Issues
https://tracker.ceph.com/
https://tracker.ceph.com/favicon.ico
2024-03-29T10:45:28Z
Ceph
Redmine
Ceph - Bug #65226 (Fix Under Review): qa: add a test - peer status show "failed" status for makin...
https://tracker.ceph.com/issues/65226
2024-03-29T10:45:28Z
Jos Collin
CephFS - Bug #65225 (New): ceph_assert on dn->get_projected_linkage()->is_remote
https://tracker.ceph.com/issues/65225
2024-03-29T10:16:03Z
Abhishek Lekshmanan
abhishek.lekshmanan@gmail.com
<p>In a workload that is heavily hardlinking and moving files, we see ceph-mds assert like the following</p>
<pre>
16.2.9/src/mds/Locker.cc: In function 'bool Locker::acquire_locks(MDRequestRef&, Mutati> /var/log/ceph/ceph-mds.minilevinson-983db664a3.log-20240329.gz
ceph-mds[2693077]: /builddir/build/BUILD/ceph-16.2.9/src/mds/Locker.cc: 404: FAILED ceph_assert(dn->get_projected_linkage()->is_remot>
ceph-mds[2693077]: ceph version 16.2.9-2 (4c3647a322c0ff5a1dd2344e039859dcbd28c830) pacific (stable)
ceph-mds[2693077]: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f40e912ed98]
ceph-mds[2693077]: 2: /usr/lib64/ceph/libceph-common.so.2(+0x276fb2) [0x7f40e912efb2]
ceph-mds[2693077]: 3: (Locker::acquire_locks(boost::intrusive_ptr<MDRequestImpl>&, MutationImpl::LockOpVec&, CInode*, bool)+0x131e) >
ceph-mds[2693077]: 4: (Server::handle_client_unlink(boost::intrusive_ptr<MDRequestImpl>&)+0xb43) [0x55eeadce4f73]
ceph-mds[2693077]: 5: (Server::dispatch_client_request(boost::intrusive_ptr<MDRequestImpl>&)+0xf1b) [0x55eeadd12d3b]
ceph-mds[2693077]: 6: (Server::handle_client_request(boost::intrusive_ptr<MClientRequest const> const&)+0x3fc) [0x55eeadd1348c]
ceph-mds[2693077]: 7: (Server::dispatch(boost::intrusive_ptr<Message const> const&)+0x12b) [0x55eeadd179fb]
ceph-mds[2693077]: 8: (MDSRank::handle_message(boost::intrusive_ptr<Message const> const&)+0xbb4) [0x55eeadc6e224]
ceph-mds[2693077]: 9: (MDSRank::_dispatch(boost::intrusive_ptr<Message const> const&, bool)+0x7bb) [0x55eeadc70bdb]
ceph-mds[2693077]: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr<Message const> const&)+0x55) [0x55eeadc711d5]
ceph-mds[2693077]: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x108) [0x55eeadc60dc8]
ceph-mds[2693077]: 12: (DispatchQueue::entry()+0x126a) [0x7f40e9374b5a]
ceph-mds[2693077]: 13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f40e9426841]
ceph-mds[2693077]: 14: /lib64/libpthread.so.0(+0x81ca) [0x7f40e810f1ca]
</pre>
<p>The full logs look like the following<br /><pre>
-1 /builddir/build/BUILD/ceph-16.2.9/src/mds/Locker.cc: In function 'bool Locker::acquire_locks(MDRequestRef&, MutationImpl::LockOpV
ec&, CInode*, bool)' thread 7f40e06ef700 time 2024-03-28T19:20:43.701971+0100
/builddir/build/BUILD/ceph-16.2.9/src/mds/Locker.cc: 404: FAILED ceph_assert(dn->get_projected_linkage()->is_remote())
ceph version 16.2.9-2 (4c3647a322c0ff5a1dd2344e039859dcbd28c830) pacific (stable)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x158) [0x7f40e912ed98]
2: /usr/lib64/ceph/libceph-common.so.2(+0x276fb2) [0x7f40e912efb2]
3: (Locker::acquire_locks(boost::intrusive_ptr<MDRequestImpl>&, MutationImpl::LockOpVec&, CInode*, bool)+0x131e) [0x55eeade5669e]
4: (Server::handle_client_unlink(boost::intrusive_ptr<MDRequestImpl>&)+0xb43) [0x55eeadce4f73]
5: (Server::dispatch_client_request(boost::intrusive_ptr<MDRequestImpl>&)+0xf1b) [0x55eeadd12d3b]
6: (Server::handle_client_request(boost::intrusive_ptr<MClientRequest const> const&)+0x3fc) [0x55eeadd1348c]
7: (Server::dispatch(boost::intrusive_ptr<Message const> const&)+0x12b) [0x55eeadd179fb]
8: (MDSRank::handle_message(boost::intrusive_ptr<Message const> const&)+0xbb4) [0x55eeadc6e224]
9: (MDSRank::_dispatch(boost::intrusive_ptr<Message const> const&, bool)+0x7bb) [0x55eeadc70bdb]
10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr<Message const> const&)+0x55) [0x55eeadc711d5]
11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x108) [0x55eeadc60dc8]
12: (DispatchQueue::entry()+0x126a) [0x7f40e9374b5a]
13: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f40e9426841]
14: /lib64/libpthread.so.0(+0x81ca) [0x7f40e810f1ca]
15: clone()
0> 2024-03-28T19:20:43.709+0100 7f40e06ef700 -1 *** Caught signal (Aborted) **
in thread 7f40e06ef700 thread_name:ms_dispatch
ceph version 16.2.9-2 (4c3647a322c0ff5a1dd2344e039859dcbd28c830) pacific (stable)
1: /lib64/libpthread.so.0(+0x12cf0) [0x7f40e8119cf0]
2: gsignal()
3: abort()
4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1a9) [0x7f40e912ede9]
5: /usr/lib64/ceph/libceph-common.so.2(+0x276fb2) [0x7f40e912efb2]
6: (Locker::acquire_locks(boost::intrusive_ptr<MDRequestImpl>&, MutationImpl::LockOpVec&, CInode*, bool)+0x131e) [0x55eeade5669e]
7: (Server::handle_client_unlink(boost::intrusive_ptr<MDRequestImpl>&)+0xb43) [0x55eeadce4f73]
8: (Server::dispatch_client_request(boost::intrusive_ptr<MDRequestImpl>&)+0xf1b) [0x55eeadd12d3b]
9: (Server::handle_client_request(boost::intrusive_ptr<MClientRequest const> const&)+0x3fc) [0x55eeadd1348c]
10: (Server::dispatch(boost::intrusive_ptr<Message const> const&)+0x12b) [0x55eeadd179fb]
11: (MDSRank::handle_message(boost::intrusive_ptr<Message const> const&)+0xbb4) [0x55eeadc6e224]
12: (MDSRank::_dispatch(boost::intrusive_ptr<Message const> const&, bool)+0x7bb) [0x55eeadc70bdb]
13: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr<Message const> const&)+0x55) [0x55eeadc711d5]
14: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr<Message> const&)+0x108) [0x55eeadc60dc8]
15: (DispatchQueue::entry()+0x126a) [0x7f40e9374b5a]
16: (DispatchQueue::DispatchThread::entry()+0x11) [0x7f40e9426841]
17: /lib64/libpthread.so.0(+0x81ca) [0x7f40e810f1ca]
18: clone()
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
</pre></p>
<p>Some interesting log snippets from before <br /><pre>
-434> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2024-03-28T19:18:03.483427+0100)
-433> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 log_client log_queue is 6 last_log 1380 sent 1374 num 6 unsent 6 sending 6
-432> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 log_client will send 2024-03-28T19:18:32.701758+0100 mds.minilevinson-983db664a3 (mds.0) 1375 : cluster [WRN] 5 slow requests, 5
included below; oldest blocked for > 13.152129 secs
-431> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 log_client will send 2024-03-28T19:18:32.701765+0100 mds.minilevinson-983db664a3 (mds.0) 1376 : cluster [WRN] slow request 7.203
507 seconds old, received at 2024-03-28T19:18:25.497610+0100: client_request(mds.0:181210 rename #0x100026f338d/sh15637 #0x606/1000295a11a caller_uid=0, caller_gid=0{}) currently sub
mit entry: journal_and_reply
-430> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 log_client will send 2024-03-28T19:18:32.701771+0100 mds.minilevinson-983db664a3 (mds.0) 1377 : cluster [WRN] slow request 6.519
057 seconds old, received at 2024-03-28T19:18:26.182061+0100: client_request(mds.0:181239 rename #0x100026f338d/sh15811 #0x608/1000295a58f caller_uid=0, caller_gid=0{}) currently sub
mit entry: journal_and_reply
-429> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 log_client will send 2024-03-28T19:18:32.701774+0100 mds.minilevinson-983db664a3 (mds.0) 1378 : cluster [WRN] slow request 6.372
089 seconds old, received at 2024-03-28T19:18:26.329029+0100: client_request(mds.0:181300 rename #0x1000185ba3c/file117_hardmv_hard #0x603/10002567d82 caller_uid=0, caller_gid=0{}) c
urrently failed to wrlock, waiting
-428> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 log_client will send 2024-03-28T19:18:32.701776+0100 mds.minilevinson-983db664a3 (mds.0) 1379 : cluster [WRN] slow request 6.371
098 seconds old, received at 2024-03-28T19:18:26.330020+0100: client_request(mds.0:181320 rename #0x1000185ba3c/file373_hardmv_hard #0x608/1000256807c caller_uid=0, caller_gid=0{}) c
urrently failed to wrlock, waiting
-427> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 log_client will send 2024-03-28T19:18:32.701781+0100 mds.minilevinson-983db664a3 (mds.0) 1380 : cluster [WRN] slow request 5.565
985 seconds old, received at 2024-03-28T19:18:27.135132+0100: client_request(mds.0:181349 rename #0x100026f338d/sh15708 #0x606/1000295a111 caller_uid=0, caller_gid=0{}) currently fai
led to wrlock, waiting
-426> 2024-03-28T19:18:33.482+0100 7f40df6ed700 10 monclient: _send_mon_message to mon.minilevinson-f52cbe8096 at v2:188.184.88.255:3300/0
-425> 2024-03-28T19:18:33.765+0100 7f40e06ef700 10 log_client handle_log_ack log(last 1380) v1
-424> 2024-03-28T19:18:33.765+0100 7f40e06ef700 10 log_client logged 2024-03-28T19:18:32.701758+0100 mds.minilevinson-983db664a3 (mds.0) 1375 : cluster [WRN] 5 slow requests, 5 in
cluded below; oldest blocked for > 13.152129 secs
-423> 2024-03-28T19:18:33.765+0100 7f40e06ef700 10 log_client logged 2024-03-28T19:18:32.701765+0100 mds.minilevinson-983db664a3 (mds.0) 1376 : cluster [WRN] slow request 7.203507
seconds old, received at 2024-03-28T19:18:25.497610+0100: client_request(mds.0:181210 rename #0x100026f338d/sh15637 #0x606/1000295a11a caller_uid=0, caller_gid=0{}) currently submit
entry: journal_and_reply
-422> 2024-03-28T19:18:33.765+0100 7f40e06ef700 10 log_client logged 2024-03-28T19:18:32.701771+0100 mds.minilevinson-983db664a3 (mds.0) 1377 : cluster [WRN] slow request 6.519057
seconds old, received at 2024-03-28T19:18:26.182061+0100: client_request(mds.0:181239 rename #0x100026f338d/sh15811 #0x608/1000295a58f caller_uid=0, caller_gid=0{}) currently submit
entry: journal_and_reply
-421> 2024-03-28T19:18:33.765+0100 7f40e06ef700 10 log_client logged 2024-03-28T19:18:32.701774+0100 mds.minilevinson-983db664a3 (mds.0) 1378 : cluster [WRN] slow request 6.372089
seconds old, received at 2024-03-28T19:18:26.329029+0100: client_request(mds.0:181300 rename #0x1000185ba3c/file117_hardmv_hard #0x603/10002567d82 caller_uid=0, caller_gid=0{}) curr
ently failed to wrlock, waiting
-420> 2024-03-28T19:18:33.765+0100 7f40e06ef700 10 log_client logged 2024-03-28T19:18:32.701776+0100 mds.minilevinson-983db664a3 (mds.0) 1379 : cluster [WRN] slow request 6.371098
seconds old, received at 2024-03-28T19:18:26.330020+0100: client_request(mds.0:181320 rename #0x1000185ba3c/file373_hardmv_hard #0x608/1000256807c caller_uid=0, caller_gid=0{}) curr
ently failed to wrlock, waiting
-419> 2024-03-28T19:18:33.765+0100 7f40e06ef700 10 log_client logged 2024-03-28T19:18:32.701781+0100 mds.minilevinson-983db664a3 (mds.0) 1380 : cluster [WRN] slow request 5.565985
seconds old, received at 2024-03-28T19:18:27.135132+0100: client_request(mds.0:181349 rename #0x100026f338d/sh15708 #0x606/1000295a111 caller_uid=0, caller_gid=0{}) currently failed
to wrlock, waiting
</pre></p>
<p>We've configured the mds `mds_op_complaint_time` to 5s in order to track some potential slow locks</p>
CephFS - Bug #65224 (New): mds: fs subvolume rm fails
https://tracker.ceph.com/issues/65224
2024-03-29T09:52:40Z
Milind Changire
<p>`fs subvolume rm` fails when subvolume dir attempted to move to a different dir where the following code fails in src/mds/Server.cc:Server::handle_client_rename</p>
<pre>
if (src_realm != dest_realm &&
src_realm->get_subvolume_ino() != dest_realm->get_subvolume_ino()) {
respond_to_request(mdr, -CEPHFS_EXDEV);
return;
}
</pre>
CephFS - Backport #65223 (New): squid: cephfs-mirror: use snapdiff api for efficient tree traversal
https://tracker.ceph.com/issues/65223
2024-03-29T08:23:14Z
Backport Bot
CephFS - Backport #65222 (New): reef: cephfs-mirror: use snapdiff api for efficient tree traversal
https://tracker.ceph.com/issues/65222
2024-03-29T08:23:06Z
Backport Bot
Dashboard - Backport #65221 (New): reef: ceph-mixin: Add RBD Mirror monitoring alerts
https://tracker.ceph.com/issues/65221
2024-03-29T08:01:18Z
Backport Bot
Dashboard - Backport #65220 (New): squid: ceph-mixin: Add RBD Mirror monitoring alerts
https://tracker.ceph.com/issues/65220
2024-03-29T08:01:08Z
Backport Bot
Dashboard - Bug #65219 (Pending Backport): ceph-mixin: Add RBD Mirror monitoring alerts
https://tracker.ceph.com/issues/65219
2024-03-29T07:57:35Z
Avan Thakkar
Dashboard - Bug #65218 (New): mgr/dashboard: Grafana ceph-cluster.json doesn't support cluster label
https://tracker.ceph.com/issues/65218
2024-03-29T07:56:13Z
Ankush Behl
<a name="Description-of-problem"></a>
<h3 >Description of problem<a href="#Description-of-problem" class="wiki-anchor">¶</a></h3>
<p>Grafana ceph-cluster.json doesn't support cluster label.</p>
<p>We don't have jsonnet for now for ceph-cluster.json file. So the Fix just adds the label to JSON only.</p>
<a name="Environment"></a>
<h3 >Environment<a href="#Environment" class="wiki-anchor">¶</a></h3>
<ul>
<li><code>ceph version</code> string:</li>
<li>Platform (OS/distro/release):</li>
<li>Cluster details (nodes, monitors, OSDs):</li>
<li>Did it happen on a stable environment or after a migration/upgrade?:</li>
<li>Browser used (e.g.: <code>Version 86.0.4240.198 (Official Build) (64-bit)</code>):</li>
</ul>
<a name="How-reproducible"></a>
<h3 >How reproducible<a href="#How-reproducible" class="wiki-anchor">¶</a></h3>
<p>Steps:</p>
<ol>
<li> Add multiple cluster via ceph dashboard</li>
<li>The graphs are breaking on grafana Ceph Cluster dashboard.</li>
<li>...</li>
</ol>
<a name="Actual-results"></a>
<h3 >Actual results<a href="#Actual-results" class="wiki-anchor">¶</a></h3>
<p>Single graph shows multiple values</p>
<a name="Expected-results"></a>
<h3 >Expected results<a href="#Expected-results" class="wiki-anchor">¶</a></h3>
<p>Graphs should be adaptable to cluster label.</p>
CephFS - Bug #65217 (Fix Under Review): cephfs: add fscrypt protection support from non-fscrypt c...
https://tracker.ceph.com/issues/65217
2024-03-29T03:12:41Z
Xiubo Li
xiubli@redhat.com
<p>Clients that do not support fscrypt can execute operations that may cause unrecoverable data loss. Add protection on the MDS so that it prevents these clients from executing some operations.</p>
<p>Note, however, that clients will still be able corrupt encrypted files by appending data to them. And they will still be able to read encrypted data from those files.</p>
<p>For the non-fscrypt support client we will allow it to read the encrypted files and directories, but couldn't change the contents of them. For the directories we won't allow to create new sub directories and file under a encrypted file, else in the kclient it will fail to dencrypt the dentry names:</p>
<pre>
125 <7>[201192.339126] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] __prepare_send_request: 0000000071b24ca5 tid 30 readdir (attempt 1)
126 <7>[201192.339144] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] set_request_path_attr: inode 0000000039d46bc2 10000007491.fffffffffffffffe
127 <7>[201192.339345] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] ceph_encode_inode_release: 0000000039d46bc2 10000007491.fffffffffffffffe mds0 used|dirty p drop Fx unless -
128 <7>[201192.339366] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] ceph_encode_inode_release: 0000000039d46bc2 10000007491.fffffffffffffffe cap 00000000b0491451 pAsLsXsFs (force)
129 <7>[201192.339386] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] __prepare_send_request: r_parent = 0000000000000000
130 <7>[201192.339448] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] ceph_mdsc_wait_request: do_request waiting
131 <7>[201192.342097] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] handle_reply: handle_reply 0000000071b24ca5
132 <7>[201192.342118] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] __unregister_request: 0000000071b24ca5 tid 30
133 <7>[201192.342134] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] handle_reply: tid 30 result 0
134 <7>[201192.342214] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] parse_reply_info_readdir: parsed dir dname 'fscrypt_crash_file'
135 <3>[201192.342232] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426]: unable to decode ~Ç+Ê<9b>pt_crash_file, got -5
136 <3>[201192.342245] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426]: problem parsing dir contents -5
137 <3>[201192.342256] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426]: mds parse_reply err -5
138 <7>[201192.342269] header: 00000000: 71 00 00 00 00 00 00 00 1e 00 00 00 00 00 00 00 q...............
139 <7>[201192.342281] header: 00000010: 1a 00 7f 00 01 00 65 03 00 00 00 00 00 00 00 00 ......e.........
140 <7>[201192.342292] header: 00000020: 00 00 00 00 02 00 00 00 00 00 00 00 00 01 00 00 ................
141 <7>[201192.342299] header: 00000030: 00 00 00 00 00 .....
142 <7>[201192.342309] front: 00000000: 05 03 00 00 00 00 00 00 f2 00 00 00 01 00 01 7a ...............z
143 <7>[201192.342320] front: 00000010: 01 00 00 07 01 74 01 00 00 91 74 00 00 00 01 00 .....t....t.....
144 <7>[201192.342327] front: 00000020: 00 fe ff ff ff ff ff ff ff 00 00 00 00 0a 0a 0a ................
145 <7>[201192.342338] front: 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 55 01 00 .............U..
146 <7>[201192.342348] front: 00000040: 00 00 00 00 00 01 00 00 00 00 00 00 00 08 00 00 ................
147 <7>[201192.342359] front: 00000050: 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 ................
148 <7>[201192.342370] front: 00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
149 <7>[201192.342391] front: 00000070: 00 00 00 00 00 00 00 00 00 00 2f f1 03 66 4f 6f ........../..fOo
150 <7>[201192.342402] front: 00000080: c9 2a 2f f1 03 66 4f 6f c9 2a 1c f1 03 66 b7 5c .*/..fOo.*...f.\
151 <7>[201192.342412] front: 00000090: ec 2e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
152 <7>[201192.342423] front: 000000a0: 00 00 00 00 00 00 ff ff ff ff ff ff ff ff 01 00 ................
153 <7>[201192.342433] front: 000000b0: 00 00 ed 41 00 00 e8 03 00 00 e8 03 00 00 01 00 ...A............
154 <7>[201192.342444] front: 000000c0: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
155 <7>[201192.342454] front: 000000d0: 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 ................
156 <7>[201192.342465] front: 000000e0: 00 00 01 00 00 00 00 00 00 00 2f f1 03 66 54 18 ........../..fT.
157 <7>[201192.342475] front: 000000f0: c0 2b 00 00 00 00 00 00 00 00 02 00 00 00 00 00 .+..............
158 <7>[201192.342486] front: 00000100: 00 00 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 ................
159 <7>[201192.342497] front: 00000110: 00 00 01 01 10 00 00 00 00 00 00 00 00 00 00 00 ................
160 <7>[201192.342507] front: 00000120: 00 00 00 00 00 00 00 00 00 00 00 00 1c f1 03 66 ...............f
161 <7>[201192.342518] front: 00000130: b7 5c ec 2e 01 00 00 00 00 00 00 00 ff ff ff ff .\..............
162 <7>[201192.342529] front: 00000140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
163 <7>[201192.342539] front: 00000150: 00 00 00 00 01 30 00 00 00 01 00 00 00 28 00 00 .....0.......(..
164 <7>[201192.342550] front: 00000160: 00 02 01 04 00 00 00 00 00 81 66 70 7d 58 e2 3b ..........fp}X.;
165 <7>[201192.342560] front: 00000170: 91 3b bc 4d 82 30 5b 68 a2 fd 80 c0 16 ac cb f5 .;.M.0[h........
166 <7>[201192.342570] front: 00000180: 38 bd ea de e4 f3 c4 e3 57 00 00 00 00 90 01 00 8.......W.......
167 <7>[201192.342581] front: 00000190: 00 01 01 0c 00 00 00 00 00 00 00 ff ff ff ff 00 ................
168 <7>[201192.342591] front: 000001a0: 00 00 00 01 00 00 00 01 07 12 00 00 00 7e c7 2b .............~.+
169 <7>[201192.342602] front: 000001b0: ca 9b 70 74 5f 63 72 61 73 68 5f 66 69 6c 65 02 ..pt_crash_file.
170 <7>[201192.342611] front: 000001c0: 01 0e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
171 <7>[201192.342618] front: 000001d0: 00 00 00 07 01 48 01 00 00 df 7a 00 00 00 01 00 .....H....z.....
172 <7>[201192.342624] front: 000001e0: 00 fe ff ff ff ff ff ff ff 00 00 00 00 08 00 00 ................
173 <7>[201192.342632] front: 000001f0: 00 00 00 00 00 01 00 00 00 00 00 00 00 55 0d 00 .............U..
174 <7>[201192.342640] front: 00000200: 00 00 00 00 00 d2 18 00 00 00 00 00 00 03 00 00 ................
175 <7>[201192.342646] front: 00000210: 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 ................
176 <7>[201192.342651] front: 00000220: 40 00 01 00 00 00 00 00 40 00 00 00 00 00 00 00 @.......@.......
177 <7>[201192.342657] front: 00000230: 00 00 00 00 00 00 03 00 00 00 2f f1 03 66 54 18 ........../..fT.
178 <7>[201192.342663] front: 00000240: c0 2b 2f f1 03 66 d7 26 73 2b 2f f1 03 66 d7 26 .+/..f.&s+/..f.&
179 <7>[201192.342668] front: 00000250: 73 2b 02 00 00 00 00 00 00 00 00 00 00 00 00 00 s+..............
180 <7>[201192.342674] front: 00000260: 00 00 00 00 00 00 ff ff ff ff ff ff ff ff 01 00 ................
181 <7>[201192.342680] front: 00000270: 00 00 a4 81 00 00 e8 03 00 00 e8 03 00 00 01 00 ................
182 <7>[201192.342685] front: 00000280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
183 <7>[201192.342691] front: 00000290: 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 ................
184 <7>[201192.342696] front: 000002a0: 00 00 00 00 00 00 00 00 00 00 2f f1 03 66 54 18 ........../..fT.
185 <7>[201192.342702] front: 000002b0: c0 2b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .+..............
186 <7>[201192.342707] front: 000002c0: 00 00 04 00 00 00 00 00 00 00 ff ff ff ff ff ff ................
187 <7>[201192.342713] front: 000002d0: ff ff 00 00 00 00 01 01 10 00 00 00 00 00 00 00 ................
188 <7>[201192.342719] front: 000002e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
189 <7>[201192.342724] front: 000002f0: 2f f1 03 66 4f 6f c9 2a 01 00 00 00 00 00 00 00 /..fOo.*........
190 <7>[201192.342730] front: 00000300: ff ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 ................
191 <7>[201192.342736] front: 00000310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
192 <7>[201192.342741] front: 00000320: 00 40 00 00 00 01 00 00 00 00 00 00 00 03 00 00 .@..............
193 <7>[201192.342747] front: 00000330: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 ................
194 <7>[201192.342752] front: 00000340: 00 00 00 00 00 03 00 00 00 00 00 00 00 02 00 00 ................
195 <7>[201192.342758] front: 00000350: 00 00 00 00 00 03 00 00 00 00 00 00 00 02 00 00 ................
196 <7>[201192.342763] front: 00000360: 00 00 00 00 00 .....
197 <7>[201192.342769] footer: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
198 <7>[201192.342775] footer: 00000010: 00 00 00 00 00 .....
199 <3>[201192.342780] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426]: got corrupt reply mds0(tid:30)
200 <7>[201192.342788] header: 00000000: 71 00 00 00 00 00 00 00 1e 00 00 00 00 00 00 00 q...............
201 <7>[201192.342793] header: 00000010: 1a 00 7f 00 01 00 65 03 00 00 00 00 00 00 00 00 ......e.........
202 <7>[201192.342799] header: 00000020: 00 00 00 00 02 00 00 00 00 00 00 00 00 01 00 00 ................
203 <7>[201192.342804] header: 00000030: 00 00 00 00 00 .....
204 <7>[201192.342810] front: 00000000: 05 03 00 00 00 00 00 00 f2 00 00 00 01 00 01 7a ...............z
205 <7>[201192.342815] front: 00000010: 01 00 00 07 01 74 01 00 00 91 74 00 00 00 01 00 .....t....t.....
206 <7>[201192.342821] front: 00000020: 00 fe ff ff ff ff ff ff ff 00 00 00 00 0a 0a 0a ................
207 <7>[201192.342827] front: 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 55 01 00 .............U..
208 <7>[201192.342832] front: 00000040: 00 00 00 00 00 01 00 00 00 00 00 00 00 08 00 00 ................
209 <7>[201192.342838] front: 00000050: 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 ................
210 <7>[201192.342843] front: 00000060: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
211 <7>[201192.342849] front: 00000070: 00 00 00 00 00 00 00 00 00 00 2f f1 03 66 4f 6f ........../..fOo
212 <7>[201192.342855] front: 00000080: c9 2a 2f f1 03 66 4f 6f c9 2a 1c f1 03 66 b7 5c .*/..fOo.*...f.\
213 <7>[201192.342860] front: 00000090: ec 2e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
214 <7>[201192.342866] front: 000000a0: 00 00 00 00 00 00 ff ff ff ff ff ff ff ff 01 00 ................
215 <7>[201192.342871] front: 000000b0: 00 00 ed 41 00 00 e8 03 00 00 e8 03 00 00 01 00 ...A............
216 <7>[201192.342877] front: 000000c0: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
217 <7>[201192.342883] front: 000000d0: 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 ................
218 <7>[201192.342888] front: 000000e0: 00 00 01 00 00 00 00 00 00 00 2f f1 03 66 54 18 ........../..fT.
219 <7>[201192.342894] front: 000000f0: c0 2b 00 00 00 00 00 00 00 00 02 00 00 00 00 00 .+..............
220 <7>[201192.342900] front: 00000100: 00 00 00 00 00 00 ff ff ff ff ff ff ff ff 00 00 ................
221 <7>[201192.342905] front: 00000110: 00 00 01 01 10 00 00 00 00 00 00 00 00 00 00 00 ................
222 <7>[201192.342911] front: 00000120: 00 00 00 00 00 00 00 00 00 00 00 00 1c f1 03 66 ...............f
223 <7>[201192.342916] front: 00000130: b7 5c ec 2e 01 00 00 00 00 00 00 00 ff ff ff ff .\..............
224 <7>[201192.342922] front: 00000140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
225 <7>[201192.342928] front: 00000150: 00 00 00 00 01 30 00 00 00 01 00 00 00 28 00 00 .....0.......(..
226 <7>[201192.342933] front: 00000160: 00 02 01 04 00 00 00 00 00 81 66 70 7d 58 e2 3b ..........fp}X.;
227 <7>[201192.342939] front: 00000170: 91 3b bc 4d 82 30 5b 68 a2 fd 80 c0 16 ac cb f5 .;.M.0[h........
228 <7>[201192.342945] front: 00000180: 38 bd ea de e4 f3 c4 e3 57 00 00 00 00 90 01 00 8.......W.......
229 <7>[201192.342950] front: 00000190: 00 01 01 0c 00 00 00 00 00 00 00 ff ff ff ff 00 ................
230 <7>[201192.342956] front: 000001a0: 00 00 00 01 00 00 00 01 07 12 00 00 00 7e c7 2b .............~.+
231 <7>[201192.342962] front: 000001b0: ca 9b 70 74 5f 63 72 61 73 68 5f 66 69 6c 65 02 ..pt_crash_file.
232 <7>[201192.342967] front: 000001c0: 01 0e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
233 <7>[201192.342973] front: 000001d0: 00 00 00 07 01 48 01 00 00 df 7a 00 00 00 01 00 .....H....z.....
234 <7>[201192.342978] front: 000001e0: 00 fe ff ff ff ff ff ff ff 00 00 00 00 08 00 00 ................
235 <7>[201192.342984] front: 000001f0: 00 00 00 00 00 01 00 00 00 00 00 00 00 55 0d 00 .............U..
236 <7>[201192.342990] front: 00000200: 00 00 00 00 00 d2 18 00 00 00 00 00 00 03 00 00 ................
237 <7>[201192.342996] front: 00000210: 00 00 00 00 00 01 00 00 00 00 00 00 00 01 00 00 ................
238 <7>[201192.343001] front: 00000220: 40 00 01 00 00 00 00 00 40 00 00 00 00 00 00 00 @.......@.......
239 <7>[201192.343007] front: 00000230: 00 00 00 00 00 00 03 00 00 00 2f f1 03 66 54 18 ........../..fT.
240 <7>[201192.343012] front: 00000240: c0 2b 2f f1 03 66 d7 26 73 2b 2f f1 03 66 d7 26 .+/..f.&s+/..f.&
241 <7>[201192.343018] front: 00000250: 73 2b 02 00 00 00 00 00 00 00 00 00 00 00 00 00 s+..............
242 <7>[201192.343024] front: 00000260: 00 00 00 00 00 00 ff ff ff ff ff ff ff ff 01 00 ................
243 <7>[201192.343029] front: 00000270: 00 00 a4 81 00 00 e8 03 00 00 e8 03 00 00 01 00 ................
244 <7>[201192.343035] front: 00000280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
245 <7>[201192.343041] front: 00000290: 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 00 ................
246 <7>[201192.343047] front: 000002a0: 00 00 00 00 00 00 00 00 00 00 2f f1 03 66 54 18 ........../..fT.
247 <7>[201192.343053] front: 000002b0: c0 2b 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .+..............
248 <7>[201192.343058] front: 000002c0: 00 00 04 00 00 00 00 00 00 00 ff ff ff ff ff ff ................
249 <7>[201192.343064] front: 000002d0: ff ff 00 00 00 00 01 01 10 00 00 00 00 00 00 00 ................
250 <7>[201192.343069] front: 000002e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
251 <7>[201192.343075] front: 000002f0: 2f f1 03 66 4f 6f c9 2a 01 00 00 00 00 00 00 00 /..fOo.*........
252 <7>[201192.343083] front: 00000300: ff ff ff ff 00 00 00 00 00 00 00 00 00 00 00 00 ................
253 <7>[201192.343088] front: 00000310: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
254 <7>[201192.343094] front: 00000320: 00 40 00 00 00 01 00 00 00 00 00 00 00 03 00 00 .@..............
255 <7>[201192.343100] front: 00000330: 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 ................
256 <7>[201192.343105] front: 00000340: 00 00 00 00 00 03 00 00 00 00 00 00 00 02 00 00 ................
257 <7>[201192.343111] front: 00000350: 00 00 00 00 00 03 00 00 00 00 00 00 00 02 00 00 ................
258 <7>[201192.343116] front: 00000360: 00 00 00 00 00 .....
259 <7>[201192.343122] footer: 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
260 <7>[201192.343128] footer: 00000010: 00 00 00 00 00 .....
261 <7>[201192.343195] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] ceph_mdsc_wait_request: do_request waited, got 0
262 <7>[201192.343206] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] ceph_mdsc_do_request: do_request 0000000071b24ca5 done, result -5
263 <7>[201192.343289] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] __ceph_put_cap_refs: 0000000039d46bc2 10000007491.fffffffffffffffe had p
264 <7>[201192.343302] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] ceph_unreserve_caps: ctx=000000008741ca3c count=20
265 <7>[201192.343311] ceph: [9a8fd138-5876-4325-af3b-ba7f972e5776 9426] __ceph_unreserve_caps: caps 25 = 5 used + 0 resv + 20 avail
</pre>
rgw - Bug #65216 (New): rgw: only accept valid ipv4 from host header
https://tracker.ceph.com/issues/65216
2024-03-29T00:30:04Z
Seena Fallah
<p>Right now the validation for ipv4 from the host header is based on the number of periods - this leads to accepting invalid ips.</p>
Ceph QA - QA Run #65215 (QA Testing): wip-batrick-testing-20240328.192822
https://tracker.ceph.com/issues/65215
2024-03-28T19:36:30Z
Patrick Donnelly
pdonnell@redhat.com
CephFS - Backport #65214 (In Progress): squid: mds: quiesce_inode op waiting on remote auth pins ...
https://tracker.ceph.com/issues/65214
2024-03-28T18:45:10Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56564">https://github.com/ceph/ceph/pull/56564</a></p>
rgw - Bug #65212 (Fix Under Review): pubsub: validate Name in CreateTopic api
https://tracker.ceph.com/issues/65212
2024-03-28T16:24:24Z
Casey Bodley
cbodley@redhat.com
<p>prevent topic names that would confuse things like ARN parsing and rados object namespacing</p>
<p>from <a class="external" href="https://docs.aws.amazon.com/sns/latest/api/API_CreateTopic.html#API_CreateTopic_RequestParameters">https://docs.aws.amazon.com/sns/latest/api/API_CreateTopic.html#API_CreateTopic_RequestParameters</a></p>
<pre>
Name
The name of the topic you want to create.
Constraints: Topic names must be made up of only uppercase and lowercase ASCII letters, numbers, underscores, and hyphens, and must be between 1 and 256 characters long.
For a FIFO (first-in-first-out) topic, the name must end with the .fifo suffix.
Type: String
Required: Yes
</pre>
Dashboard - Backport #65211 (New): reef: mgr/dashboard: Mark placement targets as non-required
https://tracker.ceph.com/issues/65211
2024-03-28T16:05:55Z
Backport Bot