Ceph : Issues
https://tracker.ceph.com/
https://tracker.ceph.com/favicon.ico
2024-03-28T10:29:12Z
Ceph
Redmine
RADOS - Backport #65198 (In Progress): squid: Failed to encode map X with expected CRC
https://tracker.ceph.com/issues/65198
2024-03-28T10:29:12Z
Backport Bot
Dashboard - Backport #65197 (In Progress): quincy: mgr/dashboard: sanitize dashboard user input f...
https://tracker.ceph.com/issues/65197
2024-03-28T08:29:32Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56551">https://github.com/ceph/ceph/pull/56551</a></p>
Dashboard - Backport #65196 (In Progress): reef: mgr/dashboard: Cloning of subvolume with the sam...
https://tracker.ceph.com/issues/65196
2024-03-28T08:22:05Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56550">https://github.com/ceph/ceph/pull/56550</a></p>
Dashboard - Backport #65195 (In Progress): squid: mgr/dashboard: Cloning of subvolume with the sa...
https://tracker.ceph.com/issues/65195
2024-03-28T08:21:57Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56549">https://github.com/ceph/ceph/pull/56549</a></p>
Dashboard - Backport #65194 (In Progress): reef: mgr/dashboard: rgw display name with dot gives v...
https://tracker.ceph.com/issues/65194
2024-03-28T08:21:49Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56548">https://github.com/ceph/ceph/pull/56548</a></p>
Dashboard - Backport #65193 (In Progress): squid: mgr/dashboard: rgw display name with dot gives ...
https://tracker.ceph.com/issues/65193
2024-03-28T08:21:41Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56547">https://github.com/ceph/ceph/pull/56547</a></p>
Dashboard - Backport #65192 (In Progress): reef: mgr/dashboard: s/active_mds/active_nfs in fs att...
https://tracker.ceph.com/issues/65192
2024-03-28T08:21:28Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56546">https://github.com/ceph/ceph/pull/56546</a></p>
Dashboard - Backport #65191 (In Progress): squid: mgr/dashboard: s/active_mds/active_nfs in fs at...
https://tracker.ceph.com/issues/65191
2024-03-28T08:21:15Z
Backport Bot
<p><a class="external" href="https://github.com/ceph/ceph/pull/56545">https://github.com/ceph/ceph/pull/56545</a></p>
bluestore - Support #65190 (New): Can ceph with SPDK driver has to be connected with local NVMe d...
https://tracker.ceph.com/issues/65190
2024-03-28T03:28:48Z
Alice Wang
<p>Hi everyone,<br />I am using ceph 18.2.2 to create a cluster with NVMe device under SPDK driver.<br />When the device is connect to the host, there is a PCIe number, such as 0000:3c:00.0, then I can start the cluster by the below command</p>
<p>vstart.sh --new -X --localhost --bluestore-spdk "trtype:PCIe traddr:0000:3c:00.0" --bluestore</p>
<p>The above is OK. But<br />If Ceph is using the SPDK engine, does it have to have a local PCIe device?<br />If the PCIe device is connected to host through NVMe-oF, can we still use bluestore-spdk to start the cluster with NVMe-oF device?<br />If it is the malloc bdev created by SPDK, can we still use bluestore-spdk to start the cluster with malloc bdev?</p>
<p>Thank you.</p>
mgr - Bug #65189 (New): Telemetry pacific-x upgrade test pauses when upgrading to squid
https://tracker.ceph.com/issues/65189
2024-03-27T22:36:30Z
Laura Flores
<p>/a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7615987<br /><pre><code class="text syntaxhl"><span class="CodeRay">2024-03-22T06:49:01.882 INFO:journalctl@ceph.mgr.y.smithi023.stdout:Mar 22 06:49:01 smithi023 ceph-e86c638a-e816-11ee-95cd-87774f69a715-mgr-y[39704]: debug 2024-03-22T06:49:01.516+0000 7fb5985ac700 -1 log_channel(cephadm) log [ERR] : Upgrade: Paused due to UPGRADE_BAD_TARGET_VERSION: Upgrade: cannot upgrade/downgrade to 19.0.0-1667-gdb0330b1
</span></code></pre></p>
<p>The solution would be to update the test so it upgrades from reef instead of pacific. We already have a quincy-x upgrade test that works fine since it follows the "n-2" upgrade rule.</p>
rgw - Bug #65188 (Fix Under Review): rgwlc: Executing radosgw-admin lc process --bucket <bkt-name...
https://tracker.ceph.com/issues/65188
2024-03-27T22:30:11Z
Matt Benjamin
mbenjamin@redhat.com
<p>[LC-Process]: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault</p>
<p>Description of problem:<br />[LC-Process]: Executing radosgw-admin lc process --bucket <bkt-name> without setting lc rule results in Segmentation fault</p>
<p>Version-Release number of selected component (if applicable):<br />ceph version 18.2.1-73.el9cp</p>
<p>How reproducible:<br />3/3</p>
<p>Steps to Reproduce:<br />1. Deploy cluster with: ceph version 18.2.1-73.el9cp<br />2. Create a bucket: <bkt_name><br />3. Upload object to the bucket<br />4. Perform: radosgw-admin lc process --bucket <bkt_name></p>
<p>Actual results:<br />Throwing error:</p>
<ul>
<li>Caught signal (Segmentation fault) <b><br /> in thread 7f74eed29800 thread_name:radosgw-admin<br /> ceph version 18.2.1-73.el9cp (16d1bc4bed21ede5993c301b4626fa21cbe97cff) reef (stable)<br /> 1: /lib64/libc.so.6(+0x54db0) [0x7f74ef254db0]<br /> 2: (RGWLC::process_bucket(int, int, RGWLC::LCWorker*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool)+0x2b6) [0x556e82e69626]<br /> 3: (RGWLC::process(RGWLC::LCWorker*, std::unique_ptr<rgw::sal::Bucket, std::default_delete<rgw::sal::Bucket> > const&, bool)+0xb7) [0x556e82e6d1a7]<br /> 4: (RGWRados::process_lc(std::unique_ptr<rgw::sal::Bucket, std::default_delete<rgw::sal::Bucket> > const&)+0xdd) [0x556e831d409d]<br /> 5: main()<br /> 6: /lib64/libc.so.6(+0x3feb0) [0x7f74ef23feb0]<br /> 7: __libc_start_main()<br /> 8: _start()<br />2024-03-20T02:17:03.968-0400 7f74eed29800 -1 <strong></b> Caught signal (Segmentation fault) *</strong><br /> in thread 7f74eed29800 thread_name:radosgw-admin</li>
</ul>
Orchestrator - Bug #65187 (New): upgrade/quincy-x/stress-split: upgrade test fails to install qui...
https://tracker.ceph.com/issues/65187
2024-03-27T22:18:47Z
Laura Flores
<pre><code class="text syntaxhl"><span class="CodeRay">2024-03-22T06:52:10.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=quincy
2024-03-22T06:52:10.571 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) <sage@newdream.net>
2024-03-22T06:52:10.571 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph.com (release key) <security@ceph.com>
2024-03-22T06:52:10.572 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64
2024-03-22T06:52:10.572 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch
2024-03-22T06:52:10.572 INFO:teuthology.packaging:ref: None
2024-03-22T06:52:10.572 INFO:teuthology.packaging:tag: None
2024-03-22T06:52:10.572 INFO:teuthology.packaging:branch: quincy
2024-03-22T06:52:10.572 INFO:teuthology.packaging:sha1: db0330b1e4e2470d52b750e251e55a522b4f7d69
2024-03-22T06:52:10.572 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=quincy
2024-03-22T06:52:10.709 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/contextutil.py", line 30, in nested
vars.append(enter())
File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
return next(self.gen)
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 218, in install
install_packages(ctx, package_list, config)
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/__init__.py", line 81, in install_packages
p.spawn(
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 84, in __exit__
for result in self:
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 98, in __next__
resurrect_traceback(result)
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 30, in resurrect_traceback
raise exc.exc_info[1]
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/parallel.py", line 23, in capture_traceback
return func(*args, **kwargs)
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/task/install/deb.py", line 79, in _update_package_list_and_install
log.info('Pulling from %s', builder.base_url)
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/packaging.py", line 554, in base_url
return self._get_base_url()
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/packaging.py", line 856, in _get_base_url
self.assert_result()
File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/packaging.py", line 937, in assert_result
raise VersionNotFoundError(self._result.url)
teuthology.exceptions.VersionNotFoundError: Failed to fetch package version from https://shaman.ceph.com/api/search/?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=quincy
2024-03-22T06:52:10.711 ERROR:teuthology.run_tasks:Saw exception from tasks.
</span></code></pre>
RADOS - Bug #65186 (New): OSDs unreachable in upgrade test
https://tracker.ceph.com/issues/65186
2024-03-27T20:28:19Z
Laura Flores
<p>/a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616011/remote/smithi087/log/a8e8c570-e819-11ee-95cd-87774f69a715<br /><pre><code class="text syntaxhl"><span class="CodeRay">2024-03-22T07:19:18.215315+0000 mon.a (mon.0) 10 : cluster 0 Standby manager daemon x restarted
2024-03-22T07:19:18.215450+0000 mon.a (mon.0) 11 : cluster 0 Standby manager daemon x started
2024-03-22T07:19:18.215315+0000 mon.a (mon.0) 10 : cluster 0 Standby manager daemon x restarted
2024-03-22T07:19:18.215450+0000 mon.a (mon.0) 11 : cluster 0 Standby manager daemon x started
2024-03-22T07:19:18.277027+0000 mon.a (mon.0) 12 : cluster 0 mgrmap e33: y(active, since 63s), standbys: x
2024-03-22T07:19:18.414028+0000 mon.a (mon.0) 13 : cluster 1 Active manager daemon y restarted
2024-03-22T07:19:18.414630+0000 mon.a (mon.0) 14 : cluster 4 Health check failed: 8 osds(s) are not reachable (OSD_UNREACHABLE)
2024-03-22T07:19:18.414953+0000 mon.a (mon.0) 15 : cluster 1 Activating manager daemon y
2024-03-22T07:19:18.427127+0000 mon.a (mon.0) 16 : cluster 0 osdmap e81: 8 total, 8 up, 8 in
2024-03-22T07:19:18.277027+0000 mon.a (mon.0) 12 : cluster 0 mgrmap e33: y(active, since 63s), standbys: x
2024-03-22T07:19:18.427673+0000 mon.a (mon.0) 17 : cluster 0 mgrmap e34: y(active, starting, since 0.0129348s), standbys: x
2024-03-22T07:19:18.414028+0000 mon.a (mon.0) 13 : cluster 1 Active manager daemon y restarted
2024-03-22T07:19:18.433869+0000 osd.4 (osd.4) 3 : cluster 3 failed to encode map e81 with expected crc
2024-03-22T07:19:18.435418+0000 osd.2 (osd.2) 3 : cluster 3 failed to encode map e81 with expected crc
2024-03-22T07:19:18.414630+0000 mon.a (mon.0) 14 : cluster 4 Health check failed: 8 osds(s) are not reachable (OSD_UNREACHABLE)
2024-03-22T07:19:18.443967+0000 osd.4 (osd.4) 4 : cluster 3 failed to encode map e81 with expected crc
</span></code></pre></p>
<p>Likely connected to <a class="external" href="https://tracker.ceph.com/issues/63389">https://tracker.ceph.com/issues/63389</a>.</p>
RADOS - Bug #65185 (New): OSD_SCRUB_ERROR, inconsistent pg in upgrade tests
https://tracker.ceph.com/issues/65185
2024-03-27T20:21:29Z
Laura Flores
<p>/a/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/7616025/remote/smithi098/log/b1f19696-e81a-11ee-95cd-87774f69a715/ceph.log.gz<br /><pre><code class="text syntaxhl"><span class="CodeRay">2024-03-22T09:20:00.000187+0000 mon.a (mon.0) 7863 : cluster 4 [ERR] OSD_SCRUB_ERRORS: 1 scrub errors
2024-03-22T09:20:00.000194+0000 mon.a (mon.0) 7864 : cluster 4 [ERR] PG_DAMAGED: Possible data damage: 1 pg inconsistent
2024-03-22T09:19:59.897409+0000 mon.a (mon.0) 7860 : cluster 0 osdmap e3595: 8 total, 8 up, 8 in
2024-03-22T09:20:00.000202+0000 mon.a (mon.0) 7865 : cluster 4 pg 103.14 is active+clean+inconsistent, acting [5,1,2]
2024-03-22T09:20:00.000151+0000 mon.a (mon.0) 7861 : cluster 4 Health detail: HEALTH_ERR noscrub flag(s) set; 1 scrub errors; Possible data damage: 1 pg inconsistent
</span></code></pre></p>
<p>More in this run: <a class="external" href="https://pulpito.ceph.com/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/">https://pulpito.ceph.com/teuthology-2024-03-22_02:08:13-upgrade-squid-distro-default-smithi/</a></p>
RADOS - Bug #65183 (Fix Under Review): Overriding an EC pool needs the "--yes-i-really-mean-it" f...
https://tracker.ceph.com/issues/65183
2024-03-27T16:23:12Z
Laura Flores
<p>/a/yuriw-2024-03-26_14:32:05-rados-wip-yuri8-testing-2024-03-25-1419-distro-default-smithi/7623454<br /><pre><code class="text syntaxhl"><span class="CodeRay">2024-03-26T20:13:29.028 INFO:tasks.workunit.client.0.smithi110.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh:35: expect_false: set -x
2024-03-26T20:13:29.028 INFO:tasks.workunit.client.0.smithi110.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh:36: expect_false: ceph osd erasure-code-profile set fooprofile a=b c=d e=f
2024-03-26T20:13:29.301 INFO:tasks.workunit.client.0.smithi110.stderr:Error EPERM: will not override erasure code profile fooprofile because the existing profile {a=b,c=d,crush-device-class=,crush-failure-domain=osd,crush-num-failure-domains=0,crush-osds-per-failure-domain=0,crush-root=default,jerasure-per-chunk-alignment=false,k=2,m=1,plugin=jerasure,technique=reed_sol_van,w=8} is different from the proposed profile {a=b,c=d,crush-device-class=,crush-failure-domain=osd,crush-num-failure-domains=0,crush-osds-per-failure-domain=0,crush-root=default,e=f,jerasure-per-chunk-alignment=false,k=2,m=1,plugin=jerasure,technique=reed_sol_van,w=8}
2024-03-26T20:13:29.304 INFO:tasks.workunit.client.0.smithi110.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh:36: expect_false: return 0
2024-03-26T20:13:29.304 INFO:tasks.workunit.client.0.smithi110.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh:2493: test_mon_osd_erasure_code: ceph osd erasure-code-profile set fooprofile a=b c=d e=f --force
2024-03-26T20:13:29.581 INFO:tasks.workunit.client.0.smithi110.stderr:Error EPERM: overriding erasure code profile can be DANGEROUS; add --yes-i-really-mean-it to do it anyway
2024-03-26T20:13:29.585 INFO:tasks.workunit.client.0.smithi110.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/cephtool/test.sh:1: test_mon_osd_erasure_code: rm -fr /tmp/cephtool.wZw
2024-03-26T20:13:29.586 DEBUG:teuthology.orchestra.run:got remote process result: 1
2024-03-26T20:13:29.587 INFO:tasks.workunit:Stopping ['cephtool'] on client.0...
</span></code></pre></p>
<p>Here's the test that fails:<br />qa/workunits/cephtool/test.sh<br /><pre><code class="text syntaxhl"><span class="CodeRay">function test_mon_osd_erasure_code()
{
ceph osd erasure-code-profile set fooprofile a=b c=d
ceph osd erasure-code-profile set fooprofile a=b c=d
expect_false ceph osd erasure-code-profile set fooprofile a=b c=d e=f
ceph osd erasure-code-profile set fooprofile a=b c=d e=f --force ---------------------> this one
ceph osd erasure-code-profile set fooprofile a=b c=d e=f
expect_false ceph osd erasure-code-profile set fooprofile a=b c=d e=f g=h
# make sure rule-foo doesn't work anymore
expect_false ceph osd erasure-code-profile set barprofile ruleset-failure-domain=host
ceph osd erasure-code-profile set barprofile crush-failure-domain=host
# clean up
ceph osd erasure-code-profile rm fooprofile
ceph osd erasure-code-profile rm barprofile
# try weird k and m values
expect_false ceph osd erasure-code-profile set badk k=1 m=1
expect_false ceph osd erasure-code-profile set badk k=1 m=2
expect_false ceph osd erasure-code-profile set badk k=0 m=2
expect_false ceph osd erasure-code-profile set badk k=-1 m=2
expect_false ceph osd erasure-code-profile set badm k=2 m=0
expect_false ceph osd erasure-code-profile set badm k=2 m=-1
ceph osd erasure-code-profile set good k=2 m=1
ceph osd erasure-code-profile rm good
}
</span></code></pre></p>