Activity
From 12/27/2022 to 01/25/2023
01/25/2023
- 06:59 AM Bug #58097 (Resolved): qa/workunits/post-file.sh: kex_exchange_identification: read: Connection r...
- This is fixed for now; the postfile sftp account is up on drop.ceph.com. At the moment, the files are not stored on ...
01/24/2023
- 03:20 PM sepia Support #58437: Sepia Lab Access Request
- I am unable to ssh, I see this error:
debug2: resolving "teuthology.front.sepia.ceph.com" port 22
debug3: resolve... - 02:04 PM sepia Support #58437: Sepia Lab Access Request
- Hey please try now
- 07:35 AM Bug #58560 (Pending Backport): test_envlibrados_for_rocksdb.sh failed to subscribe to repo
/a//yuriw-2023-01-23_17:16:25-rados-wip-yuri6-testing-2023-01-22-0750-distro-default-smithi/7133989...
01/23/2023
- 10:53 PM sepia Support #58521 (Resolved): Sepia Lab Access Request
- I helped out today and got it working. So far we don't think Gabriella will need an ssh key, but we'll cross that br...
- 04:08 PM sepia Support #58521 (In Progress): Sepia Lab Access Request
- 04:08 PM sepia Support #58521: Sepia Lab Access Request
- Hey Gabriella, please paste you ssh public key
Thanks - 04:50 PM Bug #58483: Unable to access node mero010.ceph.redhat.com
- Fixed
- 04:47 PM Bug #58483 (In Progress): Unable to access node mero010.ceph.redhat.com
- 04:23 PM sepia Support #58516 (Resolved): Sepia Lab Access Request
- Duplicate issue
- 11:13 AM sepia Support #58437: Sepia Lab Access Request
- prsrivas@fedora 13dmTf+3sbpan4iffbf1cw 65ea22c8ba7f49c3619ca9cf552a04400528b4114a53eafee715884cb3a04dea
- 10:29 AM sepia Support #58437: Sepia Lab Access Request
- If you re-run the new-client script, It's unfortunately not idempotent so if you re-ran it and still have the output,...
- 07:45 AM sepia Support #58437: Sepia Lab Access Request
- adam kraitman wrote:
> Hey Pritha Srivastava,
>
> You should have access to the Sepia lab now. Please verify you'...
01/22/2023
- 04:44 PM sepia Support #58493 (In Progress): Sepia Lab Access Request
- 04:44 PM sepia Support #58493: Sepia Lab Access Request
- Hey, Welby McRoberts is there an existing community or core developer you've worked with who has reviewed your work a...
- 04:40 PM sepia Support #58372 (In Progress): Sepia Lab Access Request
- 04:40 PM sepia Support #58372: Sepia Lab Access Request
- Hey Kotresh Hiremath Ravishankar , Are these new/additional or replacement credentials?
- 04:38 PM sepia Support #58437: Sepia Lab Access Request
- Hey Pritha Srivastava,
You should have access to the Sepia lab now. Please verify you're able to connect to the vp... - 07:38 AM Bug #55443: "SELinux denials found.." in rados run
- /a/yuriw-2023-01-21_17:58:46-rados-wip-yuri6-testing-2023-01-20-0728-distro-default-smithi/7132689
01/20/2023
- 08:49 PM Bug #58498: ceph: pgs stuck backfilling
- I opened https://tracker.ceph.com/issues/58529 to track the slow push reply behavior since it's probably unrelated to...
- 05:02 PM Bug #58498: ceph: pgs stuck backfilling
- I took a look at the logs for pg114.d6 attached to this tracker. The cost for the push replies is calculated to over
... - 02:01 PM sepia Support #58521: Sepia Lab Access Request
- I might have messed up.....
- 01:46 PM sepia Support #58521 (Resolved): Sepia Lab Access Request
- 1) Do you just need VPN access or will you also be running teuthology jobs?
just vpn
2) Desired Username:
groma...
01/19/2023
- 10:22 PM sepia Support #58516 (Resolved): Sepia Lab Access Request
- 1) Do you just need VPN access or will you also be running teuthology jobs?
vpn access
2) Desired Username:
gro... - 09:34 PM Bug #58498: ceph: pgs stuck backfilling
- mclock really shouldn't be delaying replies.
- 09:18 PM Bug #58498: ceph: pgs stuck backfilling
- Re-downed osd.128 with the op tracker enabled:...
- 09:09 PM Bug #58498: ceph: pgs stuck backfilling
- Attached logs from 128 from before marking down to after backfill appears to stall. (Deleted earlier attached logs f...
- 06:09 PM Bug #58498: ceph: pgs stuck backfilling
- One pg, the one on osd.128, continued to hang after marking the osd down. I enabled logging on the primary and marke...
- 04:29 AM Bug #58498 (New): ceph: pgs stuck backfilling
- Observed on the LRC cluster after resolving https://tracker.ceph.com/issues/58460...
- 12:54 PM Bug #58435: LRC cluster : Over 1000 PGs not deep-scrubbed in time
- Hey Prashant I don't have cephadm on ivan01
root@ivan01:~# root@ivan01:~# cephadm shell
Command 'cephadm' not... - 04:07 AM sepia Support #58437: Sepia Lab Access Request
- These are replacement credentials. The old one isn't needed as I had to regenerate the key.
01/18/2023
- 07:17 PM Bug #58497 (New): teuthology.log files grow to ~1GB due to traceback repeatedly printing
- /a/yuriw-2023-01-12_20:11:41-rados-main-distro-default-smithi/7138859 - 1.1GB
/a/yuriw-2023-01-12_20:11:41-rados-mai... - 05:20 PM sepia Support #58493 (In Progress): Sepia Lab Access Request
- 1) Do you just need VPN access or will you also be running teuthology jobs?
VPN Access only
2) Desired Username... - 04:16 PM sepia Bug #58492 (Resolved): Error reimaging machines: Fog has no smithi_centos_9.stream image.
- i'm trying to add centos 9 stream testing to ceph's qa suite in https://github.com/ceph/ceph/pull/49443
all centos... - 05:28 AM Bug #58483 (In Progress): Unable to access node mero010.ceph.redhat.com
- Unable to access node mero010.ceph.redhat.com which was added as osd, working fine but node went on offline, then una...
01/17/2023
- 10:33 PM Bug #57754 (Closed): test_envlibrados_for_rocksdb.sh: update-alternatives: error: alternative pat...
- "Closed" since we merged a workaround rather than a fix.
- 06:08 PM Bug #58435: LRC cluster : Over 1000 PGs not deep-scrubbed in time
- From ceph pg dump output :
* PG size for largest pool on the LRC cluster
Avg PG size for pool 119 is around 110 G... - 06:00 PM Bug #57731: Problem: package container-selinux conflicts with udica < 0.2.6-1 provided by udica-0...
- @Adam Kraitman the issue is still there:
/a/yuriw-2023-01-12_20:11:41-rados-main-distro-default-smithi/7138810
01/16/2023
- 12:52 PM sepia Support #58437 (In Progress): Sepia Lab Access Request
- Hey Pritha Srivastava Are these new/additional or replacement credentials?
01/15/2023
- 10:19 AM Bug #58435: LRC cluster : Over 1000 PGs not deep-scrubbed in time
- ceph health detail
HEALTH_WARN 1 clients failing to respond to capability release; 1 clients failing to advance olde... - 10:18 AM Bug #58435 (In Progress): LRC cluster : Over 1000 PGs not deep-scrubbed in time
- ceph -s
cluster:
id: 28f7427e-5558-4ffd-ae1a-51ec3042759a
health: HEALTH_WARN
1 clients...
01/14/2023
- 12:50 AM sepia Tasks #58459 (Resolved): Sepia Lab Access Request
- Key added and verified on teuthology. Add to gibba in progress.
OVPN auth added and verified. PR https://github.... - 12:37 AM Bug #58460: LRC cluster: cluster is in HEALTH_ERR
- Adam and I discussed this issue over g-chat last week. The LRC cluster is now in the healthy state.
Documenting s... - 12:21 AM Bug #58460: LRC cluster: cluster is in HEALTH_ERR
- osd.137 util was over 97% :...
- 12:09 AM Bug #58460 (New): LRC cluster: cluster is in HEALTH_ERR
- - cluster status : ...
01/13/2023
- 11:50 PM sepia Tasks #58459 (Fix Under Review): Sepia Lab Access Request
- https://github.com/ceph/keys/pull/377
- 11:39 PM sepia Tasks #58459: Sepia Lab Access Request
- Hi Adam,
These are my replacement VPN credentials as I changed my laptop. Thanks. - 11:38 PM sepia Tasks #58459 (Resolved): Sepia Lab Access Request
- 1) Do you just need VPN access or will you also be running teuthology jobs?
Both
2) Desired Username:
pdhange
... - 10:15 PM sepia Bug #58456 (Duplicate): "Socket exception: Connection reset by peer" when trying to install linux...
- 08:07 PM sepia Bug #58456 (Duplicate): "Socket exception: Connection reset by peer" when trying to install linux...
- /a/yuriw-2023-01-13_16:43:00-rados-pacific_16.2.11_RC6.6-distro-default-smithi/7141528...
- 10:15 PM Bug #58232 (Resolved): Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporaril...
- Zack fixed nodes that were double-locked due to a second copy of paddles that was started up again, using the old met...
- 05:36 PM Bug #58232: Could not get lock /var/lib/apt/lists/lock - open (11: Resource temporarily unavailable)
- @Adam Kraitman this is still happening in the Pacific RC run.
http://pulpito.front.sepia.ceph.com/yuriw-2023-01-13... - 06:29 PM Bug #54993: ansible task failed: testnode : Install packages
- This problem seems to be transient-- a rerun usually gets rid of it.
- 04:34 PM Bug #58371 (Resolved): https://telemetry.ceph.com endpoint not available
- Hi Jan,
Adam Kraitman and Dan Mick restored the service and now the telemetry endpoints are up and running again. ... - 05:18 AM sepia Support #58437 (In Progress): Sepia Lab Access Request
- 1) Do you just need VPN access or will you also be running teuthology jobs?
I need VPN access and I will also be r...
01/12/2023
- 08:32 PM Bug #58435: LRC cluster : Over 1000 PGs not deep-scrubbed in time
- @Adam, I have assigned this tracker to you for now. Let's track the progress of PGs not deep-scrubbed issue through t...
- 08:27 PM Bug #58435 (In Progress): LRC cluster : Over 1000 PGs not deep-scrubbed in time
- This tracker is to investigate and troubleshoot over 1000 PGs not deep-scrubbed in time.
"ceph version 17.2.5 (983...
01/11/2023
- 05:20 PM sepia Bug #58382 (Resolved): centos 8 builds fail on main from cmake "Error executing java -version"
- the centos 8 x86_64 builds are succeeding again, so i'm changing status to resolved. thanks to everybody that helped ...
- 09:39 AM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- I don't understand exactly why it was failing, but I managed to update all the Centos8 nodes to have v352 packages by...
- 04:40 AM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- https://github.com/ceph/ceph/pull/49711 disables cephfs_java for the time being. The only user is the cephfs jni bin...
01/10/2023
- 04:16 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- Samuel Just wrote:
> Are those machines running centos 8 or centos stream 8? Presumably the latter?
centos stream 8 - 02:16 AM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- Are those machines running centos 8 or centos stream 8? Presumably the latter?
- 01:16 AM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- Looks like ceph.spec.in specifies java-devel jpackage-utils and sharutils. java-devel is presumably pulling java-1.8...
01/09/2023
- 11:51 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- Latest developments:
When cmake tries to execute `java -version` with the latest java version, we get this error:
...
01/06/2023
- 10:13 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- Got a bit further to understanding what's wrong.
With this change to ceph.spec.in:... - 05:14 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- This line in the build script used to install dependenies (https://github.com/ceph/ceph-build/blob/main/scripts/build...
- 06:20 PM sepia Support #58277: Sepia Lab Access Request
- Hi Adam,
As per the steps mentioned "here":https://docs.ceph.com/en/latest/dev/developer_guide/testing_integration_t...
01/05/2023
- 11:23 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- The symlink is created right after package install:...
- 10:38 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- I reset the symlink in my local environment with `ln -s /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-2.el8_5.x86_64 ...
- 10:29 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- I was able to reproduce this locally in a centos 8 container.
I simulated the situation manually by first installi... - 06:52 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- -DJAVA_HOME is set to "/usr/lib/jvm/java-1.8.0-openjdk", which is where the installed java version should be.
Acco... - 04:49 PM Bug #58390 (New): Account request for Magna002- Please need access to magna002 as i could not tak...
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMXdwoU3lxsYDiexbcgLIzXKDHigk3JWvZpfOaZSVmckycUyOIjzj+WYRO7PgCej4euFrAhiAeLtAOy...
- 02:17 PM sepia Support #58277: Sepia Lab Access Request
- Thanks Adam.
We were able to connect to vpn and able to ssh to test nodes. - 06:47 AM Bug #58384 (New): Octo_lab-Magna002 machine access for Teuthology
- Hi,
Please add attached ssh_key of my laptop to access "Octo lab - Magna002 machine".
Username: vdas
Email: vd...
01/04/2023
- 09:27 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- This seems to be the script that installs the dependencies based on distro, when a branch is pushed to shaman: https:...
- 08:26 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- The difference is that the successful builds were installing java v1.8.0.322, while the newer builds have installed j...
- 08:21 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- This is what the output normally looks like, when cmake is successful:...
- 08:19 PM sepia Bug #58382: centos 8 builds fail on main from cmake "Error executing java -version"
- This build from 12 days ago was the last one that was still okay.
https://shaman.ceph.com/builds/ceph/main/461ac2c30... - 07:59 PM sepia Bug #58382 (Resolved): centos 8 builds fail on main from cmake "Error executing java -version"
- https://shaman.ceph.com/builds/ceph/main/4d8388a3d8bbb62d5d3678728508a4487fc1e94c/default/328335/...
01/03/2023
- 08:03 PM Bug #58371: https://telemetry.ceph.com endpoint not available
- I restored the DB and the public dashboards (https://telemetry-public.ceph.com/).
Adam Kraitman and Dan Mick are w...
01/02/2023
- 02:11 PM sepia Support #58372 (In Progress): Sepia Lab Access Request
- 1) Do you just need VPN access or will you also be running teuthology jobs?
Both. I will also be running teuthology ...
12/30/2022
- 07:07 PM Bug #58371: https://telemetry.ceph.com endpoint not available
- Hi Jan,
Many thanks for reporting this. We are working on fixing it.
Thanks,
Yaarit - 01:40 PM Bug #58371 (Resolved): https://telemetry.ceph.com endpoint not available
- hello, it looks like telemetry submission endpoint is not available
```
2022-12-30T08:24:18.412+0000 7f0bd0cbb700...
12/27/2022
- 01:06 PM Support #58363 (In Progress): Octo lab - Magna002 machine access
- I am running it now you should have access in a few hours
- 10:08 AM Support #58363 (In Progress): Octo lab - Magna002 machine access
- Requesting access to Octo lab- Magna002
ssh-key:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNapen2HQlMD/PSHUB3bXNtG...
Also available in: Atom