Project

General

Profile

Activity

From 04/16/2020 to 05/15/2020

05/15/2020

10:00 PM Bug #45570 (Fix Under Review): Make "Failed to power on" fatal
http://qa-proxy.ceph.com/teuthology/aemerson-2020-05-15_18:43:20-rados-wip-objection-triumphant-distro-basic-smithi/5... David Galloway
08:34 PM Bug #45512 (In Progress): Jobs are running on the same machine at the same time
This is still happening :(
http://pulpito.ceph.com/nodes/smithi040.front.sepia.ceph.com/
5058215 started at 202...
David Galloway
04:13 PM Feature #45569 (Resolved): Show 'up' status in `teuthology-lock --brief` output
e.g, Instead of... David Galloway
04:43 AM Bug #45559 (Rejected): teuthology seems to ignore py2 teuthology_branch directive in yaml file
Josh reminded me I need to check out the branch locally since the teuthology command line is based on the local copy. Brad Hubbard
03:01 AM Bug #45559 (Rejected): teuthology seems to ignore py2 teuthology_branch directive in yaml file
http://pulpito.ceph.com/bhubbard-2020-05-15_01:07:37-rados-wip-badone-testing-mimic-distro-basic-smithi/5055147/
htt...
Brad Hubbard

05/14/2020

10:25 PM Bug #45341 (Fix Under Review): Have provision.fog wait for rc.local to stop running
https://github.com/ceph/teuthology/pull/1480 Zack Cerza
09:04 PM Bug #45341: Have provision.fog wait for rc.local to stop running
Zack Cerza wrote:
> Would you mind making the fog/ansible change first, and then I can whip up a teuthology branch t...
David Galloway
07:09 PM Bug #45341: Have provision.fog wait for rc.local to stop running
Right, of course, that makes sense. I'd propose using @fog.sentinel_file@ as the configuration item just to be slight... Zack Cerza
07:48 PM Bug #45438: teuthology/orchestra/connection: connection retry misses some exceptions
Another run pair which approve this:
http://pulpito.ceph.com/kyr-2020-05-14_18:54:28-multimds-wip-yuri-testing-202...
Kyrylo Shatskyy
07:02 PM Bug #45438: teuthology/orchestra/connection: connection retry misses some exceptions
I've tried to reproduce the error.
Running the job against py2 teuthology it produces failure
http://pulpito.ce...
Kyrylo Shatskyy
06:57 PM Bug #45556 (Closed): ConnectionLostError: SSH connection to smithixxx was lost
... Neha Ojha
06:14 PM Bug #45543: powercycle: NoValidConnectionsError: [Errno None] Unable to connect to port x
This affects master as well.
http://pulpito.ceph.com/?suite=powercycle&branch=master
First instance seen in: ht...
Neha Ojha
06:04 PM Bug #45543: powercycle: NoValidConnectionsError: [Errno None] Unable to connect to port x
Neha Ojha wrote:
> Based on http://pulpito.ceph.com/?suite=powercycle&branch=octopus
>
> First instance seen in: ...
Neha Ojha
05:58 PM Bug #45543: powercycle: NoValidConnectionsError: [Errno None] Unable to connect to port x
Based on http://pulpito.ceph.com/?suite=powercycle&branch=octopus
First instance seen in: /a/teuthology-2020-04-11...
Neha Ojha

05/13/2020

10:04 PM Bug #45438: teuthology/orchestra/connection: connection retry misses some exceptions
Is this the same thing? https://tracker.ceph.com/issues/45543 David Galloway
03:52 PM Bug #45438 (Resolved): teuthology/orchestra/connection: connection retry misses some exceptions
Kefu Chai
09:55 PM Bug #45543 (New): powercycle: NoValidConnectionsError: [Errno None] Unable to connect to port x
http://pulpito.ceph.com/yuriw-2020-05-13_14:51:27-powercycle-wip-yuri-octopus_15.2.2_RC0-distro-basic-smithi/
Look...
David Galloway
06:00 AM Bug #45255 (Resolved): Teuthology seems to timeout too soon after reboot and downstream tests fail
hi Georgios, i am closing this issue as that offending commit was reverted. please reopen it if this issue surfaces a... Kefu Chai

05/12/2020

05:19 PM Bug #43291: teuthology run gets stuck with first_in_suite or last_in_suite jobs in queued state
Probable fix https://github.com/ceph/teuthology/pull/1472 Kyrylo Shatskyy
11:53 AM Bug #43291: teuthology run gets stuck with first_in_suite or last_in_suite jobs in queued state
Another possible fix is to do NOT add this kind of job to paddles at all so there is no need to delete them, another ... Kyrylo Shatskyy
05:18 PM Bug #45520: The --first-in-suite and --last-in-suite jobs are scheduled too many times with --num
Probable fix https://github.com/ceph/teuthology/pull/1473 Kyrylo Shatskyy
05:15 PM Bug #45520 (Resolved): The --first-in-suite and --last-in-suite jobs are scheduled too many times...
When teuthology-suite is used with --num N argument then
the last_in_suite and first_in_suite special purpose tasks
...
Kyrylo Shatskyy
02:00 PM Bug #45512 (Resolved): Jobs are running on the same machine at the same time
Kefu Chai
01:15 PM Bug #45512: Jobs are running on the same machine at the same time
https://github.com/ceph/teuthology/pull/1471 Kefu Chai
12:54 PM Bug #45512: Jobs are running on the same machine at the same time
Another instance: smithi146
http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-12_02:30:02-upgrade:mimic-x-nau...
David Galloway
11:49 AM Feature #43043 (Resolved): Add support of 'pelagos' provisioning system
Kyrylo Shatskyy
11:49 AM Feature #43043: Add support of 'pelagos' provisioning system
Roman I guess we can close this ticket Kyrylo Shatskyy

05/11/2020

09:00 PM Bug #45512: Jobs are running on the same machine at the same time
I don't have the answer yet, but here's some information from the paddles server:
@
ubuntu@pulpito:~$ grep -a smi...
Zack Cerza
08:15 PM Bug #45512: Jobs are running on the same machine at the same time
Related?... David Galloway
08:03 PM Bug #45512 (Resolved): Jobs are running on the same machine at the same time
http://qa-proxy.ceph.com/teuthology/teuthology-2020-05-11_02:01:02-rbd-master-distro-basic-smithi/5043278/teuthology.... David Galloway
07:58 PM Bug #45511: Huge ugly traceback when killing jobs
Not sure if this is related?... David Galloway
07:57 PM Bug #45511 (New): Huge ugly traceback when killing jobs
... David Galloway

05/08/2020

12:07 PM Bug #45255 (Fix Under Review): Teuthology seems to timeout too soon after reboot and downstream t...
reverted in https://github.com/ceph/teuthology/pull/1469 Kefu Chai
07:43 AM Bug #45442 (New): ubuntu 20.02: Hang on: "The following packages will be REMOVED:"
http://pulpito.ceph.com/swagner-2020-05-07_16:10:50-rados-wip-swagner2-testing-2020-05-07-1308-distro-basic-smithi/50... Sebastian Wagner

05/07/2020

10:15 PM Bug #45438 (Fix Under Review): teuthology/orchestra/connection: connection retry misses some exce...
https://github.com/ceph/teuthology/pull/1466 Patrick Donnelly
10:11 PM Bug #45438 (Closed): teuthology/orchestra/connection: connection retry misses some exceptions
... Patrick Donnelly

05/05/2020

12:58 PM Bug #45384: bootstrapping teuthology on Ubuntu 20.04 does not work
Here's the output:
teuthworker@kvm06:~$ worker_start magna 1
magna 1
Starting 1 workers for magna
teuthworker@k...
Michael Roesch
12:39 PM Bug #45384 (Fix Under Review): bootstrapping teuthology on Ubuntu 20.04 does not work
We try to get teuthology working on Ubuntu 20.04, following this guide: https://docs.ceph.com/teuthology/docs/LAB_SET... Michael Roesch

04/30/2020

12:51 PM Bug #45341: Have provision.fog wait for rc.local to stop running
Zack Cerza wrote:
> In the cloud provisioner we use this method:
> https://github.com/ceph/teuthology/blob/master/t...
David Galloway

04/29/2020

10:16 PM Bug #45341: Have provision.fog wait for rc.local to stop running
In the cloud provisioner we use this method:
https://github.com/ceph/teuthology/blob/master/teuthology/provision/clo...
Zack Cerza
08:50 PM Bug #45341 (Resolved): Have provision.fog wait for rc.local to stop running
I _think_ what is occasionally happening is a job proceeds as soon as teuthology can SSH to the host. However, rc.lo... David Galloway
09:36 PM Bug #45297 (Resolved): sudo docker kill -s Signals.SIGHUP osd fails with 'Error: Invalid signal: ...
Thanks Sebastian Brad Hubbard
11:48 AM Bug #45297 (Fix Under Review): sudo docker kill -s Signals.SIGHUP osd fails with 'Error: Invalid ...
Sebastian Wagner
11:48 AM Bug #45297: sudo docker kill -s Signals.SIGHUP osd fails with 'Error: Invalid signal: Signals.SIG...
https://github.com/ceph/teuthology/pull/1462 Sebastian Wagner
11:36 AM Bug #45297: sudo docker kill -s Signals.SIGHUP osd fails with 'Error: Invalid signal: Signals.SIG...
https://github.com/ceph/teuthology/blob/6cf405938efbe617c2316e93efd87b7984500174/teuthology/orchestra/daemon/cephadmu... Sebastian Wagner

04/28/2020

07:33 PM Bug #45297: sudo docker kill -s Signals.SIGHUP osd fails with 'Error: Invalid signal: Signals.SIG...
Looks like something's being escaped when it shouldn't? (Signals.SIGHUP has got to be a Python variable that needs to... Greg Farnum
01:01 AM Bug #45297 (Resolved): sudo docker kill -s Signals.SIGHUP osd fails with 'Error: Invalid signal: ...
/a/teuthology-2020-04-26_07:01:02-rados-master-distro-basic-smithi/4985894... Brad Hubbard

04/24/2020

06:07 PM Bug #35951 (Duplicate): Recently merged "$" feature broke --filter and --rerun
Over at #45119 I just learned that "--rerun" will be fixed by (or as a side effect of?) the migration to Python 3. Nathan Cutler
12:44 PM Bug #45255: Teuthology seems to timeout too soon after reboot and downstream tests fail
Attatching the log files so they don't get deleted Georgios Kyratsas
10:47 AM Bug #45255 (Resolved): Teuthology seems to timeout too soon after reboot and downstream tests fail
I noticed that PR https://github.com/ceph/teuthology/pull/1426 introduced a regression on downstream tests that inclu... Georgios Kyratsas

04/22/2020

08:13 PM Bug #45119 (Won't Fix): problems rerunning failed jobs
Josh Durgin
07:24 PM Bug #45119: problems rerunning failed jobs
Josh Durgin wrote:
> The root cause is the random algorithm changing: https://bugs.python.org/issue27742
>
> This...
Kyrylo Shatskyy
07:26 PM Bug #45192 (New): teuthology through an exception while trying to collect coredump with missing ....

Log excerpt:...
Kyrylo Shatskyy

04/20/2020

05:11 AM Bug #45132 (Resolved): Can not create teuthology venv with the bootstrap script and python 3.8
Thomas Bechtold

04/19/2020

06:29 AM Bug #45132 (In Progress): Can not create teuthology venv with the bootstrap script and python 3.8
PR for fixing this is: https://github.com/ceph/teuthology/pull/1451 Thomas Bechtold

04/17/2020

08:23 PM Bug #45119: problems rerunning failed jobs
The root cause is the random algorithm changing: https://bugs.python.org/issue27742
This is clear from examining a...
Josh Durgin
06:20 PM Bug #45119: problems rerunning failed jobs
Bisected this - it was working before the py3 conversion:
16ccba3ee4b6be7f17e9c38eeaa7e4eb5640ffe7 is the first ba...
Josh Durgin
03:12 PM Bug #45119: problems rerunning failed jobs
i updated #35951 with my findings. but it's still a mystery to me. no clues so far. Kefu Chai
03:05 PM Bug #45119: problems rerunning failed jobs
update https://tracker.ceph.com/issues/35951#note-6 Yuri Weinstein
03:07 PM Bug #45132 (Resolved): Can not create teuthology venv with the bootstrap script and python 3.8
I think we need to update cffi to a version that works with python 3.8 (cffi >= 1.12.3 might work according to https:... Thomas Bechtold
04:31 AM Bug #35951: Recently merged "$" feature broke --filter and --rerun
https://github.com/ceph/teuthology/pull/1198 should be able to address this issue. the idea was to persist the seed a... Kefu Chai

04/16/2020

10:23 PM Bug #45119: problems rerunning failed jobs
It could be been masked and seems to be related to using `subset`
Main point it is not right and not expected beha...
Yuri Weinstein
10:10 PM Bug #45119: problems rerunning failed jobs
Yuri, are you sure this is a new issue?
I think it is an old bug related to '$' magic, since it is generating rand...
Kyrylo Shatskyy
09:53 PM Bug #45119: problems rerunning failed jobs
Yuri, okay, I've got where it from:
http://qa-proxy.ceph.com/teuthology/yuriw-2020-04-15_00:09:40-kcephfs-wip-yuri-t...
Kyrylo Shatskyy
09:43 PM Bug #45119: problems rerunning failed jobs
I guess subset things comes from results.log
Yuri can you attach it to the bug, I hope you haven't removed it. Or ju...
Kyrylo Shatskyy
07:37 PM Bug #45119 (Won't Fix): problems rerunning failed jobs
consider this use case:
Run http://pulpito.ceph.com/yuriw-2020-04-15_00:09:40-kcephfs-wip-yuri-testing-2020-04-14-...
Yuri Weinstein
05:07 PM Bug #35951: Recently merged "$" feature broke --filter and --rerun
I looked at this again today, and I believe the bug is still present. What happens is: jobs with the "magic" $ have a... Nathan Cutler
02:30 PM Feature #40824: Add --no-fog to `teuthology-lock`
Also, in the light of lame locking in teuthology, it would be cool to isolate locking from any kind of provisioning. Kyrylo Shatskyy
02:29 PM Feature #40824: Add --no-fog to `teuthology-lock`
Since we could have different ways of provisioning a node, there make sense to work out something smarter rather than... Kyrylo Shatskyy
02:05 PM Bug #45105 (Resolved): teuthology-lock --summary - is broken
Kefu Chai
01:43 PM Bug #45105: teuthology-lock --summary - is broken
Corresponding fix https://github.com/ceph/teuthology/pull/1449 Kyrylo Shatskyy
01:34 PM Bug #45105: teuthology-lock --summary - is broken
In py2 the 'None' type is converted to 'None' str in many functions by default, that's not a case in py3 anymore.
S...
Kyrylo Shatskyy
01:03 PM Bug #45105: teuthology-lock --summary - is broken
Yuri Weinstein wrote:
> @kyr can you take a look pls ?
Thanks for reporting this, the issue is reproducible to me...
Kyrylo Shatskyy
 

Also available in: Atom