Project

General

Profile

Activity

From 02/01/2015 to 03/02/2015

03/02/2015

07:39 PM Feature #10989 (New): Optionally upgrade ceph libraries but not test programs on client
Oops, not pluralized. 'ceph-test' is the name of the package containing many of ceph's functional tests, including th... Josh Durgin
07:32 PM Feature #10989 (Need More Info): Optionally upgrade ceph libraries but not test programs on client
What/where are 'ceph-tests' ? Zack Cerza
07:28 PM Feature #10989: Optionally upgrade ceph libraries but not test programs on client
Related suite - upgrade/client-upgrade Yuri Weinstein
07:21 PM Feature #10989 (Resolved): Optionally upgrade ceph libraries but not test programs on client
In the client upgrade tests (where the cluster is e.g. firefly and only the client is upgraded to master), we could g... Josh Durgin
07:32 PM Bug #10600 (Resolved): PATH issues on RHEL7 nodes?
This is on hammer, firefly, dumpling now. Greg Farnum
04:53 PM Feature #10945: Enable teuthology to re-run only failed jobs
Also:
http://paddles.front.sepia.ceph.com/runs/teuthology-2015-03-01_23:18:01-multimds-hammer-testing-basic-multi/jo...
Zack Cerza
03:16 PM Feature #10945: Enable teuthology to re-run only failed jobs
Nice use of --filter, Loic. I'd think we could probably make a simple call to paddles, get the jobs that have failed... Andrew Schoen

02/28/2015

03:09 PM Feature #10945: Enable teuthology to re-run only failed jobs
The simpler way is to use the --filter argument of teuthology-suite with the value of the description: field found in... Loïc Dachary

02/27/2015

07:38 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
I was able to get mod_proxy_fcgi to work with tcp on rhel 7. The apache configs in the above comments were correct, ... Andrew Schoen
05:22 PM Feature #10963 (Rejected): Add v0.87.1.8 to giant stable suites
I don't see in /upgrade/giant we even use any giant dotted release versions Yuri Weinstein

02/26/2015

10:36 PM Bug #10918 (Resolved): teuthology-suite --subset mangles roles?
Just a flaky yaml file. Samuel Just
05:43 PM Feature #10963 (Rejected): Add v0.87.1.8 to giant stable suites
Yuri Weinstein
05:10 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
I just tried adding those two different ways in teuthology and got a 500 back from apache when running the s3 tests. ... Andrew Schoen

02/25/2015

09:55 PM Bug #10918: teuthology-suite --subset mangles roles?
on ubuntu@teuthology:/var/lib/teuthworker/archive/sage-2015-02-25_11:39:12-rados-hammer-distro-basic-multi/778311
wh...
Sage Weil
07:16 PM Revision e29a49d2 (teuthology): Merge pull request #447 from ceph/upgrade-issue
Do not specify a version when upgrading ceph on rpm-based systems. Zack Cerza
06:54 PM Revision ce3f4a89 (teuthology): Do not specify a version when upgrading ceph on rpm-based systems.
Because of the split of ceph-devel, upgrading ceph-devel with an
explicit version does not allow check_obsoletes to w...
Andrew Schoen
06:02 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
radosgw is not using tcp for fastcgi here. Need to update ceph.conf with the following:... Yehuda Sadeh
03:48 PM Revision c786e9f6 (teuthology): Merge pull request #448 from ceph/hadoop-workunits
hadoop: support rhel and ubuntu Andrew Schoen
02:54 AM Revision 1625c987 (teuthology): hadoop: support rhel and ubuntu
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Noah Watkins

02/24/2015

10:14 PM Feature #10945 (Resolved): Enable teuthology to re-run only failed jobs
Currently there is no a simple way to do so.
It will help a lot and also will use our resources more efficiently if ...
Yuri Weinstein
09:34 PM Feature #10939 (Duplicate): capture more information about kernel crashes with kdump
Didn't see that Ilya already filed http://tracker.ceph.com/issues/10937 Josh Durgin
06:22 PM Feature #10939 (Duplicate): capture more information about kernel crashes with kdump
kdb is ok for debugging some problems, but it'd often be better if we had the full dmesg leading up to the crash. Ins... Josh Durgin
08:05 PM Revision 5b88d4e0 (teuthology): Merge pull request #427 from ceph/travisci
add Travis CI integration Zack Cerza
07:15 PM Bug #10600 (Pending Backport): PATH issues on RHEL7 nodes?
This is in master but we'll want it in our other test branches as well. Just waiting to let it get through a few runs... Greg Farnum
06:20 PM Bug #10600 (Resolved): PATH issues on RHEL7 nodes?
https://github.com/ceph/ceph-qa-suite/commit/7e5d8cb61aaa755aa1504cb70ade23b57235a584 Zack Cerza
05:07 AM Bug #10600 (Fix Under Review): PATH issues on RHEL7 nodes?
https://github.com/ceph/ceph-qa-suite/pull/341 Greg Farnum
07:14 PM Feature #10940 (New): support running commands with the same environment a user would
This comes out of #10600. While we were able to generate the necessary PATH info in the workunit.py task to solve tha... Greg Farnum
06:56 PM Bug #10816 (Resolved): "mds e5 Missing health data for MDS 4117" endless loop in upgrade:firefly-...
Don't see this in latest run so resolving for now
http://pulpito.front.sepia.ceph.com/teuthology-2015-02-22_17:13:...
Yuri Weinstein
06:42 PM Bug #10500 (Resolved): "HEALTH_WARN 18 pgs peering" in upgrade:firefly-firefly-distro-basic-multi...
Looked good on the latest runs Yuri Weinstein
06:15 PM Bug #10893 (Resolved): Install failing on RHEL-family machines (due to mixed package sources?)
Zack Cerza
05:29 PM Feature #10680: qemu task relies on virtio-9p, which is not available in centos/rhel
https://github.com/ceph/ceph-qa-suite/pull/343 Andrew Schoen
05:28 PM Feature #10937 (New): enable and collect kernel crash dumps
Enable kdump instead of kdb, figure out where to store dumps and kernels for later inspection.
(kdump may at times b...
Ilya Dryomov
04:55 PM Bug #10936 (Resolved): No package ceph-devel-0.92 during upgrades
This is the same issue as #10926. Except this happens during upgrades instead of installing.... Andrew Schoen
04:03 PM Feature #10704 (Resolved): Implement "expand test coverage for the upgrade suites" for giant-x suite
Yuri Weinstein

02/23/2015

10:23 PM Bug #10926 (Resolved): No package ceph-devel-0.92 available.
https://github.com/ceph/teuthology/commit/7d002de0da388df6f2dbf0d2c03aaca2484bcb7a Andrew Schoen
09:31 PM Bug #10926 (Fix Under Review): No package ceph-devel-0.92 available.
https://github.com/ceph/teuthology/pull/446
It tested this out in octo with rhel 7 nodes and the install task pass...
Andrew Schoen
08:44 PM Bug #10926: No package ceph-devel-0.92 available.
Sandon, I can take the teuthology work on this if you'd like. Thanks for looking into this! Andrew Schoen
07:24 PM Bug #10926: No package ceph-devel-0.92 available.
Sounds good. Thanks for looking into this! Ken Dreyer
06:57 PM Bug #10926: No package ceph-devel-0.92 available.
I actually don't think we need the version appended anymore. I think the change to the priorities config file to chec... Sandon Van Ness
06:42 PM Bug #10926: No package ceph-devel-0.92 available.
If it's done to override EPEL, I wonder if that technique of appending the version number was done long before we sta... Ken Dreyer
06:31 PM Bug #10926: No package ceph-devel-0.92 available.
Ok so appending the version is what is causing the problem. I think we were doing that to insure it was getting the r... Sandon Van Ness
06:25 PM Bug #10926: No package ceph-devel-0.92 available.
From playing around manually with this, "yum install ceph-devel-0.92" fails, whereas "yum install ceph-devel" succeeds. Ken Dreyer
06:14 PM Bug #10926: No package ceph-devel-0.92 available.
Talking to Ken it looks like the obsoletes should handle this. I will definitely do some manual testing as we want th... Sandon Van Ness
06:08 PM Bug #10926: No package ceph-devel-0.92 available.
Not sure how this is handled since so many packages are obsoleting it if via yum you are telling it to install the ob... Sandon Van Ness
06:04 PM Bug #10926: No package ceph-devel-0.92 available.
The -devel package was removed from this commit last week:
https://github.com/ceph/ceph/commit/c341c52984ca7c7b814...
Sandon Van Ness
10:14 PM Revision 7d002de0 (teuthology): Merge pull request #446 from ceph/wip-10926-andrew
Do not install packages with a specified version, fixes #10926 Sage Weil
09:42 PM Revision c8a20b10 (teuthology): Do not install packages with a specified version, fixes #10926
Now that we've got yum setup to check for obsoletes we don't need to
specify a version. Specifying the version actual...
Andrew Schoen
07:13 PM Revision 45c89952 (teuthology): Don't specify ceph package version on yum command line.
It messes with obseleted packages. I think we did this in the first
place as an incorrect workaround. Since setting t...
Sandon Van Ness
06:17 PM Revision 029cbed2 (teuthology): Merge pull request #445 from ceph/hadoop
hadoop: clean-up and sigkill delay fix Andrew Schoen
01:41 AM Revision af22f59e (teuthology): hadoop: use dict-to-conf converter
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Noah Watkins
01:41 AM Revision 62c1d8c9 (teuthology): hadoop: add yarn sigkill delay hack
This instructs yarn to wait more time (10sec) than the default (250ms)
before killing containers to give ceph clients...
Noah Watkins

02/22/2015

02:58 AM Bug #10926 (Resolved): No package ceph-devel-0.92 available.
... Sage Weil

02/20/2015

05:56 PM Revision c2c95876 (teuthology): install: use newly-split ceph-devel package names
The ceph-devel package was split up on the hammer branch in
ceph.git's c341c52984ca7c7b814b1adc269b7fb8d4c57a29. Upda...
Ken Dreyer

02/19/2015

06:51 PM Feature #10704 (Fix Under Review): Implement "expand test coverage for the upgrade suites" for gi...
Yuri Weinstein
06:51 PM Feature #10704: Implement "expand test coverage for the upgrade suites" for giant-x suite
Here is the run - http://pulpito.front.sepia.ceph.com/teuthology-2015-02-18_17:05:01-upgrade:giant-x-hammer-distro-ba... Yuri Weinstein
06:47 PM Bug #10918: teuthology-suite --subset mangles roles?
... Sage Weil
06:42 PM Bug #10918 (Resolved): teuthology-suite --subset mangles roles?
... Sage Weil
06:07 PM Bug #10600: PATH issues on RHEL7 nodes?
And I guess I'm stuck shepherding this even if I can't solve it on my own right now. Greg Farnum
06:06 PM Bug #10600: PATH issues on RHEL7 nodes?
Forgot to update this yesterday: ssh sets up the environment differently depending on whether it's a login shell (or ... Greg Farnum

02/18/2015

11:08 PM Bug #10600 (In Progress): PATH issues on RHEL7 nodes?
gregf@magna002:/a/gregf-2015-02-17_14:18:29-fs-wip-firefly-flock---basic-magna/49584/teuthology.log:... Greg Farnum
09:18 PM Feature #10736 (Resolved): teuthology testing task
This is complete. Schedule the 'teuthology' suite to run these tests. The suite locks, does some provisioning and ru... Andrew Schoen
07:36 PM Feature #10910 (Resolved): Allow Remote.get_file() to use original filename locally
https://github.com/ceph/teuthology/commit/9160768ccfad32f7da5ef17e41204ae90058f3fe Zack Cerza
06:50 PM Feature #10910 (Fix Under Review): Allow Remote.get_file() to use original filename locally
https://github.com/ceph/teuthology/pull/443 Zack Cerza
04:55 PM Feature #10910 (Resolved): Allow Remote.get_file() to use original filename locally
Currently @Remote.get_file()@ calls @tempfile.mkstemp()@ to generate a local filename. I think it should allow using ... Zack Cerza
06:55 PM Revision 9160768c (teuthology): Merge pull request #443 from ceph/wip-10910
Allow Remote.get_file() to use original filename Andrew Schoen
06:48 PM Revision f10446d8 (teuthology): Allow Remote.get_file() to use original filename
If dest_dir != '/tmp', attempt to use the original filename.
Signed-off-by: Zack Cerza <zack@redhat.com>
Zack Cerza
06:48 PM Revision 24f40507 (teuthology): Update out-of-date integration test
Signed-off-by: Zack Cerza <zack@redhat.com> Zack Cerza
06:37 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
I've been testing this with rhel 7.0 nodes in octo. When I attempt to use mod_proxy_fcgi with TCP and run s3 tests a... Andrew Schoen

02/17/2015

09:21 PM Feature #10551 (Need More Info): test RGW with mod_proxy_fcgi instead of mod_fastcgi
I've got the machinery in place that will now use mod_proxy_fcgi by default, but allow the usage of mod_fastcgi by ad... Andrew Schoen
04:52 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
A nice thread on ceph-devel about using mod_proxy_fcgi with rgw.
http://article.gmane.org/gmane.comp.file-systems....
Andrew Schoen
04:15 PM Feature #10551 (In Progress): test RGW with mod_proxy_fcgi instead of mod_fastcgi
Andrew Schoen
09:07 PM Bug #10856 (Resolved): teuthology will happily destroy a VM it doesn't own
Zack Cerza
08:58 PM Bug #7364 (Resolved): apt fails to update from ceph apt mirror
https://github.com/ceph/teuthology/commit/27e31a026b81b4f93d56c2af392b71d208cc9c91 Zack Cerza
08:35 PM Bug #7364: apt fails to update from ceph apt mirror
Testing this before I ask for review:
http://pulpito.ceph.redhat.com/zack-2015-02-17_15:33:15-teuthology-master---ba...
Zack Cerza
08:30 PM Bug #7364 (Fix Under Review): apt fails to update from ceph apt mirror
https://github.com/ceph/teuthology/pull/442 Zack Cerza
08:25 PM Bug #7364 (In Progress): apt fails to update from ceph apt mirror
This has been happening more often. I'm going to implement a workaround causing @apt-get update@ failures to not fail... Zack Cerza
08:57 PM Revision 27e31a02 (teuthology): Merge pull request #442 from ceph/wip-7364
Don't fail just because apt-get update does Andrew Schoen
08:26 PM Revision f3954768 (teuthology): Don't fail just because apt-get update does
If we fail later anyway, fine. If not, yay!
Signed-off-by: Zack Cerza <zack@redhat.com>
Zack Cerza
06:23 PM Feature #9759 (Resolved): Teuthology suite needed for Calamari testing
The suite exists. I think that if we find we are missing anything, we should open specific tickets. So I am closing... Anonymous
05:37 PM Revision fa5ffbae (teuthology): Don't fail if task runs twice
This was happening because _yum_unset_check_obsoletes() was failing
Signed-off-by: Zack Cerza <zack@redhat.com>
Zack Cerza

02/16/2015

09:32 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
In order to keep this Teuthology implementation issue separate from the packaging work that needs to get merged to ce... Ken Dreyer
08:18 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
We think the teuthology change will be enough to fix this; the packaging work is ongoing Zack Cerza
08:17 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
teuthology workaround merged:
https://github.com/ceph/teuthology/commit/5634406557114ef4e511e1f2aac2099cad06971f
Zack Cerza
08:11 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
https://github.com/ceph/teuthology/pull/440 Zack Cerza
07:33 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
I've filed https://bugzilla.redhat.com/1193182 "ceph has unversioned obsoletes", in order to get this fixed downstrea... Ken Dreyer
06:35 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
We got together to discuss this, and here's what we decided:
Since this is blocking testing and therefore merges, ...
Zack Cerza
06:07 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
Note that if we add @check_obsoletes@ to @priorities.conf@, I'm pretty sure that's not something we'd need to revert ... Ken Dreyer
05:07 PM Bug #10893: Install failing on RHEL-family machines (due to mixed package sources?)
The downstream packages in Fedora and EPEL have split the main "python-ceph" RPM into separate modules: "python-rados... Ken Dreyer
04:53 PM Bug #10893 (Resolved): Install failing on RHEL-family machines (due to mixed package sources?)
... Greg Farnum
08:15 PM Revision 56344065 (teuthology): Merge pull request #440 from ceph/wip-10893
Set check_obsoletes = 1 in yum's priorities.conf Andrew Schoen
08:10 PM Revision c1a6f903 (teuthology): Set check_obsoletes = 1 in yum's priorities.conf
Signed-off-by: Zack Cerza <zack@redhat.com> Zack Cerza
07:56 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
We discussed this in standup today and it was decided that we'll have ceph-qa-chef install both mod_proxy_fcgi and mo... Andrew Schoen
03:07 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
It sounds like teuthology needs to support installing and configuring both mod_fastcgi and mod_proxy_fcgi. Is that r... Andrew Schoen
04:03 PM Revision 3f9d10b2 (teuthology): Merge pull request #439 from ceph/pytest-better-errors
put test failures in failure_reason; skip test_correct_os_version for debian Zack Cerza

02/14/2015

07:15 AM Bug #10600 (Need More Info): PATH issues on RHEL7 nodes?
Greg Farnum

02/13/2015

09:51 PM Bug #10878: downburst gives debian 7.1 when asking for 7.0
This is a know behavior as we discussed in standup today. There isn't even a 7.0 image and we're ok with that as lon... Andrew Schoen
05:21 PM Bug #10878 (Won't Fix): downburst gives debian 7.1 when asking for 7.0
When asking for a debian 7.0 vps machine downburst actually provisions a debian 7.1 machine.
See failure here:
...
Andrew Schoen
09:07 PM Revision 4a233609 (teuthology): Include test failures in ctx.summary['failure_reason']
I also added a new line to make reading the log nicer
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Andrew Schoen
09:07 PM Revision a52039c1 (teuthology): Make an exception for debian in tests.test_correct_os_version
This is because of a known issue where downburst gives us 7.1 when we
ask for 7.0. We're ok with this behavior for n...
Andrew Schoen
08:59 PM Bug #10879 (Resolved): teuthology-nuke --stale race condition
https://github.com/ceph/teuthology/commit/6960a01350823d0927f45888145e4f06cba7eba8 Zack Cerza
08:21 PM Bug #10879 (Fix Under Review): teuthology-nuke --stale race condition
https://github.com/ceph/teuthology/pull/438 Zack Cerza
05:51 PM Bug #10879 (Resolved): teuthology-nuke --stale race condition
The race condition is far easier to hit when vps nodes are involved.
# job enters the waiting state
# teuthology ...
Zack Cerza
08:58 PM Revision 6960a013 (teuthology): Merge pull request #438 from ceph/wip-10879
Avoid race condition in find_stale_locks() Andrew Schoen
08:49 PM Revision 48a52443 (teuthology): Avoid race condition in find_stale_locks()
Because of the way we were checking nodes against running jobs, it was
possible to falsely report nodes as stale if t...
Zack Cerza
08:31 PM Bug #10748 (Resolved): "ConnectionLostError: SSH connection to vpm" error
I believe this is resolved. Zack Cerza
05:18 PM Bug #10869 (Resolved): Workunit task should not use the same filename to store the list of workunits
Yuri Weinstein
03:36 PM Revision 35de47fc (teuthology): Merge pull request #437 from ceph/pytest-noterminalreporter
Fixes the issue of pytest failing with scheduled jobs Zack Cerza
03:23 PM Bug #9409 (In Progress): libvirt: QEMU errors operation is not valid: domain is not running -...
Thanks for the great explanation.
Okay, so I think @task.internal.lock_machines()@, @provision.create_if_vm()@ and...
Zack Cerza
09:39 AM Bug #9409: libvirt: QEMU errors operation is not valid: domain is not running -- cannot un...
I have a strong objection to disabling that. For whatever reason on rhel6/centos6 (haven't seen it on any other distr... Sandon Van Ness
03:17 AM Bug #9409 (Need More Info): libvirt: QEMU errors operation is not valid: domain is not running ...
I'm strongly leaning toward just disabling this "Re-creating guest" feature, as I've never seen it work. I have, howe... Zack Cerza
03:00 PM Revision c7094539 (teuthology): Remove the pytest default TerminalReporter in the tests task
This fixes the issue of pytest IO Error failure in a scheduled job
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Andrew Schoen
12:05 AM Bug #10856: teuthology will happily destroy a VM it doesn't own
Merge commits:
https://github.com/ceph/paddles/commit/0bfa5ef456b38e96dc1b60e33a93d67f764990d2
https://github.com/c...
Zack Cerza

02/12/2015

10:42 PM Bug #10869 (In Progress): Workunit task should not use the same filename to store the list of wor...
PR to fix this - https://github.com/ceph/ceph-qa-suite/pull/332 Yuri Weinstein
09:52 PM Bug #10869 (Resolved): Workunit task should not use the same filename to store the list of workunits
Release giant v0.87.1
Run: http://pulpito.front.sepia.ceph.com/teuthology-2015-02-11_15:40:51-upgrade:firefly-x-gi...
Yuri Weinstein
09:56 PM Bug #10821 (Resolved): failed mkfs.btrfs as part of OSD setup
This took some investigation but the cause was kind of an unexpected one. The device mapper was utilizing the devices... Sandon Van Ness
04:42 PM Bug #10821: failed mkfs.btrfs as part of OSD setup
mira059:
http://qa-proxy.ceph.com/teuthology/teuthology-2015-02-11_15:59:22-fs-giant-distro-basic-multi/751441/
htt...
Greg Farnum
09:14 PM Revision b3486612 (teuthology): If there is an error running pytest, mark job as dead
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
07:48 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
Here's the latest that I've heard:
Upstream we are switching to mod_proxy_fcgi entirely for all distros (RHEL and ...
Ken Dreyer
06:42 PM Revision 2850a532 (teuthology): Merge pull request #435 from ceph/wip-10856
Safer vps unlocking Andrew Schoen
06:21 PM Bug #10856 (Fix Under Review): teuthology will happily destroy a VM it doesn't own
This should cover the teuthology side:
https://github.com/ceph/teuthology/pull/435
Zack Cerza
04:42 AM Bug #10856: teuthology will happily destroy a VM it doesn't own
This should help stop the spread of the problem on the paddles side:
https://github.com/ceph/paddles/pull/52
I st...
Zack Cerza
01:24 AM Bug #10856 (Resolved): teuthology will happily destroy a VM it doesn't own
Zack Cerza
06:15 PM Revision 10e46a11 (teuthology): Add optional description arg to destroy_if_vm()
If it is passed and doesn't match the one received from the lock server,
don't destroy the VM.
Signed-off-by: Zack C...
Zack Cerza
06:15 PM Revision 1c4b2df1 (teuthology): Add optional description arg to unlock_one()
It gets passed to the lock server.
Signed-off-by: Zack Cerza <zack@redhat.com>
Zack Cerza
06:15 PM Revision 04dc751a (teuthology): Pass a description to unlock_one()
Signed-off-by: Zack Cerza <zack@redhat.com> Zack Cerza
06:14 PM Revision 828f462b (teuthology): Merge pull request #436 from ceph/wip-hadoop-linter
hadoop: remove parallel from import; fixes linter warning Zack Cerza
06:06 PM Revision a5ac6143 (teuthology): hadoop: remove parallel from import; fixes linter warning
Signed-off-by: Greg Farnum <gfarnum@redhat.com> Greg Farnum
04:39 PM Bug #10814: downburst doesn't work on Fedora
I removed libvirt-python from the install_requires list.
downburst bootstrapped when I did that. Then:...
Greg Farnum
04:36 PM Revision e34f1df2 (teuthology): Ensure our version of pytest_runtest_makereport is ran first
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
05:03 AM Revision db7ae119 (teuthology): Merge pull request #434 from ceph/hadoop
Hadoop 2.0 Task
This will need more updating, but pulling it in-tree gets better in-situ testing, and the hadoop tas...
Greg Farnum

02/11/2015

10:35 PM Revision d79d4d7e (teuthology): Merge pull request #432 from ceph/pytest-stdout
Make pytest capture stdout when running the tests task Zack Cerza
10:23 PM Revision ed865477 (teuthology): Make pytest capture stdout when running the tests task
This makes the logs easier to read.
Signed-off-by: Andrew Schoen <aschoen@redhat.com>
Andrew Schoen
09:44 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
We really need to know if we're going to continue supporting both modules. If we're going to continue to support mod_... Zack Cerza
09:39 PM Feature #10551 (Need More Info): test RGW with mod_proxy_fcgi instead of mod_fastcgi
Andrew Schoen
09:17 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
Ken, thanks. Those configs are helpful. FastCgiIPCDir and FastCgiExternalServer are specific to mod_fastcgi so I assu... Andrew Schoen
08:33 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
Yehuda is telling me that we will in fact need to support both modules. Here's the breakdown as I understand it now:
...
Zack Cerza
06:19 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
Here's the sample configs that Yehuda gave to John and me a while back.
Note that there's a difference here depend...
Ken Dreyer
05:52 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
We're also setting a few things in the apache.conf for mod_fastcgi that we'll need mod_proxy_fcgi equivalents for.
...
Andrew Schoen
03:29 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
We'll need to add this config to the apache template in ceph-qa-suite to enable mod_proxy and mod_proxy_fcgi:
Load...
Andrew Schoen
09:35 PM Revision 30ff46f6 (teuthology): hadoop: 2x
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Noah Watkins
09:35 PM Revision 1dd91832 (teuthology): misc: create prepend_lines helper
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Noah Watkins
09:35 PM Revision 8e0d1437 (teuthology): hadoop: easier config creation
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Noah Watkins
09:35 PM Revision 10445429 (teuthology): hadoop: support cephfs
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Noah Watkins
09:35 PM Revision 4d1032ea (teuthology): hadoop: separate ceph/hdfs config actions
Signed-off-by: Noah Watkins <noahwatkins@gmail.com> Noah Watkins
09:34 PM Feature #10680: qemu task relies on virtio-9p, which is not available in centos/rhel
There's no need to keep virtio-9p around. Using the same mechanism on each platform would simplify things a bit. Josh Durgin
09:32 PM Feature #10680 (New): qemu task relies on virtio-9p, which is not available in centos/rhel
The main other option would be samba. Whichever is easier to configure in the host and via cloudinit (where the tests... Josh Durgin
08:27 PM Feature #10680 (Need More Info): qemu task relies on virtio-9p, which is not available in centos/...
Changing to a feature so we can plan for it as such.
We'll also want some more specific requirements. Is NFS defin...
Zack Cerza
08:32 PM Bug #10825 (Resolved): task.install: Removing ceph sources lists hangs forever
https://github.com/ceph/teuthology/commit/81dbfdfbca549b7c32ba89ebf60c1c5e40cc081e Zack Cerza
08:17 PM Bug #10825 (Fix Under Review): task.install: Removing ceph sources lists hangs forever
https://github.com/ceph/teuthology/pull/431 Zack Cerza
07:27 PM Bug #10825: task.install: Removing ceph sources lists hangs forever
My branch is https://github.com/ceph/teuthology/tree/wip-10825
Test run is http://pulpito.ceph.redhat.com/zack-2015-...
Zack Cerza
08:31 PM Revision 81dbfdfb (teuthology): Merge pull request #431 from ceph/wip-10825
Smarter sources.list/.repo removal Andrew Schoen
07:39 PM Revision d56e7bff (teuthology): Don't run two apt-get updates in parallel
Signed-off-by: Zack Cerza <zack@redhat.com> Zack Cerza
07:39 PM Revision 73e67caa (teuthology): Tweak _remove_sources_list_{deb,rpm}()
Make their implementations more concise, their logging more verbose, and
avoid running apt-get update if no sources l...
Zack Cerza
06:56 PM Feature #10594 (Resolved): RHEL 7.1 support for chef
This mainly involved making the repos available so we aren't forced to just do this on octo and making sure they were... Sandon Van Ness
12:16 AM Bug #10814: downburst doesn't work on Fedora
I briefly looked at this just to see how much work it would be.
It appears like precise fedora20 does not include ...
Sandon Van Ness

02/10/2015

10:28 PM Bug #10825 (In Progress): task.install: Removing ceph sources lists hangs forever
This is being caused by this commit:
https://github.com/ceph/teuthology/commit/d71a87452f6450ba5d749630ee4ba476bce543c7
Zack Cerza
01:52 PM Bug #10825 (Resolved): task.install: Removing ceph sources lists hangs forever
http://pulpito.ceph.com/loic-2015-02-02_23:31:31-rados-giant-backports---basic-multi/736802/
was killed after >1d be...
Loïc Dachary
09:21 PM Bug #10404: thrashosds: failed to decode JSON
None of us have seen this recently; if it starts popping up again I suggest we have the author of the Thrasher class ... Zack Cerza
09:19 PM Bug #10622 (Resolved): kernel install failures on sha1 length checks
Forgot to mention I merged the debugging branch:
https://github.com/ceph/teuthology/pull/422
https://github.com/cep...
Zack Cerza
09:17 PM Bug #10777: ceph-deploy task doesnot create osd as per ctx.cluster.remotes
This does look like a bug, but it's not really problematic for us. I'd be happy to review patches, but can't justify ... Zack Cerza
09:02 PM Feature #10829 (New): better support for assertion errors
It is nice that we use plain asserts in tasks but those are a pain to debug because there is no repr/diff that would ... Alfredo Deza
07:14 PM Bug #10687 (Closed): chef failure: error downloading python-libs-2.6.6-52.el6.x86_64
Zack Cerza
07:11 PM Bug #10651 (Closed): stale pycs causing issues
Yeah, this was related to clock skew. Closing this now as we've resolved those issues for now. Andrew Schoen
06:40 PM Bug #10715 (Resolved): qemu not found on el7
Please re-open if this is still an issue Zack Cerza
06:39 PM Bug #10577 (Resolved): "Failure: ImportError" in upgrade:dumpling-firefly-x-next-distro-basic-mul...
Yuri Weinstein
05:52 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
Will we need issue #10808 for this to be considered completely resolved? I imagine so. Andrew Schoen
05:11 PM Feature #10551: test RGW with mod_proxy_fcgi instead of mod_fastcgi
Another aspect of this is getting rid of our fork of apache and using the distro provided version. We've decided it'... Andrew Schoen
04:14 PM Bug #10816: "mds e5 Missing health data for MDS 4117" endless loop in upgrade:firefly-x-hammer-di...
I can't think of a reason why erasure code would cause failure in this situation. And even if it did, I can't hink of... Loïc Dachary
03:29 PM Revision 451b2f7e (teuthology): Merge pull request #430 from tchaikov/fix-readme
README: update with teuthology-suite Andrew Schoen
07:49 AM Bug #10821 (Resolved): failed mkfs.btrfs as part of OSD setup
http://pulpito.ceph.com/teuthology-2015-02-08_23:04:01-fs-hammer-testing-basic-multi/745187/
http://pulpito.ceph.com...
Greg Farnum
06:34 AM Bug #10600: PATH issues on RHEL7 nodes?
Well, I tried reproducing this in a VPS setup and was unable to do so. So I've given in and pushed a patch to master ... Greg Farnum
01:35 AM Revision b291e40e (teuthology): README: update with teuthology-suite
* replace sample usage of deprecated schedule_suite.sh with that of
teuthology-suite
* add a pointer to `teuthology...
Kefu Chai

02/09/2015

11:27 PM Bug #10816: "mds e5 Missing health data for MDS 4117" endless loop in upgrade:firefly-x-hammer-di...
Loic - it's coming from https://github.com/ceph/ceph-qa-suite/tree/hammer/suites/upgrade/firefly-x/stress-split-erasu... Yuri Weinstein
10:55 PM Bug #10816: "mds e5 Missing health data for MDS 4117" endless loop in upgrade:firefly-x-hammer-di...
At a guess, you've upgraded the monitors and the MDS has not been upgraded yet. (Or some but not all of the monitors?... Greg Farnum
10:43 PM Bug #10816 (Resolved): "mds e5 Missing health data for MDS 4117" endless loop in upgrade:firefly-...
Run: http://pulpito.ceph.com/teuthology-2015-02-08_17:13:01-upgrade:firefly-x-hammer-distro-basic-multi/
Job: 744272...
Yuri Weinstein
10:32 PM Bug #10814 (Won't Fix): downburst doesn't work on Fedora
I was trying to get this working last week and no matter what packages I installed the bootstrap script failed on bui... Greg Farnum
10:08 PM Revision 096f6f0a (teuthology): Merge pull request #429 from ceph/tests-task-cleanup
Adds a machine_type property to orchestra.remote.Remote Zack Cerza
06:49 PM Revision d2d68524 (teuthology): Adds a machine_type property to orchestra.remote.Remote
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
12:41 PM Bug #10797 (New): ceph.py fallback fails because mkfs.ext4 does not have -f option on trusty
https://github.com/ceph/ceph-qa-suite/blob/master/tasks/ceph.py#L530
will run mkfs once more after adding the \-f fl...
Loïc Dachary
07:46 AM Documentation #8678: missing documentation for schedule_suite.sh replacement
i also posted a PR at https://github.com/ceph/teuthology/pull/430 to address this. Kefu Chai

02/06/2015

08:06 PM Feature #10781 (Rejected): upgrade:hammer-x suite
well, there is no next branch :) Tamilarasi muthamizhan
06:59 PM Feature #10781: upgrade:hammer-x suite
well, i thought it might be a good idea to add upgrade suite from hammer to x[where x is next] Tamilarasi muthamizhan
06:46 PM Feature #10781 (Resolved): upgrade:hammer-x suite
Per Tamil's request added, the idea is to run hammer against next branch.
Sage, pls comment and assign to me if yo...
Yuri Weinstein
03:05 PM Feature #10736: teuthology testing task
The teuthology work is pretty much complete, https://github.com/ceph/teuthology/pull/426
Working on the qa suite n...
Andrew Schoen
06:09 AM Bug #10777 (New): ceph-deploy task doesnot create osd as per ctx.cluster.remotes
Have observerd while creating ceph cluster using ceph-deploy task,
the way osds are associated in ctx.cluster.remote...
Shambhu Rajak
05:26 AM Bug #10600 (In Progress): PATH issues on RHEL7 nodes?
I've got a branch to get path info and will run some tests once it's available for install. Greg Farnum

02/05/2015

11:19 PM Bug #10776 (Can't reproduce): paramiko: "Exception: Remote transport is ignoring rekey requests" ...
Run: http://pulpito.ceph.com/teuthology-2015-02-03_02:35:02-smoke-master-distro-basic-multi/
Job: 738480
Logs: http...
Yuri Weinstein
10:22 PM Bug #10774 (Resolved): teuthology unlocks VMs before destroying
https://github.com/ceph/teuthology/commit/bda9cb993f372116c804ea49daefda6b816650d5 Zack Cerza
09:59 PM Bug #10774 (Fix Under Review): teuthology unlocks VMs before destroying
https://github.com/ceph/teuthology/pull/428 Zack Cerza
09:12 PM Bug #10774 (Resolved): teuthology unlocks VMs before destroying
See https://github.com/ceph/teuthology/blob/02b7d28a5eb7e797ed9e39019a2cf03a51712761/teuthology/lock.py#L440
This ...
Zack Cerza
10:02 PM Revision bda9cb99 (teuthology): Merge pull request #428 from ceph/wip-10774
Don't unlock VMs before destroying them Andrew Schoen
09:58 PM Revision f4e47e98 (teuthology): Don't destroy a vps that is owned by someone else
This only affects the unlock_one() codepath at this point.
Signed-off-by: Zack Cerza <zack@redhat.com>
Zack Cerza
09:14 PM Revision 91d59805 (teuthology): Destroy VMs before unlocking, not after
Signed-off-by: Zack Cerza <zack@redhat.com> Zack Cerza
09:02 PM Bug #10748: "ConnectionLostError: SSH connection to vpm" error
I guess I'm probably wrong about memory being the issue in this case. Wow, this is frustrating! Zack Cerza
06:36 PM Bug #10748: "ConnectionLostError: SSH connection to vpm" error
Running ceph-qa-chef manually I ran a free -m loop every 2 seconds and memory (after disk cache/buffers) got no where... Sandon Van Ness
04:47 AM Bug #10748: "ConnectionLostError: SSH connection to vpm" error
I have never seen a machine run out of memory on chef. Ram would definitely not cause the apt failures we have seen w... Sandon Van Ness
04:26 AM Bug #10748: "ConnectionLostError: SSH connection to vpm" error
I just manually locked a vps and ran chef on it. It got within 70MB of running out of RAM, with no swap, again just d... Zack Cerza
08:54 PM Revision 498c75e6 (teuthology): Merge pull request #426 from ceph/wip-10736
create a teuthology integration test task Zack Cerza
08:19 PM Revision 187803d3 (teuthology): Correctly pull the default machine_type if machine_type is not de...
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
08:19 PM Revision aaed783b (teuthology): Moved task.tests into it's own module and created tests.test_run
This sets up the basic structure for teuthology integration testing.
Any tests put in teuthology/task/tests will be a...
Andrew Schoen
08:19 PM Revision 594a3521 (teuthology): Added integration tests to ensure locking is working correctly
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
08:19 PM Revision 51114ffb (teuthology): Do not log skipped tests as failed
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
08:19 PM Revision 4f64f5f3 (teuthology): Use pytest to auto discover and run tests in task.tests
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
07:23 PM Revision 853228e4 (teuthology): install: include libcephfs-jni in .deb list as well as RPM one
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Reviewed-by: Zack Cerza <zack@redhat.com>
Greg Farnum
06:20 PM Revision 20bb34e3 (teuthology): add Travis CI integration
This will allow Travis CI to run tox to check for test failures. Ken Dreyer
05:16 PM Feature #10753 (Resolved): Use run()'s new label arg for a couple common task failures
Andrew Schoen
03:57 PM Feature #10767: simplify the ceph-deploy suite
All of these subcommands should get tested. Minimal cluster needed for all of these (whatever minimal means, I think ... Alfredo Deza
03:55 PM Feature #10767 (New): simplify the ceph-deploy suite
The current ceph-deploy suite requires 9 nodes to complete it's tests. Because of this it typically eats up a ton of... Andrew Schoen
02:00 AM Bug #10500: "HEALTH_WARN 18 pgs peering" in upgrade:firefly-firefly-distro-basic-multi run
Update - https://github.com/ceph/ceph-qa-suite/pull/323#issuecomment-72977621 Yuri Weinstein
12:57 AM Bug #10752 (Resolved): Move VPMs to static leases
This is completed. I sent out an email to the storage-eng list for people who might be effected by it (existing vms) Sandon Van Ness

02/04/2015

10:07 PM Revision c5da67b6 (teuthology): Merge pull request #425 from ceph/wip-10753
Add a label to the ceph-qa-chef run in task.internal Zack Cerza
10:07 PM Revision 967ea02c (teuthology): Add a label to the ceph-qa-chef run in task.internal
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
09:56 PM Feature #10753: Use run()'s new label arg for a couple common task failures
https://github.com/ceph/teuthology/pull/425
https://github.com/ceph/ceph-qa-suite/pull/322
Andrew Schoen
09:23 PM Feature #10753 (Resolved): Use run()'s new label arg for a couple common task failures
Following up from #10699, let's add labels to a couple of the more frequent and confusing CommandFailedErrors that we... Zack Cerza
09:44 PM Feature #10754 (Resolved): paddles: add percentage to queue_stats command
https://github.com/ceph/paddles/commit/172a7b926faa419773995d9921a50d512daa3ba2 Zack Cerza
09:38 PM Feature #10754 (Fix Under Review): paddles: add percentage to queue_stats command
https://github.com/ceph/paddles/pull/51 Zack Cerza
09:36 PM Feature #10754 (Resolved): paddles: add percentage to queue_stats command
Zack Cerza
08:56 PM Bug #10752 (Resolved): Move VPMs to static leases
Just to insure IP addresses never change and there isn't any DNS weirdness going on. This also means we can stop runn... Sandon Van Ness
05:50 PM Feature #10699 (Resolved): make teuthology failure reasons more clear
https://github.com/ceph/teuthology/commit/018cbc899f0c69d0409db3609c1a1f117dca9223 Andrew Schoen
05:15 PM Revision 018cbc89 (teuthology): Merge pull request #423 from ceph/teuth-testing
Add a label to run.run Zack Cerza
05:04 PM Bug #10748: "ConnectionLostError: SSH connection to vpm" error
The RepresenterErrors are a Sentry issue and I filed #10750 to track them.
Sandon, what's wrong with our VMs? :(
Zack Cerza
04:20 PM Bug #10748 (Resolved): "ConnectionLostError: SSH connection to vpm" error
Lots of dead runs in http://pulpito.front.sepia.ceph.com/teuthology-2015-02-02_18:18:01-upgrade:firefly-x-giant-distr... Yuri Weinstein
05:01 PM Bug #10749: "Could not get lock /var/lib/dpkg/lock" error
The RepresenterErrors are a Sentry issue and I filed #10750 to track them. The dpkg lock issue looks like a chef prob... Zack Cerza
04:24 PM Bug #10749 (Closed): "Could not get lock /var/lib/dpkg/lock" error
Many errors in run http://pulpito.front.sepia.ceph.com/teuthology-2015-02-02_18:18:01-upgrade:firefly-x-giant-distro-... Yuri Weinstein
05:00 PM Bug #10750 (Resolved): RepresenterError in sentry codepath
http://qa-proxy.ceph.com/teuthology/teuthology-2015-02-02_18:18:01-upgrade:firefly-x-giant-distro-basic-vps/737593/te... Zack Cerza
03:54 PM Revision 3ae8ed7d (teuthology): Added a 'tests' task that we can use to test teuthology features
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
03:54 PM Revision 2ad47537 (teuthology): Add a label kwarg to run.run
This can be used to label or annotate commands to remotes so we can
print a meaningful log message if the command fai...
Andrew Schoen

02/03/2015

10:38 PM Bug #10600: PATH issues on RHEL7 nodes?
I meant from the script Zack Cerza
10:00 PM Feature #10704 (In Progress): Implement "expand test coverage for the upgrade suites" for giant-x...
Yuri Weinstein
08:50 PM Feature #10736 (Resolved): teuthology testing task
We want to create a teuthology task that will act as a set of integration tests for teuthology itself. We'd run this... Andrew Schoen
05:57 PM Feature #10699: make teuthology failure reasons more clear
One option we discussed was adding a label kwarg to run.run so that we can print a descriptive label when a command f... Andrew Schoen
05:26 PM Revision 4aa2e937 (teuthology): Use pytest to auto discover and run tests in task.tests
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
03:03 PM Revision dcfcfdcb (teuthology): Added a 'tests' task that we can use to test teuthology features
Signed-off-by: Andrew Schoen <aschoen@redhat.com> Andrew Schoen
02:51 PM Revision 29aeb42a (teuthology): Add a label kwarg to run.run
This can be used to label or annotate commands to remotes so we can
print a meaningful log message if the command fai...
Andrew Schoen

02/02/2015

10:46 PM Bug #10500: "HEALTH_WARN 18 pgs peering" in upgrade:firefly-firefly-distro-basic-multi run
This is the peering wq bug most likely. We need to disable the config option which makes that bug show up for these ... Samuel Just
10:34 PM Revision 9cdc3578 (teuthology): Make 'branch' a tag in Sentry
Signed-off-by: Zack Cerza <zack@redhat.com> Zack Cerza
08:40 PM Bug #10715: qemu not found on el7
commit:05ce2aa1bf030ea225300b48e7914577a412b38c Josh Durgin
12:43 PM Bug #10715 (Resolved): qemu not found on el7
ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2015-02-01_23:00:02-rbd-next-distro-basic-multi/735303
c...
Sage Weil
08:28 PM Revision ad51511a (teuthology): Add branch to Sentry info
Signed-off-by: Zack Cerza <zack@redhat.com> Zack Cerza
 

Also available in: Atom