Project

General

Profile

Activity

From 10/31/2016 to 11/29/2016

11/29/2016

10:03 PM Bug #18067 (Resolved): libtool missing on xenial nodes
David Galloway
07:02 PM Bug #18067: libtool missing on xenial nodes
https://github.com/ceph/ceph-cm-ansible/pull/294 David Galloway
06:51 PM Bug #18067 (In Progress): libtool missing on xenial nodes
libtool is installed. This is from smithi106's teuthology.log?... David Galloway
02:59 PM Bug #18067 (Resolved): libtool missing on xenial nodes
See failures like this:
http://pulpito.ceph.com/teuthology-2016-11-27_10:10:01-fs-jewel---basic-smithi/580462/
John Spray
08:38 PM Bug #18069: rados suite upgrade tests fail on xenial nodes
Sage Weil wrote:
> I forced the rebuild bc (I assumed) it wasn't actually built before.
Nah, that gitbuilder neve...
David Galloway
08:22 PM Bug #18069: rados suite upgrade tests fail on xenial nodes
I forced the rebuild bc (I assumed) it wasn't actually built before. I suspect the workaround here is just to not ru... Sage Weil
06:45 PM Bug #18069: rados suite upgrade tests fail on xenial nodes
Looks like somebody forced a rebuild on v0.67.10
infernalis is failing due to ...
David Galloway
03:05 PM Bug #18069: rados suite upgrade tests fail on xenial nodes
Similar failures when other jobs try to get infernalis branch John Spray
03:05 PM Bug #18069 (Resolved): rados suite upgrade tests fail on xenial nodes
For example:... John Spray
04:31 PM Bug #17977: Numerous KVM failures: "failed to initialize KVM: Permission denied"
Any test that includes "ec-data-pool" is most likely unstable right now. Jason Dillaman
03:42 PM Bug #17977: Numerous KVM failures: "failed to initialize KVM: Permission denied"
I re-ran the rbd suite with ... David Galloway

11/26/2016

07:43 AM Support #15340 (Resolved): Update smithi P3700 NVMe card firmware
Completed this weekend during download, tracker, and docs site migration. David Galloway
07:28 AM Tasks #17927 (Resolved): need additional addresses for Sepia RefArch lab in RDU2
This is complete. David Galloway

11/24/2016

07:09 PM Bug #18039 (Resolved): can't nuke mira095
... Yuri Weinstein
06:58 PM Bug #18038 (Resolved): can't nuke mira041
... Yuri Weinstein

11/23/2016

08:07 PM Bug #17917: can't nuke mira039
Drive 1 is missing or dead. LUN0 is not showing up in HDD boot drive order.... David Galloway
07:50 PM Bug #10472 (Won't Fix): ccache misconfigured on gitbuilders
gitbuilders are being retired soon. We can re-open if this needs to be addressed on jenkins slaves. David Galloway
07:37 PM Tasks #17927 (In Progress): need additional addresses for Sepia RefArch lab in RDU2
I'll take care of this during some lab maintenance on Friday. David Galloway

11/22/2016

11:51 PM Bug #17977 (In Progress): Numerous KVM failures: "failed to initialize KVM: Permission denied"
Found this: https://bugzilla.redhat.com/show_bug.cgi?id=1333159 David Galloway
04:25 PM Bug #17977: Numerous KVM failures: "failed to initialize KVM: Permission denied"
I scheduled a run that would output the ACLs of /dev/kvm before attempting to run qemu-kvm. http://pulpito.ceph.com/... David Galloway

11/21/2016

06:12 PM Bug #17977: Numerous KVM failures: "failed to initialize KVM: Permission denied"
I *think* this is the first run we encountered this in: http://pulpito.ceph.com/teuthology-2016-11-13_11:00:02-rbd-ma... David Galloway
05:34 PM Bug #17977: Numerous KVM failures: "failed to initialize KVM: Permission denied"
I am only seeing this on smithis running CentOS. I've marked smithi010 for debugging.
Link to run: http://pulpito...
David Galloway
04:02 PM Bug #17977: Numerous KVM failures: "failed to initialize KVM: Permission denied"
It's a recent failure -- and it only happens on some nodes and not others. Therefore, something most likely changed o... Jason Dillaman
03:40 PM Bug #17977: Numerous KVM failures: "failed to initialize KVM: Permission denied"
Guessing maybe a 'sudo' needs to be added to this block? https://github.com/ceph/ceph-qa-suite/blob/master/tasks/qem... David Galloway
02:35 PM Bug #17977 (Resolved): Numerous KVM failures: "failed to initialize KVM: Permission denied"
Many RBD QEMU test cases are failing due to KVM permission denied issues. Example: http://qa-proxy.ceph.com/teutholog... Jason Dillaman
03:32 PM Bug #10675 (In Progress): failures related to clock skew
Found this in the latest example:... David Galloway

11/20/2016

10:32 PM Bug #10675: failures related to clock skew
Seen again in http://pulpito.front.sepia.ceph.com/smithfarm-2016-11-18_09:59:00-fs-hammer-backports---basic-smithi/55... Nathan Cutler

11/17/2016

08:56 PM Bug #17941 (Resolved): smithi052 can't be nuked (stuck at power cycle)
Host was hanging during boot trying to mount a network mount. Presumably a non-existent ceph cluster.
Reimaged host.
David Galloway
05:03 PM Bug #17941: smithi052 can't be nuked (stuck at power cycle)
... Yuri Weinstein
04:55 PM Bug #17941 (Resolved): smithi052 can't be nuked (stuck at power cycle)
Yuri Weinstein
01:57 AM Tasks #17927 (Resolved): need additional addresses for Sepia RefArch lab in RDU2
Hello,
I have this IP range for OpenStack Floating IP addresses in the
Seipa/Ceph lab:
172.21.1.151 - 200
...
Dave Cain

11/15/2016

08:56 PM Bug #17917 (Resolved): can't nuke mira039
... Yuri Weinstein

11/14/2016

08:01 PM Bug #17887 (Resolved): mira034 and mira054 stale and unresponsive to ipmi power cycle
System wasn't power cycling despite sending power commands.
I sent an ...
David Galloway
03:28 PM Bug #17887: mira034 and mira054 stale and unresponsive to ipmi power cycle
You should be using conserver instead of ipmitool sol activate directly. David Galloway
03:45 PM Bug #17890 (Resolved): gitbuilder does not build latest Jewel packages for xenial
Since I don't think we're 100% ready to retire all the gitbuilders, I increased the gitbuilder VM's disk size and for... David Galloway
12:33 PM Bug #17890: gitbuilder does not build latest Jewel packages for xenial
You should no longer rely on Gitbuilders CI output. They are on their way to being turned off completely.
Please u...
Alfredo Deza
11:54 AM Bug #17890: gitbuilder does not build latest Jewel packages for xenial
For the record I tried to force rebuild the "jewel" branch this morning and the issue is still here. @Brett : yes, I ... Loïc Dachary
10:56 AM Bug #17890: gitbuilder does not build latest Jewel packages for xenial
So the situation will be resolved by rebuilding? Brett Niver
09:48 AM Bug #17890 (Resolved): gitbuilder does not build latest Jewel packages for xenial
The latest Jewel commit https://github.com/ceph/ceph/commit/427f357f0eed32c9ce17590ae9303a94e8b710e7
is included in ...
Ramana Raja

11/13/2016

09:24 PM Bug #17887 (Resolved): mira034 and mira054 stale and unresponsive to ipmi power cycle
Both marked down... Yuri Weinstein

11/12/2016

12:28 AM Support #17848: Sepia Lab Access Request
Requested VPN credentials from Owen via IRC. Waiting to hear back. David Galloway
12:28 AM Bug #14674 (Resolved): mira097 has bad drives
All disks replaced. David Galloway

11/11/2016

11:59 PM Bug #17873 (Rejected): CentOS 7.2 failing to install in Sepia lab
This is actually a result of a bad/leftover partition table on some refurbished drives we bought. Running ... David Galloway
08:54 PM Bug #17873 (Rejected): CentOS 7.2 failing to install in Sepia lab
... David Galloway
08:44 PM Bug #14214: mira115 has bad disks (marked down)
On-board NIC port 1 is flapping. Changed to port 2. David Galloway
08:12 PM Cleanup #14528 (Resolved): Track down usage and purpose of mira{123..126} aka dubia{001..004}
David Galloway
07:24 PM Cleanup #14528 (Fix Under Review): Track down usage and purpose of mira{123..126} aka dubia{001.....
Finished last few host renames / disk replacements.
https://github.com/ceph/ceph-sepia-secrets/pull/185
*Final Su...
David Galloway

11/09/2016

05:05 PM Support #17848 (Closed): Sepia Lab Access Request
1) Do you just need VPN access or will you also be running teuthology jobs?
I hope to just run teuthology jobs, an...
Owen Synge

11/03/2016

03:12 PM Fix #17780: update joao's ssh key throughout the lab
Thanks David! Joao Eduardo Luis

11/02/2016

08:05 PM Bug #14840 (In Progress): mira091 is not accessible
This machine locked up again. We'll have to retire it. David Galloway
08:03 PM Support #17740 (Resolved): Sepia Lab Access Request
David Galloway
08:03 PM Fix #17780 (Resolved): update joao's ssh key throughout the lab
I updated your public key on the teuthology host and VPSHOSTs. Any jobs you run from here on will copy your new key ... David Galloway
02:44 PM Fix #17780 (Fix Under Review): update joao's ssh key throughout the lab
https://github.com/ceph/keys/pull/61 David Galloway
12:10 PM Fix #17780 (Resolved): update joao's ssh key throughout the lab
Today I realized that:
1. openssh deprecated DSA
2. teuthology - and thus i'm assuming the whole lab - does not a...
Joao Eduardo Luis
03:21 PM Support #17427: Sepia Lab Access Request
Hi Daniel,
I'm hoping to gather a bit more information about your interest in accessing the Sepia test lab. Are y...
David Galloway

10/31/2016

04:10 PM Bug #14840 (Resolved): mira091 is not accessible
Reimaged this host on 4OCT and brought VMs back up today. David Galloway
03:19 PM Bug #14546 (Resolved): mira033 kernel panic from MCE
Machine seems to be passing jobs w/o issue. David Galloway
02:55 PM Cleanup #14868 (Resolved): Audit user list
https://github.com/ceph/ceph-sepia-secrets/pull/178 David Galloway
02:54 PM Bug #16875 (Closed): apama001 load spikes and OSD 65 dies
This host is being reinstalled with RHEL7 and repurposed as a docker host. David Galloway
02:53 PM Bug #17415: RHEV Network or Storage issues
We're still limping along with the arbiter brick in place. We had to disable PCP reporting for now, though. The PCP... David Galloway
 

Also available in: Atom