Project

General

Profile

Activity

From 10/23/2013 to 11/21/2013

11/21/2013

05:22 PM devops Feature #6836 (Resolved): Get all Ceph installation package dependencies from EPEL etc, into eith...
Neil Levine
04:00 PM Bug #6796 (In Progress): ceph mons interpretting pg splits very wrong
Greg Farnum
03:40 PM Feature #6835 (Resolved): EC: ec pgs will need to be able to specify temp primaries other than ac...
OSDMap interface must allow primariness to be specified seperately from acting set position for ec pools. Samuel Just
03:35 PM Bug #6834: nightlies: monitor crashed in emperor
From this backtrace it looks like either there was a hardware problem, or the monitor was using so much memory it cou... Greg Farnum
03:24 PM Bug #6834 (Can't reproduce): nightlies: monitor crashed in emperor
logs: ubuntu@teuthology:/a/teuthology-2013-11-18_19:31:27-upgrade-parallel-next-testing-basic-plana/107772... Tamilarasi muthamizhan
03:12 PM devops Feature #6310: Get Dumpling into CentOS Ceph repo
Dumpling is not yet in the Centos repo.
I reran the mock build on centos 6.3 and 6.4 to verify that there should n...
Anonymous
03:11 PM Bug #6833: `/etc/init.d/ceph status` occasionally exists silently
Hmm. `/etc/init.d/ceph restart` is doing the same thing. Zack Cerza
02:55 PM Bug #6833 (Can't reproduce): `/etc/init.d/ceph status` occasionally exists silently
I have a cluster that got wedged somehow, and when I run `/etc/init.d/ceph status` it simply exits with status 0. Tha... Zack Cerza
03:11 PM Bug #6820: Bad commandline usage crashed my monitor
nevermind, got it. Joao Eduardo Luis
01:59 PM Feature #5991: EC: [link] Backfill peers should not be included in the acting set
Ian Colle
01:57 PM Feature #5990 (In Progress): EC: [link] Factor out the ReplciatedPG object replication and client...
Ian Colle
01:39 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
Yeah, I think a note is fine. Mark Kirkwood
12:33 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
`delete=True` is the default, and we are explicitly setting that flag to `delete=False` because of that reason.
Wo...
Alfredo Deza
01:38 PM Feature #6832 (Resolved): EC: Adapt pg log to include information necessary for rollback
This feature includes adapting the ReplicatedBackend to allow xattr, append operations to be rollback-able. It also ... Samuel Just
01:35 PM Feature #6831 (Resolved): EC: Adapt ReplicatedPG read path to handle async reads
This is a bit challenging because there might be a sequence of reads at different offsets within the transaction. Th... Samuel Just
12:40 PM Bug #6807 (In Progress): Debian Wheezy Teuthology Ceph-deploy run failed.
Alfredo Deza
07:28 AM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
It sounds like there was an earlier problem with the test or a different failure — why is it trying to delete the cep... Greg Farnum
07:21 AM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
It looks like the reason we were enforcing the *single file system* was because we might still have OSDs mounted (hen... Alfredo Deza
09:46 AM devops Feature #5282 (Closed): Get Dumpling into EPEL
Ceph 0.67.3 dumpling is in the epel repository. Anonymous
09:25 AM rgw Bug #6830 (Resolved): S3 CompleteMultipartUploadResult has empty ETag element
RHEL 6.4, Ceph 0.67.
The S3 Complete Multipart Upload operation returns a result that looks like this:...
Benjamin Gilbert
09:12 AM Feature #6828: osd should not silently fail to start when journal partition has no UUID
I now understand this is not a bug but a feature. It probably deserves a warning of some kind ? http://dachary.org/?p... Loïc Dachary
04:53 AM Feature #6828 (Rejected): osd should not silently fail to start when journal partition has no UUID
Providing a journal that is a partition that does not have the expected journal UUID should trigger an error.... Loïc Dachary
06:50 AM rgw Bug #6829 (Resolved): rgw: missing RGWUserAdminOpState::system_specified initialization
By inspecting code I noticed that, which means that when modifying user configuration the system settings of the user... Yehuda Sadeh
03:16 AM Bug #6827 (Resolved): ceph-disk hangs on blkid -s TYPE /dev/fd0
On a supermicro hardware running ubuntu 12.04.3 ceph-disk list hangs forever trying to *blkid -s TYPE /dev/fd0*
<pre...
Loïc Dachary
03:01 AM Bug #6826 (Duplicate): Non-equal performance of 'freshly joined' OSDs
Was just a bare eye observation for a long time, but I`ll try to formalize it here.
For OSDs entered recently perfom...
Andrey Korolyov
12:06 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
I use ceph version 0.72-3-g5e1e02c (5e1e02c99b620fa4ffd2b455eb8e005b172fa05c), which is the "hotfix" for http://track... Corin Langosch

11/20/2013

07:32 PM CephFS Bug #6742: failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
This has popped up several more times in the nightlies; it looks to be a regular occurrence now. :/ Greg Farnum
07:11 PM CephFS Bug #6742: failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
/teuthology-2013-11-14_23:01:38-fs-next-testing-basic-plana/{100526|100525|100487} Greg Farnum
11:34 AM CephFS Bug #6742: failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
/a/teuthology-2013-11-18_23:01:16-fs-master-testing-basic-plana/108236 (no logs) Greg Farnum
07:24 PM rbd Bug #6368 (Resolved): rbd nosetests keep failing with AttributeError
This turned out to be a silly version skew problem. centos and rhel have older versions of nosetests that needed diff... Josh Durgin
10:32 AM rbd Bug #6368: rbd nosetests keep failing with AttributeError
Changing the priority of this so that we can (hopefully) get it fixed soon.
This is still an issue for the nightly...
Alfredo Deza
07:16 PM Bug #6795 (Resolved): osd: 'tell bench' write size argument is ignored
commit:40a76ef0d09f8ecbea13712410d9d34f25b91935 Josh Durgin
07:11 PM CephFS Bug #6773 (Duplicate): libcephfs interface tests maybe hanging
Greg Farnum
06:30 PM devops Bug #6790 (Resolved): /ref/branch/version mismatch
So it looks like oh RHEL just using the version value (without the dist part) does not work so as we discussed earlie... Sandon Van Ness
04:53 PM Bug #6820: Bad commandline usage crashed my monitor
John, that's the client being unable to contact the monitor (most likely because the monitor indeed crashed). We'd ne... Joao Eduardo Luis
03:54 PM Bug #6820 (Resolved): Bad commandline usage crashed my monitor
I was looking to dump my CRUSH map without JSON syntax and I used the -f command to specify a format.
ceph osd cru...
John Wilkins
04:34 PM Bug #6824 (Resolved): Removal of an OSD that is not down should set non-successful status code
A customer reports that some ceph commands return a status code of 0, even when they failed. The provided example was... Brian Andrus
03:35 PM Fix #6780: monitor errors when checking for quorum status
Reason for this: the code in place to keep compatibility with previous versions of the monitor with regard to the Cep... Joao Eduardo Luis
10:50 AM Fix #6780 (In Progress): monitor errors when checking for quorum status
this happens when some osds and mons are upgraded to next branch [emperor]
recent logs: ubuntu@teuthology:/var/lib...
Tamilarasi muthamizhan
10:21 AM Bug #6808 (Duplicate): ceph-deploy/rbd/.../tasks/rbd_cli_tests.yaml fails on non-debian distros
Issues already opened for this: #6648 #6649 and the one used as the main issue to track this problem is #6368
Alfredo Deza
10:15 AM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
Good catch, I think that this is the culprit:... Alfredo Deza
10:09 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
Corin, I forgot one step that would be wonderful if you could do: install google-perftools and run 'google-pprof <pat... Joao Eduardo Luis
09:56 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
Corin, forgot to ask: what version is this happening on exactly and are you using our packaged binaries? Joao Eduardo Luis
09:06 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
I just did what you wrote, please see attachment. Corin Langosch
08:47 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
Can you please obtain a heap dump out of the monitor?
$ ceph heap start_profiler -m 10.0.0.7:6789
wait some tim...
Joao Eduardo Luis
08:04 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
... Corin Langosch
08:00 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
... Corin Langosch
08:00 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
cluster 4ac0e21b-6ea2-4ac7-8114-122bd9ba55d6
health HEALTH_OK
monmap e5: 3 mons at {a=10.0.0.5:6789/0...
Corin Langosch
07:58 AM Bug #6810 (Can't reproduce): very high monitor memory usage after upgrade dumpling -> emperor
As you know I upgraded a few days ago from dumpling to emperor. All deamons are now running emperor. I have 3 monitor... Corin Langosch
09:44 AM rbd Bug #5426: librbd: mutex assert in perfcounters::tinc in librbd::AioCompletion::complete()
Upgraded to Urgent since causing failures in nightlies Ian Colle
07:09 AM Linux kernel client Bug #6809 (New): 3.11 kernel panic: Workqueue: ceph-msgr con_work
The ceph cluster is very unstable ( hosts going up and down frequently ) and has high latency ( > 10ms ) on more than... Loïc Dachary
02:10 AM Bug #6797: ceph osd out does not migrate properly
I cannot agree that the question is just about user experience. Drive replacement almost always don`t mean any remapp... Andrey Korolyov

11/19/2013

05:38 PM Bug #6808 (Duplicate): ceph-deploy/rbd/.../tasks/rbd_cli_tests.yaml fails on non-debian distros
Coredumps are generated. See
/a/teuthology-2013-11-19_01:10:01-ceph-deploy-next-testing-basic-vps/{108357. 108353,...
Anonymous
05:22 PM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
That error should read:... Anonymous
05:18 PM Bug #6807 (Resolved): Debian Wheezy Teuthology Ceph-deploy run failed.
/a/teuthology-2013-11-19_01:10:01-ceph-deploy-next-testing-basic-vps/108339 needs to be investigated. The run report... Anonymous
04:58 PM Bug #6806: mon: audit cmd_getval() calls to make sure they handle failures correctly
Sigh. Yes, this was intentional, so that there was any value at all to doing the validation in the front end. I gue... Dan Mick
04:42 PM Bug #6806 (Resolved): mon: audit cmd_getval() calls to make sure they handle failures correctly
During #6796 we noticed that most calls to cmd_getval() do not care for the function's return value, which indicates ... Joao Eduardo Luis
04:46 PM Bug #6796: ceph mons interpretting pg splits very wrong
The real issue here is that the Dumpling code is not considering the return value of cmd_getval() when obtaining the ... Joao Eduardo Luis
04:39 PM Feature #6805 (Resolved): mon: find a way to properly extend/change mon commands without breaking...
This need stems from #6796. It became obvious that changing variable types will break compatibility with previous ver... Joao Eduardo Luis
04:17 PM rgw Fix #6804 (Resolved): Overly verbose logging: "setting object write_tag=" - rgw_rados.cc
Logging of "setting object write_tag=" at level=0, seems much too verbose considering 10's of thousands of these even... Ron Allred
03:59 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
This will be fine for temporary files opened with 'delete=False' - if we start using delete=True then they will be po... Mark Kirkwood
12:56 PM devops Bug #6701 (Fix Under Review): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18...
Alfredo Deza
12:55 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
There was a PR addressing the problem for using shutil.move and I just opened another one to fix the missing `close()... Alfredo Deza
02:07 PM Bug #6803 (Can't reproduce): rados test failing in the nightlies on next branch
logs: ubuntu@teuthology:/a/teuthology-2013-11-16_23:00:11-rados-next-testing-basic-plana/103763... Tamilarasi muthamizhan
12:20 PM rgw Bug #6802 (Rejected): ARM: rgw_swift failure (internal server error, 500)
Looks like almost all the tests are fialing with a 500 error (internal server error):
config:...
Sandon Van Ness
12:20 PM devops Feature #6752 (Resolved): ceph-deploy to install yum or apt source with custom repo hostname
And that just got merged with hash: 9388f83 Alfredo Deza
11:41 AM devops Feature #6752 (Fix Under Review): ceph-deploy to install yum or apt source with custom repo hostname
Opened pull request: https://github.com/ceph/ceph-deploy/pull/136 Alfredo Deza
11:33 AM devops Feature #6752 (In Progress): ceph-deploy to install yum or apt source with custom repo hostname
Found a couple of small items that needed to be fixed. Going to update in a different PR. Alfredo Deza
10:50 AM devops Feature #6752 (Resolved): ceph-deploy to install yum or apt source with custom repo hostname
changeset was merged into ceph-deploy's master branch with hash: 3331b6f
A bunch of documentation was added as w...
Alfredo Deza
06:52 AM devops Feature #6752 (Fix Under Review): ceph-deploy to install yum or apt source with custom repo hostname
Pull request opened: https://github.com/ceph/ceph-deploy/pull/135 Alfredo Deza
12:14 PM rgw Bug #6801 (Rejected): RGW on arm: 'saw valgrind issues'
Config:... Sandon Van Ness
11:50 AM rbd Bug #6800 (Resolved): rbd/qemu-iotests.sh Failing on Arm.
Config:... Sandon Van Ness
11:04 AM Bug #6797: ceph osd out does not migrate properly
Yeah, odd as it seems this is actually a user experience conflict — marking an OSD out does not change the CRUSH weig... Greg Farnum
09:45 AM Bug #6797: ceph osd out does not migrate properly
+1, migration overhead may be reduced by doing these actions in a short order, but generally it introduces _two_ peer... Andrey Korolyov
12:28 AM Bug #6797 (Won't Fix): ceph osd out does not migrate properly
ok, the subject might be misleading; here is what is happening:
- ceph osd out $id
- wait until migration finishe...
Zoltan Arnold Nagy
10:53 AM rgw Feature #6799 (Closed): rgw: test keystone integration on RHEL
Run any applicable tempest tests and swift tests. Josh Durgin
06:38 AM rgw Bug #5931: radosgw crashes when deleting object
I believe you need to open another bug with link to this one to get any feedback. Artem Salpagarov
05:25 AM rgw Bug #5931: radosgw crashes when deleting object
we have exactly the same problem
and i also upgraded to emperor because i thought this will fix the issue
but the i...
ramon makkelie

11/18/2013

06:17 PM Bug #6796: ceph mons interpretting pg splits very wrong
Summarizing discussion with Sam and Dan:
the problem is caused by commit:2fe0d0d97af95c22db80800f5b9da51f672d9407, w...
Greg Farnum
05:09 PM Bug #6796: ceph mons interpretting pg splits very wrong
The actual problem is that the mon sees
2013-11-18T15:04:50.683 DEBUG:teuthology.orchestra.run:Running [10.214.135...
Samuel Just
05:08 PM Bug #6796 (Resolved): ceph mons interpretting pg splits very wrong
logs are copied to mira055.front.sepia.ceph.com:/home/ubuntu/bug_6776_1
to reproduce the bug, ...
Tamilarasi muthamizhan
03:30 PM Bug #6795: osd: 'tell bench' write size argument is ignored
https://github.com/ceph/ceph/pull/854 Josh Durgin
03:25 PM Bug #6795 (Resolved): osd: 'tell bench' write size argument is ignored
osd bench is supposed to take a second argument as the block size in bytes, but the default block size of 4MB is alwa... Josh Durgin
01:13 PM Bug #6786 (Resolved): os/FileJournal.cc: 1533: FAILED assert(seq >= last_committed_seq)
Samuel Just
01:13 PM Bug #6756: journal full hang on startup
Samuel Just
11:57 AM devops Bug #6790: /ref/branch/version mismatch
I have a couple concerns.
If we add the rpm release to the version, will that break anything that expects version ...
Anonymous
11:12 AM Bug #6794 (Rejected): ceph-deploy needs both prepare and activate for paths
I thought the docs where talking about filesystem paths, not device paths. The docs are actually correct, so closing ... Alfredo Deza
11:04 AM Bug #6794 (Rejected): ceph-deploy needs both prepare and activate for paths
The docs currently give an example for `create` that uses paths which is not correct. Alfredo Deza

11/17/2013

06:40 AM CephFS Bug #6791: mds assert after startup - CDir::commit error (want > commited version)
Thanks for the advice.
The "mds log_max_segments = 100000" avoided the assertion.
I'm starting to copy the data o...
Maros Vegh
06:12 AM CephFS Bug #6791: mds assert after startup - CDir::commit error (want > commited version)
Looks like the FS get corrupted. I suggest copying the data out and re-creating the FS.
add folowing line to ceph...
Zheng Yan
04:35 AM CephFS Bug #6791: mds assert after startup - CDir::commit error (want > commited version)
On a higher log level i can see that this happens during "try_to_expire" on a journal LogSegment:
-4> 2013-11-...
Maros Vegh

11/16/2013

07:35 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
Managed to provoke this again, this time creating a keyring for an osd on a host that is not a monitor. The tiggering... Mark Kirkwood
01:43 PM CephFS Bug #6791 (Won't Fix): mds assert after startup - CDir::commit error (want > commited version)
On upgrade from 0.67 to 0.72 i experienced the bug 6755.
I repaired the system with the ceph_filestore_tool as descr...
Maros Vegh
04:07 AM Documentation #2684: doc: ceph and all daemons take --show-config
"src/common/config.cc":https://github.com/ceph/ceph/blob/7e53473a7aaa68e03843c521022b9e60b33e4ef3/src/common/config.c... Loïc Dachary
03:17 AM Documentation #2684: doc: ceph and all daemons take --show-config
... Loïc Dachary

11/15/2013

05:49 PM Bug #6777: nightlies: gem dependency error
I wonder if the rubygem mirror was having trouble. I just tried to reproduce this on a brand new debian wheezy (7.0) ... Sandon Van Ness
03:52 PM devops Bug #6790: /ref/branch/version mismatch
logs: ubuntu@teuthology:/a/teuthology-2013-11-15_01:35:02-upgrade-small-next-testing-basic-vps/100676
this has to ...
Tamilarasi muthamizhan
03:47 PM devops Bug #6790 (Resolved): /ref/branch/version mismatch
So currently the version to install with yum is grabbed from sha1/$sha1/version or /ref/$branch/version
Basically ...
Sandon Van Ness
03:09 PM Bug #6003 (Can't reproduce): journal Unable to read past sequence 406 ...
Samuel Just
03:08 PM Bug #6776 (Duplicate): nightly failure: timed out waiting for admin_socket after osd restarted
Samuel Just
03:07 PM Bug #6778 (Won't Fix): log bound mismatch errors seen
Added to whitelist. Samuel Just
02:49 PM Bug #6786: os/FileJournal.cc: 1533: FAILED assert(seq >= last_committed_seq)
logs: ubuntu@teuthology:/a/teuthology-2013-11-14_23:00:22-rados-next-testing-basic-plana/100293... Tamilarasi muthamizhan
10:14 AM Bug #6786 (Resolved): os/FileJournal.cc: 1533: FAILED assert(seq >= last_committed_seq)
ceph version 0.72-204-g878f354 (878f3540d1a69d0ca7d2014ba7a9f3cb5cfd986d)
1: (FileJournal::committed_thru(unsigned...
Samuel Just
02:23 PM Bug #6789 (Resolved): cannot remove the leader when there only are two monitors
On Ubuntu precise with dumpling 0.67.4, create a new cluster with two monitors. ceph mon remove name_of_the_leader wi... Loïc Dachary
11:58 AM Bug #6761 (Resolved): emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Samuel Just
11:17 AM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
Now a fully working version with environment variables:... Alfredo Deza
10:38 AM Bug #6787 (Won't Fix): upstart is restarting daemons which we want to be dead
It looks like upstart's auto service management stuff is restarting OSDs which have committed suicide due to getting ... Greg Farnum
10:21 AM Fix #6780 (Need More Info): monitor errors when checking for quorum status
Joao Eduardo Luis
08:40 AM Fix #6780: monitor errors when checking for quorum status
what version was this on? I think sage fixed this particular issue last sprint. Joao Eduardo Luis
09:57 AM rgw Bug #6713 (Can't reproduce): rgw master nosetests failure.
Does not seem to be happening anymore. Anonymous
09:55 AM devops Bug #6745 (Can't reproduce): ceph-deploy gatherkeys on ubuntu dumpling
It turns out that i did not have a quorum. I'll close this.
The WARNIN is still a spelling error.
Anonymous
07:19 AM rbd Bug #5469: qemu-io: segfault when tried IO with invalid arguments
I'm having a simular issue atm:
char device redirected to /dev/pts/2
xen be: qdisk-768: error: unknown operation ...
Bram Pieters
12:45 AM rgw Support #6785 (Closed): integration of radosgw and keystone
hi, I tried to integrate radosgw and keystone according to the guide here(http://ceph.com/docs/master/radosgw/config/... welmess gao
12:44 AM rgw Support #6784 (Closed): integration of radosgw and keystone
hi, I tried to integrate radosgw and keystone according to the guide here(http://ceph.com/docs/master/radosgw/config/... welmess gao

11/14/2013

06:09 PM Bug #6781 (Can't reproduce): timed out waiting for recovery - probably ceph command hang
ubuntu@teuthology:/a/samuelj-2013-11-14_15:22:25-rados-wip-6761-emperor-testing-basic-plana/99764
ceph.log indicat...
Samuel Just
05:44 PM Fix #6780 (Closed): monitor errors when checking for quorum status
logs: ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97245
pasting th...
Tamilarasi muthamizhan
05:39 PM devops Bug #6779: fix typo on the modfastcgi repo for fedora18
this also applies for other fedora packages. Tamilarasi muthamizhan
05:37 PM devops Bug #6779 (Resolved): fix typo on the modfastcgi repo for fedora18
currently, it is called 'feodra-fcgi-ceph' repo.
INFO:teuthology.orchestra.run.out:[10.214.138.59]: [2013-11-13T20...
Tamilarasi muthamizhan
04:50 PM Bug #6778 (Won't Fix): log bound mismatch errors seen
logs:ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97229... Tamilarasi muthamizhan
04:33 PM Bug #6777: nightlies: gem dependency error
ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97233 Tamilarasi muthamizhan
04:32 PM Bug #6777 (Can't reproduce): nightlies: gem dependency error
logs: ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97184
This issue...
Tamilarasi muthamizhan
04:27 PM Bug #6776: nightly failure: timed out waiting for admin_socket after osd restarted
... Tamilarasi muthamizhan
04:26 PM Bug #6776 (Duplicate): nightly failure: timed out waiting for admin_socket after osd restarted
logs: ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97235... Tamilarasi muthamizhan
04:17 PM rbd Bug #6775 (Rejected): kvm backtrace on rbd task
On these tests:
/var/lib/teuthworker/archive/teuthology-2013-11-07_19:00:48-rbd-cuttlefish-testing-basic-plana/893...
Sandon Van Ness
03:14 PM Bug #6769: rados upgrade tests failing in the nightlies
teuthology bug, that is Samuel Just
03:13 PM Bug #6769 (Resolved): rados upgrade tests failing in the nightlies
4aaa908d92168e2e46b8802d6042b1d3ffb9bc54
Pushed to master and next.
Samuel Just
02:08 PM Bug #6769 (In Progress): rados upgrade tests failing in the nightlies
Samuel Just
03:13 PM Documentation #6774 (Resolved): Documentation: osd scrub load threshold incorrect.
In rados/configuration/osd-config-ref.rst ... Tyler Brekke
02:06 PM Bug #6674 (Resolved): Busted client locking
1212a2119f3681de40cf947dae9c3b0d3f19e6fe Samuel Just
12:07 PM Bug #6674: Busted client locking
This is my theory. When CephContext is being cleaned-up, we first disable lockdep, then wait for the service thread t... Noah Watkins
11:38 AM Bug #6674: Busted client locking
I just saw it a couple more times going over fs nightlies from this week, and it broke several rados tests last night... Greg Farnum
11:26 AM Bug #6674: Busted client locking
Has this been triggered again since the first time we saw it? Noah Watkins
11:12 AM Bug #6674: Busted client locking
commit:c7d975aeadf908d11577c480fa0a2e831d069c55 is the one I was discussing. Greg Farnum
01:19 PM CephFS Bug #5753: ceph-fuse: segfault when getting back a traceless rename op
teuthology-2013-11-08_19:01:24-fs-dumpling-testing-basic-plana/90910/
teuthology-2013-11-09_19:01:04-fs-cuttlefish-t...
Greg Farnum
12:55 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
A bunch of progress, we are now allowing a couple of flags: --repo-url and --gpg-url. When GPG is not passed in it wi... Alfredo Deza
12:29 PM Bug #6756 (Pending Backport): journal full hang on startup
Samuel Just
12:28 PM Bug #6758 (Pending Backport): clone_range missing head src
Samuel Just
11:34 AM CephFS Bug #6773 (Duplicate): libcephfs interface tests maybe hanging
/a/teuthology-2013-11-09_23:01:13-fs-next-testing-basic-plana, 92248 and 92209 both hung at the same point:... Greg Farnum
11:21 AM CephFS Bug #6394: teuthology: bad dereference in mds thrasher
/teuthology-2013-11-08_19:01:24-fs-dumpling-testing-basic-plana/90904/ Greg Farnum
11:12 AM Bug #6738: lockdep segfault when running rados test.sh
#6674 Greg Farnum
11:12 AM Bug #6738 (Duplicate): lockdep segfault when running rados test.sh
Samuel Just
06:25 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Likewise for me, following the instructions everything is now back up and running without issues, thanks! Damien Churchill
02:46 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Now some hours later everything is still running perfectly fine. Thanks again! Corin Langosch

11/13/2013

08:30 PM Cleanup #6766: keyring leading spaces
The only difference I see there is quoting on the caps mon value, not spaces. Is that what you meant to say? Dan Mick
10:48 AM Cleanup #6766 (New): keyring leading spaces
A keyring generated with ceph-deploy has no leading spaces. A keyring generated with ceph-authtool does have leading ... John Wilkins
07:44 PM CephFS Documentation #6771 (Closed): add mds configuration

may should update document about how to configure mds.
Anonymous
07:42 PM CephFS Bug #6770 (Can't reproduce): ceph fscache: write file more than a page size to orignal file cause...
When mount -o fsc on cephfs, write more than page size content to a file (less than a page), when each time writing 8... Min Chen
07:28 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
After installing the gitbuilder version and checking all osds reported between 50 - 200 lost objects. Repair went wit... Corin Langosch
06:57 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
tested wip-6761-emperor on mira056 and mira074 . It works fine! Tamilarasi muthamizhan
05:32 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
I've got a repair mechanism for you to try:
Install wip-6761-emperor on all osd machines
stop all osds
for each ...
Samuel Just
04:38 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
wip-6761-emperor tool appears to work:... Greg Farnum
04:30 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
(04:25:58 PM) samuelj@newdream.net/laptop: yes
(04:26:09 PM) samuelj@newdream.net/laptop: you need to create a clust...
Samuel Just
03:47 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Yup, it's easy to reproduce. I did it with vstart, notes follow.... Greg Farnum
03:21 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
We messed up when changing the encoding for adding flags to the object_info_t. Sam has a patch which looks good to me... Greg Farnum
01:36 PM Bug #6761 (In Progress): emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Ian Colle
01:07 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
We need to understand why the upgrade suites did not catch this. Samuel Just
01:06 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs

(12:56:37 PM) samuelj@newdream.net/home: the object_info encoding changed.
(01:07:07 PM) samuelj@newdream.net/home...
Samuel Just
11:47 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
System-Wide limits seem good on all hosts:... Corin Langosch
10:19 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
The log shows the clients getting an error (which should be handled better):... Josh Durgin
09:55 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
I just attached it here too. Sorry. Corin Langosch
09:54 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Here the full log ('debug objecter = 20' and 'debug objectcacher = 20' on the client) when doing the "rbd export" htt... Corin Langosch
08:48 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Same happens for another image that cannot be booted anymore:... Corin Langosch
08:43 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
I just tried to export the image a file so I can boot the vm using that. But it fails to:... Corin Langosch
08:36 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
This is the backtrace when I try to start from another rbd image. Looks the same to me.... Corin Langosch
08:02 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
Same but nicely formatted:... Corin Langosch
08:00 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
# gdb --args /opt/qemu/1.4.0/bin/qemu-system-x86_64 -smp sockets=1,cores=2 -m 512 -vga cirrus -drive id=drive14183,if... Corin Langosch
07:26 AM Bug #6761 (Resolved): emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
All my systems run ubuntu 12.10. I was running dumpling for a few months without any errors. My kvm guests use qemu-r... Corin Langosch
07:03 PM devops Bug #6698 (In Progress): new osds added to the cluster not starting up due to crush lookup failure
this bug was caused by crush lookup failure.
I see similar issue with one of my local clusters: mira047 and mira048.
Tamilarasi muthamizhan
05:39 PM Bug #6768 (Resolved): Misleading log messages using "data store" for monitor storage
It was building properly on the gitbuilders and looked good to me; merged! Greg Farnum
05:23 PM Bug #6768 (Fix Under Review): Misleading log messages using "data store" for monitor storage
David Zafman
02:10 PM Bug #6768 (Resolved): Misleading log messages using "data store" for monitor storage
Change the following messages to use "local monitor storage" instead of "data store"
reached critical levels of av...
David Zafman
04:54 PM Bug #6755 (Duplicate): mds assert soon after startup (while recovering)
dup 6761 Zheng Yan
04:23 PM Bug #6769 (Resolved): rados upgrade tests failing in the nightlies
logs: /a/teuthology-2013-11-07_05:30:02-upgrade-next-testing-basic-plana/87919... Tamilarasi muthamizhan
01:44 PM Feature #6767 (Resolved): mon should have "version" command so ceph tell mon.* version works
ceph tell osd.* version is handy as a "am I running what I think I'm running". It would be trivial to implement this... Dan Mick
01:32 PM Subtask #5858 (In Progress): Backfill should be able to handle multiple backfill peers
David Zafman
01:32 PM Subtask #5855 (Fix Under Review): Backfill peers should not be included in the acting set
David Zafman
12:41 PM Feature #5991 (Fix Under Review): EC: [link] Backfill peers should not be included in the acting set
Ian Colle
12:40 PM Feature #5994 (In Progress): EC: [link] Backfill should be able to handle multiple backfill peers
Ian Colle
10:35 AM rgw Bug #6765 (Rejected): ARM: RGW s3 tests fail.
Seems to be hitting a 500 code or get connection refused to the web API during the test. Not sure if apache is crashi... Sandon Van Ness
09:55 AM Fix #6763 (Resolved): crushtool: don't warn so harshly when enabling CRUSH_TUNABLES
Apparently the crushtool says "tunables are DANGEROUS and NOT YET RECOMMENDED". Change that to be something less frig... Greg Farnum
09:23 AM RADOS Fix #6762: OSD 'numpg_*' performance counter inaccurate after adding PGs, until OSD map next changes
Setting this to the next release so we don't forget it. Greg Farnum
07:55 AM RADOS Fix #6762 (New): OSD 'numpg_*' performance counter inaccurate after adding PGs, until OSD map nex...

After creating some new PGs, I notice that the 'numpg' and associated counters don't reflect the newly increased PG...
John Spray
06:06 AM rgw Feature #3454: Support temp URLs for Swift API
I have just run into this as well.
Is there a timeline for this?
Wayne E Seguin
04:24 AM rgw Feature #3454: Support temp URLs for Swift API
Samen problem as Ryan
so if this could be picked up that would be great
ramon makkelie
05:29 AM Bug #6598: osd crash after recreating pool with same name (cuttlefish + bobtail?)
It was our mistake resulted in race/duplication for mkcoll() call and resulting crash on EEXIST. Sorry for noise. Andrey Korolyov
03:19 AM rgw Bug #6760: rgw incompatible with gsutil, authorization signature wrongly computed
There is a related issue here: https://github.com/ceph/ceph/pull/498 Valery Tschopp
03:17 AM rgw Bug #6760 (New): rgw incompatible with gsutil, authorization signature wrongly computed
The Google Cloud Storage tool 'gsutil' sends the 'x-goog-api-version: 2' header to RadosGW. For an unknown reason, rg... Valery Tschopp

11/12/2013

05:25 PM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
Squeeze has a depdency on python-requests which is in the squeeze backports distro, but not everyone enables that. Anonymous
05:24 PM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
The httpd-mmn problem with teuthology went away with a rebuild of the target machine. So it is probably a package or... Anonymous
04:12 PM Feature #6759 (New): Allow partial dump of pg statistics
The pool and osd sections of the pg dump are very useful. The only issue is that the pg dump itself can be very large... Carson Anderson
03:58 PM Bug #6755: mds assert soon after startup (while recovering)
I think the mds will function after deleting all lost objects Zheng Yan
10:43 AM Bug #6755: mds assert soon after startup (while recovering)
.. upgrading from 0.67.4 to 0.72 .. Markus Blank-Burian
10:39 AM Bug #6755 (Duplicate): mds assert soon after startup (while recovering)
Since I debugged this one a bit, I try to summarize what I could gather. I was in the process of upgrading from 0.64.... Markus Blank-Burian
03:28 PM Bug #6758 (Resolved): clone_range missing head src
osd/ReplicatedPG.cc: 5225: FAILED assert(attrs || !pg_log.get_missing().is_missing(soid) || (pg_log.get_log().objects... Samuel Just
02:36 PM Bug #6744: make install fails when --prefix specified
I spent some time looking at this with Yehuda; I suspect that at least one of the
problems is this section from Make...
Dan Mick
01:56 PM Bug #6756 (Resolved): journal full hang on startup
2013-11-10 22:51:06.073780 7fd833945780 2 journal open /var/lib/ceph/osd/ceph-4/journal fsid ed7f3df7-52a6-4fc7-a6ca... Samuel Just
01:46 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
For DEBs we just write the correct URL to... Alfredo Deza
01:36 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
The key can be a regular file too, but then I would need to push that to nodes.
Also, it should be noted that for ...
Alfredo Deza
01:25 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
Yes, I suddenly thought about the keys too. Filename can also be an arg (--use-repo-key) ? Neil Levine
01:11 PM devops Feature #6752 (In Progress): ceph-deploy to install yum or apt source with custom repo hostname
Alfredo Deza
01:11 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
A convention needs to be followed to retrieve the correct GPG Key from the local repo. Currently, ceph-deploy uses th... Alfredo Deza
07:12 AM devops Feature #6752 (Resolved): ceph-deploy to install yum or apt source with custom repo hostname
The goal is to have a user grab an ISO or .tar.gz consisting of all the files they need to run Ceph which would be do... Neil Levine
12:35 PM Documentation #6465: admin/build-doc should have some kind of build check for broken links
Certainly easy to test, and yeah, I was thinking even some external thing like wget or curl or something. Whatever w... Dan Mick
10:12 AM Documentation #6465: admin/build-doc should have some kind of build check for broken links
Maybe if we enabled the `-n` or 'nitpicky' option?
This could probably break the build, so we would need to check ...
Alfredo Deza
10:31 AM Bug #6740: ceph-disk --dmcrypt flag does not work when journals and backing stores are co-located
I believe this is the same bug as issue #6700, which has a patch. Once the OSD with dmcrypt has been set up, issue #6... Jan Harkes
10:04 AM Bug #6751: Pool 'df' statistics go bad after changing PG count
Oh, I missed the object counts entirely. Have you waited a while and made sure these wrong values persist? I have a s... Greg Farnum
10:01 AM Bug #6751: Pool 'df' statistics go bad after changing PG count
Yes: the USED and OBJECTS values for pbench3 both shoot up.
(sorry it's a bit difficult to read, forgot to put the...
John Spray
09:44 AM Bug #6751: Pool 'df' statistics go bad after changing PG count
This is the "USED" value on the pool pbench3 that you're looking at, right? Greg Farnum
06:12 AM Bug #6751 (Resolved): Pool 'df' statistics go bad after changing PG count

To reproduce:
# Create a pool with few PGs
# Create some objects in the pool
# Note the 'ceph df' output
# In...
John Spray
09:57 AM Fix #6754 (Resolved): erasure-code: jerasure plugin does not check parameters properly
"proposed fix":https://github.com/ceph/ceph/pull/2442
wrong parameter checks which can make the whole thing SEGV.
...
Loïc Dachary
09:07 AM devops Bug #6720 (Rejected): mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
mkcephfs is deprecated Ian Colle
08:20 AM devops Bug #6720: mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
Agree with Alfredo. It also seems a bit unsustainable to keep working on deprecated tools in the long run. If we have... Joao Eduardo Luis
07:54 AM devops Bug #6720: mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
Why are we opening bugs (or even attempting to work on) something that is deprecated?
If I'm a user who wants to g...
Alfredo Deza
08:16 AM CephFS Fix #6753: cephx authentication for mds seem to accept both "allow" and "allow *"
I opened https://github.com/ceph/ceph/pull/844 to fix the typo in the documentation. David Moreau Simard
08:15 AM CephFS Fix #6753 (Closed): cephx authentication for mds seem to accept both "allow" and "allow *"
Documentation at http://ceph.com/docs/master/rados/operations/authentication/ refers to both "mds 'allow'" and "mds '... David Moreau Simard
05:23 AM devops Bug #6745: ceph-deploy gatherkeys on ubuntu dumpling
I didn't see output for the monitors. How did those where deployed? did they have quorum before attempting to `gather... Alfredo Deza
12:35 AM Documentation #6749 (Resolved): Documentation says CRUSH bucket weights are integers
The documentation (http://ceph.com/docs/master/rados/operations/crush-map/) says
bucket weights are "double integers...
Christian Theune

11/11/2013

04:18 AM Bug #5239: osd: Segmentation fault in ceph-osd / tcmalloc
For future reference the culprit seems be the version of leveldb in debian wheezy. We've built a newer leveldb packag... Emil Renner Berthing

11/10/2013

04:49 PM CephFS Bug #6609: teuthology rsync workunit failure
> 2013-11-01T13:37:12.841 DEBUG:teuthology.orchestra.run:Running [10.214.133.35]: 'sudo rm -rf -- /home/ubuntu/cephte... Zheng Yan
12:50 PM rgw Feature #6748 (Resolved): Return bucket name in response header
Make a configurable setting where you can enable the RGW to return the bucket name of the bucket the request came for... Wido den Hollander
12:49 PM rgw Feature #6747 (Fix Under Review): PowerDNS backend for RGW bucket directing
See this blueprint: http://wiki.ceph.com/01Planning/02Blueprints/Firefly/PowerDNS_backend_for_RGW Wido den Hollander

11/09/2013

10:27 AM devops Bug #6726: Official packages do not appear to be available for Saucy
Conversation I had with Mark and Peter on IRC seemed to indicate it had just been overlooked, since the packages are ... Graeme Nordgren
09:26 AM devops Bug #6726: Official packages do not appear to be available for Saucy
Is there something blocking this, or just have to get to it? :) Greg Farnum
12:11 AM devops Bug #6726: Official packages do not appear to be available for Saucy
This is even more important now that 0.72 (emperor) packages are available in the repo for previous releases, but not... Graeme Nordgren
07:50 AM devops Bug #6746: ceph-release rpm not playing well with yum-plugin-priorities
whoops sorry this isn't really devops related. feel free to move... John Kinsella
07:35 AM devops Bug #6746 (Resolved): ceph-release rpm not playing well with yum-plugin-priorities
We use the yum-plugin-priorities to organize things across 10ish public/private repos we define on our servers. Just ... John Kinsella

11/08/2013

10:43 PM Bug #6683 (Resolved): mon: MonmapMonitor: specify epoch in 'mon getmap'
Sage Weil
05:53 PM devops Bug #6745 (Can't reproduce): ceph-deploy gatherkeys on ubuntu dumpling
bootstrap-osd/ceph.keyring and bootstrap-mds/ceph.keyring are not created.
Also there is a spelling error (WARNIN)...
Anonymous
04:21 PM Bug #6744 (Closed): make install fails when --prefix specified
Tries to install stuff into /sbin Yehuda Sadeh
04:14 PM CephFS Bug #6609: teuthology rsync workunit failure
/a/teuthology-2013-10-31_23:01:45-kcephfs-next-testing-basic-plana/78406
I haven't checked what this is doing any ...
Greg Farnum
03:53 PM CephFS Bug #5753: ceph-fuse: segfault when getting back a traceless rename op
/a/teuthology-2013-11-04_19:01:34-fs-dumpling-testing-basic-plana/84738 Greg Farnum
03:41 PM CephFS Bug #6742 (Resolved): failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
/a/teuthology-2013-11-07_23:01:12-fs-next-testing-basic-plana/89851 (I copied the MDS and client logs, which have ful... Greg Farnum
03:33 PM rgw Bug #6611: RGW: Using underscores when setting headers returns 403
Did some investigation. The problem originates in apache at ap_create_environment() that normalizes all the env vars ... Yehuda Sadeh
03:30 PM rgw Bug #6152: New S3 auth code fails when using response-* query string params to override response ...
This is still waiting for merge (sent a pull request a week ago). Yehuda Sadeh
03:27 PM CephFS Bug #6741 (Can't reproduce): failed snaptest-2.sh; got ENOTEMPTY on should-be empty dir
http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-07_23:01:12-fs-next-testing-basic-plana/89873/... Greg Farnum
02:27 PM rgw Bug #6672 (Resolved): normal PUT object requests got 405 sometimes
Added signed-off by (with approval of author) and merged into dumpling at commit:372f62717c56d9ab883ae2942e13d6d8d37c... Yehuda Sadeh
02:12 PM Documentation #6727: pg docs still imply pg_num must be set at creation, but we have split
IMO http://ceph.com/docs/master/rados/operations/pools/ should also be updated in the "set" section to specifically e... Graeme Nordgren
01:42 PM CephFS Bug #6608: samba teuthology dbench failure
None of these failed tests are running it in parallel:... Greg Farnum
12:47 PM Fix #6673 (Resolved): 'osd pool set metadata pg_num 34' broken
This got merged a while ago. Greg Farnum
11:10 AM Bug #6737 (Resolved): ceph package rebuild needed for emperor on fedora18
The fedora18 gitbuilder had crashed, so had not built the request sha1. The gitbuilder has been restarted. Anonymous
09:20 AM Bug #6737 (Resolved): ceph package rebuild needed for emperor on fedora18
logs: ubuntu@teuthology:/a/teuthology-2013-11-08_05:35:02-upgrade-small-next-testing-basic-vps/90343... Tamilarasi muthamizhan
11:01 AM Bug #6740 (Resolved): ceph-disk --dmcrypt flag does not work when journals and backing stores are...
See ceph-users thread: http://www.mail-archive.com/ceph-users@lists.ceph.com/msg05426.html
It appears that the com...
Greg Farnum
09:59 AM Bug #6738: lockdep segfault when running rados test.sh
ubuntu@teuthology:/a/teuthology-2013-11-07_23:00:08-rados-next-testing-basic-plana/89753 Tamilarasi muthamizhan
09:58 AM Bug #6738: lockdep segfault when running rados test.sh
ubuntu@teuthology:/a/teuthology-2013-11-07_23:00:08-rados-next-testing-basic-plana/89749 Tamilarasi muthamizhan
09:56 AM Bug #6738 (Duplicate): lockdep segfault when running rados test.sh
logs: ubuntu@teuthology:/a/teuthology-2013-11-07_23:00:08-rados-next-testing-basic-plana/89747... Tamilarasi muthamizhan
09:23 AM rbd Bug #5426: librbd: mutex assert in perfcounters::tinc in librbd::AioCompletion::complete()
latest logs: ubuntu@teuthology:/a/teuthology-2013-11-08_05:35:02-upgrade-small-next-testing-basic-vps/90358 Tamilarasi muthamizhan
05:46 AM Bug #6736 (Resolved): Bugs in per pool IOPs/recovery statistics

So I'm playing with the new 'ceph osd pool stats' in Emperor.
Initially I had a healthy cluster (3 OSDs, 3 mons,...
John Spray

11/07/2013

10:29 PM Bug #6715: ceph is conflicting with mongodb on ubuntu 12.04 LTS
I'm using the ubuntu cloud archive for havana on top of precise. I've double checked the ceph packaging and you're in... Zoltan Arnold Nagy
10:48 AM Bug #6715: ceph is conflicting with mongodb on ubuntu 12.04 LTS
I can't find libgoogle-perftools4 in the Precise repositories (just browsing http://packages.ubuntu.com/precise/ and ... Greg Farnum
07:18 PM Documentation #3839: SSD crushmap example will not compile
It seems that the types are missing and rulesets for ssd and ssd-primary are still the same. Alexandre Marangone
11:38 AM Documentation #3839: SSD crushmap example will not compile
I believe I fixed this some time ago. Please verify. John Wilkins
05:08 PM Fix #6705 (Fix Under Review): mon: test for ping
wip-6705 ; https://github.com/ceph/ceph/pull/835 Joao Eduardo Luis
01:57 PM Feature #6735 (Resolved): scrub documentation needs to be improved (see below conversation or as ...
(12:55:58 PM) tyler.brekke@newdream.net: I think I might be missing something,
https://github.com/ceph/ceph/blob/mas...
Samuel Just
01:24 PM Documentation #789 (Resolved): Document config options
Have documented so many since 2011, we need a specific bug to address anything new.
John Wilkins
01:03 PM rgw Bug #6733: rgw readwrite test fails on next branch
it seems to be some kind of miscommunication between radosgw and apache, most likely apache is going down Tamilarasi muthamizhan
10:41 AM rgw Bug #6733 (Closed): rgw readwrite test fails on next branch
logs: ubuntu@teuthology:/a/teuthology-2013-11-05_23:01:20-rgw-next-testing-basic-plana/86307... Tamilarasi muthamizhan
11:46 AM Documentation #6734 (Closed): better TOC/version highlighting
Several users have complained that it's not intuitive which branch they are on (next/master), especially if they come... Patrick McGarry
11:26 AM Documentation #6454 (Resolved): Add dumpling to the documentation on OS/Kernel Recommendations
See http://ceph.com/docs/master/start/os-recommendations/#dumpling-0-67 John Wilkins
11:22 AM Documentation #5853 (Resolved): quickstart is freaking people out about mons and osds on same mac...
Quick-start has been rewritten. John Wilkins
10:58 AM CephFS Bug #6718 (Duplicate): handle not found error running dbench.sh
#6608 Greg Farnum
10:53 AM devops Bug #6717: Failure to remove samba packages
Is there something in that log which makes you think it's a test issue? dpkg locking conflicts are generally some inf... Greg Farnum
10:26 AM devops Bug #6720: mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
Bug appears to be caused by the following commit:
https://github.com/ceph/ceph/commit/177e2ab1cad325b875249a514bc1...
Mark Nelson
09:41 AM rgw Fix #6615 (Resolved): radosgw: data log list admin api does not include any markers
backported, commit:6917b02530103b8c86ed75592da33144b0dea168 Yehuda Sadeh
09:40 AM rgw Bug #6604 (Resolved): radosgw-agent: opstate tracking error
backported, merge commit:6917b02530103b8c86ed75592da33144b0dea168 Yehuda Sadeh
09:35 AM rgw Fix #6616 (Resolved): radosgw: system users are not handled well by read_policy()
Was backported, commit:f1fa8116d441924d44c99624829f3daa090c821c Yehuda Sadeh
09:15 AM rgw Feature #6678 (Resolved): rgw: reject writes to secondary zones
merged, commit:84fb1bf3eefe88c0f5f15034d69c171e6531bf76. Yehuda Sadeh
08:25 AM Feature #6732 (Rejected): mon: 'mon_status' should provide as much insight as 'ping'
'mon_status' is currently providing just a fraction of the information that would be useful to troubleshoot a monitor... Joao Eduardo Luis
06:47 AM Documentation #6727: pg docs still imply pg_num must be set at creation, but we have split
There is also a bug in ceph that after changing pg_num and pgp_num of a pool will display incorrect results on ceph -... Florian Wiessner
05:16 AM rbd Bug #6368: rbd nosetests keep failing with AttributeError
Issue #6648 was opened recently with a `Can't reproduce` status.
I believe that re-writing these tests to avoid gl...
Alfredo Deza

11/06/2013

09:22 PM Documentation #6731 (Closed): Document the hashpspool setting and config options
We have some fairly nice documentation on how the CRUSH tunable settings work. The very similar HASHPSPOOL flag is on... Greg Farnum
07:43 PM Bug #6725: objecter: kick_requests() resends ops that should be paused
Thanks, that's a lot nicer. It looks good to me if it survives more testing. Josh Durgin
04:56 PM Bug #6725: objecter: kick_requests() resends ops that should be paused
I looked and tested it, the fix itself didn't do the work, but I was able to find some other issues. I pushed a secon... Yehuda Sadeh
04:22 PM Bug #6730 (Won't Fix): BUG: MAX_LOCKDEP_ENTRIES too low!

As this is kernel code I don't think it is related to the branch I'm testing. This was rebased yesterday on next b...
David Zafman
03:07 PM Feature #6729 (Resolved): Make pg statistics less wrong after split
Apparently this is pretty annoying to users. As a start, we could split the parent pg's stats evenly among its child... Samuel Just
02:36 PM Bug #6633: osd: pgls vs osd restart/peering race misses objects
wip-pgls testing Samuel Just
02:35 PM Bug #6722 (Resolved): osd/PGLog.cc: 368: FAILED assert(p->version > newhead)
Samuel Just
01:43 PM Bug #6728 (Won't Fix): ceph: sbindir always points to /sbin, even if configured with --prefix
Yehuda Sadeh
01:33 PM Documentation #6727 (Resolved): pg docs still imply pg_num must be set at creation, but we have s...
[22:16] * mobile (~pvsa@82.113.121.253) has joined #ceph
[22:20] <JoeGruher> i can run 'ceph osd pool set <pool_na...
Florian Wiessner
12:00 PM devops Bug #6726: Official packages do not appear to be available for Saucy
Mark filed this for me based on a conversation on IRC. It was pointed out that the latest packages current exist in t... Graeme Nordgren
12:00 PM devops Bug #6726: Official packages do not appear to be available for Saucy
I suggest seeing this in a more general way. Was this merely an oversight or is the current workflow lacking in some... Peter Matulis
11:41 AM devops Bug #6726: Official packages do not appear to be available for Saucy
This appears to be the case for dumpling as well. Mark Nelson
11:40 AM devops Bug #6726 (Resolved): Official packages do not appear to be available for Saucy
It appears we are missing saucy packages here:
http://ceph.com/debian-emperor/dists/
But we do have them in the...
Mark Nelson
09:32 AM rbd Documentation #5006: doc: openstack configuration changes for havana
This is complete now. The bug status should be changed to resolved. John Wilkins
09:32 AM rgw Documentation #5165 (Resolved): rgw: multisite: regions and global namespace documentation
Doc branch merged. Document is now public. John Wilkins
09:31 AM rgw Documentation #5166 (Resolved): rgw: dr: async repl and DR documentation
Doc branch merged and made public. John Wilkins
08:40 AM Bug #6636: sockaddr_storage and uuid_t are not portable to other platforms
Added pull request with this patch for easier discussion
https://github.com/ceph/ceph/pull/828
Noah Watkins
08:34 AM CephFS Feature #3426 (In Progress): ceph-fuse: build/run on os x
Noah Watkins
06:11 AM devops Bug #6654 (Rejected): ceph-deploy: bootstrap requires python-virtualenv on raring
I'm closing this since this is the way it has always been. I am not sure if we really want to put effort into install... Alfredo Deza
05:31 AM devops Bug #6701 (Resolved): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invali...
Pull Request opened: https://github.com/ceph/ceph-deploy/pull/126
And merged into ceph-deploy's master branch with...
Alfredo Deza
01:02 AM devops Feature #6365: Package up newer version of Qemu for RHEL & EL6.3 and 6.4
Any update on this ? the qemu-kvm version you distribute is 6 month old.
The cleaning of async vs not async package ...
Thomas O

11/05/2013

10:11 PM Bug #6719 (Resolved): 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
Samuel Just
03:41 PM Bug #6719: 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
wip-6719 Samuel Just
02:21 PM Bug #6719: 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
Variant with logs possibly at slider:/home/samuelj/bug_logs/6719-readerror Samuel Just
02:20 PM Bug #6719 (Resolved): 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
2013-11-05T09:57:50.225 INFO:teuthology.task.rados.rados.0.out:[10.214.132.26]: rollback oid 49 to 320
2013-11-05T09...
Samuel Just
09:50 PM Bug #6722: osd/PGLog.cc: 368: FAILED assert(p->version > newhead)
Fix in wip-6719 Samuel Just
05:46 PM Bug #6722 (Resolved): osd/PGLog.cc: 368: FAILED assert(p->version > newhead)
2013-11-05 17:30:54.916068 7f9f42a66700 -1 osd/PGLog.cc: In function 'void PGLog::rewind_divergent_log(ObjectStore::T... Samuel Just
06:48 PM Bug #6725: objecter: kick_requests() resends ops that should be paused
Possible fix is in the wip-objecter-full branch, but I can't test it at the moment due to bad network conditions. Josh Durgin
06:37 PM Bug #6725 (Resolved): objecter: kick_requests() resends ops that should be paused
Operations may be paused because the osd map has a flag manually set, or the cluster is too full. In either case, ope... Josh Durgin
06:36 PM rgw Feature #6677 (Fix Under Review): rgw: add compatibility for MultipartUpload
Yehuda Sadeh
06:25 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
Done. Mark Kirkwood
06:04 AM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
Thanks for the ticket and the resolution Mark!
Would you mind sending a pull request to https://github.com/ceph/ce...
Alfredo Deza
06:14 PM rgw Feature #6678 (Fix Under Review): rgw: reject writes to secondary zones
Yehuda Sadeh
05:48 PM rgw Bug #6713: rgw master nosetests failure.
I'll keep an eye on these and close if they stay away. Anonymous
02:59 PM CephFS Bug #6655 (Can't reproduce): readdir() fails on CephFS mount symlinked directories
Zheng Yan
11:46 AM CephFS Bug #6655: readdir() fails on CephFS mount symlinked directories
I haven't been able to reproduce it again either since I remounted CephFS on the client a while ago. (And probably re... Pieter Steyn
02:52 PM devops Bug #6720 (Rejected): mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
Script that was able to produce a working cluster in 0.67.4 failed to work in 0.72rc1. This was reported externally ... Mark Nelson
02:13 PM Bug #6714 (Resolved): ENOTEMPTY on replay
Samuel Just
01:47 PM CephFS Bug #6718 (Duplicate): handle not found error running dbench.sh
http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-03_23:01:12-fs-next-testing-basic-plana/83672/teuthology.log
...
Zack Cerza
01:42 PM devops Bug #6717 (Closed): Failure to remove samba packages
http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-03_23:01:12-fs-next-testing-basic-plana/83671/teuthology.log
...
Zack Cerza
01:30 PM CephFS Bug #6613: samba is crashing in teuthology
Still happening: http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-03_23:01:12-fs-next-testing-basic-plana/83668... Zack Cerza
12:57 PM devops Bug #6708 (Resolved): apt-get key fails
Merged into ceph-deploy's master branch with hash: e6f3a3b
Idiotic that Python + subprocess freak out with quote...
Alfredo Deza
12:25 PM devops Bug #6708 (Fix Under Review): apt-get key fails
Opened Pull Request: https://github.com/ceph/ceph-deploy/pull/124 Alfredo Deza
08:06 AM devops Bug #6708 (In Progress): apt-get key fails
I was able to confirm the problem. It is possible we have not caught this before because GPG keys may already exist i... Alfredo Deza
06:46 AM devops Bug #6708: apt-get key fails
Was able to reproduce *running as root* on that same box:... Alfredo Deza
06:22 AM devops Bug #6708: apt-get key fails
Loic, is it possible you are using a proxy for those hosts? Alfredo Deza
12:34 PM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
For the debian packages, it looks like we only need to supply libgoogle-perftools for squeeze and for arm on quantal ... Anonymous
09:48 AM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
httpd-mmn is not a real package. It's a virtual dependency that was created to version the apache plug-in api. Apac... Anonymous
09:43 AM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
The rpm-emperor repo on ceph.com has been updated with the addition of the following packages:
gdisk-0.8.2-1.el6.x...
Anonymous
12:25 PM rgw Bug #6152 (Pending Backport): New S3 auth code fails when using response-* query string params to...
Yehuda Sadeh
12:25 PM rgw Bug #6152: New S3 auth code fails when using response-* query string params to override response ...
I sent a pull request a few days ago, so this should be in the next dumpling release. Yehuda Sadeh
07:09 AM Bug #6636: sockaddr_storage and uuid_t are not portable to other platforms
Awesome, thanks Alan. I'll pull this into wip-port for the time being. Noah Watkins
05:05 AM Bug #6715 (Won't Fix): ceph is conflicting with mongodb on ubuntu 12.04 LTS
I'm not sure who's to blame here, but ceph depends on libgoogle-perftools0, while mongo depends on libgoogle-perftool... Zoltan Arnold Nagy

11/04/2013

08:05 PM devops Bug #6701 (Fix Under Review): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18...
Ian Colle
08:00 PM rgw Bug #6710 (Fix Under Review): radosgw init script does not exit 1 and tell the user if the hostna...
Ian Colle
06:37 PM devops Bug #6698 (Resolved): new osds added to the cluster not starting up due to crush lookup failure
Sage Weil
02:40 PM Bug #6714: ENOTEMPTY on replay
wip-enotempty Samuel Just
11:40 AM Bug #6714 (Resolved): ENOTEMPTY on replay
Occurred while deleting a temp collection. The remaining object was a temp object which had been recreated as part o... Samuel Just
01:27 PM Feature #6568: ceph-rest-api authentication
I'm not aware of any such plans, no, Matt; we sort of look at the ceph-rest-api as
a way of allowing internal-net a...
Dan Mick
12:40 PM rgw Bug #6672 (Pending Backport): normal PUT object requests got 405 sometimes
Yehuda Sadeh
12:28 PM rgw Bug #6709: rgw upgrade test fails during readwrite
It's hard to determine what exactly happened, as we don't turn on the gateway logging anymore. But generally it looks... Yehuda Sadeh
11:51 AM rbd Feature #6432 (Resolved): Change TGT Plugin to allow setting client name for authentication
Upstream patches have been accepted. Dan Mick
11:37 AM Bug #6685 (Resolved): osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
Samuel Just
10:52 AM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
commit:72e2874f402575846d1bd84f2af865d28947bc67 ("ReplicatedPG::recover_backfill: adjust last_backfill to HEAD if sna... Greg Farnum
11:34 AM Bug #6681 (Resolved): osd recovery hung
Samuel Just
10:54 AM Bug #6681: osd recovery hung
commit:cf4e00ff5c6d0c548cb766f94288277d1f661094 ("OSD: don't clear peering_wait_for_split in advance_map()") looks go... Greg Farnum
11:25 AM Bug #6712 (Resolved): 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function '...
Samuel Just
11:19 AM Bug #6712: 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function 'OSDMapRef O...
The code looks fine, although we're arguing over how to write a clear commit message. ;) Greg Farnum
11:22 AM Bug #6711 (Duplicate): osd/OSD.h: 509: FAILED assert(ret)
#6712 Greg Farnum
11:12 AM Feature #6466 (Resolved): stgt: unlimit number of images supported
Patches accepted by upstream; ready for rebuild/rerelease. Dan Mick
10:37 AM rgw Bug #6713 (Can't reproduce): rgw master nosetests failure.
Using the following yaml files:... Anonymous
10:36 AM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
Just testing SSL installation on CentOS. When installing mod_ssl, it picks up mod_ssl.x86_64 1:2.2.22-1.ceph.el6, but... John Wilkins
09:40 AM devops Bug #6482 (Resolved): ceph-deploy install on Centos: can't find wget?
Merged into ceph-deploy's master branch with hash: 91618ba Alfredo Deza
07:54 AM devops Bug #6482 (Fix Under Review): ceph-deploy install on Centos: can't find wget?
Opened pull request: https://github.com/ceph/ceph-deploy/pull/123 Alfredo Deza
05:43 AM devops Bug #6482: ceph-deploy install on Centos: can't find wget?
After discussing with ksingh, I can confirm this is still a bug and ceph-deploy should install wget before it attempt... Alfredo Deza

11/03/2013

09:09 PM Bug #6712: 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function 'OSDMapRef O...
wip-6685 also Samuel Just
08:50 PM Bug #6712 (Resolved): 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function '...
ubuntu@teuthology:/a/samuelj-2013-11-03_17:07:25-rados-wip-6685-testing-basic-plana/82408/remote
-4> 2013-11-0...
Samuel Just
08:47 PM Bug #6711 (Duplicate): osd/OSD.h: 509: FAILED assert(ret)
2013-11-03 19:33:02.081330 7f10e5289700 1 -- 10.214.132.16:6804/7111 >> :/0 pipe(0x2a57780 sd=55 :6804 s=0 pgs=0 cs=... Samuel Just
11:12 AM Bug #6681: osd recovery hung
Samuel Just
11:11 AM Bug #6681: osd recovery hung
I may have a fix in wip-6685 Samuel Just
11:12 AM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
Samuel Just
05:25 AM rgw Bug #6710: radosgw init script does not exit 1 and tell the user if the hostname does not match
Went a bit beyond the scope, saw that there were more improvements that could be made.
https://github.com/ceph/ceph/...
David Moreau Simard
04:14 AM rgw Bug #6710: radosgw init script does not exit 1 and tell the user if the hostname does not match
I will submit a pull request for this. David Moreau Simard
04:14 AM rgw Bug #6710 (Resolved): radosgw init script does not exit 1 and tell the user if the hostname does ...
If the hostname of the machine and the host configured for radosgw in ceph.conf, we exit 0 without telling the user a... David Moreau Simard

11/02/2013

09:34 PM rgw Bug #6709 (Can't reproduce): rgw upgrade test fails during readwrite
... Sage Weil
07:54 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
Figured out what the issue with shutil.move was - needed to close the temp file before moving. Not an issue with os.r... Mark Kirkwood
06:25 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
I now know why the original error is happening. My previous musings were not really on the mark (as it were):
cons...
Mark Kirkwood
05:40 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
I'm possibly causing the issue using shutil.move (can't see how mind you)... Mark Kirkwood
05:28 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
Further on this (post the os.rename -> shutil.move), the next problem is:... Mark Kirkwood
05:46 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
wip-6685 Samuel Just
01:56 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
Got it with logging, working on branch. Samuel Just
01:36 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
ubuntu@teuthology:/a/teuthology-2013-11-01_23:00:05-rados-next-testing-basic-plana/80229/remote Samuel Just
04:31 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
Ivan Kudryavtsev wrote:
> is it possible you've done any shenanigans like 'ceph pg force_create_pg ...' at some poin...
Sage Weil
11:56 AM devops Bug #6708 (Resolved): apt-get key fails
on a brand new precise-12.04.3 : ... Loïc Dachary
12:15 AM Feature #6707 (Fix Under Review): Backport per pool iops data to 0.67
Sage Weil

11/01/2013

09:58 PM Feature #6707 (Rejected): Backport per pool iops data to 0.67
Neil Levine
07:06 PM rgw Fix #6615: radosgw: data log list admin api does not include any markers
backport: https://github.com/ceph/ceph/pull/804 Josh Durgin
07:06 PM rgw Fix #6616: radosgw: system users are not handled well by read_policy()
backport: https://github.com/ceph/ceph/pull/804 Josh Durgin
07:06 PM rgw Bug #6604: radosgw-agent: opstate tracking error
https://github.com/ceph/ceph/pull/805 Josh Durgin
06:33 PM rgw Bug #6694 (Duplicate): radosgw upstart script doesn't provide the -n parameter
The name is determined by looking for client sections in ceph.conf that have 'host' matching the machine name. You ha... Josh Durgin
04:44 PM devops Bug #6698 (Fix Under Review): new osds added to the cluster not starting up due to crush lookup f...
Sage Weil
03:49 PM Fix #6673 (Fix Under Review): 'osd pool set metadata pg_num 34' broken
https://github.com/ceph/ceph/pull/802 Greg Farnum
03:27 PM Fix #6673: 'osd pool set metadata pg_num 34' broken
Actually, looking at this again, I've realized that the teuthology test is running the rados api tests, which involve... Greg Farnum
02:35 PM Fix #6673 (In Progress): 'osd pool set metadata pg_num 34' broken
We saw this again; /a/dzafman-2013-10-31_14:29:25-rados-wip-flush-5855-testing-basic-plana/77511.
The PGs were indee...
Greg Farnum
03:01 PM Bug #6636: sockaddr_storage and uuid_t are not portable to other platforms
Here's a patch that fixes the problem for struct sockaddr_storage. I haven't looked at uuid_t yet. Googling suggest... Alan Somers
02:04 PM Bug #6003 (Need More Info): journal Unable to read past sequence 406 ...
af0c5b415ad5a863940add7c2e8f3c1ac8040ef2 should fix the most recent iteration, but it doesn't explain the instances p... Samuel Just
01:53 PM rgw Bug #6706 (Resolved): radosgw init script and hostname variations
https://github.com/ceph/ceph/pull/801 Sage Weil
01:30 PM rgw Bug #6706 (Resolved): radosgw init script and hostname variations
Relevant snippet:... David Moreau Simard
12:08 PM Fix #6705 (Resolved): mon: test for ping
Sage Weil
11:13 AM RADOS Feature #6704 (New): osd: ability to move a pg to the front (or back?) of the backfill/recovery q...
sometimes the admin wants to explicitly prioritize backfill/recovery on a specific pg. Sage Weil
10:23 AM Bug #5951 (Resolved): osd: next: EEXIST on mkcoll
Samuel Just
09:30 AM Bug #5818: leveldb 1.12: hang on shutdown (mon)
ubuntu@plana83:~$ dpkg -l | grep leveldb
rc libleveldb1 1.12.0-1precise.cep...
Sage Weil
09:30 AM Bug #5818: leveldb 1.12: hang on shutdown (mon)
well, observed this hang with 1.12.
ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2013-10-31_23:00:13-...
Sage Weil
09:01 AM devops Bug #6699 (Fix Under Review): sysvinit script setting incorrect OSD weights
if it works on rhel and ubuntu, that is good enough for now. will push to next. Sage Weil
08:54 AM devops Bug #6699: sysvinit script setting incorrect OSD weights
Changing 'df' to 'df -P' in the ceph sysvinit script fixes the issue, not sure how portable this flag is. Jan Harkes
08:52 AM devops Bug #6703 (Resolved): OSDs with dmcrypt fail to start at boot
RHEL6.4 ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7)
On boot the OSDs are not started when th...
Jan Harkes
08:23 AM devops Bug #6482: ceph-deploy install on Centos: can't find wget?
Hi Alfredo
You can close this this is now fixed
karan singh
01:17 AM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
The particular issue is caused by os.rename in ceph_deploy/hosts/remotes.py line 54. replacing that with shutil.move... Mark Kirkwood

10/31/2013

11:08 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
Omitted the probably significant fact that /data2 is a partition in a different disk from /var Mark Kirkwood
11:05 PM devops Bug #6701 (Resolved): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invali...
Ceph version is 0.71-234-g1f02d00 built from src on bunti 13.10.
The desired setup is osd data in /data2/cephdata...
Mark Kirkwood
08:39 PM Bug #6700 (Resolved): ceph-disk prepare fails to set up an encrypted disk
RHEL 6.4 ceph 0.67.4
Ceph-disk prepare fails when setting up encrypted disks when the data and journal are locat...
Jan Harkes
07:30 PM devops Bug #6699 (Resolved): sysvinit script setting incorrect OSD weights
I noticed that the sysvinit script on a RHEL6.4 system was giving more weight to disks that were fuller. This is caus... Jan Harkes
05:50 PM Bug #6003: journal Unable to read past sequence 406 ...
This looks like the same problem seen in a unit test.
2013-10-31T15:23:41.910 INFO:teuthology.orchestra.run.err:[1...
David Zafman
10:21 AM Bug #6003: journal Unable to read past sequence 406 ...
Sage Weil
10:09 AM Bug #6003: journal Unable to read past sequence 406 ...
Kernel: 3.8.0-21-generic Samuel Just
10:07 AM Bug #6003: journal Unable to read past sequence 406 ...
slider:~samuelj/bug_logs/13-10-31-04:05:43/remote
With journal logging! I seem to be able to reproduce this prett...
Samuel Just
08:28 AM Bug #6003: journal Unable to read past sequence 406 ...
/a/teuthology-2013-10-28_23:01:45-kcephfs-master-testing-basic-plana/73927 Greg Farnum
05:10 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
is it possible you've done any shenanigans like 'ceph pg force_create_pg ...' at some point in the past?
Yes, inde...
Ivan Kudryavtsev
01:52 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
oh, looking more closely, it looks like osd.42 is trying to activate but has no data for this pg. is it possible you... Sage Weil
11:06 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
Any chance I can start these OSD without losing data on them? Ivan Kudryavtsev
11:01 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
... Sage Weil
10:30 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
Ivan Kudryavtsev wrote:
> I also have
>
> 2013-11-01 00:13:25.635829 mon.0 [WRN] mon.2 10.252.0.4:6789/0 clock s...
Sage Weil
10:14 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
I also have
2013-11-01 00:13:25.635829 mon.0 [WRN] mon.2 10.252.0.4:6789/0 clock skew 0.870987s > max 0.05s
2013...
Ivan Kudryavtsev
10:14 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
Here it is, attached. Ivan Kudryavtsev
09:36 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
can you turn up the logs on osd.42 too, and then restart this same osd (to reproduce the same crash), and then attach... Sage Weil
04:59 PM devops Bug #6698: new osds added to the cluster not starting up due to crush lookup failure
... Tamilarasi muthamizhan
04:58 PM devops Bug #6698 (Resolved): new osds added to the cluster not starting up due to crush lookup failure
ceph version: next branch [0.72-rc1]
test setup : mira025, mira038
had a cluster running on master branch, upgr...
Tamilarasi muthamizhan
03:26 PM Bug #6697 (Resolved): strncmp(3) must not be used on binary data
strncmp(3) is intended to compare C strings, so it quits comparing after encountering a null character. The Linux ma... Alan Somers
03:18 PM Bug #5951: osd: next: EEXIST on mkcoll
592a99ccd8da2d7843ebb7ce6439566732580b7a Samuel Just
01:24 PM Bug #5951: osd: next: EEXIST on mkcoll
Found it. See wip-eexist. Samuel Just
02:55 PM rgw Bug #6696 (Can't reproduce): Upgrade rgw failure in nightly tests. (/home/ubuntu/cephtest/s3-test...
Suite tests using the following sets of yaml files:... Anonymous
02:43 PM rbd Bug #6695 (Won't Fix): Upgrade rbd failure in nightly tests. (mkdir --p ..)
The overnight test for the following set of yaml files:... Anonymous
12:18 PM rgw Bug #6694 (Duplicate): radosgw upstart script doesn't provide the -n parameter
I'm using the Ceph Chef Cookbook to create and manage the configuration of a new Ceph 0.67.4 cluster on Ubuntu 12.10,... Walter Huf
11:19 AM Bug #6680 (Resolved): libcephfs.h broken for gcc
Noah Watkins
11:19 AM Bug #6680: libcephfs.h broken for gcc
d3b56918698803ce441d9b1ef0185caebed4d433 Noah Watkins
09:40 AM Bug #6680: libcephfs.h broken for gcc
it was a cleanup for clang to not mix class vs struct. that's secondary, though.. this should definitely build in c. Sage Weil
11:04 AM rgw Bug #5843: swift api: x-container-meta-{key} should not be allowed on an object
It came from a customer, after discussions it is not causing any issue at all. We can leave it how it is. Alexandre Marangone
09:50 AM rbd Bug #6693: "rbd ls" returns error if the pool empty
I agree the exit code should be 0 in this case, but I'm downgrading the priority since there's an easy workaround and... Josh Durgin
05:33 AM rbd Bug #6693 (Resolved): "rbd ls" returns error if the pool empty
if the pool exists but only empty, instead of not returning anything, it returns an error message that it's empty, an... Zoltan Arnold Nagy
09:14 AM CephFS Bug #6613 (Need More Info): samba is crashing in teuthology
Zheng Yan
09:05 AM CephFS Bug #6608: samba teuthology dbench failure
not sambe issue. It's wrong to run two instances of dbench on the same test directory. Zheng Yan
08:01 AM CephFS Bug #6608 (Need More Info): samba teuthology dbench failure
If samba is broken in this configuration we'll need to change our test (and report upstream, if we aren't using it in... Greg Farnum
07:45 AM devops Bug #6691 (Resolved): debian wheezy fails to be detected by ceph-deploy
Opened pull request: https://github.com/ceph/ceph-deploy/pull/121
Merged into ceph-deploy master branch with hash;...
Alfredo Deza

10/30/2013

11:44 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
It's on dumpling node.
root@ceph-osd-1-2:/# ceph -v
ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7)
Ivan Kudryavtsev
10:17 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
Here it is. Ivan Kudryavtsev
09:34 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
Can you set 'debug osd = 20' and 'debug ms = 1', and restart and osd so that we have a complete log leading up to the... Sage Weil
09:06 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
During recovery some osds crashing and after crash unable to up again, looks like in topic "Domino crash" regarding o... Ivan Kudryavtsev
07:35 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
Also it seems, that my meta information is corrupted, I have a lot of images in RBD and now, I don't see any:
rbd:...
Ivan Kudryavtsev
06:25 AM Bug #6684 (Rejected): osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log....
Hello, Guys, I'm in a big trouble. After upgrading Bobtail to cuttlefish I met in a
great troubles, unable to make ...
Ivan Kudryavtsev
09:03 PM Bug #6003: journal Unable to read past sequence 406 ...
I hit this same problem in recovery testing. Attached is the complete log. David Zafman
07:29 PM rgw Bug #6672: normal PUT object requests got 405 sometimes
Send the pull request against the 'next' branch at https://github.com/ceph/ceph/pull/795 Xiangyu Lv
04:56 PM rgw Bug #5374 (Resolved): Avoid relying on keystone's admin token
Merged, commit:b20d1bf33bdf6ed25ef9bb37afc3890282ece6d4 Yehuda Sadeh
04:52 PM devops Bug #6690 (Resolved): ceph-deploy: bogus apt sources.list.d/ceph.list file on wheezy
Sage Weil
03:24 PM devops Bug #6690: ceph-deploy: bogus apt sources.list.d/ceph.list file on wheezy
... Sage Weil
02:25 PM devops Bug #6690 (Resolved): ceph-deploy: bogus apt sources.list.d/ceph.list file on wheezy
... Sage Weil
04:23 PM Bug #6692 (Resolved): Documentation: Command line incorrect.
In http://ceph.com/docs/master/rados/operations/authentication/
the enabling cephx command #2 that reads:
ceph-...
Anonymous
04:11 PM Fix #5612 (Resolved): down mons prevents osd hosts from booting properly
Sage Weil
04:10 PM devops Bug #6691 (Resolved): debian wheezy fails to be detected by ceph-deploy
platform module in Python does not mention `wheezy` at all.
Alfredo Deza
12:29 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
It did happen immediately after transitioning from being the backfill target Samuel Just
12:27 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
Or not, we process the OP_BACKFILL_PROGRESS ordered vs the removal ops. Samuel Just
10:50 AM Bug #6685 (In Progress): osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
Samuel Just
10:50 AM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
Urgh, this is because we can advance last_backfill past uncompleted removal ops during backfill. Working on it. Samuel Just
09:56 AM Bug #6685 (Resolved): osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
... Sage Weil
12:29 PM Bug #6689 (Resolved): osd: remove_redundant_pg_temp() can be slow on big clusters
it blocks the cpu, which prevents us from processing lease acks, which makes us call a new election.
observed on b...
Sage Weil
12:26 PM Fix #6116: osd: incomplete pg from thrashing on next
Removed the teuthology workaround as well. Samuel Just
12:24 PM Fix #6116 (Resolved): osd: incomplete pg from thrashing on next
I was way off on this one. We do ack the backfill completion. I suspect that the actual problem was probably fixed ... Samuel Just
11:37 AM rbd Documentation #5006: doc: openstack configuration changes for havana
Should get this updated prior to HK Summit if at all possible. Ian Colle
10:42 AM Feature #6687 (New): Ability to set up/down/in/out based on CRUSH hierarchy
It would be nice if the cli tools supported the ability to perform OSD state transitions based of CRUSH hierarchy. Fo... Kyle Bader
10:10 AM Bug #6686 (Resolved): segfault in prioritized queue dequeue
I've been hitting the following fault on OSX. At first I suspected a nasty bug mixing libc++ with libstdc++ on accide... Noah Watkins
05:36 AM Bug #6683 (Fix Under Review): mon: MonmapMonitor: specify epoch in 'mon getmap'
Joao Eduardo Luis
05:36 AM Bug #6683: mon: MonmapMonitor: specify epoch in 'mon getmap'
wip-6683, https://github.com/ceph/ceph/pull/789 and commit:e5efd882e7eba360158c2218bbd197766d082b02 Joao Eduardo Luis
05:30 AM Bug #6683: mon: MonmapMonitor: specify epoch in 'mon getmap'
mon/MonCommands.h specifies that 'mon getmap' receives an epoch number, but we don't handle it and always return the ... Joao Eduardo Luis
04:16 AM Bug #6683 (Resolved): mon: MonmapMonitor: specify epoch in 'mon getmap'
Joao Eduardo Luis
04:13 AM Subtask #3605 (Resolved): mon: print lookup path when reporting -ENOENT to user-space
AFAICT this has been addressed in multiple occasions throughout the last year. Joao Eduardo Luis
04:12 AM Subtask #2674 (Rejected): mon: Single-Paxos: mon commits suicide after remove&add
works just fine on current next Joao Eduardo Luis
04:08 AM Bug #5967 (Resolved): monitor caps parser should accept '.' as a legal unquoted character
See commit:1f573c885cce776a7e8b2f3d4b91a304a8bc15c5 Joao Eduardo Luis
01:24 AM Bug #6614 (Resolved): unittest_bufferlist failed
Good to know, thanks for the feedback :-) Loïc Dachary

10/29/2013

09:40 PM Documentation #6682 (Resolved): Adjustments to the Ceph Quick installation guide
Hello,
Following the Quick installation guide, I ran into some issues which I think should be addressed:
[PREFL...
Alex Mendes
09:27 PM Bug #6614: unittest_bufferlist failed
Test ”make check“ succeed now. huang jun
08:42 PM rgw Bug #6672: normal PUT object requests got 405 sometimes
Updated with a new commit. Please help review it at https://github.com/ceph/ceph/pull/783 Xiangyu Lv
09:30 AM rgw Bug #6672 (Fix Under Review): normal PUT object requests got 405 sometimes
Ian Colle
08:00 AM rgw Bug #6672: normal PUT object requests got 405 sometimes
Thanks, the initialization code is indeed not safe. See my comment in the pull request. Yehuda Sadeh
04:57 AM rgw Bug #6672: normal PUT object requests got 405 sometimes
A quick fix is to set initialized to true after populating table in hex_to_num().
Please help review the fix at: h...
Xiangyu Lv
04:53 AM rgw Bug #6672 (Resolved): normal PUT object requests got 405 sometimes
Under load testing, normal PUT object requests got 405 sometimes. The root cause is that there is a defect in hex_to_... Xiangyu Lv
07:45 PM CephFS Bug #6608 (Rejected): samba teuthology dbench failure
running dbench on local FS in parallel also results in similar failures. Zheng Yan
11:43 AM CephFS Bug #6608: samba teuthology dbench failure
http://qa-proxy.ceph.com/teuthology/teuthology-2013-10-27_19:01:26-fs-dumpling-testing-basic-plana/71285/
http://qa-...
Greg Farnum
05:09 PM Bug #6681: osd recovery hung
Not an rc blocker. This is actually a fairly old bug and can be worked around by restarting one of the effected osds... Samuel Just
05:07 PM Bug #6681 (Resolved): osd recovery hung
ubuntu@teuthology:/a/teuthology-2013-10-29_10:45:19-rados-next-testing-basic-plana
ubuntu@plana31:~$ sudo ceph -s
...
Samuel Just
04:36 PM Bug #6605 (Resolved): mon: remove full osd state on "osd rm"
merged into master Joao Eduardo Luis
04:29 PM Bug #6605 (Fix Under Review): mon: remove full osd state on "osd rm"
wip-6605, pull request 787, commit:e02740ac5da7c9f5e4c1fdd603918e56c05123de
Greg Farnum wrote:
> It turns out t...
Joao Eduardo Luis
04:21 PM Bug #6680 (Resolved): libcephfs.h broken for gcc
This patch https://github.com/ceph/ceph/commit/e1666d0400ecef464d33480e4290896404b7a7bd did the following:... Noah Watkins
04:08 PM Bug #6679 (Resolved): throttle: transient unit test failure
... Sage Weil
03:49 PM rgw Feature #6678 (Resolved): rgw: reject writes to secondary zones
Non-master zones are intended to be read-only. For disaster recovery, it may be useful to expose a secondary zone as ... Josh Durgin
03:13 PM Bug #6633: osd: pgls vs osd restart/peering race misses objects
list.cc actually has slightly flaky tests if they are concurrent with pg splitting. wip-6633 so far has fixes to tol... Samuel Just
02:40 PM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
removing 'related to' link from this ticket to #5869 Joao Eduardo Luis
10:22 AM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
Is it possible to recover without those osds? Samuel Just
10:22 AM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
This is not actually related to 5869. Samuel Just
04:13 AM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
If this is indeed the same bug, and not a different iteration, then maybe we should backport the fix at least for Dum... Joao Eduardo Luis
02:37 PM rgw Feature #6677 (Resolved): rgw: add compatibility for MultipartUpload
The AWS-PHP-SDK used MultipartUpload instead of CompleteMultipartUpload.
https://github.com/aws/aws-sdk-php/issues...
Tyler Brekke
02:03 PM RADOS Fix #6676 (New): osd should never block getting filestore throttle while holding the pg lock
Currently, it is possible for the filestore finisher threads to block trying to acquire a pg lock in order to complet... Samuel Just
12:45 PM rgw Bug #6152: New S3 auth code fails when using response-* query string params to override response ...
This is still broken on 0.67.4. Please consider backporting the fix (7a7361d7) to Dumpling. Benjamin Gilbert
12:12 PM devops Bug #6587 (Resolved): ceph-deploy does not use cluster arg for mon error checking
Merged into ceph-deploy master branch with hash: da03ab7 Alfredo Deza
12:12 PM devops Bug #6675 (Resolved): Fedora fails to import keys on ceph-deploy install
Merged into ceph-deploy master branch with hash: 5145d94
It makes sure that all `rpm --import` commands are the ...
Alfredo Deza
11:24 AM devops Bug #6675 (Fix Under Review): Fedora fails to import keys on ceph-deploy install
PR opened: https://github.com/ceph/ceph-deploy/pull/119
Alfredo Deza
09:34 AM devops Bug #6675 (Resolved): Fedora fails to import keys on ceph-deploy install
The command was fixed for CentOS but not for fedora:... Alfredo Deza
11:15 AM Fix #6673 (Resolved): 'osd pool set metadata pg_num 34' broken
Sage Weil
09:40 AM Fix #6673 (Fix Under Review): 'osd pool set metadata pg_num 34' broken
Sage Weil
08:57 AM Fix #6673: 'osd pool set metadata pg_num 34' broken
the test later fails with... Sage Weil
08:55 AM Fix #6673: 'osd pool set metadata pg_num 34' broken
What's broken about this, besides the ridiculous parsing output? We deliberately prevent splitting while creating the... Greg Farnum
08:50 AM Fix #6673 (Resolved): 'osd pool set metadata pg_num 34' broken
... Sage Weil
11:05 AM Bug #6207 (Resolved): Found incorrect object contents
This can be explained by prematurely updating last_backfill based on backfill_pos and backfills_in_flight.
ad5655b...
Samuel Just
10:28 AM Bug #6674: Busted client locking
I don't think the theory from the previous comment has any merit! I wonder if it's possible that lockdep is missing a... Noah Watkins
09:51 AM Bug #6674: Busted client locking
Looking at initializations of CephContext, I don't see any explicit freeing, which begs the question which thread is ... Noah Watkins
09:24 AM Bug #6674 (Resolved): Busted client locking
... Greg Farnum
08:34 AM devops Feature #3311 (Resolved): ceph-deploy: pushy bug triggers on interpreter shutdown
We no longer use Pushy Alfredo Deza
08:33 AM devops Feature #3312 (Resolved): ceph-deploy: pushy uses pickle, that's a security problem
We no longer use Pushy. Alfredo Deza
07:57 AM devops Fix #4953 (Resolved): ceph-deploy: dns mismatches can cause gatherkeys to fail
ceph-deploy will now check for common errors (like the one mentioned) when deploying monitors. Alfredo Deza
07:46 AM devops Feature #4998 (Resolved): ceph-deploy should allow user specified installation sources
with the new `--no-adjust-repos` we rely on users setting up their own mirrors.
A new 'firewall-install` command i...
Alfredo Deza
07:45 AM devops Feature #3922 (Resolved): ceph-deploy: version command
ceph-deploy has a `--version` command now. Alfredo Deza
01:42 AM Bug #6400: osd crashed in dumpling due to unexpected error (EEXIST?)
I can reproduce this issue on CentOS 6.4 machine every time.
My steps are:
# Build Ceph with dumpling release c...
Tengwei Cai

10/28/2013

11:16 PM Bug #6671 (Can't reproduce): FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)

We were on 0.56.1; we shut down all of the OSDs and mon on a particular node, rebooted the node for maintenance an...
Tom Lanyon
09:03 PM CephFS Bug #6613: samba is crashing in teuthology
tail of client log:
---
2013-10-22 08:05:27.405155 7ff1167fc700 20 client.4105 trim_cache size 0 max 0
2013-10-22 ...
Zheng Yan
10:05 AM CephFS Bug #6613: samba is crashing in teuthology
This is happening regularly on dumpling and next, but I don't think I've seen it on cuttlefish. We've clearly done so... Greg Farnum
04:15 PM Bug #6003: journal Unable to read past sequence 406 ...
ubuntu@teuthology:/a/teuthology-2013-10-27_19:00:21-rados-dumpling-testing-basic-plana/71126 Sage Weil
04:06 PM Bug #6585 (Resolved): osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
Sage Weil
01:51 PM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
I've merged wip-6585, but it's not quite fixed yet. Samuel Just
09:51 AM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
We think we might have fixed this, or at least most of it — but testing is shaking out a lot of long-standing bugs in... Greg Farnum
04:06 PM rgw Bug #6621 (Resolved): quota: the max-size and max-objects value when zero
Sage Weil
01:54 PM rgw Bug #6621: quota: the max-size and max-objects value when zero
There was an actual issue with setting negative values, so moved that back into the bug tracker. A fix was pushed to ... Yehuda Sadeh
12:00 PM rgw Bug #6621: quota: the max-size and max-objects value when zero
ok got it, if its by design, i think we should add this to the radosgw-admin help page saying that we allow max size ... Tamilarasi muthamizhan
11:53 AM rgw Bug #6621: quota: the max-size and max-objects value when zero
Oh, I thought the problem was the other way around (that it wasn't accepting 0 or negative numbers). That's by design... Yehuda Sadeh
01:37 PM devops Bug #6587 (Fix Under Review): ceph-deploy does not use cluster arg for mon error checking
Opened pull request: https://github.com/ceph/ceph-deploy/pull/116 Alfredo Deza
01:13 PM devops Bug #6650 (Resolved): ceph-deploy: should exit/error when apt fails
Merged into ceph-deploy's master branch with hash: 1a72df3 Alfredo Deza
10:58 AM devops Bug #6650 (Fix Under Review): ceph-deploy: should exit/error when apt fails
Pull request opened: https://github.com/ceph/ceph-deploy/pull/115 Alfredo Deza
06:48 AM devops Bug #6650 (In Progress): ceph-deploy: should exit/error when apt fails
Just to be clear, this is not ceph-deploy failing to install, it is APT having issues with the install.
The first ...
Alfredo Deza
11:59 AM devops Bug #6654: ceph-deploy: bootstrap requires python-virtualenv on raring
the bootstrap script does not install python-virtualenv but it does check for it just to error out. Alfredo Deza
11:42 AM Bug #6633: osd: pgls vs osd restart/peering race misses objects
Logs on slider:/~samuelj/buglogs/13-10-27-15:50:48 Samuel Just
10:30 AM Bug #6598: osd crash after recreating pool with same name (cuttlefish + bobtail?)
Seems that we had accidentaly triggered existing assert() and in the wild it would be almost impossible to reproduce ... Andrey Korolyov
10:04 AM CephFS Bug #6608: samba teuthology dbench failure
http://qa-proxy.ceph.com/teuthology/teuthology-2013-10-25_23:01:10-fs-master-testing-basic-plana/69202/
http://qa-pr...
Greg Farnum
09:40 AM CephFS Bug #6655 (Need More Info): readdir() fails on CephFS mount symlinked directories
Sage Weil
09:21 AM rbd Bug #5425: krbd: xfstest 89 hang, 'read_partial_message skipping long message'
ubuntu@teuthology:/a/teuthology-2013-10-26_23:01:27-krbd-next-testing-basic-plana/70455 Sage Weil
06:43 AM devops Bug #6588 (Resolved): use WARNING level when keyring does not exist
Merged into ceph-deploy master branch with hash: cea6139 Alfredo Deza
04:42 AM rbd Bug #5876: Assertion failure in rbd_img_obj_callback() : rbd_assert(which >= img_request->next_co...
Hi,
I had this bug just now (see below), with a 3.10.16 kernel, with patches from your Git (cf attached file), and...
Olivier Bonvalet

10/27/2013

07:15 PM Bug #5823: cpu load on cluster node is very high, client can't get data on pg from primary node ...
I saw in the log file "osd is marked wrongly", but i checked its process was running... Khanh Nguyen Dang Quoc
09:53 AM devops Bug #6650: ceph-deploy: should exit/error when apt fails
a very annoying bug. I don't see how this could have made it into any release at all, since it is basically one of th... Zoltan Arnold Nagy

10/26/2013

04:38 AM CephFS Bug #6655: readdir() fails on CephFS mount symlinked directories
I can't reproduce this locally. Which kernel did you use? please try ceph-fuse and recent kernel. Zheng Yan
02:12 AM CephFS Bug #6655: readdir() fails on CephFS mount symlinked directories
If struggling to reproduce, it seems like readdir() works directly after other access to the symlink, but only once.
...
Pieter Steyn
01:38 AM CephFS Bug #6655 (Can't reproduce): readdir() fails on CephFS mount symlinked directories
Background:
* Ubuntu Server 12.04 64bit.
* CephFS Dumpling 0.67.4
* We moved from local filesystem to CephFS ...
Pieter Steyn

10/25/2013

08:10 PM Bug #6043: upstart does not reflect running ceph-osd daemons (ubuntu 13.04 only)
We're using Chef to install our nodes (which uses ceph-disk tools to manage the disks) so:... Hunter Nield
04:49 PM Bug #6043: upstart does not reflect running ceph-osd daemons (ubuntu 13.04 only)
Hunter, what are the osd data directories named? Samuel Just
04:31 PM Bug #6043 (Can't reproduce): upstart does not reflect running ceph-osd daemons (ubuntu 13.04 only)
I am not able to reproduce this issue on raring with latest stable dumpling branch [v0.67.4]
test setup tried: vpm...
Tamilarasi muthamizhan
05:53 PM Bug #6635 (Resolved): ceph pool rename needs to succeed if the source pool does not exist and the...
Sage Weil
05:00 PM Fix #5612 (Fix Under Review): down mons prevents osd hosts from booting properly
Sage Weil
04:58 PM Fix #5612 (New): down mons prevents osd hosts from booting properly
https://github.com/ceph/ceph/pull/772
Sage Weil
04:58 PM Fix #5612 (Fix Under Review): down mons prevents osd hosts from booting properly
Sage Weil
04:58 PM devops Bug #6654 (Rejected): ceph-deploy: bootstrap requires python-virtualenv on raring
ceph-deploy bootstrap prompts for "installing python-virtualenv".
I thought bootstrap takes care of this and we di...
Tamilarasi muthamizhan
04:40 PM Bug #6614: unittest_bufferlist failed
Compiled against 289b7903407ce1b34f1afe9e0c769093c14d0ba9 on centos-6.4 the test completes successfully. But the test... Loïc Dachary
02:32 PM Bug #6614 (In Progress): unittest_bufferlist failed
The error originaly reported is not an actual error, it is the output of an assert that was expected to happen as par... Loïc Dachary
04:34 PM Bug #6598: osd crash after recreating pool with same name (cuttlefish + bobtail?)
occurs on cuttlefish, not dumpling. Sage Weil
04:24 PM Fix #4942 (Resolved): librados: do not hang on auth failure on start
this works on cuttlefish and dumpling and later. non-trivial backport to fix it on bobtail. Sage Weil
04:13 PM devops Bug #6650: ceph-deploy: should exit/error when apt fails
this happens on precise as well.... Tamilarasi muthamizhan
03:47 PM devops Bug #6650 (Resolved): ceph-deploy: should exit/error when apt fails
ceph-deploy version: 1.2.7... Tamilarasi muthamizhan
04:06 PM Bug #6649 (Can't reproduce): ceph-deploy + rbd export tests fail
Sage Weil
03:34 PM Bug #6649 (Can't reproduce): ceph-deploy + rbd export tests fail
... Sage Weil
04:06 PM Bug #6648 (Can't reproduce): ceph-deploy librbd test fails on centos
Sage Weil
03:15 PM Bug #6648 (Can't reproduce): ceph-deploy librbd test fails on centos
... Sage Weil
03:08 PM Bug #6622 (Resolved): nightly runs:apt-get failures on ubuntu
resolving this; curl is now in hte repo for dumpling. i suspect teh branch just hadn't rebuilt yet. Sage Weil
02:56 PM Bug #6272 (Closed): ceph command usage missing setcrushmap
David Zafman
02:18 PM Bug #6390 (Can't reproduce): ENOTEMPTY on TEMP coll on replay
Samuel Just
02:15 PM Bug #5823: cpu load on cluster node is very high, client can't get data on pg from primary node ...
I mean, was it always osds on a particular node which get marked down? Samuel Just
02:11 PM Bug #5951 (Can't reproduce): osd: next: EEXIST on mkcoll
Samuel Just
02:09 PM Feature #6645 (Closed): EC: [link] BPC (basic pyramid code)
Loïc Dachary
02:00 PM Fix #6059 (Resolved): osd: block reads while repgather is writing across replicas
Sage Weil
01:56 PM Feature #6644 (Resolved): cachepool: evict
Sage Weil
01:56 PM Feature #6643 (Fix Under Review): cachepool: flush
Sage Weil
01:56 PM Feature #6643 (Resolved): cachepool: flush
Sage Weil
01:55 PM Feature #6188 (Fix Under Review): cachepool: osd: promote on write and mark object dirty
Sage Weil
01:35 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
Memory usage on the host looks stable. Traffic on the host dips because it loses the crashed guest's traffic. Disk ut... Mike Dawson
01:19 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
Mike Dawson wrote:
> Just discovered that the load on the host spikes when the crash occurs. I don't have info on th...
Sage Weil
01:14 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
Just discovered that the load on the host spikes when the crash occurs. I don't have info on the first, but the past ... Mike Dawson
12:43 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
One more crash today. Again, different host and guest with similar config and workloads.
*** Error in `qemu-system...
Mike Dawson
01:25 PM rgw Bug #6604 (Pending Backport): radosgw-agent: opstate tracking error
Sage Weil
01:11 PM rgw Feature #5170: RGW: Object restriping tool to fix large objects from argonaut.
Sage Weil
01:11 PM rgw Feature #4466 (Resolved): Quotas per bucket - synchronous
Sage Weil
01:08 PM rbd Feature #6432: Change TGT Plugin to allow setting client name for authentication
Sage Weil
12:49 PM Bug #6574 (Resolved): osd/ReplicatedPG.cc: 7851: FAILED assert(0)
847ea6059211b1ce104207584a57badf830103d6 Samuel Just
11:48 AM devops Feature #6067 (Resolved): ceph-deploy: make mon create catch common errors
#6638 was create to tackle adding monitors. Everything else in this ticket has been already implemented in ceph-deplo... Alfredo Deza
11:38 AM devops Feature #3347 (Resolved): ceph-deploy: allow setting ssh user
Merged into ceph-deploy's master branch with hash: e623b1e Alfredo Deza
11:23 AM devops Bug #6595: Hardcoded install path in ceph-disk
This is a custom install we are building ceph from source. Did not provide the --prefix argument to ./configure and e... Adam Manzanares
11:03 AM devops Bug #6595: Hardcoded install path in ceph-disk
How did you manage to get ceph executables in that path? Is this a custom install?
*Everywhere* in ceph-disk it re...
Alfredo Deza
11:18 AM Bug #6636 (Resolved): sockaddr_storage and uuid_t are not portable to other platforms
... Sage Weil
10:54 AM devops Bug #6587: ceph-deploy does not use cluster arg for mon error checking
This needs to be fixed ASAP before the next release. It is annoying (and completely misleading) for users that are us... Alfredo Deza
10:14 AM rbd Bug #6072 (Resolved): librbd image rename breaks child backwards reference
in next and backported! Sage Weil

10/24/2013

09:08 PM rbd Bug #6631: disabling writethrough until flush appears to disable RBD cache
More repetition of tests..
// IOPS for Sequential 4KB Write _with_ "rbd cache writethrough until flush = true"
Se...
Amit Vijairania
09:06 PM rbd Bug #6631: disabling writethrough until flush appears to disable RBD cache
On Nova node..
// Ceph.conf
[global]
fsid = c14be55f-f608-4f76-aaa3-925b07a72e43
mon initial members = alln01-c...
Amit Vijairania
02:19 PM rbd Bug #6631 (Closed): disabling writethrough until flush appears to disable RBD cache
Recently we saw a report that when using fio to perform 4K sequential direct IO writes to files on an XFS filesystem ... Mark Nelson
09:01 PM rbd Bug #6630: fio tests against raw RBD volumes show strange results
With multiple runs of same test on RBD device with XFS, we get following outputs..
fio --rw=write -ioengine=libaio...
Amit Vijairania
08:57 PM rbd Bug #6630: fio tests against raw RBD volumes show strange results
FIO workload:
[root@sm-rhel6-template ceph-perf]# cat fio-small-write.sh
#!/bin/bash
mkdir -p /root/ceph-per...
Amit Vijairania
02:12 PM rbd Bug #6630 (Resolved): fio tests against raw RBD volumes show strange results
Recently we've seen a report that when performing direct io sequential write tests against raw RBD volumes, RBD cache... Mark Nelson
07:33 PM Bug #6635 (Fix Under Review): ceph pool rename needs to succeed if the source pool does not exist...
https://github.com/ceph/ceph/pull/765 Joao Eduardo Luis
05:08 PM Bug #6635 (Resolved): ceph pool rename needs to succeed if the source pool does not exist and the...
That way, the command will be somewhat idempotent. Samuel Just
06:18 PM rbd Bug #6072 (Fix Under Review): librbd image rename breaks child backwards reference
This was just a needlessly small limit in the python bindings. https://github.com/ceph/ceph/pull/764 Josh Durgin
05:29 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
Hit a similar crash with a slightly different backtrace this today. Different host and vm. Both hosts have similar co... Mike Dawson
05:07 PM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
65817 FAIL scheduled_teuthology@teuthology rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay.... Samuel Just
05:06 PM Bug #6627 (Resolved): segfault at PGMap::dirty_all during upgrade mon
Samuel Just
01:13 PM Bug #6627: segfault at PGMap::dirty_all during upgrade mon
hi Joao, reproduced this issue when upgrading monitors one at a time from cuttlefish to next with debugs on - logs ar... Tamilarasi muthamizhan
09:53 AM Bug #6627 (Resolved): segfault at PGMap::dirty_all during upgrade mon
logs: ubuntu@teuthology:/a/teuthology-2013-10-22_01:30:02-upgrade-next-testing-basic-plana/64049... Tamilarasi muthamizhan
04:44 PM rgw Bug #6621: quota: the max-size and max-objects value when zero
same applies to max-objects as well... Tamilarasi muthamizhan
03:46 PM Cleanup #6634 (Resolved): MOSDSubOp: remove unused snapc/snapcontext members
These members are never accessed, except to encode, decode, and print. Since they are printed in logs, their being em... Greg Farnum
03:11 PM Bug #6003: journal Unable to read past sequence 406 ...
ubuntu@teuthology:/a/teuthology-2013-10-23_19:00:21-rados-dumpling-testing-basic-plana/65408 Sage Weil
03:10 PM Bug #6633 (Resolved): osd: pgls vs osd restart/peering race misses objects
test saw a sequence like:
- create object
- start osd
- things peer
- pgls
-> pgls returns empty result
I exp...
Sage Weil
02:08 PM Bug #6629: fd cache and external changes to recently-modified files don't behave nicely
Maybe we could use inotify to watch for relevant changes to the filestore and close the cached FDs when their files e... Dan Mick
01:28 PM Bug #6629 (Won't Fix): fd cache and external changes to recently-modified files don't behave nicely
A user reported that they were trying to test pg repair functionality and it did not appear to be restoring files del... Greg Farnum
12:13 PM rbd Bug #6628 (Resolved): krbd: BUG during ceph_osdc_stop() sometimes when rbd_add() fails
BUG at osd_client.c:978:... Josh Durgin
11:32 AM rbd Fix #6079: libceph: osd_client does not handle PAUSERD or PAUSEWR or FULL flags in osdmap
This is worse than I thought - for the FULL flag, the OSD will return -ENOSPC, which will get translated into -EIO by... Josh Durgin
09:49 AM rgw Fix #6616: radosgw: system users are not handled well by read_policy()
https://github.com/ceph/ceph/pull/763 Josh Durgin
09:49 AM rgw Fix #6615 (Fix Under Review): radosgw: data log list admin api does not include any markers
https://github.com/ceph/ceph/pull/763 Josh Durgin
09:49 AM rgw Bug #6604 (Fix Under Review): radosgw-agent: opstate tracking error
https://github.com/ceph/ceph/pull/763 Josh Durgin
09:25 AM rbd Feature #6626 (Resolved): openstack: cinder: allow users to delete snapshots that have clones
Hide the dependency on the backend, and do auto-flattening as needed (similar to the existing auto-flattening for the... Josh Durgin
08:39 AM Bug #6620 (Pending Backport): mon: MDSMonitor/MDSMap: 'ceph report' leads to segfault on MDSMap::...
commit:0e8182edd850f061421777988974efbaa3575b9f
We should probably backport this to dumpling.
Joao Eduardo Luis
06:59 AM devops Bug #6592 (In Progress): 3.8 kernel + /dev/cciss/c0d1 + precise : fail to show in /dev/disk/by-pa...
*blkid -o udev /dev/cciss/c0d1p2* does not return anything. Note, however, that after a reboot the OSDs are running f... Loïc Dachary
01:33 AM CephFS Feature #3541 (Resolved): mds: robust ino lookup using file backpointers
Zheng Yan
01:33 AM CephFS Feature #4295 (Resolved): mds: Actually purge deleted directories
Zheng Yan
12:19 AM Bug #5823: cpu load on cluster node is very high, client can't get data on pg from primary node ...
It occurred as having multiple write requests to cluster...
I deployed 10 osds/node with default configuration.
...
Khanh Nguyen Dang Quoc

10/23/2013

11:23 PM rbd Bug #6576 (Resolved): librbd: test_librbd_fsx segfaults on start up on 32-bit arm
teuthology.git commit:705a77f5d1c3159ac61200283dd50c154fcc55aa Josh Durgin
07:25 PM rbd Bug #6576 (Fix Under Review): librbd: test_librbd_fsx segfaults on start up on 32-bit arm
https://github.com/ceph/teuthology/pull/143 Sage Weil
06:49 PM CephFS Bug #6609: teuthology rsync workunit failure
files were synced appropriately. rsync only sync directory share/doc/ 's timestamp or mode when it was executed for t... Zheng Yan
03:18 PM CephFS Bug #6609: teuthology rsync workunit failure
I didn't look at the details much (even to figure out what the file transfer issues were). What kind of timestamp iss... Greg Farnum
06:23 PM Bug #6622: nightly runs:apt-get failures on ubuntu
the ceph-build-deb-native.sh script has been updated to include the curl gnutls packages in the generated repo. i ra... Sage Weil
06:02 PM Bug #6622: nightly runs:apt-get failures on ubuntu
I manually recreated what was happening on a new vm to get the full apt output. here is the problem:
Reading state...
Sandon Van Ness
05:32 PM Bug #6622 (Resolved): nightly runs:apt-get failures on ubuntu
logs:ubuntu@teuthology:/a/teuthology-2013-10-23_01:35:02-upgrade-small-next-testing-basic-vps/64863... Tamilarasi muthamizhan
05:43 PM CephFS Bug #6623 (Resolved): mds: update backtraces on existing clusters
The backtrace code doesn't update existing clusters as it touches them, unless the paths actually change.
Zheng fi...
Greg Farnum
05:19 PM rgw Bug #6621 (Resolved): quota: the max-size and max-objects value when zero
ceph version: 0.71-249-g31a9492 (31a94922a9ada132bea06be308484ead84e4d879)
while setting quota, the max-size field...
Tamilarasi muthamizhan
04:20 PM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
Sam liked it; have squashed and am scheduling a suite run now. Greg Farnum
04:00 PM Bug #6620: mon: MDSMonitor/MDSMap: 'ceph report' leads to segfault on MDSMap::dump_info
... Joao Eduardo Luis
03:42 PM Bug #6620 (Resolved): mon: MDSMonitor/MDSMap: 'ceph report' leads to segfault on MDSMap::dump_info
Triggered at least on 0.67.4 and beyond. Happens for a store from burnupi02, not for other stores. Currently assess... Joao Eduardo Luis
03:37 PM devops Feature #6618 (Rejected): gitbuilder: run 'make check' before building debs and rpms
The non-package gitbuilders do this, and it helps catch platform-specific errors. There are several platforms buildin... Josh Durgin
03:31 PM Bug #6614: unittest_bufferlist failed
We talked on irc last night, this was on CentOS 6.4. The deb/rpm gitbuilders aren't running make check - we should ad... Josh Durgin
01:19 PM Bug #6614: unittest_bufferlist failed
Where are you running this, and where are you getting the source (I can't find that commit)? Our gitbuilders (http://... Greg Farnum
11:03 AM Feature #5992 (Resolved): EC: [link] Refactor Backfill to use PGBackend methods
Samuel Just
11:00 AM Subtask #6391 (Duplicate): stuck incomplete
Looks like these are the same problem (which, luckily, users are seriously unlikely to come across). Greg Farnum
10:55 AM Fix #6116: osd: incomplete pg from thrashing on next
The workaround I put into teuthology was inadequate, I'm going to put this in the backlog and downgrade it now that i... Samuel Just
10:19 AM Documentation #5618 (Resolved): radosgw pool size guidelines
There's a full section on pools in the configuration reference now.
http://ceph.com/docs/master/radosgw/config-ref/...
John Wilkins
10:04 AM rgw Fix #6616 (Resolved): radosgw: system users are not handled well by read_policy()
System users should be able to read suspended buckets and get 404 for deleted buckets instead of 403. Josh Durgin
10:02 AM rgw Fix #6615 (Resolved): radosgw: data log list admin api does not include any markers
The last marker of the log entries fetched is needed to determine the position in the log. Previously the radosgw-age... Josh Durgin
09:59 AM devops Bug #6595: Hardcoded install path in ceph-disk
It does not match the default install path that ceph is currently using. Ceph was installed to /usr/local/bin by defa... Adam Manzanares
07:43 AM RADOS Fix #6570: osd: do not keep full pg log entries in memory
Thank your for taking care of this. This is really a huge problem for us.
I don't quite understand your statement...
Corin Langosch
02:10 AM RADOS Fix #6570: osd: do not keep full pg log entries in memory
Is there any danger for increasing peering time? It would be awesome to make this feature configurable since some peo... Andrey Korolyov
02:31 AM Feature #5984: mon: probe monitors to check on their status regardless of quorum
commit:c2cf8489bc9c8fa40153d8ddb163e9b25e72bcd5 Joao Eduardo Luis
02:29 AM Feature #5984 (Resolved): mon: probe monitors to check on their status regardless of quorum
Joao Eduardo Luis
 

Also available in: Atom