Activity
From 10/23/2013 to 11/21/2013
11/21/2013
- 05:22 PM devops Feature #6836 (Resolved): Get all Ceph installation package dependencies from EPEL etc, into eith...
- 04:00 PM Bug #6796 (In Progress): ceph mons interpretting pg splits very wrong
- 03:40 PM Feature #6835 (Resolved): EC: ec pgs will need to be able to specify temp primaries other than ac...
- OSDMap interface must allow primariness to be specified seperately from acting set position for ec pools.
- 03:35 PM Bug #6834: nightlies: monitor crashed in emperor
- From this backtrace it looks like either there was a hardware problem, or the monitor was using so much memory it cou...
- 03:24 PM Bug #6834 (Can't reproduce): nightlies: monitor crashed in emperor
- logs: ubuntu@teuthology:/a/teuthology-2013-11-18_19:31:27-upgrade-parallel-next-testing-basic-plana/107772...
- 03:12 PM devops Feature #6310: Get Dumpling into CentOS Ceph repo
- Dumpling is not yet in the Centos repo.
I reran the mock build on centos 6.3 and 6.4 to verify that there should n... - 03:11 PM Bug #6833: `/etc/init.d/ceph status` occasionally exists silently
- Hmm. `/etc/init.d/ceph restart` is doing the same thing.
- 02:55 PM Bug #6833 (Can't reproduce): `/etc/init.d/ceph status` occasionally exists silently
- I have a cluster that got wedged somehow, and when I run `/etc/init.d/ceph status` it simply exits with status 0. Tha...
- 03:11 PM Bug #6820: Bad commandline usage crashed my monitor
- nevermind, got it.
- 01:59 PM Feature #5991: EC: [link] Backfill peers should not be included in the acting set
- 01:57 PM Feature #5990 (In Progress): EC: [link] Factor out the ReplciatedPG object replication and client...
- 01:39 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- Yeah, I think a note is fine.
- 12:33 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- `delete=True` is the default, and we are explicitly setting that flag to `delete=False` because of that reason.
Wo... - 01:38 PM Feature #6832 (Resolved): EC: Adapt pg log to include information necessary for rollback
- This feature includes adapting the ReplicatedBackend to allow xattr, append operations to be rollback-able. It also ...
- 01:35 PM Feature #6831 (Resolved): EC: Adapt ReplicatedPG read path to handle async reads
- This is a bit challenging because there might be a sequence of reads at different offsets within the transaction. Th...
- 12:40 PM Bug #6807 (In Progress): Debian Wheezy Teuthology Ceph-deploy run failed.
- 07:28 AM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
- It sounds like there was an earlier problem with the test or a different failure — why is it trying to delete the cep...
- 07:21 AM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
- It looks like the reason we were enforcing the *single file system* was because we might still have OSDs mounted (hen...
- 09:46 AM devops Feature #5282 (Closed): Get Dumpling into EPEL
- Ceph 0.67.3 dumpling is in the epel repository.
- 09:25 AM rgw Bug #6830 (Resolved): S3 CompleteMultipartUploadResult has empty ETag element
- RHEL 6.4, Ceph 0.67.
The S3 Complete Multipart Upload operation returns a result that looks like this:... - 09:12 AM Feature #6828: osd should not silently fail to start when journal partition has no UUID
- I now understand this is not a bug but a feature. It probably deserves a warning of some kind ? http://dachary.org/?p...
- 04:53 AM Feature #6828 (Rejected): osd should not silently fail to start when journal partition has no UUID
- Providing a journal that is a partition that does not have the expected journal UUID should trigger an error....
- 06:50 AM rgw Bug #6829 (Resolved): rgw: missing RGWUserAdminOpState::system_specified initialization
- By inspecting code I noticed that, which means that when modifying user configuration the system settings of the user...
- 03:16 AM Bug #6827 (Resolved): ceph-disk hangs on blkid -s TYPE /dev/fd0
- On a supermicro hardware running ubuntu 12.04.3 ceph-disk list hangs forever trying to *blkid -s TYPE /dev/fd0*
<pre... - 03:01 AM Bug #6826 (Duplicate): Non-equal performance of 'freshly joined' OSDs
- Was just a bare eye observation for a long time, but I`ll try to formalize it here.
For OSDs entered recently perfom... - 12:06 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- I use ceph version 0.72-3-g5e1e02c (5e1e02c99b620fa4ffd2b455eb8e005b172fa05c), which is the "hotfix" for http://track...
11/20/2013
- 07:32 PM CephFS Bug #6742: failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
- This has popped up several more times in the nightlies; it looks to be a regular occurrence now. :/
- 07:11 PM CephFS Bug #6742: failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
- /teuthology-2013-11-14_23:01:38-fs-next-testing-basic-plana/{100526|100525|100487}
- 11:34 AM CephFS Bug #6742: failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
- /a/teuthology-2013-11-18_23:01:16-fs-master-testing-basic-plana/108236 (no logs)
- 07:24 PM rbd Bug #6368 (Resolved): rbd nosetests keep failing with AttributeError
- This turned out to be a silly version skew problem. centos and rhel have older versions of nosetests that needed diff...
- 10:32 AM rbd Bug #6368: rbd nosetests keep failing with AttributeError
- Changing the priority of this so that we can (hopefully) get it fixed soon.
This is still an issue for the nightly... - 07:16 PM Bug #6795 (Resolved): osd: 'tell bench' write size argument is ignored
- commit:40a76ef0d09f8ecbea13712410d9d34f25b91935
- 07:11 PM CephFS Bug #6773 (Duplicate): libcephfs interface tests maybe hanging
- 06:30 PM devops Bug #6790 (Resolved): /ref/branch/version mismatch
- So it looks like oh RHEL just using the version value (without the dist part) does not work so as we discussed earlie...
- 04:53 PM Bug #6820: Bad commandline usage crashed my monitor
- John, that's the client being unable to contact the monitor (most likely because the monitor indeed crashed). We'd ne...
- 03:54 PM Bug #6820 (Resolved): Bad commandline usage crashed my monitor
- I was looking to dump my CRUSH map without JSON syntax and I used the -f command to specify a format.
ceph osd cru... - 04:34 PM Bug #6824 (Resolved): Removal of an OSD that is not down should set non-successful status code
- A customer reports that some ceph commands return a status code of 0, even when they failed. The provided example was...
- 03:35 PM Fix #6780: monitor errors when checking for quorum status
- Reason for this: the code in place to keep compatibility with previous versions of the monitor with regard to the Cep...
- 10:50 AM Fix #6780 (In Progress): monitor errors when checking for quorum status
- this happens when some osds and mons are upgraded to next branch [emperor]
recent logs: ubuntu@teuthology:/var/lib... - 10:21 AM Bug #6808 (Duplicate): ceph-deploy/rbd/.../tasks/rbd_cli_tests.yaml fails on non-debian distros
- Issues already opened for this: #6648 #6649 and the one used as the main issue to track this problem is #6368
- 10:15 AM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
- Good catch, I think that this is the culprit:...
- 10:09 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- Corin, I forgot one step that would be wonderful if you could do: install google-perftools and run 'google-pprof <pat...
- 09:56 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- Corin, forgot to ask: what version is this happening on exactly and are you using our packaged binaries?
- 09:06 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- I just did what you wrote, please see attachment.
- 08:47 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- Can you please obtain a heap dump out of the monitor?
$ ceph heap start_profiler -m 10.0.0.7:6789
wait some tim... - 08:04 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- ...
- 08:00 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- ...
- 08:00 AM Bug #6810: very high monitor memory usage after upgrade dumpling -> emperor
- cluster 4ac0e21b-6ea2-4ac7-8114-122bd9ba55d6
health HEALTH_OK
monmap e5: 3 mons at {a=10.0.0.5:6789/0... - 07:58 AM Bug #6810 (Can't reproduce): very high monitor memory usage after upgrade dumpling -> emperor
- As you know I upgraded a few days ago from dumpling to emperor. All deamons are now running emperor. I have 3 monitor...
- 09:44 AM rbd Bug #5426: librbd: mutex assert in perfcounters::tinc in librbd::AioCompletion::complete()
- Upgraded to Urgent since causing failures in nightlies
- 07:09 AM Linux kernel client Bug #6809 (New): 3.11 kernel panic: Workqueue: ceph-msgr con_work
- The ceph cluster is very unstable ( hosts going up and down frequently ) and has high latency ( > 10ms ) on more than...
- 02:10 AM Bug #6797: ceph osd out does not migrate properly
- I cannot agree that the question is just about user experience. Drive replacement almost always don`t mean any remapp...
11/19/2013
- 05:38 PM Bug #6808 (Duplicate): ceph-deploy/rbd/.../tasks/rbd_cli_tests.yaml fails on non-debian distros
- Coredumps are generated. See
/a/teuthology-2013-11-19_01:10:01-ceph-deploy-next-testing-basic-vps/{108357. 108353,... - 05:22 PM Bug #6807: Debian Wheezy Teuthology Ceph-deploy run failed.
- That error should read:...
- 05:18 PM Bug #6807 (Resolved): Debian Wheezy Teuthology Ceph-deploy run failed.
- /a/teuthology-2013-11-19_01:10:01-ceph-deploy-next-testing-basic-vps/108339 needs to be investigated. The run report...
- 04:58 PM Bug #6806: mon: audit cmd_getval() calls to make sure they handle failures correctly
- Sigh. Yes, this was intentional, so that there was any value at all to doing the validation in the front end. I gue...
- 04:42 PM Bug #6806 (Resolved): mon: audit cmd_getval() calls to make sure they handle failures correctly
- During #6796 we noticed that most calls to cmd_getval() do not care for the function's return value, which indicates ...
- 04:46 PM Bug #6796: ceph mons interpretting pg splits very wrong
- The real issue here is that the Dumpling code is not considering the return value of cmd_getval() when obtaining the ...
- 04:39 PM Feature #6805 (Resolved): mon: find a way to properly extend/change mon commands without breaking...
- This need stems from #6796. It became obvious that changing variable types will break compatibility with previous ver...
- 04:17 PM rgw Fix #6804 (Resolved): Overly verbose logging: "setting object write_tag=" - rgw_rados.cc
- Logging of "setting object write_tag=" at level=0, seems much too verbose considering 10's of thousands of these even...
- 03:59 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- This will be fine for temporary files opened with 'delete=False' - if we start using delete=True then they will be po...
- 12:56 PM devops Bug #6701 (Fix Under Review): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18...
- 12:55 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- There was a PR addressing the problem for using shutil.move and I just opened another one to fix the missing `close()...
- 02:07 PM Bug #6803 (Can't reproduce): rados test failing in the nightlies on next branch
- logs: ubuntu@teuthology:/a/teuthology-2013-11-16_23:00:11-rados-next-testing-basic-plana/103763...
- 12:20 PM rgw Bug #6802 (Rejected): ARM: rgw_swift failure (internal server error, 500)
- Looks like almost all the tests are fialing with a 500 error (internal server error):
config:... - 12:20 PM devops Feature #6752 (Resolved): ceph-deploy to install yum or apt source with custom repo hostname
- And that just got merged with hash: 9388f83
- 11:41 AM devops Feature #6752 (Fix Under Review): ceph-deploy to install yum or apt source with custom repo hostname
- Opened pull request: https://github.com/ceph/ceph-deploy/pull/136
- 11:33 AM devops Feature #6752 (In Progress): ceph-deploy to install yum or apt source with custom repo hostname
- Found a couple of small items that needed to be fixed. Going to update in a different PR.
- 10:50 AM devops Feature #6752 (Resolved): ceph-deploy to install yum or apt source with custom repo hostname
- changeset was merged into ceph-deploy's master branch with hash: 3331b6f
A bunch of documentation was added as w... - 06:52 AM devops Feature #6752 (Fix Under Review): ceph-deploy to install yum or apt source with custom repo hostname
- Pull request opened: https://github.com/ceph/ceph-deploy/pull/135
- 12:14 PM rgw Bug #6801 (Rejected): RGW on arm: 'saw valgrind issues'
- Config:...
- 11:50 AM rbd Bug #6800 (Resolved): rbd/qemu-iotests.sh Failing on Arm.
- Config:...
- 11:04 AM Bug #6797: ceph osd out does not migrate properly
- Yeah, odd as it seems this is actually a user experience conflict — marking an OSD out does not change the CRUSH weig...
- 09:45 AM Bug #6797: ceph osd out does not migrate properly
- +1, migration overhead may be reduced by doing these actions in a short order, but generally it introduces _two_ peer...
- 12:28 AM Bug #6797 (Won't Fix): ceph osd out does not migrate properly
- ok, the subject might be misleading; here is what is happening:
- ceph osd out $id
- wait until migration finishe... - 10:53 AM rgw Feature #6799 (Closed): rgw: test keystone integration on RHEL
- Run any applicable tempest tests and swift tests.
- 06:38 AM rgw Bug #5931: radosgw crashes when deleting object
- I believe you need to open another bug with link to this one to get any feedback.
- 05:25 AM rgw Bug #5931: radosgw crashes when deleting object
- we have exactly the same problem
and i also upgraded to emperor because i thought this will fix the issue
but the i...
11/18/2013
- 06:17 PM Bug #6796: ceph mons interpretting pg splits very wrong
- Summarizing discussion with Sam and Dan:
the problem is caused by commit:2fe0d0d97af95c22db80800f5b9da51f672d9407, w... - 05:09 PM Bug #6796: ceph mons interpretting pg splits very wrong
- The actual problem is that the mon sees
2013-11-18T15:04:50.683 DEBUG:teuthology.orchestra.run:Running [10.214.135... - 05:08 PM Bug #6796 (Resolved): ceph mons interpretting pg splits very wrong
- logs are copied to mira055.front.sepia.ceph.com:/home/ubuntu/bug_6776_1
to reproduce the bug, ... - 03:30 PM Bug #6795: osd: 'tell bench' write size argument is ignored
- https://github.com/ceph/ceph/pull/854
- 03:25 PM Bug #6795 (Resolved): osd: 'tell bench' write size argument is ignored
- osd bench is supposed to take a second argument as the block size in bytes, but the default block size of 4MB is alwa...
- 01:13 PM Bug #6786 (Resolved): os/FileJournal.cc: 1533: FAILED assert(seq >= last_committed_seq)
- 01:13 PM Bug #6756: journal full hang on startup
- 11:57 AM devops Bug #6790: /ref/branch/version mismatch
- I have a couple concerns.
If we add the rpm release to the version, will that break anything that expects version ... - 11:12 AM Bug #6794 (Rejected): ceph-deploy needs both prepare and activate for paths
- I thought the docs where talking about filesystem paths, not device paths. The docs are actually correct, so closing ...
- 11:04 AM Bug #6794 (Rejected): ceph-deploy needs both prepare and activate for paths
- The docs currently give an example for `create` that uses paths which is not correct.
11/17/2013
- 06:40 AM CephFS Bug #6791: mds assert after startup - CDir::commit error (want > commited version)
- Thanks for the advice.
The "mds log_max_segments = 100000" avoided the assertion.
I'm starting to copy the data o... - 06:12 AM CephFS Bug #6791: mds assert after startup - CDir::commit error (want > commited version)
- Looks like the FS get corrupted. I suggest copying the data out and re-creating the FS.
add folowing line to ceph... - 04:35 AM CephFS Bug #6791: mds assert after startup - CDir::commit error (want > commited version)
- On a higher log level i can see that this happens during "try_to_expire" on a journal LogSegment:
-4> 2013-11-...
11/16/2013
- 07:35 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- Managed to provoke this again, this time creating a keyring for an osd on a host that is not a monitor. The tiggering...
- 01:43 PM CephFS Bug #6791 (Won't Fix): mds assert after startup - CDir::commit error (want > commited version)
- On upgrade from 0.67 to 0.72 i experienced the bug 6755.
I repaired the system with the ceph_filestore_tool as descr... - 04:07 AM Documentation #2684: doc: ceph and all daemons take --show-config
- "src/common/config.cc":https://github.com/ceph/ceph/blob/7e53473a7aaa68e03843c521022b9e60b33e4ef3/src/common/config.c...
- 03:17 AM Documentation #2684: doc: ceph and all daemons take --show-config
- ...
11/15/2013
- 05:49 PM Bug #6777: nightlies: gem dependency error
- I wonder if the rubygem mirror was having trouble. I just tried to reproduce this on a brand new debian wheezy (7.0) ...
- 03:52 PM devops Bug #6790: /ref/branch/version mismatch
- logs: ubuntu@teuthology:/a/teuthology-2013-11-15_01:35:02-upgrade-small-next-testing-basic-vps/100676
this has to ... - 03:47 PM devops Bug #6790 (Resolved): /ref/branch/version mismatch
- So currently the version to install with yum is grabbed from sha1/$sha1/version or /ref/$branch/version
Basically ... - 03:09 PM Bug #6003 (Can't reproduce): journal Unable to read past sequence 406 ...
- 03:08 PM Bug #6776 (Duplicate): nightly failure: timed out waiting for admin_socket after osd restarted
- 03:07 PM Bug #6778 (Won't Fix): log bound mismatch errors seen
- Added to whitelist.
- 02:49 PM Bug #6786: os/FileJournal.cc: 1533: FAILED assert(seq >= last_committed_seq)
- logs: ubuntu@teuthology:/a/teuthology-2013-11-14_23:00:22-rados-next-testing-basic-plana/100293...
- 10:14 AM Bug #6786 (Resolved): os/FileJournal.cc: 1533: FAILED assert(seq >= last_committed_seq)
- ceph version 0.72-204-g878f354 (878f3540d1a69d0ca7d2014ba7a9f3cb5cfd986d)
1: (FileJournal::committed_thru(unsigned... - 02:23 PM Bug #6789 (Resolved): cannot remove the leader when there only are two monitors
- On Ubuntu precise with dumpling 0.67.4, create a new cluster with two monitors. ceph mon remove name_of_the_leader wi...
- 11:58 AM Bug #6761 (Resolved): emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- 11:17 AM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
- Now a fully working version with environment variables:...
- 10:38 AM Bug #6787 (Won't Fix): upstart is restarting daemons which we want to be dead
- It looks like upstart's auto service management stuff is restarting OSDs which have committed suicide due to getting ...
- 10:21 AM Fix #6780 (Need More Info): monitor errors when checking for quorum status
- 08:40 AM Fix #6780: monitor errors when checking for quorum status
- what version was this on? I think sage fixed this particular issue last sprint.
- 09:57 AM rgw Bug #6713 (Can't reproduce): rgw master nosetests failure.
- Does not seem to be happening anymore.
- 09:55 AM devops Bug #6745 (Can't reproduce): ceph-deploy gatherkeys on ubuntu dumpling
- It turns out that i did not have a quorum. I'll close this.
The WARNIN is still a spelling error. - 07:19 AM rbd Bug #5469: qemu-io: segfault when tried IO with invalid arguments
- I'm having a simular issue atm:
char device redirected to /dev/pts/2
xen be: qdisk-768: error: unknown operation ... - 12:45 AM rgw Support #6785 (Closed): integration of radosgw and keystone
- hi, I tried to integrate radosgw and keystone according to the guide here(http://ceph.com/docs/master/radosgw/config/...
- 12:44 AM rgw Support #6784 (Closed): integration of radosgw and keystone
- hi, I tried to integrate radosgw and keystone according to the guide here(http://ceph.com/docs/master/radosgw/config/...
11/14/2013
- 06:09 PM Bug #6781 (Can't reproduce): timed out waiting for recovery - probably ceph command hang
- ubuntu@teuthology:/a/samuelj-2013-11-14_15:22:25-rados-wip-6761-emperor-testing-basic-plana/99764
ceph.log indicat... - 05:44 PM Fix #6780 (Closed): monitor errors when checking for quorum status
- logs: ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97245
pasting th... - 05:39 PM devops Bug #6779: fix typo on the modfastcgi repo for fedora18
- this also applies for other fedora packages.
- 05:37 PM devops Bug #6779 (Resolved): fix typo on the modfastcgi repo for fedora18
- currently, it is called 'feodra-fcgi-ceph' repo.
INFO:teuthology.orchestra.run.out:[10.214.138.59]: [2013-11-13T20... - 04:50 PM Bug #6778 (Won't Fix): log bound mismatch errors seen
- logs:ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97229...
- 04:33 PM Bug #6777: nightlies: gem dependency error
- ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97233
- 04:32 PM Bug #6777 (Can't reproduce): nightlies: gem dependency error
- logs: ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97184
This issue... - 04:27 PM Bug #6776: nightly failure: timed out waiting for admin_socket after osd restarted
- ...
- 04:26 PM Bug #6776 (Duplicate): nightly failure: timed out waiting for admin_socket after osd restarted
- logs: ubuntu@teuthology:/a/teuthology-2013-11-13_14:42:07-upgrade-parallel-next-testing-basic-vps/97235...
- 04:17 PM rbd Bug #6775 (Rejected): kvm backtrace on rbd task
- On these tests:
/var/lib/teuthworker/archive/teuthology-2013-11-07_19:00:48-rbd-cuttlefish-testing-basic-plana/893... - 03:14 PM Bug #6769: rados upgrade tests failing in the nightlies
- teuthology bug, that is
- 03:13 PM Bug #6769 (Resolved): rados upgrade tests failing in the nightlies
- 4aaa908d92168e2e46b8802d6042b1d3ffb9bc54
Pushed to master and next. - 02:08 PM Bug #6769 (In Progress): rados upgrade tests failing in the nightlies
- 03:13 PM Documentation #6774 (Resolved): Documentation: osd scrub load threshold incorrect.
- In rados/configuration/osd-config-ref.rst ...
- 02:06 PM Bug #6674 (Resolved): Busted client locking
- 1212a2119f3681de40cf947dae9c3b0d3f19e6fe
- 12:07 PM Bug #6674: Busted client locking
- This is my theory. When CephContext is being cleaned-up, we first disable lockdep, then wait for the service thread t...
- 11:38 AM Bug #6674: Busted client locking
- I just saw it a couple more times going over fs nightlies from this week, and it broke several rados tests last night...
- 11:26 AM Bug #6674: Busted client locking
- Has this been triggered again since the first time we saw it?
- 11:12 AM Bug #6674: Busted client locking
- commit:c7d975aeadf908d11577c480fa0a2e831d069c55 is the one I was discussing.
- 01:19 PM CephFS Bug #5753: ceph-fuse: segfault when getting back a traceless rename op
- teuthology-2013-11-08_19:01:24-fs-dumpling-testing-basic-plana/90910/
teuthology-2013-11-09_19:01:04-fs-cuttlefish-t... - 12:55 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
- A bunch of progress, we are now allowing a couple of flags: --repo-url and --gpg-url. When GPG is not passed in it wi...
- 12:29 PM Bug #6756 (Pending Backport): journal full hang on startup
- 12:28 PM Bug #6758 (Pending Backport): clone_range missing head src
- 11:34 AM CephFS Bug #6773 (Duplicate): libcephfs interface tests maybe hanging
- /a/teuthology-2013-11-09_23:01:13-fs-next-testing-basic-plana, 92248 and 92209 both hung at the same point:...
- 11:21 AM CephFS Bug #6394: teuthology: bad dereference in mds thrasher
- /teuthology-2013-11-08_19:01:24-fs-dumpling-testing-basic-plana/90904/
- 11:12 AM Bug #6738: lockdep segfault when running rados test.sh
- #6674
- 11:12 AM Bug #6738 (Duplicate): lockdep segfault when running rados test.sh
- 06:25 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- Likewise for me, following the instructions everything is now back up and running without issues, thanks!
- 02:46 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- Now some hours later everything is still running perfectly fine. Thanks again!
11/13/2013
- 08:30 PM Cleanup #6766: keyring leading spaces
- The only difference I see there is quoting on the caps mon value, not spaces. Is that what you meant to say?
- 10:48 AM Cleanup #6766 (New): keyring leading spaces
- A keyring generated with ceph-deploy has no leading spaces. A keyring generated with ceph-authtool does have leading ...
- 07:44 PM CephFS Documentation #6771 (Closed): add mds configuration
-
may should update document about how to configure mds. - 07:42 PM CephFS Bug #6770 (Can't reproduce): ceph fscache: write file more than a page size to orignal file cause...
- When mount -o fsc on cephfs, write more than page size content to a file (less than a page), when each time writing 8...
- 07:28 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- After installing the gitbuilder version and checking all osds reported between 50 - 200 lost objects. Repair went wit...
- 06:57 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- tested wip-6761-emperor on mira056 and mira074 . It works fine!
- 05:32 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- I've got a repair mechanism for you to try:
Install wip-6761-emperor on all osd machines
stop all osds
for each ... - 04:38 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- wip-6761-emperor tool appears to work:...
- 04:30 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- (04:25:58 PM) samuelj@newdream.net/laptop: yes
(04:26:09 PM) samuelj@newdream.net/laptop: you need to create a clust... - 03:47 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- Yup, it's easy to reproduce. I did it with vstart, notes follow....
- 03:21 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- We messed up when changing the encoding for adding flags to the object_info_t. Sam has a patch which looks good to me...
- 01:36 PM Bug #6761 (In Progress): emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- 01:07 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- We need to understand why the upgrade suites did not catch this.
- 01:06 PM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
(12:56:37 PM) samuelj@newdream.net/home: the object_info encoding changed.
(01:07:07 PM) samuelj@newdream.net/home...- 11:47 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- System-Wide limits seem good on all hosts:...
- 10:19 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- The log shows the clients getting an error (which should be handled better):...
- 09:55 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- I just attached it here too. Sorry.
- 09:54 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- Here the full log ('debug objecter = 20' and 'debug objectcacher = 20' on the client) when doing the "rbd export" htt...
- 08:48 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- Same happens for another image that cannot be booted anymore:...
- 08:43 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- I just tried to export the image a file so I can boot the vm using that. But it fails to:...
- 08:36 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- This is the backtrace when I try to start from another rbd image. Looks the same to me....
- 08:02 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- Same but nicely formatted:...
- 08:00 AM Bug #6761: emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- # gdb --args /opt/qemu/1.4.0/bin/qemu-system-x86_64 -smp sockets=1,cores=2 -m 512 -vga cirrus -drive id=drive14183,if...
- 07:26 AM Bug #6761 (Resolved): emperor's "dirty" flag is being interpreted as "lost" by Dumpling OSDs
- All my systems run ubuntu 12.10. I was running dumpling for a few months without any errors. My kvm guests use qemu-r...
- 07:03 PM devops Bug #6698 (In Progress): new osds added to the cluster not starting up due to crush lookup failure
- this bug was caused by crush lookup failure.
I see similar issue with one of my local clusters: mira047 and mira048. - 05:39 PM Bug #6768 (Resolved): Misleading log messages using "data store" for monitor storage
- It was building properly on the gitbuilders and looked good to me; merged!
- 05:23 PM Bug #6768 (Fix Under Review): Misleading log messages using "data store" for monitor storage
- 02:10 PM Bug #6768 (Resolved): Misleading log messages using "data store" for monitor storage
- Change the following messages to use "local monitor storage" instead of "data store"
reached critical levels of av... - 04:54 PM Bug #6755 (Duplicate): mds assert soon after startup (while recovering)
- dup 6761
- 04:23 PM Bug #6769 (Resolved): rados upgrade tests failing in the nightlies
- logs: /a/teuthology-2013-11-07_05:30:02-upgrade-next-testing-basic-plana/87919...
- 01:44 PM Feature #6767 (Resolved): mon should have "version" command so ceph tell mon.* version works
- ceph tell osd.* version is handy as a "am I running what I think I'm running". It would be trivial to implement this...
- 01:32 PM Subtask #5858 (In Progress): Backfill should be able to handle multiple backfill peers
- 01:32 PM Subtask #5855 (Fix Under Review): Backfill peers should not be included in the acting set
- 12:41 PM Feature #5991 (Fix Under Review): EC: [link] Backfill peers should not be included in the acting set
- 12:40 PM Feature #5994 (In Progress): EC: [link] Backfill should be able to handle multiple backfill peers
- 10:35 AM rgw Bug #6765 (Rejected): ARM: RGW s3 tests fail.
- Seems to be hitting a 500 code or get connection refused to the web API during the test. Not sure if apache is crashi...
- 09:55 AM Fix #6763 (Resolved): crushtool: don't warn so harshly when enabling CRUSH_TUNABLES
- Apparently the crushtool says "tunables are DANGEROUS and NOT YET RECOMMENDED". Change that to be something less frig...
- 09:23 AM RADOS Fix #6762: OSD 'numpg_*' performance counter inaccurate after adding PGs, until OSD map next changes
- Setting this to the next release so we don't forget it.
- 07:55 AM RADOS Fix #6762 (New): OSD 'numpg_*' performance counter inaccurate after adding PGs, until OSD map nex...
After creating some new PGs, I notice that the 'numpg' and associated counters don't reflect the newly increased PG...- 06:06 AM rgw Feature #3454: Support temp URLs for Swift API
- I have just run into this as well.
Is there a timeline for this? - 04:24 AM rgw Feature #3454: Support temp URLs for Swift API
- Samen problem as Ryan
so if this could be picked up that would be great - 05:29 AM Bug #6598: osd crash after recreating pool with same name (cuttlefish + bobtail?)
- It was our mistake resulted in race/duplication for mkcoll() call and resulting crash on EEXIST. Sorry for noise.
- 03:19 AM rgw Bug #6760: rgw incompatible with gsutil, authorization signature wrongly computed
- There is a related issue here: https://github.com/ceph/ceph/pull/498
- 03:17 AM rgw Bug #6760 (New): rgw incompatible with gsutil, authorization signature wrongly computed
- The Google Cloud Storage tool 'gsutil' sends the 'x-goog-api-version: 2' header to RadosGW. For an unknown reason, rg...
11/12/2013
- 05:25 PM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
- Squeeze has a depdency on python-requests which is in the squeeze backports distro, but not everyone enables that.
- 05:24 PM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
- The httpd-mmn problem with teuthology went away with a rebuild of the target machine. So it is probably a package or...
- 04:12 PM Feature #6759 (New): Allow partial dump of pg statistics
- The pool and osd sections of the pg dump are very useful. The only issue is that the pg dump itself can be very large...
- 03:58 PM Bug #6755: mds assert soon after startup (while recovering)
- I think the mds will function after deleting all lost objects
- 10:43 AM Bug #6755: mds assert soon after startup (while recovering)
- .. upgrading from 0.67.4 to 0.72 ..
- 10:39 AM Bug #6755 (Duplicate): mds assert soon after startup (while recovering)
- Since I debugged this one a bit, I try to summarize what I could gather. I was in the process of upgrading from 0.64....
- 03:28 PM Bug #6758 (Resolved): clone_range missing head src
- osd/ReplicatedPG.cc: 5225: FAILED assert(attrs || !pg_log.get_missing().is_missing(soid) || (pg_log.get_log().objects...
- 02:36 PM Bug #6744: make install fails when --prefix specified
- I spent some time looking at this with Yehuda; I suspect that at least one of the
problems is this section from Make... - 01:56 PM Bug #6756 (Resolved): journal full hang on startup
- 2013-11-10 22:51:06.073780 7fd833945780 2 journal open /var/lib/ceph/osd/ceph-4/journal fsid ed7f3df7-52a6-4fc7-a6ca...
- 01:46 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
- For DEBs we just write the correct URL to...
- 01:36 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
- The key can be a regular file too, but then I would need to push that to nodes.
Also, it should be noted that for ... - 01:25 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
- Yes, I suddenly thought about the keys too. Filename can also be an arg (--use-repo-key) ?
- 01:11 PM devops Feature #6752 (In Progress): ceph-deploy to install yum or apt source with custom repo hostname
- 01:11 PM devops Feature #6752: ceph-deploy to install yum or apt source with custom repo hostname
- A convention needs to be followed to retrieve the correct GPG Key from the local repo. Currently, ceph-deploy uses th...
- 07:12 AM devops Feature #6752 (Resolved): ceph-deploy to install yum or apt source with custom repo hostname
- The goal is to have a user grab an ISO or .tar.gz consisting of all the files they need to run Ceph which would be do...
- 12:35 PM Documentation #6465: admin/build-doc should have some kind of build check for broken links
- Certainly easy to test, and yeah, I was thinking even some external thing like wget or curl or something. Whatever w...
- 10:12 AM Documentation #6465: admin/build-doc should have some kind of build check for broken links
- Maybe if we enabled the `-n` or 'nitpicky' option?
This could probably break the build, so we would need to check ... - 10:31 AM Bug #6740: ceph-disk --dmcrypt flag does not work when journals and backing stores are co-located
- I believe this is the same bug as issue #6700, which has a patch. Once the OSD with dmcrypt has been set up, issue #6...
- 10:04 AM Bug #6751: Pool 'df' statistics go bad after changing PG count
- Oh, I missed the object counts entirely. Have you waited a while and made sure these wrong values persist? I have a s...
- 10:01 AM Bug #6751: Pool 'df' statistics go bad after changing PG count
- Yes: the USED and OBJECTS values for pbench3 both shoot up.
(sorry it's a bit difficult to read, forgot to put the... - 09:44 AM Bug #6751: Pool 'df' statistics go bad after changing PG count
- This is the "USED" value on the pool pbench3 that you're looking at, right?
- 06:12 AM Bug #6751 (Resolved): Pool 'df' statistics go bad after changing PG count
To reproduce:
# Create a pool with few PGs
# Create some objects in the pool
# Note the 'ceph df' output
# In...- 09:57 AM Fix #6754 (Resolved): erasure-code: jerasure plugin does not check parameters properly
- "proposed fix":https://github.com/ceph/ceph/pull/2442
wrong parameter checks which can make the whole thing SEGV.
... - 09:07 AM devops Bug #6720 (Rejected): mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
- mkcephfs is deprecated
- 08:20 AM devops Bug #6720: mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
- Agree with Alfredo. It also seems a bit unsustainable to keep working on deprecated tools in the long run. If we have...
- 07:54 AM devops Bug #6720: mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
- Why are we opening bugs (or even attempting to work on) something that is deprecated?
If I'm a user who wants to g... - 08:16 AM CephFS Fix #6753: cephx authentication for mds seem to accept both "allow" and "allow *"
- I opened https://github.com/ceph/ceph/pull/844 to fix the typo in the documentation.
- 08:15 AM CephFS Fix #6753 (Closed): cephx authentication for mds seem to accept both "allow" and "allow *"
- Documentation at http://ceph.com/docs/master/rados/operations/authentication/ refers to both "mds 'allow'" and "mds '...
- 05:23 AM devops Bug #6745: ceph-deploy gatherkeys on ubuntu dumpling
- I didn't see output for the monitors. How did those where deployed? did they have quorum before attempting to `gather...
- 12:35 AM Documentation #6749 (Resolved): Documentation says CRUSH bucket weights are integers
- The documentation (http://ceph.com/docs/master/rados/operations/crush-map/) says
bucket weights are "double integers...
11/11/2013
- 04:18 AM Bug #5239: osd: Segmentation fault in ceph-osd / tcmalloc
- For future reference the culprit seems be the version of leveldb in debian wheezy. We've built a newer leveldb packag...
11/10/2013
- 04:49 PM CephFS Bug #6609: teuthology rsync workunit failure
- > 2013-11-01T13:37:12.841 DEBUG:teuthology.orchestra.run:Running [10.214.133.35]: 'sudo rm -rf -- /home/ubuntu/cephte...
- 12:50 PM rgw Feature #6748 (Resolved): Return bucket name in response header
- Make a configurable setting where you can enable the RGW to return the bucket name of the bucket the request came for...
- 12:49 PM rgw Feature #6747 (Fix Under Review): PowerDNS backend for RGW bucket directing
- See this blueprint: http://wiki.ceph.com/01Planning/02Blueprints/Firefly/PowerDNS_backend_for_RGW
11/09/2013
- 10:27 AM devops Bug #6726: Official packages do not appear to be available for Saucy
- Conversation I had with Mark and Peter on IRC seemed to indicate it had just been overlooked, since the packages are ...
- 09:26 AM devops Bug #6726: Official packages do not appear to be available for Saucy
- Is there something blocking this, or just have to get to it? :)
- 12:11 AM devops Bug #6726: Official packages do not appear to be available for Saucy
- This is even more important now that 0.72 (emperor) packages are available in the repo for previous releases, but not...
- 07:50 AM devops Bug #6746: ceph-release rpm not playing well with yum-plugin-priorities
- whoops sorry this isn't really devops related. feel free to move...
- 07:35 AM devops Bug #6746 (Resolved): ceph-release rpm not playing well with yum-plugin-priorities
- We use the yum-plugin-priorities to organize things across 10ish public/private repos we define on our servers. Just ...
11/08/2013
- 10:43 PM Bug #6683 (Resolved): mon: MonmapMonitor: specify epoch in 'mon getmap'
- 05:53 PM devops Bug #6745 (Can't reproduce): ceph-deploy gatherkeys on ubuntu dumpling
- bootstrap-osd/ceph.keyring and bootstrap-mds/ceph.keyring are not created.
Also there is a spelling error (WARNIN)... - 04:21 PM Bug #6744 (Closed): make install fails when --prefix specified
- Tries to install stuff into /sbin
- 04:14 PM CephFS Bug #6609: teuthology rsync workunit failure
- /a/teuthology-2013-10-31_23:01:45-kcephfs-next-testing-basic-plana/78406
I haven't checked what this is doing any ... - 03:53 PM CephFS Bug #5753: ceph-fuse: segfault when getting back a traceless rename op
- /a/teuthology-2013-11-04_19:01:34-fs-dumpling-testing-basic-plana/84738
- 03:41 PM CephFS Bug #6742 (Resolved): failed libcephfs_interface_tests (LibCephFS.ReaddirRCB hangs)
- /a/teuthology-2013-11-07_23:01:12-fs-next-testing-basic-plana/89851 (I copied the MDS and client logs, which have ful...
- 03:33 PM rgw Bug #6611: RGW: Using underscores when setting headers returns 403
- Did some investigation. The problem originates in apache at ap_create_environment() that normalizes all the env vars ...
- 03:30 PM rgw Bug #6152: New S3 auth code fails when using response-* query string params to override response ...
- This is still waiting for merge (sent a pull request a week ago).
- 03:27 PM CephFS Bug #6741 (Can't reproduce): failed snaptest-2.sh; got ENOTEMPTY on should-be empty dir
- http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-07_23:01:12-fs-next-testing-basic-plana/89873/...
- 02:27 PM rgw Bug #6672 (Resolved): normal PUT object requests got 405 sometimes
- Added signed-off by (with approval of author) and merged into dumpling at commit:372f62717c56d9ab883ae2942e13d6d8d37c...
- 02:12 PM Documentation #6727: pg docs still imply pg_num must be set at creation, but we have split
- IMO http://ceph.com/docs/master/rados/operations/pools/ should also be updated in the "set" section to specifically e...
- 01:42 PM CephFS Bug #6608: samba teuthology dbench failure
- None of these failed tests are running it in parallel:...
- 12:47 PM Fix #6673 (Resolved): 'osd pool set metadata pg_num 34' broken
- This got merged a while ago.
- 11:10 AM Bug #6737 (Resolved): ceph package rebuild needed for emperor on fedora18
- The fedora18 gitbuilder had crashed, so had not built the request sha1. The gitbuilder has been restarted.
- 09:20 AM Bug #6737 (Resolved): ceph package rebuild needed for emperor on fedora18
- logs: ubuntu@teuthology:/a/teuthology-2013-11-08_05:35:02-upgrade-small-next-testing-basic-vps/90343...
- 11:01 AM Bug #6740 (Resolved): ceph-disk --dmcrypt flag does not work when journals and backing stores are...
- See ceph-users thread: http://www.mail-archive.com/ceph-users@lists.ceph.com/msg05426.html
It appears that the com... - 09:59 AM Bug #6738: lockdep segfault when running rados test.sh
- ubuntu@teuthology:/a/teuthology-2013-11-07_23:00:08-rados-next-testing-basic-plana/89753
- 09:58 AM Bug #6738: lockdep segfault when running rados test.sh
- ubuntu@teuthology:/a/teuthology-2013-11-07_23:00:08-rados-next-testing-basic-plana/89749
- 09:56 AM Bug #6738 (Duplicate): lockdep segfault when running rados test.sh
- logs: ubuntu@teuthology:/a/teuthology-2013-11-07_23:00:08-rados-next-testing-basic-plana/89747...
- 09:23 AM rbd Bug #5426: librbd: mutex assert in perfcounters::tinc in librbd::AioCompletion::complete()
- latest logs: ubuntu@teuthology:/a/teuthology-2013-11-08_05:35:02-upgrade-small-next-testing-basic-vps/90358
- 05:46 AM Bug #6736 (Resolved): Bugs in per pool IOPs/recovery statistics
So I'm playing with the new 'ceph osd pool stats' in Emperor.
Initially I had a healthy cluster (3 OSDs, 3 mons,...
11/07/2013
- 10:29 PM Bug #6715: ceph is conflicting with mongodb on ubuntu 12.04 LTS
- I'm using the ubuntu cloud archive for havana on top of precise. I've double checked the ceph packaging and you're in...
- 10:48 AM Bug #6715: ceph is conflicting with mongodb on ubuntu 12.04 LTS
- I can't find libgoogle-perftools4 in the Precise repositories (just browsing http://packages.ubuntu.com/precise/ and ...
- 07:18 PM Documentation #3839: SSD crushmap example will not compile
- It seems that the types are missing and rulesets for ssd and ssd-primary are still the same.
- 11:38 AM Documentation #3839: SSD crushmap example will not compile
- I believe I fixed this some time ago. Please verify.
- 05:08 PM Fix #6705 (Fix Under Review): mon: test for ping
- wip-6705 ; https://github.com/ceph/ceph/pull/835
- 01:57 PM Feature #6735 (Resolved): scrub documentation needs to be improved (see below conversation or as ...
- (12:55:58 PM) tyler.brekke@newdream.net: I think I might be missing something,
https://github.com/ceph/ceph/blob/mas... - 01:24 PM Documentation #789 (Resolved): Document config options
- Have documented so many since 2011, we need a specific bug to address anything new.
- 01:03 PM rgw Bug #6733: rgw readwrite test fails on next branch
- it seems to be some kind of miscommunication between radosgw and apache, most likely apache is going down
- 10:41 AM rgw Bug #6733 (Closed): rgw readwrite test fails on next branch
- logs: ubuntu@teuthology:/a/teuthology-2013-11-05_23:01:20-rgw-next-testing-basic-plana/86307...
- 11:46 AM Documentation #6734 (Closed): better TOC/version highlighting
- Several users have complained that it's not intuitive which branch they are on (next/master), especially if they come...
- 11:26 AM Documentation #6454 (Resolved): Add dumpling to the documentation on OS/Kernel Recommendations
- See http://ceph.com/docs/master/start/os-recommendations/#dumpling-0-67
- 11:22 AM Documentation #5853 (Resolved): quickstart is freaking people out about mons and osds on same mac...
- Quick-start has been rewritten.
- 10:58 AM CephFS Bug #6718 (Duplicate): handle not found error running dbench.sh
- #6608
- 10:53 AM devops Bug #6717: Failure to remove samba packages
- Is there something in that log which makes you think it's a test issue? dpkg locking conflicts are generally some inf...
- 10:26 AM devops Bug #6720: mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
- Bug appears to be caused by the following commit:
https://github.com/ceph/ceph/commit/177e2ab1cad325b875249a514bc1... - 09:41 AM rgw Fix #6615 (Resolved): radosgw: data log list admin api does not include any markers
- backported, commit:6917b02530103b8c86ed75592da33144b0dea168
- 09:40 AM rgw Bug #6604 (Resolved): radosgw-agent: opstate tracking error
- backported, merge commit:6917b02530103b8c86ed75592da33144b0dea168
- 09:35 AM rgw Fix #6616 (Resolved): radosgw: system users are not handled well by read_policy()
- Was backported, commit:f1fa8116d441924d44c99624829f3daa090c821c
- 09:15 AM rgw Feature #6678 (Resolved): rgw: reject writes to secondary zones
- merged, commit:84fb1bf3eefe88c0f5f15034d69c171e6531bf76.
- 08:25 AM Feature #6732 (Rejected): mon: 'mon_status' should provide as much insight as 'ping'
- 'mon_status' is currently providing just a fraction of the information that would be useful to troubleshoot a monitor...
- 06:47 AM Documentation #6727: pg docs still imply pg_num must be set at creation, but we have split
- There is also a bug in ceph that after changing pg_num and pgp_num of a pool will display incorrect results on ceph -...
- 05:16 AM rbd Bug #6368: rbd nosetests keep failing with AttributeError
- Issue #6648 was opened recently with a `Can't reproduce` status.
I believe that re-writing these tests to avoid gl...
11/06/2013
- 09:22 PM Documentation #6731 (Closed): Document the hashpspool setting and config options
- We have some fairly nice documentation on how the CRUSH tunable settings work. The very similar HASHPSPOOL flag is on...
- 07:43 PM Bug #6725: objecter: kick_requests() resends ops that should be paused
- Thanks, that's a lot nicer. It looks good to me if it survives more testing.
- 04:56 PM Bug #6725: objecter: kick_requests() resends ops that should be paused
- I looked and tested it, the fix itself didn't do the work, but I was able to find some other issues. I pushed a secon...
- 04:22 PM Bug #6730 (Won't Fix): BUG: MAX_LOCKDEP_ENTRIES too low!
As this is kernel code I don't think it is related to the branch I'm testing. This was rebased yesterday on next b...- 03:07 PM Feature #6729 (Resolved): Make pg statistics less wrong after split
- Apparently this is pretty annoying to users. As a start, we could split the parent pg's stats evenly among its child...
- 02:36 PM Bug #6633: osd: pgls vs osd restart/peering race misses objects
- wip-pgls testing
- 02:35 PM Bug #6722 (Resolved): osd/PGLog.cc: 368: FAILED assert(p->version > newhead)
- 01:43 PM Bug #6728 (Won't Fix): ceph: sbindir always points to /sbin, even if configured with --prefix
- 01:33 PM Documentation #6727 (Resolved): pg docs still imply pg_num must be set at creation, but we have s...
- [22:16] * mobile (~pvsa@82.113.121.253) has joined #ceph
[22:20] <JoeGruher> i can run 'ceph osd pool set <pool_na... - 12:00 PM devops Bug #6726: Official packages do not appear to be available for Saucy
- Mark filed this for me based on a conversation on IRC. It was pointed out that the latest packages current exist in t...
- 12:00 PM devops Bug #6726: Official packages do not appear to be available for Saucy
- I suggest seeing this in a more general way. Was this merely an oversight or is the current workflow lacking in some...
- 11:41 AM devops Bug #6726: Official packages do not appear to be available for Saucy
- This appears to be the case for dumpling as well.
- 11:40 AM devops Bug #6726 (Resolved): Official packages do not appear to be available for Saucy
- It appears we are missing saucy packages here:
http://ceph.com/debian-emperor/dists/
But we do have them in the... - 09:32 AM rbd Documentation #5006: doc: openstack configuration changes for havana
- This is complete now. The bug status should be changed to resolved.
- 09:32 AM rgw Documentation #5165 (Resolved): rgw: multisite: regions and global namespace documentation
- Doc branch merged. Document is now public.
- 09:31 AM rgw Documentation #5166 (Resolved): rgw: dr: async repl and DR documentation
- Doc branch merged and made public.
- 08:40 AM Bug #6636: sockaddr_storage and uuid_t are not portable to other platforms
- Added pull request with this patch for easier discussion
https://github.com/ceph/ceph/pull/828 - 08:34 AM CephFS Feature #3426 (In Progress): ceph-fuse: build/run on os x
- 06:11 AM devops Bug #6654 (Rejected): ceph-deploy: bootstrap requires python-virtualenv on raring
- I'm closing this since this is the way it has always been. I am not sure if we really want to put effort into install...
- 05:31 AM devops Bug #6701 (Resolved): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invali...
- Pull Request opened: https://github.com/ceph/ceph-deploy/pull/126
And merged into ceph-deploy's master branch with... - 01:02 AM devops Feature #6365: Package up newer version of Qemu for RHEL & EL6.3 and 6.4
- Any update on this ? the qemu-kvm version you distribute is 6 month old.
The cleaning of async vs not async package ...
11/05/2013
- 10:11 PM Bug #6719 (Resolved): 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
- 03:41 PM Bug #6719: 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
- wip-6719
- 02:21 PM Bug #6719: 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
- Variant with logs possibly at slider:/home/samuelj/bug_logs/6719-readerror
- 02:20 PM Bug #6719 (Resolved): 3083: oid 16 contents (ObjNum 1034 snap 308 seq_num 1034) corrupt
- 2013-11-05T09:57:50.225 INFO:teuthology.task.rados.rados.0.out:[10.214.132.26]: rollback oid 49 to 320
2013-11-05T09... - 09:50 PM Bug #6722: osd/PGLog.cc: 368: FAILED assert(p->version > newhead)
- Fix in wip-6719
- 05:46 PM Bug #6722 (Resolved): osd/PGLog.cc: 368: FAILED assert(p->version > newhead)
- 2013-11-05 17:30:54.916068 7f9f42a66700 -1 osd/PGLog.cc: In function 'void PGLog::rewind_divergent_log(ObjectStore::T...
- 06:48 PM Bug #6725: objecter: kick_requests() resends ops that should be paused
- Possible fix is in the wip-objecter-full branch, but I can't test it at the moment due to bad network conditions.
- 06:37 PM Bug #6725 (Resolved): objecter: kick_requests() resends ops that should be paused
- Operations may be paused because the osd map has a flag manually set, or the cluster is too full. In either case, ope...
- 06:36 PM rgw Feature #6677 (Fix Under Review): rgw: add compatibility for MultipartUpload
- 06:25 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- Done.
- 06:04 AM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- Thanks for the ticket and the resolution Mark!
Would you mind sending a pull request to https://github.com/ceph/ce... - 06:14 PM rgw Feature #6678 (Fix Under Review): rgw: reject writes to secondary zones
- 05:48 PM rgw Bug #6713: rgw master nosetests failure.
- I'll keep an eye on these and close if they stay away.
- 02:59 PM CephFS Bug #6655 (Can't reproduce): readdir() fails on CephFS mount symlinked directories
- 11:46 AM CephFS Bug #6655: readdir() fails on CephFS mount symlinked directories
- I haven't been able to reproduce it again either since I remounted CephFS on the client a while ago. (And probably re...
- 02:52 PM devops Bug #6720 (Rejected): mkcephfs appears to fail to add osds to the odsmap in 0.72rc1
- Script that was able to produce a working cluster in 0.67.4 failed to work in 0.72rc1. This was reported externally ...
- 02:13 PM Bug #6714 (Resolved): ENOTEMPTY on replay
- 01:47 PM CephFS Bug #6718 (Duplicate): handle not found error running dbench.sh
- http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-03_23:01:12-fs-next-testing-basic-plana/83672/teuthology.log
... - 01:42 PM devops Bug #6717 (Closed): Failure to remove samba packages
- http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-03_23:01:12-fs-next-testing-basic-plana/83671/teuthology.log
... - 01:30 PM CephFS Bug #6613: samba is crashing in teuthology
- Still happening: http://qa-proxy.ceph.com/teuthology/teuthology-2013-11-03_23:01:12-fs-next-testing-basic-plana/83668...
- 12:57 PM devops Bug #6708 (Resolved): apt-get key fails
- Merged into ceph-deploy's master branch with hash: e6f3a3b
Idiotic that Python + subprocess freak out with quote... - 12:25 PM devops Bug #6708 (Fix Under Review): apt-get key fails
- Opened Pull Request: https://github.com/ceph/ceph-deploy/pull/124
- 08:06 AM devops Bug #6708 (In Progress): apt-get key fails
- I was able to confirm the problem. It is possible we have not caught this before because GPG keys may already exist i...
- 06:46 AM devops Bug #6708: apt-get key fails
- Was able to reproduce *running as root* on that same box:...
- 06:22 AM devops Bug #6708: apt-get key fails
- Loic, is it possible you are using a proxy for those hosts?
- 12:34 PM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
- For the debian packages, it looks like we only need to supply libgoogle-perftools for squeeze and for arm on quantal ...
- 09:48 AM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
- httpd-mmn is not a real package. It's a virtual dependency that was created to version the apache plug-in api. Apac...
- 09:43 AM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
- The rpm-emperor repo on ceph.com has been updated with the addition of the following packages:
gdisk-0.8.2-1.el6.x... - 12:25 PM rgw Bug #6152 (Pending Backport): New S3 auth code fails when using response-* query string params to...
- 12:25 PM rgw Bug #6152: New S3 auth code fails when using response-* query string params to override response ...
- I sent a pull request a few days ago, so this should be in the next dumpling release.
- 07:09 AM Bug #6636: sockaddr_storage and uuid_t are not portable to other platforms
- Awesome, thanks Alan. I'll pull this into wip-port for the time being.
- 05:05 AM Bug #6715 (Won't Fix): ceph is conflicting with mongodb on ubuntu 12.04 LTS
- I'm not sure who's to blame here, but ceph depends on libgoogle-perftools0, while mongo depends on libgoogle-perftool...
11/04/2013
- 08:05 PM devops Bug #6701 (Fix Under Review): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18...
- 08:00 PM rgw Bug #6710 (Fix Under Review): radosgw init script does not exit 1 and tell the user if the hostna...
- 06:37 PM devops Bug #6698 (Resolved): new osds added to the cluster not starting up due to crush lookup failure
- 02:40 PM Bug #6714: ENOTEMPTY on replay
- wip-enotempty
- 11:40 AM Bug #6714 (Resolved): ENOTEMPTY on replay
- Occurred while deleting a temp collection. The remaining object was a temp object which had been recreated as part o...
- 01:27 PM Feature #6568: ceph-rest-api authentication
- I'm not aware of any such plans, no, Matt; we sort of look at the ceph-rest-api as
a way of allowing internal-net a... - 12:40 PM rgw Bug #6672 (Pending Backport): normal PUT object requests got 405 sometimes
- 12:28 PM rgw Bug #6709: rgw upgrade test fails during readwrite
- It's hard to determine what exactly happened, as we don't turn on the gateway logging anymore. But generally it looks...
- 11:51 AM rbd Feature #6432 (Resolved): Change TGT Plugin to allow setting client name for authentication
- Upstream patches have been accepted.
- 11:37 AM Bug #6685 (Resolved): osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- 10:52 AM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- commit:72e2874f402575846d1bd84f2af865d28947bc67 ("ReplicatedPG::recover_backfill: adjust last_backfill to HEAD if sna...
- 11:34 AM Bug #6681 (Resolved): osd recovery hung
- 10:54 AM Bug #6681: osd recovery hung
- commit:cf4e00ff5c6d0c548cb766f94288277d1f661094 ("OSD: don't clear peering_wait_for_split in advance_map()") looks go...
- 11:25 AM Bug #6712 (Resolved): 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function '...
- 11:19 AM Bug #6712: 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function 'OSDMapRef O...
- The code looks fine, although we're arguing over how to write a clear commit message. ;)
- 11:22 AM Bug #6711 (Duplicate): osd/OSD.h: 509: FAILED assert(ret)
- #6712
- 11:12 AM Feature #6466 (Resolved): stgt: unlimit number of images supported
- Patches accepted by upstream; ready for rebuild/rerelease.
- 10:37 AM rgw Bug #6713 (Can't reproduce): rgw master nosetests failure.
- Using the following yaml files:...
- 10:36 AM devops Bug #6590: Ceph Package Dependencies not Included in Ceph Extras
- Just testing SSL installation on CentOS. When installing mod_ssl, it picks up mod_ssl.x86_64 1:2.2.22-1.ceph.el6, but...
- 09:40 AM devops Bug #6482 (Resolved): ceph-deploy install on Centos: can't find wget?
- Merged into ceph-deploy's master branch with hash: 91618ba
- 07:54 AM devops Bug #6482 (Fix Under Review): ceph-deploy install on Centos: can't find wget?
- Opened pull request: https://github.com/ceph/ceph-deploy/pull/123
- 05:43 AM devops Bug #6482: ceph-deploy install on Centos: can't find wget?
- After discussing with ksingh, I can confirm this is still a bug and ceph-deploy should install wget before it attempt...
11/03/2013
- 09:09 PM Bug #6712: 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function 'OSDMapRef O...
- wip-6685 also
- 08:50 PM Bug #6712 (Resolved): 0> 2013-11-03 19:33:02.062588 7f10f9bcf700 -1 osd/OSD.h: In function '...
- ubuntu@teuthology:/a/samuelj-2013-11-03_17:07:25-rados-wip-6685-testing-basic-plana/82408/remote
-4> 2013-11-0... - 08:47 PM Bug #6711 (Duplicate): osd/OSD.h: 509: FAILED assert(ret)
- 2013-11-03 19:33:02.081330 7f10e5289700 1 -- 10.214.132.16:6804/7111 >> :/0 pipe(0x2a57780 sd=55 :6804 s=0 pgs=0 cs=...
- 11:12 AM Bug #6681: osd recovery hung
- 11:11 AM Bug #6681: osd recovery hung
- I may have a fix in wip-6685
- 11:12 AM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- 05:25 AM rgw Bug #6710: radosgw init script does not exit 1 and tell the user if the hostname does not match
- Went a bit beyond the scope, saw that there were more improvements that could be made.
https://github.com/ceph/ceph/... - 04:14 AM rgw Bug #6710: radosgw init script does not exit 1 and tell the user if the hostname does not match
- I will submit a pull request for this.
- 04:14 AM rgw Bug #6710 (Resolved): radosgw init script does not exit 1 and tell the user if the hostname does ...
- If the hostname of the machine and the host configured for radosgw in ceph.conf, we exit 0 without telling the user a...
11/02/2013
- 09:34 PM rgw Bug #6709 (Can't reproduce): rgw upgrade test fails during readwrite
- ...
- 07:54 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- Figured out what the issue with shutil.move was - needed to close the temp file before moving. Not an issue with os.r...
- 06:25 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- I now know why the original error is happening. My previous musings were not really on the mark (as it were):
cons... - 05:40 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- I'm possibly causing the issue using shutil.move (can't see how mind you)...
- 05:28 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- Further on this (post the os.rename -> shutil.move), the next problem is:...
- 05:46 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- wip-6685
- 01:56 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- Got it with logging, working on branch.
- 01:36 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- ubuntu@teuthology:/a/teuthology-2013-11-01_23:00:05-rados-next-testing-basic-plana/80229/remote
- 04:31 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- Ivan Kudryavtsev wrote:
> is it possible you've done any shenanigans like 'ceph pg force_create_pg ...' at some poin... - 11:56 AM devops Bug #6708 (Resolved): apt-get key fails
- on a brand new precise-12.04.3 : ...
- 12:15 AM Feature #6707 (Fix Under Review): Backport per pool iops data to 0.67
11/01/2013
- 09:58 PM Feature #6707 (Rejected): Backport per pool iops data to 0.67
- 07:06 PM rgw Fix #6615: radosgw: data log list admin api does not include any markers
- backport: https://github.com/ceph/ceph/pull/804
- 07:06 PM rgw Fix #6616: radosgw: system users are not handled well by read_policy()
- backport: https://github.com/ceph/ceph/pull/804
- 07:06 PM rgw Bug #6604: radosgw-agent: opstate tracking error
- https://github.com/ceph/ceph/pull/805
- 06:33 PM rgw Bug #6694 (Duplicate): radosgw upstart script doesn't provide the -n parameter
- The name is determined by looking for client sections in ceph.conf that have 'host' matching the machine name. You ha...
- 04:44 PM devops Bug #6698 (Fix Under Review): new osds added to the cluster not starting up due to crush lookup f...
- 03:49 PM Fix #6673 (Fix Under Review): 'osd pool set metadata pg_num 34' broken
- https://github.com/ceph/ceph/pull/802
- 03:27 PM Fix #6673: 'osd pool set metadata pg_num 34' broken
- Actually, looking at this again, I've realized that the teuthology test is running the rados api tests, which involve...
- 02:35 PM Fix #6673 (In Progress): 'osd pool set metadata pg_num 34' broken
- We saw this again; /a/dzafman-2013-10-31_14:29:25-rados-wip-flush-5855-testing-basic-plana/77511.
The PGs were indee... - 03:01 PM Bug #6636: sockaddr_storage and uuid_t are not portable to other platforms
- Here's a patch that fixes the problem for struct sockaddr_storage. I haven't looked at uuid_t yet. Googling suggest...
- 02:04 PM Bug #6003 (Need More Info): journal Unable to read past sequence 406 ...
- af0c5b415ad5a863940add7c2e8f3c1ac8040ef2 should fix the most recent iteration, but it doesn't explain the instances p...
- 01:53 PM rgw Bug #6706 (Resolved): radosgw init script and hostname variations
- https://github.com/ceph/ceph/pull/801
- 01:30 PM rgw Bug #6706 (Resolved): radosgw init script and hostname variations
- Relevant snippet:...
- 12:08 PM Fix #6705 (Resolved): mon: test for ping
- 11:13 AM RADOS Feature #6704 (New): osd: ability to move a pg to the front (or back?) of the backfill/recovery q...
- sometimes the admin wants to explicitly prioritize backfill/recovery on a specific pg.
- 10:23 AM Bug #5951 (Resolved): osd: next: EEXIST on mkcoll
- 09:30 AM Bug #5818: leveldb 1.12: hang on shutdown (mon)
- ubuntu@plana83:~$ dpkg -l | grep leveldb
rc libleveldb1 1.12.0-1precise.cep... - 09:30 AM Bug #5818: leveldb 1.12: hang on shutdown (mon)
- well, observed this hang with 1.12.
ubuntu@teuthology:/var/lib/teuthworker/archive/teuthology-2013-10-31_23:00:13-... - 09:01 AM devops Bug #6699 (Fix Under Review): sysvinit script setting incorrect OSD weights
- if it works on rhel and ubuntu, that is good enough for now. will push to next.
- 08:54 AM devops Bug #6699: sysvinit script setting incorrect OSD weights
- Changing 'df' to 'df -P' in the ceph sysvinit script fixes the issue, not sure how portable this flag is.
- 08:52 AM devops Bug #6703 (Resolved): OSDs with dmcrypt fail to start at boot
- RHEL6.4 ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7)
On boot the OSDs are not started when th... - 08:23 AM devops Bug #6482: ceph-deploy install on Centos: can't find wget?
- Hi Alfredo
You can close this this is now fixed - 01:17 AM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- The particular issue is caused by os.rename in ceph_deploy/hosts/remotes.py line 54. replacing that with shutil.move...
10/31/2013
- 11:08 PM devops Bug #6701: ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invalid cross-dev...
- Omitted the probably significant fact that /data2 is a partition in a different disk from /var
- 11:05 PM devops Bug #6701 (Resolved): ceph-deploy osd prepare on directory path fails: OSError: [Errno 18] Invali...
- Ceph version is 0.71-234-g1f02d00 built from src on bunti 13.10.
The desired setup is osd data in /data2/cephdata... - 08:39 PM Bug #6700 (Resolved): ceph-disk prepare fails to set up an encrypted disk
- RHEL 6.4 ceph 0.67.4
Ceph-disk prepare fails when setting up encrypted disks when the data and journal are locat... - 07:30 PM devops Bug #6699 (Resolved): sysvinit script setting incorrect OSD weights
- I noticed that the sysvinit script on a RHEL6.4 system was giving more weight to disks that were fuller. This is caus...
- 05:50 PM Bug #6003: journal Unable to read past sequence 406 ...
- This looks like the same problem seen in a unit test.
2013-10-31T15:23:41.910 INFO:teuthology.orchestra.run.err:[1... - 10:21 AM Bug #6003: journal Unable to read past sequence 406 ...
- 10:09 AM Bug #6003: journal Unable to read past sequence 406 ...
- Kernel: 3.8.0-21-generic
- 10:07 AM Bug #6003: journal Unable to read past sequence 406 ...
- slider:~samuelj/bug_logs/13-10-31-04:05:43/remote
With journal logging! I seem to be able to reproduce this prett... - 08:28 AM Bug #6003: journal Unable to read past sequence 406 ...
- /a/teuthology-2013-10-28_23:01:45-kcephfs-master-testing-basic-plana/73927
- 05:10 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- is it possible you've done any shenanigans like 'ceph pg force_create_pg ...' at some point in the past?
Yes, inde... - 01:52 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- oh, looking more closely, it looks like osd.42 is trying to activate but has no data for this pg. is it possible you...
- 11:06 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- Any chance I can start these OSD without losing data on them?
- 11:01 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- ...
- 10:30 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- Ivan Kudryavtsev wrote:
> I also have
>
> 2013-11-01 00:13:25.635829 mon.0 [WRN] mon.2 10.252.0.4:6789/0 clock s... - 10:14 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- I also have
2013-11-01 00:13:25.635829 mon.0 [WRN] mon.2 10.252.0.4:6789/0 clock skew 0.870987s > max 0.05s
2013... - 10:14 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- Here it is, attached.
- 09:36 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- can you turn up the logs on osd.42 too, and then restart this same osd (to reproduce the same crash), and then attach...
- 04:59 PM devops Bug #6698: new osds added to the cluster not starting up due to crush lookup failure
- ...
- 04:58 PM devops Bug #6698 (Resolved): new osds added to the cluster not starting up due to crush lookup failure
- ceph version: next branch [0.72-rc1]
test setup : mira025, mira038
had a cluster running on master branch, upgr... - 03:26 PM Bug #6697 (Resolved): strncmp(3) must not be used on binary data
- strncmp(3) is intended to compare C strings, so it quits comparing after encountering a null character. The Linux ma...
- 03:18 PM Bug #5951: osd: next: EEXIST on mkcoll
- 592a99ccd8da2d7843ebb7ce6439566732580b7a
- 01:24 PM Bug #5951: osd: next: EEXIST on mkcoll
- Found it. See wip-eexist.
- 02:55 PM rgw Bug #6696 (Can't reproduce): Upgrade rgw failure in nightly tests. (/home/ubuntu/cephtest/s3-test...
- Suite tests using the following sets of yaml files:...
- 02:43 PM rbd Bug #6695 (Won't Fix): Upgrade rbd failure in nightly tests. (mkdir --p ..)
- The overnight test for the following set of yaml files:...
- 12:18 PM rgw Bug #6694 (Duplicate): radosgw upstart script doesn't provide the -n parameter
- I'm using the Ceph Chef Cookbook to create and manage the configuration of a new Ceph 0.67.4 cluster on Ubuntu 12.10,...
- 11:19 AM Bug #6680 (Resolved): libcephfs.h broken for gcc
- 11:19 AM Bug #6680: libcephfs.h broken for gcc
- d3b56918698803ce441d9b1ef0185caebed4d433
- 09:40 AM Bug #6680: libcephfs.h broken for gcc
- it was a cleanup for clang to not mix class vs struct. that's secondary, though.. this should definitely build in c.
- 11:04 AM rgw Bug #5843: swift api: x-container-meta-{key} should not be allowed on an object
- It came from a customer, after discussions it is not causing any issue at all. We can leave it how it is.
- 09:50 AM rbd Bug #6693: "rbd ls" returns error if the pool empty
- I agree the exit code should be 0 in this case, but I'm downgrading the priority since there's an easy workaround and...
- 05:33 AM rbd Bug #6693 (Resolved): "rbd ls" returns error if the pool empty
- if the pool exists but only empty, instead of not returning anything, it returns an error message that it's empty, an...
- 09:14 AM CephFS Bug #6613 (Need More Info): samba is crashing in teuthology
- 09:05 AM CephFS Bug #6608: samba teuthology dbench failure
- not sambe issue. It's wrong to run two instances of dbench on the same test directory.
- 08:01 AM CephFS Bug #6608 (Need More Info): samba teuthology dbench failure
- If samba is broken in this configuration we'll need to change our test (and report upstream, if we aren't using it in...
- 07:45 AM devops Bug #6691 (Resolved): debian wheezy fails to be detected by ceph-deploy
- Opened pull request: https://github.com/ceph/ceph-deploy/pull/121
Merged into ceph-deploy master branch with hash;...
10/30/2013
- 11:44 PM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- It's on dumpling node.
root@ceph-osd-1-2:/# ceph -v
ceph version 0.67.4 (ad85b8bfafea6232d64cb7ba76a8b6e8252fa0c7)
- 10:17 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- Here it is.
- 09:34 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- Can you set 'debug osd = 20' and 'debug ms = 1', and restart and osd so that we have a complete log leading up to the...
- 09:06 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- During recovery some osds crashing and after crash unable to up again, looks like in topic "Domino crash" regarding o...
- 07:35 AM Bug #6684: osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log.tail)
- Also it seems, that my meta information is corrupted, I have a lot of images in RBD and now, I don't see any:
rbd:... - 06:25 AM Bug #6684 (Rejected): osd/PGLog.cc: 402: FAILED assert(log.head >= olog.tail && olog.head >= log....
- Hello, Guys, I'm in a big trouble. After upgrading Bobtail to cuttlefish I met in a
great troubles, unable to make ... - 09:03 PM Bug #6003: journal Unable to read past sequence 406 ...
- I hit this same problem in recovery testing. Attached is the complete log.
- 07:29 PM rgw Bug #6672: normal PUT object requests got 405 sometimes
- Send the pull request against the 'next' branch at https://github.com/ceph/ceph/pull/795
- 04:56 PM rgw Bug #5374 (Resolved): Avoid relying on keystone's admin token
- Merged, commit:b20d1bf33bdf6ed25ef9bb37afc3890282ece6d4
- 04:52 PM devops Bug #6690 (Resolved): ceph-deploy: bogus apt sources.list.d/ceph.list file on wheezy
- 03:24 PM devops Bug #6690: ceph-deploy: bogus apt sources.list.d/ceph.list file on wheezy
- ...
- 02:25 PM devops Bug #6690 (Resolved): ceph-deploy: bogus apt sources.list.d/ceph.list file on wheezy
- ...
- 04:23 PM Bug #6692 (Resolved): Documentation: Command line incorrect.
- In http://ceph.com/docs/master/rados/operations/authentication/
the enabling cephx command #2 that reads:
ceph-... - 04:11 PM Fix #5612 (Resolved): down mons prevents osd hosts from booting properly
- 04:10 PM devops Bug #6691 (Resolved): debian wheezy fails to be detected by ceph-deploy
- platform module in Python does not mention `wheezy` at all.
- 12:29 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- It did happen immediately after transitioning from being the backfill target
- 12:27 PM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- Or not, we process the OP_BACKFILL_PROGRESS ordered vs the removal ops.
- 10:50 AM Bug #6685 (In Progress): osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- 10:50 AM Bug #6685: osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- Urgh, this is because we can advance last_backfill past uncompleted removal ops during backfill. Working on it.
- 09:56 AM Bug #6685 (Resolved): osd/ReplicatedPG.cc: 8345: FAILED assert(0 == "erroneously present object")
- ...
- 12:29 PM Bug #6689 (Resolved): osd: remove_redundant_pg_temp() can be slow on big clusters
- it blocks the cpu, which prevents us from processing lease acks, which makes us call a new election.
observed on b... - 12:26 PM Fix #6116: osd: incomplete pg from thrashing on next
- Removed the teuthology workaround as well.
- 12:24 PM Fix #6116 (Resolved): osd: incomplete pg from thrashing on next
- I was way off on this one. We do ack the backfill completion. I suspect that the actual problem was probably fixed ...
- 11:37 AM rbd Documentation #5006: doc: openstack configuration changes for havana
- Should get this updated prior to HK Summit if at all possible.
- 10:42 AM Feature #6687 (New): Ability to set up/down/in/out based on CRUSH hierarchy
- It would be nice if the cli tools supported the ability to perform OSD state transitions based of CRUSH hierarchy. Fo...
- 10:10 AM Bug #6686 (Resolved): segfault in prioritized queue dequeue
- I've been hitting the following fault on OSX. At first I suspected a nasty bug mixing libc++ with libstdc++ on accide...
- 05:36 AM Bug #6683 (Fix Under Review): mon: MonmapMonitor: specify epoch in 'mon getmap'
- 05:36 AM Bug #6683: mon: MonmapMonitor: specify epoch in 'mon getmap'
- wip-6683, https://github.com/ceph/ceph/pull/789 and commit:e5efd882e7eba360158c2218bbd197766d082b02
- 05:30 AM Bug #6683: mon: MonmapMonitor: specify epoch in 'mon getmap'
- mon/MonCommands.h specifies that 'mon getmap' receives an epoch number, but we don't handle it and always return the ...
- 04:16 AM Bug #6683 (Resolved): mon: MonmapMonitor: specify epoch in 'mon getmap'
- 04:13 AM Subtask #3605 (Resolved): mon: print lookup path when reporting -ENOENT to user-space
- AFAICT this has been addressed in multiple occasions throughout the last year.
- 04:12 AM Subtask #2674 (Rejected): mon: Single-Paxos: mon commits suicide after remove&add
- works just fine on current next
- 04:08 AM Bug #5967 (Resolved): monitor caps parser should accept '.' as a legal unquoted character
- See commit:1f573c885cce776a7e8b2f3d4b91a304a8bc15c5
- 01:24 AM Bug #6614 (Resolved): unittest_bufferlist failed
- Good to know, thanks for the feedback :-)
10/29/2013
- 09:40 PM Documentation #6682 (Resolved): Adjustments to the Ceph Quick installation guide
- Hello,
Following the Quick installation guide, I ran into some issues which I think should be addressed:
[PREFL... - 09:27 PM Bug #6614: unittest_bufferlist failed
- Test ”make check“ succeed now.
- 08:42 PM rgw Bug #6672: normal PUT object requests got 405 sometimes
- Updated with a new commit. Please help review it at https://github.com/ceph/ceph/pull/783
- 09:30 AM rgw Bug #6672 (Fix Under Review): normal PUT object requests got 405 sometimes
- 08:00 AM rgw Bug #6672: normal PUT object requests got 405 sometimes
- Thanks, the initialization code is indeed not safe. See my comment in the pull request.
- 04:57 AM rgw Bug #6672: normal PUT object requests got 405 sometimes
- A quick fix is to set initialized to true after populating table in hex_to_num().
Please help review the fix at: h... - 04:53 AM rgw Bug #6672 (Resolved): normal PUT object requests got 405 sometimes
- Under load testing, normal PUT object requests got 405 sometimes. The root cause is that there is a defect in hex_to_...
- 07:45 PM CephFS Bug #6608 (Rejected): samba teuthology dbench failure
- running dbench on local FS in parallel also results in similar failures.
- 11:43 AM CephFS Bug #6608: samba teuthology dbench failure
- http://qa-proxy.ceph.com/teuthology/teuthology-2013-10-27_19:01:26-fs-dumpling-testing-basic-plana/71285/
http://qa-... - 05:09 PM Bug #6681: osd recovery hung
- Not an rc blocker. This is actually a fairly old bug and can be worked around by restarting one of the effected osds...
- 05:07 PM Bug #6681 (Resolved): osd recovery hung
- ubuntu@teuthology:/a/teuthology-2013-10-29_10:45:19-rados-next-testing-basic-plana
ubuntu@plana31:~$ sudo ceph -s
... - 04:36 PM Bug #6605 (Resolved): mon: remove full osd state on "osd rm"
- merged into master
- 04:29 PM Bug #6605 (Fix Under Review): mon: remove full osd state on "osd rm"
- wip-6605, pull request 787, commit:e02740ac5da7c9f5e4c1fdd603918e56c05123de
Greg Farnum wrote:
> It turns out t... - 04:21 PM Bug #6680 (Resolved): libcephfs.h broken for gcc
- This patch https://github.com/ceph/ceph/commit/e1666d0400ecef464d33480e4290896404b7a7bd did the following:...
- 04:08 PM Bug #6679 (Resolved): throttle: transient unit test failure
- ...
- 03:49 PM rgw Feature #6678 (Resolved): rgw: reject writes to secondary zones
- Non-master zones are intended to be read-only. For disaster recovery, it may be useful to expose a secondary zone as ...
- 03:13 PM Bug #6633: osd: pgls vs osd restart/peering race misses objects
- list.cc actually has slightly flaky tests if they are concurrent with pg splitting. wip-6633 so far has fixes to tol...
- 02:40 PM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
- removing 'related to' link from this ticket to #5869
- 10:22 AM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
- Is it possible to recover without those osds?
- 10:22 AM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
- This is not actually related to 5869.
- 04:13 AM Bug #6671: FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
- If this is indeed the same bug, and not a different iteration, then maybe we should backport the fix at least for Dum...
- 02:37 PM rgw Feature #6677 (Resolved): rgw: add compatibility for MultipartUpload
- The AWS-PHP-SDK used MultipartUpload instead of CompleteMultipartUpload.
https://github.com/aws/aws-sdk-php/issues... - 02:03 PM RADOS Fix #6676 (New): osd should never block getting filestore throttle while holding the pg lock
- Currently, it is possible for the filestore finisher threads to block trying to acquire a pg lock in order to complet...
- 12:45 PM rgw Bug #6152: New S3 auth code fails when using response-* query string params to override response ...
- This is still broken on 0.67.4. Please consider backporting the fix (7a7361d7) to Dumpling.
- 12:12 PM devops Bug #6587 (Resolved): ceph-deploy does not use cluster arg for mon error checking
- Merged into ceph-deploy master branch with hash: da03ab7
- 12:12 PM devops Bug #6675 (Resolved): Fedora fails to import keys on ceph-deploy install
- Merged into ceph-deploy master branch with hash: 5145d94
It makes sure that all `rpm --import` commands are the ... - 11:24 AM devops Bug #6675 (Fix Under Review): Fedora fails to import keys on ceph-deploy install
- PR opened: https://github.com/ceph/ceph-deploy/pull/119
- 09:34 AM devops Bug #6675 (Resolved): Fedora fails to import keys on ceph-deploy install
- The command was fixed for CentOS but not for fedora:...
- 11:15 AM Fix #6673 (Resolved): 'osd pool set metadata pg_num 34' broken
- 09:40 AM Fix #6673 (Fix Under Review): 'osd pool set metadata pg_num 34' broken
- 08:57 AM Fix #6673: 'osd pool set metadata pg_num 34' broken
- the test later fails with...
- 08:55 AM Fix #6673: 'osd pool set metadata pg_num 34' broken
- What's broken about this, besides the ridiculous parsing output? We deliberately prevent splitting while creating the...
- 08:50 AM Fix #6673 (Resolved): 'osd pool set metadata pg_num 34' broken
- ...
- 11:05 AM Bug #6207 (Resolved): Found incorrect object contents
- This can be explained by prematurely updating last_backfill based on backfill_pos and backfills_in_flight.
ad5655b... - 10:28 AM Bug #6674: Busted client locking
- I don't think the theory from the previous comment has any merit! I wonder if it's possible that lockdep is missing a...
- 09:51 AM Bug #6674: Busted client locking
- Looking at initializations of CephContext, I don't see any explicit freeing, which begs the question which thread is ...
- 09:24 AM Bug #6674 (Resolved): Busted client locking
- ...
- 08:34 AM devops Feature #3311 (Resolved): ceph-deploy: pushy bug triggers on interpreter shutdown
- We no longer use Pushy
- 08:33 AM devops Feature #3312 (Resolved): ceph-deploy: pushy uses pickle, that's a security problem
- We no longer use Pushy.
- 07:57 AM devops Fix #4953 (Resolved): ceph-deploy: dns mismatches can cause gatherkeys to fail
- ceph-deploy will now check for common errors (like the one mentioned) when deploying monitors.
- 07:46 AM devops Feature #4998 (Resolved): ceph-deploy should allow user specified installation sources
- with the new `--no-adjust-repos` we rely on users setting up their own mirrors.
A new 'firewall-install` command i... - 07:45 AM devops Feature #3922 (Resolved): ceph-deploy: version command
- ceph-deploy has a `--version` command now.
- 01:42 AM Bug #6400: osd crashed in dumpling due to unexpected error (EEXIST?)
- I can reproduce this issue on CentOS 6.4 machine every time.
My steps are:
# Build Ceph with dumpling release c...
10/28/2013
- 11:16 PM Bug #6671 (Can't reproduce): FAILED assert(ret) in OSDMapRef OSDService::get_map(epoch_t)
We were on 0.56.1; we shut down all of the OSDs and mon on a particular node, rebooted the node for maintenance an...- 09:03 PM CephFS Bug #6613: samba is crashing in teuthology
- tail of client log:
---
2013-10-22 08:05:27.405155 7ff1167fc700 20 client.4105 trim_cache size 0 max 0
2013-10-22 ... - 10:05 AM CephFS Bug #6613: samba is crashing in teuthology
- This is happening regularly on dumpling and next, but I don't think I've seen it on cuttlefish. We've clearly done so...
- 04:15 PM Bug #6003: journal Unable to read past sequence 406 ...
- ubuntu@teuthology:/a/teuthology-2013-10-27_19:00:21-rados-dumpling-testing-basic-plana/71126
- 04:06 PM Bug #6585 (Resolved): osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
- 01:51 PM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
- I've merged wip-6585, but it's not quite fixed yet.
- 09:51 AM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
- We think we might have fixed this, or at least most of it — but testing is shaking out a lot of long-standing bugs in...
- 04:06 PM rgw Bug #6621 (Resolved): quota: the max-size and max-objects value when zero
- 01:54 PM rgw Bug #6621: quota: the max-size and max-objects value when zero
- There was an actual issue with setting negative values, so moved that back into the bug tracker. A fix was pushed to ...
- 12:00 PM rgw Bug #6621: quota: the max-size and max-objects value when zero
- ok got it, if its by design, i think we should add this to the radosgw-admin help page saying that we allow max size ...
- 11:53 AM rgw Bug #6621: quota: the max-size and max-objects value when zero
- Oh, I thought the problem was the other way around (that it wasn't accepting 0 or negative numbers). That's by design...
- 01:37 PM devops Bug #6587 (Fix Under Review): ceph-deploy does not use cluster arg for mon error checking
- Opened pull request: https://github.com/ceph/ceph-deploy/pull/116
- 01:13 PM devops Bug #6650 (Resolved): ceph-deploy: should exit/error when apt fails
- Merged into ceph-deploy's master branch with hash: 1a72df3
- 10:58 AM devops Bug #6650 (Fix Under Review): ceph-deploy: should exit/error when apt fails
- Pull request opened: https://github.com/ceph/ceph-deploy/pull/115
- 06:48 AM devops Bug #6650 (In Progress): ceph-deploy: should exit/error when apt fails
- Just to be clear, this is not ceph-deploy failing to install, it is APT having issues with the install.
The first ... - 11:59 AM devops Bug #6654: ceph-deploy: bootstrap requires python-virtualenv on raring
- the bootstrap script does not install python-virtualenv but it does check for it just to error out.
- 11:42 AM Bug #6633: osd: pgls vs osd restart/peering race misses objects
- Logs on slider:/~samuelj/buglogs/13-10-27-15:50:48
- 10:30 AM Bug #6598: osd crash after recreating pool with same name (cuttlefish + bobtail?)
- Seems that we had accidentaly triggered existing assert() and in the wild it would be almost impossible to reproduce ...
- 10:04 AM CephFS Bug #6608: samba teuthology dbench failure
- http://qa-proxy.ceph.com/teuthology/teuthology-2013-10-25_23:01:10-fs-master-testing-basic-plana/69202/
http://qa-pr... - 09:40 AM CephFS Bug #6655 (Need More Info): readdir() fails on CephFS mount symlinked directories
- 09:21 AM rbd Bug #5425: krbd: xfstest 89 hang, 'read_partial_message skipping long message'
- ubuntu@teuthology:/a/teuthology-2013-10-26_23:01:27-krbd-next-testing-basic-plana/70455
- 06:43 AM devops Bug #6588 (Resolved): use WARNING level when keyring does not exist
- Merged into ceph-deploy master branch with hash: cea6139
- 04:42 AM rbd Bug #5876: Assertion failure in rbd_img_obj_callback() : rbd_assert(which >= img_request->next_co...
- Hi,
I had this bug just now (see below), with a 3.10.16 kernel, with patches from your Git (cf attached file), and...
10/27/2013
- 07:15 PM Bug #5823: cpu load on cluster node is very high, client can't get data on pg from primary node ...
- I saw in the log file "osd is marked wrongly", but i checked its process was running...
- 09:53 AM devops Bug #6650: ceph-deploy: should exit/error when apt fails
- a very annoying bug. I don't see how this could have made it into any release at all, since it is basically one of th...
10/26/2013
- 04:38 AM CephFS Bug #6655: readdir() fails on CephFS mount symlinked directories
- I can't reproduce this locally. Which kernel did you use? please try ceph-fuse and recent kernel.
- 02:12 AM CephFS Bug #6655: readdir() fails on CephFS mount symlinked directories
- If struggling to reproduce, it seems like readdir() works directly after other access to the symlink, but only once.
... - 01:38 AM CephFS Bug #6655 (Can't reproduce): readdir() fails on CephFS mount symlinked directories
- Background:
* Ubuntu Server 12.04 64bit.
* CephFS Dumpling 0.67.4
* We moved from local filesystem to CephFS ...
10/25/2013
- 08:10 PM Bug #6043: upstart does not reflect running ceph-osd daemons (ubuntu 13.04 only)
- We're using Chef to install our nodes (which uses ceph-disk tools to manage the disks) so:...
- 04:49 PM Bug #6043: upstart does not reflect running ceph-osd daemons (ubuntu 13.04 only)
- Hunter, what are the osd data directories named?
- 04:31 PM Bug #6043 (Can't reproduce): upstart does not reflect running ceph-osd daemons (ubuntu 13.04 only)
- I am not able to reproduce this issue on raring with latest stable dumpling branch [v0.67.4]
test setup tried: vpm... - 05:53 PM Bug #6635 (Resolved): ceph pool rename needs to succeed if the source pool does not exist and the...
- 05:00 PM Fix #5612 (Fix Under Review): down mons prevents osd hosts from booting properly
- 04:58 PM Fix #5612 (New): down mons prevents osd hosts from booting properly
- https://github.com/ceph/ceph/pull/772
- 04:58 PM Fix #5612 (Fix Under Review): down mons prevents osd hosts from booting properly
- 04:58 PM devops Bug #6654 (Rejected): ceph-deploy: bootstrap requires python-virtualenv on raring
- ceph-deploy bootstrap prompts for "installing python-virtualenv".
I thought bootstrap takes care of this and we di... - 04:40 PM Bug #6614: unittest_bufferlist failed
- Compiled against 289b7903407ce1b34f1afe9e0c769093c14d0ba9 on centos-6.4 the test completes successfully. But the test...
- 02:32 PM Bug #6614 (In Progress): unittest_bufferlist failed
- The error originaly reported is not an actual error, it is the output of an assert that was expected to happen as par...
- 04:34 PM Bug #6598: osd crash after recreating pool with same name (cuttlefish + bobtail?)
- occurs on cuttlefish, not dumpling.
- 04:24 PM Fix #4942 (Resolved): librados: do not hang on auth failure on start
- this works on cuttlefish and dumpling and later. non-trivial backport to fix it on bobtail.
- 04:13 PM devops Bug #6650: ceph-deploy: should exit/error when apt fails
- this happens on precise as well....
- 03:47 PM devops Bug #6650 (Resolved): ceph-deploy: should exit/error when apt fails
- ceph-deploy version: 1.2.7...
- 04:06 PM Bug #6649 (Can't reproduce): ceph-deploy + rbd export tests fail
- 03:34 PM Bug #6649 (Can't reproduce): ceph-deploy + rbd export tests fail
- ...
- 04:06 PM Bug #6648 (Can't reproduce): ceph-deploy librbd test fails on centos
- 03:15 PM Bug #6648 (Can't reproduce): ceph-deploy librbd test fails on centos
- ...
- 03:08 PM Bug #6622 (Resolved): nightly runs:apt-get failures on ubuntu
- resolving this; curl is now in hte repo for dumpling. i suspect teh branch just hadn't rebuilt yet.
- 02:56 PM Bug #6272 (Closed): ceph command usage missing setcrushmap
- 02:18 PM Bug #6390 (Can't reproduce): ENOTEMPTY on TEMP coll on replay
- 02:15 PM Bug #5823: cpu load on cluster node is very high, client can't get data on pg from primary node ...
- I mean, was it always osds on a particular node which get marked down?
- 02:11 PM Bug #5951 (Can't reproduce): osd: next: EEXIST on mkcoll
- 02:09 PM Feature #6645 (Closed): EC: [link] BPC (basic pyramid code)
- 02:00 PM Fix #6059 (Resolved): osd: block reads while repgather is writing across replicas
- 01:56 PM Feature #6644 (Resolved): cachepool: evict
- 01:56 PM Feature #6643 (Fix Under Review): cachepool: flush
- 01:56 PM Feature #6643 (Resolved): cachepool: flush
- 01:55 PM Feature #6188 (Fix Under Review): cachepool: osd: promote on write and mark object dirty
- 01:35 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
- Memory usage on the host looks stable. Traffic on the host dips because it loses the crashed guest's traffic. Disk ut...
- 01:19 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
- Mike Dawson wrote:
> Just discovered that the load on the host spikes when the crash occurs. I don't have info on th... - 01:14 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
- Just discovered that the load on the host spikes when the crash occurs. I don't have info on the first, but the past ...
- 12:43 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
- One more crash today. Again, different host and guest with similar config and workloads.
*** Error in `qemu-system... - 01:25 PM rgw Bug #6604 (Pending Backport): radosgw-agent: opstate tracking error
- 01:11 PM rgw Feature #5170: RGW: Object restriping tool to fix large objects from argonaut.
- 01:11 PM rgw Feature #4466 (Resolved): Quotas per bucket - synchronous
- 01:08 PM rbd Feature #6432: Change TGT Plugin to allow setting client name for authentication
- 12:49 PM Bug #6574 (Resolved): osd/ReplicatedPG.cc: 7851: FAILED assert(0)
- 847ea6059211b1ce104207584a57badf830103d6
- 11:48 AM devops Feature #6067 (Resolved): ceph-deploy: make mon create catch common errors
- #6638 was create to tackle adding monitors. Everything else in this ticket has been already implemented in ceph-deplo...
- 11:38 AM devops Feature #3347 (Resolved): ceph-deploy: allow setting ssh user
- Merged into ceph-deploy's master branch with hash: e623b1e
- 11:23 AM devops Bug #6595: Hardcoded install path in ceph-disk
- This is a custom install we are building ceph from source. Did not provide the --prefix argument to ./configure and e...
- 11:03 AM devops Bug #6595: Hardcoded install path in ceph-disk
- How did you manage to get ceph executables in that path? Is this a custom install?
*Everywhere* in ceph-disk it re... - 11:18 AM Bug #6636 (Resolved): sockaddr_storage and uuid_t are not portable to other platforms
- ...
- 10:54 AM devops Bug #6587: ceph-deploy does not use cluster arg for mon error checking
- This needs to be fixed ASAP before the next release. It is annoying (and completely misleading) for users that are us...
- 10:14 AM rbd Bug #6072 (Resolved): librbd image rename breaks child backwards reference
- in next and backported!
10/24/2013
- 09:08 PM rbd Bug #6631: disabling writethrough until flush appears to disable RBD cache
- More repetition of tests..
// IOPS for Sequential 4KB Write _with_ "rbd cache writethrough until flush = true"
Se... - 09:06 PM rbd Bug #6631: disabling writethrough until flush appears to disable RBD cache
- On Nova node..
// Ceph.conf
[global]
fsid = c14be55f-f608-4f76-aaa3-925b07a72e43
mon initial members = alln01-c... - 02:19 PM rbd Bug #6631 (Closed): disabling writethrough until flush appears to disable RBD cache
- Recently we saw a report that when using fio to perform 4K sequential direct IO writes to files on an XFS filesystem ...
- 09:01 PM rbd Bug #6630: fio tests against raw RBD volumes show strange results
- With multiple runs of same test on RBD device with XFS, we get following outputs..
fio --rw=write -ioengine=libaio... - 08:57 PM rbd Bug #6630: fio tests against raw RBD volumes show strange results
- FIO workload:
[root@sm-rhel6-template ceph-perf]# cat fio-small-write.sh
#!/bin/bash
mkdir -p /root/ceph-per... - 02:12 PM rbd Bug #6630 (Resolved): fio tests against raw RBD volumes show strange results
- Recently we've seen a report that when performing direct io sequential write tests against raw RBD volumes, RBD cache...
- 07:33 PM Bug #6635 (Fix Under Review): ceph pool rename needs to succeed if the source pool does not exist...
- https://github.com/ceph/ceph/pull/765
- 05:08 PM Bug #6635 (Resolved): ceph pool rename needs to succeed if the source pool does not exist and the...
- That way, the command will be somewhat idempotent.
- 06:18 PM rbd Bug #6072 (Fix Under Review): librbd image rename breaks child backwards reference
- This was just a needlessly small limit in the python bindings. https://github.com/ceph/ceph/pull/764
- 05:29 PM rbd Bug #6480: librbd crashed qemu-system-x86_64
- Hit a similar crash with a slightly different backtrace this today. Different host and vm. Both hosts have similar co...
- 05:07 PM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
- 65817 FAIL scheduled_teuthology@teuthology rados/thrash/{clusters/fixed-2.yaml fs/btrfs.yaml msgr-failures/osd-delay....
- 05:06 PM Bug #6627 (Resolved): segfault at PGMap::dirty_all during upgrade mon
- 01:13 PM Bug #6627: segfault at PGMap::dirty_all during upgrade mon
- hi Joao, reproduced this issue when upgrading monitors one at a time from cuttlefish to next with debugs on - logs ar...
- 09:53 AM Bug #6627 (Resolved): segfault at PGMap::dirty_all during upgrade mon
- logs: ubuntu@teuthology:/a/teuthology-2013-10-22_01:30:02-upgrade-next-testing-basic-plana/64049...
- 04:44 PM rgw Bug #6621: quota: the max-size and max-objects value when zero
- same applies to max-objects as well...
- 03:46 PM Cleanup #6634 (Resolved): MOSDSubOp: remove unused snapc/snapcontext members
- These members are never accessed, except to encode, decode, and print. Since they are printed in logs, their being em...
- 03:11 PM Bug #6003: journal Unable to read past sequence 406 ...
- ubuntu@teuthology:/a/teuthology-2013-10-23_19:00:21-rados-dumpling-testing-basic-plana/65408
- 03:10 PM Bug #6633 (Resolved): osd: pgls vs osd restart/peering race misses objects
- test saw a sequence like:
- create object
- start osd
- things peer
- pgls
-> pgls returns empty result
I exp... - 02:08 PM Bug #6629: fd cache and external changes to recently-modified files don't behave nicely
- Maybe we could use inotify to watch for relevant changes to the filestore and close the cached FDs when their files e...
- 01:28 PM Bug #6629 (Won't Fix): fd cache and external changes to recently-modified files don't behave nicely
- A user reported that they were trying to test pg repair functionality and it did not appear to be restoring files del...
- 12:13 PM rbd Bug #6628 (Resolved): krbd: BUG during ceph_osdc_stop() sometimes when rbd_add() fails
- BUG at osd_client.c:978:...
- 11:32 AM rbd Fix #6079: libceph: osd_client does not handle PAUSERD or PAUSEWR or FULL flags in osdmap
- This is worse than I thought - for the FULL flag, the OSD will return -ENOSPC, which will get translated into -EIO by...
- 09:49 AM rgw Fix #6616: radosgw: system users are not handled well by read_policy()
- https://github.com/ceph/ceph/pull/763
- 09:49 AM rgw Fix #6615 (Fix Under Review): radosgw: data log list admin api does not include any markers
- https://github.com/ceph/ceph/pull/763
- 09:49 AM rgw Bug #6604 (Fix Under Review): radosgw-agent: opstate tracking error
- https://github.com/ceph/ceph/pull/763
- 09:25 AM rbd Feature #6626 (Resolved): openstack: cinder: allow users to delete snapshots that have clones
- Hide the dependency on the backend, and do auto-flattening as needed (similar to the existing auto-flattening for the...
- 08:39 AM Bug #6620 (Pending Backport): mon: MDSMonitor/MDSMap: 'ceph report' leads to segfault on MDSMap::...
- commit:0e8182edd850f061421777988974efbaa3575b9f
We should probably backport this to dumpling. - 06:59 AM devops Bug #6592 (In Progress): 3.8 kernel + /dev/cciss/c0d1 + precise : fail to show in /dev/disk/by-pa...
- *blkid -o udev /dev/cciss/c0d1p2* does not return anything. Note, however, that after a reboot the OSDs are running f...
- 01:33 AM CephFS Feature #3541 (Resolved): mds: robust ino lookup using file backpointers
- 01:33 AM CephFS Feature #4295 (Resolved): mds: Actually purge deleted directories
- 12:19 AM Bug #5823: cpu load on cluster node is very high, client can't get data on pg from primary node ...
- It occurred as having multiple write requests to cluster...
I deployed 10 osds/node with default configuration.
...
10/23/2013
- 11:23 PM rbd Bug #6576 (Resolved): librbd: test_librbd_fsx segfaults on start up on 32-bit arm
- teuthology.git commit:705a77f5d1c3159ac61200283dd50c154fcc55aa
- 07:25 PM rbd Bug #6576 (Fix Under Review): librbd: test_librbd_fsx segfaults on start up on 32-bit arm
- https://github.com/ceph/teuthology/pull/143
- 06:49 PM CephFS Bug #6609: teuthology rsync workunit failure
- files were synced appropriately. rsync only sync directory share/doc/ 's timestamp or mode when it was executed for t...
- 03:18 PM CephFS Bug #6609: teuthology rsync workunit failure
- I didn't look at the details much (even to figure out what the file transfer issues were). What kind of timestamp iss...
- 06:23 PM Bug #6622: nightly runs:apt-get failures on ubuntu
- the ceph-build-deb-native.sh script has been updated to include the curl gnutls packages in the generated repo. i ra...
- 06:02 PM Bug #6622: nightly runs:apt-get failures on ubuntu
- I manually recreated what was happening on a new vm to get the full apt output. here is the problem:
Reading state... - 05:32 PM Bug #6622 (Resolved): nightly runs:apt-get failures on ubuntu
- logs:ubuntu@teuthology:/a/teuthology-2013-10-23_01:35:02-upgrade-small-next-testing-basic-vps/64863...
- 05:43 PM CephFS Bug #6623 (Resolved): mds: update backtraces on existing clusters
- The backtrace code doesn't update existing clusters as it touches them, unless the paths actually change.
Zheng fi... - 05:19 PM rgw Bug #6621 (Resolved): quota: the max-size and max-objects value when zero
- ceph version: 0.71-249-g31a9492 (31a94922a9ada132bea06be308484ead84e4d879)
while setting quota, the max-size field... - 04:20 PM Bug #6585: osd: backfill vs copy-from delay badness (was osd: ENOENT on clone)
- Sam liked it; have squashed and am scheduling a suite run now.
- 04:00 PM Bug #6620: mon: MDSMonitor/MDSMap: 'ceph report' leads to segfault on MDSMap::dump_info
- ...
- 03:42 PM Bug #6620 (Resolved): mon: MDSMonitor/MDSMap: 'ceph report' leads to segfault on MDSMap::dump_info
- Triggered at least on 0.67.4 and beyond. Happens for a store from burnupi02, not for other stores. Currently assess...
- 03:37 PM devops Feature #6618 (Rejected): gitbuilder: run 'make check' before building debs and rpms
- The non-package gitbuilders do this, and it helps catch platform-specific errors. There are several platforms buildin...
- 03:31 PM Bug #6614: unittest_bufferlist failed
- We talked on irc last night, this was on CentOS 6.4. The deb/rpm gitbuilders aren't running make check - we should ad...
- 01:19 PM Bug #6614: unittest_bufferlist failed
- Where are you running this, and where are you getting the source (I can't find that commit)? Our gitbuilders (http://...
- 11:03 AM Feature #5992 (Resolved): EC: [link] Refactor Backfill to use PGBackend methods
- 11:00 AM Subtask #6391 (Duplicate): stuck incomplete
- Looks like these are the same problem (which, luckily, users are seriously unlikely to come across).
- 10:55 AM Fix #6116: osd: incomplete pg from thrashing on next
- The workaround I put into teuthology was inadequate, I'm going to put this in the backlog and downgrade it now that i...
- 10:19 AM Documentation #5618 (Resolved): radosgw pool size guidelines
- There's a full section on pools in the configuration reference now.
http://ceph.com/docs/master/radosgw/config-ref/... - 10:04 AM rgw Fix #6616 (Resolved): radosgw: system users are not handled well by read_policy()
- System users should be able to read suspended buckets and get 404 for deleted buckets instead of 403.
- 10:02 AM rgw Fix #6615 (Resolved): radosgw: data log list admin api does not include any markers
- The last marker of the log entries fetched is needed to determine the position in the log. Previously the radosgw-age...
- 09:59 AM devops Bug #6595: Hardcoded install path in ceph-disk
- It does not match the default install path that ceph is currently using. Ceph was installed to /usr/local/bin by defa...
- 07:43 AM RADOS Fix #6570: osd: do not keep full pg log entries in memory
- Thank your for taking care of this. This is really a huge problem for us.
I don't quite understand your statement... - 02:10 AM RADOS Fix #6570: osd: do not keep full pg log entries in memory
- Is there any danger for increasing peering time? It would be awesome to make this feature configurable since some peo...
- 02:31 AM Feature #5984: mon: probe monitors to check on their status regardless of quorum
- commit:c2cf8489bc9c8fa40153d8ddb163e9b25e72bcd5
- 02:29 AM Feature #5984 (Resolved): mon: probe monitors to check on their status regardless of quorum
Also available in: Atom