Ceph : Issueshttps://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2021-10-18T02:27:18ZCeph
Redmine rbd - Bug #52962 (Fix Under Review): rbd: increase overlap range when overlap occur.https://tracker.ceph.com/issues/529622021-10-18T02:27:18Zjianpeng majianpeng.ma@intel.com
<p>We use BlockGuard to avoid reorder when overlap occur. But the following<br />case don't work:<br />a: write(3,5)<br />b: write(4,9)<br />c: write(8, 11).<br />b overlap w/ a, so it append in a. c overlap w/ b.<br />In fact, c shuld append b. But currently, it can't change overlap range<br />when b overlap a occur. The c can't be detained and issue before b.<br />We should change overlap from (3,5) to (3, 9) when b overlap a.</p> RADOS - Feature #49089 (Fix Under Review): msg: add new func support_reencodehttps://tracker.ceph.com/issues/490892021-02-02T08:13:27Zjianpeng majianpeng.ma@intel.com
<p>Currently, we use Messenger::ms_can_fast_dispatch to verfiy Message whether support reencode. Now we add a new api of Message to support</p>
<p>Why MOSDMap can't support reencode: in MOSDMap::encode_payload, it clear data in<br /><a class="external" href="https://github.com/ceph/ceph/blob/master/src/messages/MOSDMap.h#L116">https://github.com/ceph/ceph/blob/master/src/messages/MOSDMap.h#L116</a><br /><a class="external" href="https://github.com/ceph/ceph/blob/master/src/messages/MOSDMap.h#L139">https://github.com/ceph/ceph/blob/master/src/messages/MOSDMap.h#L139</a>.<br />So it can't reencode.</p>
<p>But for other message, we can reencode. So we add new func to support reencode.</p> Ceph - Bug #39643 (In Progress): make jemalloc/tcmalloc work.https://tracker.ceph.com/issues/396432019-05-09T08:26:53Zjianpeng majianpeng.ma@intel.com
<p>In commit 2e01287, it let libcommon or libceph-common take care of this.<br />But in fact, libcommon/libceph-common didn't link ALLOC_LIBS which make other<br />bin-files couldn't use jemalloc/tcmalloc.</p> Ceph - Bug #24569 (Fix Under Review): Fix outorder between Thread::create and Thread::set_iopriohttps://tracker.ceph.com/issues/245692018-06-19T06:25:06Zjianpeng majianpeng.ma@intel.com
<p>In TheadPool::start_threads:</p>
<blockquote><blockquote>
<p>int r = wt->set_ioprio(ioprio_class, ioprio_priority);<br />if (r < 0)<br />lderr(cct) << " set_ioprio got " << cpp_strerror(r) << dendl;</p>
</blockquote></blockquote>
<blockquote><blockquote>
<p>wt->create(thread_name.c_str());</p>
</blockquote></blockquote>
<p>In fact, it should firstly call create.</p> Ceph - Bug #24567 (Fix Under Review): fix a race between Thread::create and Thread::set_iopriohttps://tracker.ceph.com/issues/245672018-06-19T06:07:39Zjianpeng majianpeng.ma@intel.com
<p>We may do the following process:<br />a)Thread::create/try_create<br />b)Thread::set_ioprio</p>
<p>But the entry of Thread maybe not exec after step a) which cause step b)<br />can't set currently ioprio.<br />To fix potential race, i add mutex which make step b) wait until entry<br />exec.</p> Ceph - Bug #14918 (Resolved): [BlueStore]If bluestore_sync_transaction == true, make bluestore de...https://tracker.ceph.com/issues/149182016-02-29T05:28:26Zjianpeng majianpeng.ma@intel.com
<p>In _txc_finish_io, it get the lock of osr and call _txc_state_proc.<br />If bluestore_sync_transaction true adn txd->wal_txn null, it will<br />call _txc_finish which need get lock of osr. So the deadlock occru.<br />The simple wal is put txc in kv_queue whether sync or async.</p> RADOS - Bug #14100 (New): the semantics of CEPH_OSD_OP_ZERO.https://tracker.ceph.com/issues/141002015-12-17T07:23:45Zjianpeng majianpeng.ma@intel.com
<p>Now in ReplicatedPG::do_osd_ops, for CEPH_OSD_OP_ZERO, there are the following handle:</p>
<p>a)if object din't exist, the op became no-op.<br />b)if object existed<br /><pre>
if (offset + len >= ob.size)
if (offset >= ob.size)
op became no-op
else
op became truncate and the ob.size became offset
else
zero(offset, len) like normal write opeartions
</pre></p>
<p>Those behaviors are ok for rbd. From the native semantics of zero, it write the data(off, len) w/ zero.Make later read data is zero.<br />For rbd, we know the object size. If non-exist or smaller, we will append zero data in the end.</p>
<p>rados has a api: rados_write_op_zero, it use zero as write which don't carry data from client to osd.<br />But for a normal object, <br />a)zero a non-exist object don't work<br />b)zero(offset >= ob.size) don't work(but write(offset >= ob.size can work)<br />c)zero(offset + len >= ob.size & offset < ob.size) don't work(zero become a truncate and ob.size=offset).Later read(offset, len) don't return any data.</p>
<p>In fact, for rbd, zero mean discard. But for rados, zero is a speical write.</p> CephFS - Feature #12265 (New): Add Iohint in cephfshttps://tracker.ceph.com/issues/122652015-07-10T05:27:31Zjianpeng majianpeng.ma@intel.com
<p>This is the plan of BP "Add Iohint in Cephfs"(<a class="external" href="http://tracker.ceph.com/projects/ceph/wiki/Add_IOhint_in_CephFS">http://tracker.ceph.com/projects/ceph/wiki/Add_IOhint_in_CephFS</a>).</p>
<p>By the following steps to implement this BP<br />a:add iohint in mds, mostly for mds journal. Like osd journal, most journal data don't read again and will overwrite. So w/ DONTNEED is make sense.<br />b:add iohint mount option. Different sub directory have different purpose. For example, one directory don't need cache data.<br />c:in cephfs client(kernel/client) add iohint. Hope the the posix_fadvise can effect the whole ceph cluster rather than client kernel page cache.<br />d:Maybe add iohint in cephfs repair.(Need review code).</p> RADOS - Feature #11046 (Resolved): osd: rados io hints improvementshttps://tracker.ceph.com/issues/110462015-03-06T00:48:28Zjianpeng majianpeng.ma@intel.com
<p><a class="external" href="https://wiki.ceph.com/Planning/Blueprints/Infernalis/osd%3A_rados_io_hints_improvements">https://wiki.ceph.com/Planning/Blueprints/Infernalis/osd%3A_rados_io_hints_improvements</a></p>
<p>For this BP, divided into three parts:<br />1: add iohint handle in cache tier<br />2: add option for rbd cli<br />3: add option for command about cephfs/ceph-fuse.</p> Ceph - Bug #9342 (Resolved): Different implementation for PGTransaction::get_bytes_written betwee...https://tracker.ceph.com/issues/93422014-09-04T07:20:02Zjianpeng majianpeng.ma@intel.com
<p>The l_osd_op_w_inb use this value. So the perf-dump display different value when write same object to erasure-pool/replicated-pool.<br />I think the implementation of ECTransaction is right. It should record the size of data which client wrote.</p> Ceph - Bug #9019 (Resolved): Makefile.am: error: required file './README' not foundhttps://tracker.ceph.com/issues/90192014-08-05T18:16:55Zjianpeng majianpeng.ma@intel.com
<p>commit(a923e2c9eb16823fa484c) Renamed README to README.md to render in markdown. After that, i can't generate Makefile.in.</p>
<pre><code>./autogen.sh<br />+ set -e<br />+ test -f src/ceph.in<br />+ which libtoolize<br />+ [ /usr/bin/libtoolize ]<br />+ LIBTOOLIZE=libtoolize<br />+ test -d .git<br />+ git submodule update --init<br />+ rm -f config.cache<br />+ aclocal -I m4 --install<br />+ check_for_pkg_config<br />+ which pkg-config<br />+ return<br />+ libtoolize --force --copy<br />libtoolize: putting auxiliary files in `.'.<br />libtoolize: copying file `./ltmain.sh'<br />libtoolize: putting macros in AC_CONFIG_MACRO_DIR, `m4'.<br />libtoolize: copying file `m4/libtool.m4'<br />libtoolize: copying file `m4/ltoptions.m4'<br />libtoolize: copying file `m4/ltsugar.m4'<br />libtoolize: copying file `m4/ltversion.m4'<br />libtoolize: copying file `m4/lt~obsolete.m4'<br />+ aclocal -I m4 --install<br />+ autoconf<br />+ autoheader<br />+ automake -a --add-missing -Wall<br />Makefile.am: error: required file './README' not found<br />src/test/Makefile.am:248: warning: patsubst %,$(srcdir: non-POSIX variable name<br />src/test/Makefile.am:248: (probably a GNU make extension)<br />src/Makefile.am:35: 'src/test/Makefile.am' included from here</code></pre> devops - Bug #5174 (Resolved): df: ‘/media/osd.0/.’: No such file or directoryhttps://tracker.ceph.com/issues/51742013-05-26T19:45:59Zjianpeng majianpeng.ma@intel.com
<p>In my cluster, there are two machine:<br />host1: mon/mds<br />host2: two osd/mon</p>
<p>When i exec" service ceph -a start' on host1, it will print:df: ‘/media/osd.0/.’: No such file or directory.</p>
<p>Only i create a directory /media/osd.0/ on host1.It won't print the messages.</p> devops - Bug #5161 (Resolved): daemons should create /var/run/ceph if it doesn't already existhttps://tracker.ceph.com/issues/51612013-05-23T19:49:54Zjianpeng majianpeng.ma@intel.com
<p>I wanted to add a new mon into a cluster.But when i exce "ceph-mon -i majianpeng --mkfs --monmap map --keyring key", it print the following messages:</p>
<blockquote>
<p>2013-05-24 10:37:26.795665 7f446912e7c0 -1 asok(0x1440000) AdminSocketConfigObs::init: failed: >AdminSocket::bind_and_listen: failed to bind the UNIX domain socket to '/var/run/ceph/ceph-mon.majianpeng.asok': (2) No >such file or directory<br />ceph-mon: created monfs at /var/lib/ceph/mon/mon.majianpeng for mon.majianpeng</p>
</blockquote>
<p>I think the reason is there is not /var/run/ceph.<br />After I created the directory, then exec again.It's ok.</p>
<p>In document on ceph.com,there is no guide to do this.<br />I think we should create /var/run/ceph automaticly if there is no this director.</p> devops - Bug #5150 (Resolved): How many memory need if we compile ceph?https://tracker.ceph.com/issues/51502013-05-22T17:21:37Zjianpeng majianpeng.ma@intel.com
<p>My machine hardware:<br />memory: 2G<br />Intel(R) Core(TM) i3-2120 CPU @ 3.30GHz</p>
<p>But when i compile ceph, it will cause OOM.<br />Why? Can we optimize it?</p> devops - Bug #5130 (Resolved): ceph-deploy osd create ubuntu:sdbhttps://tracker.ceph.com/issues/51302013-05-21T00:03:58Zjianpeng majianpeng.ma@intel.com
<p>root@majianpeng:/tmp/my-cluster# ceph-deploy osd create ubuntu:sdb<br />INFO:ceph-disk:Will colocate journal with data on /dev/sdb<br />Caution! After loading partitions, the CRC doesn't check out!<br />Caution: invalid main GPT header, but valid backup; regenerating main header<br />from backup!</p>
<p>Warning! Main partition table CRC mismatch! Loaded backup partition table<br />instead of main partition table!</p>
<p>Warning! One or more CRCs don't match. You should repair the disk!</p>
<p>Invalid partition data!<br />ceph-disk: Error: Command '['sgdisk', '--new=2:0:1024M', '--change-name=2:ceph journal', '--partition-guid=2:d23bc8e4-4967-4ae0-9966-99851c670580', '--typecode=2:45b0969e-9b03-4f30-b4c6-b4b80ceff106', '--', '/dev/sdb']' returned non-zero exit status 2<br />Traceback (most recent call last):<br /> File "/usr/bin/ceph-deploy", line 22, in <module><br /> main()<br /> File "/usr/lib/pymodules/python2.7/ceph_deploy/cli.py", line 112, in main<br /> return args.func(args)<br /> File "/usr/lib/pymodules/python2.7/ceph_deploy/osd.py", line 293, in osd<br /> prepare(args, cfg, activate_prepared_disk=True)<br /> File "/usr/lib/pymodules/python2.7/ceph_deploy/osd.py", line 177, in prepare<br /> dmcrypt_dir=args.dmcrypt_key_dir,<br /> File "/usr/lib/python2.7/dist-packages/pushy/protocol/proxy.py", line 255, in <lambda><br /> (conn.operator(type_, self, args, kwargs))<br /> File "/usr/lib/python2.7/dist-packages/pushy/protocol/connection.py", line 66, in operator<br /> return self.send_request(type_, (object, args, kwargs))<br /> File "/usr/lib/python2.7/dist-packages/pushy/protocol/baseconnection.py", line 323, in send_request<br /> return self.__handle(m)<br /> File "/usr/lib/python2.7/dist-packages/pushy/protocol/baseconnection.py", line 639, in __handle<br /> raise e<br />pushy.protocol.proxy.ExceptionProxy: Command '['ceph-disk-prepare', '--', '/dev/sdb']' returned non-zero exit status 1</p>