https://tracker.ceph.com/https://tracker.ceph.com/favicon.ico2021-07-13T12:41:22ZCeph Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1990162021-07-13T12:41:22ZSebastian Wagner
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/199016/diff?detail_id=208385">diff</a>)</li></ul> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1990172021-07-13T12:42:55ZSebastian Wagner
<ul><li><strong>Subject</strong> changed from <i>cephadm/rgw : RGW server is not coming up</i> to <i>cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initialize</i></li></ul> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1990182021-07-13T12:43:36ZSebastian Wagner
<ul></ul><p>the rgw log looks like so:<br /><pre>
2021-07-13T10:39:08.749+0000 7f8f5770b440 0 deferred set uid:gid to 167:167 (ceph:ceph)
2021-07-13T10:39:08.749+0000 7f8f5770b440 0 ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable), process radosgw, pid 2
2021-07-13T10:39:08.749+0000 7f8f5770b440 0 framework: beast
2021-07-13T10:39:08.749+0000 7f8f5770b440 0 framework conf key: port, val: 7080
2021-07-13T10:39:08.749+0000 7f8f5770b440 1 radosgw_Main not setting numa affinity
2021-07-13T10:44:08.750+0000 7f8f4399b700 -1 Initialization timeout, failed to initialize
(repeates 5 times)
</pre></p> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1990192021-07-13T12:48:14ZSebastian Wagner
<ul></ul><p>Try running</p>
<pre>
ceph orch daemon rm rgw.newstore1.fedora.mkhhgr
</pre>
<p>If this doesn't help, I will probably need some help form the RGW team.</p> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1990632021-07-13T18:12:39ZDimitri Savineau
<ul></ul><pre>
pgs: 100.000% pgs not active
33 undersized+peered
</pre>
<p>I'm pretty sure the issue occured because the PGs aren't active+clean.</p>
<p>The default RGW pools (.rgw.root, default.rgw.control, default.rgw.meta and default.rgw.log) should be created during the first RGW instance start.</p>
<p>But if the first RGW pool (.rgw.root) isn't in a correct state then the others aren't created.</p>
<p>@Jiffin : Would you be able to retest this but, because you have a all-in-one setup, set the osd_pool_default_size variable to 1 before deploying the OSDs ?</p>
<pre>
ceph config set global osd_pool_default_size 1
</pre> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1990652021-07-13T18:35:35ZDimitri Savineau
<ul></ul><p>Answering to myself...</p>
<pre>
# cephadm --image quay.io/ceph/ceph:v16.2.5 bootstrap --mon-ip x.x.x.x --skip-pull --skip-dashboard --skip-monitoring-stack
# cephadm shell ceph config set global mon_warn_on_pool_no_redundancy false
# cephadm shell ceph config set global osd_pool_default_size 1
# cephadm shell ceph orch apply osd --all-available-devices
Scheduled osd.all-available-devices update...
# cephadm shell ceph orch apply rgw newstore1 --port=7080 --placement=1
Scheduled rgw.newstore1 update...
# cephadm shell ceph orch ls --service_type rgw
NAME PORTS RUNNING REFRESHED AGE PLACEMENT
rgw.newstore1 ?:7080 1/1 5m ago 5m count:1
# cephadm shell ceph orch ps --service_name rgw.newstore1
NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID
rgw.newstore1.cephaio.xcyqan cephaio *:7080 running (5m) 5m ago 5m 15.5M - 16.2.5 6933c2a0b7dd c3cf0c0aa3ad
</pre>
<p>So I think we can close this issue.</p> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1991092021-07-14T14:19:38ZJiffin Tony Thottan
<ul><li><strong>Description</strong> updated (<a title="View differences" href="/journals/199109/diff?detail_id=208497">diff</a>)</li></ul><p>Dimitri Savineau wrote:</p>
<blockquote>
<p>Answering to myself...</p>
<p>[...]</p>
<p>So I think we can close this issue.</p>
</blockquote>
<p>I was able to brought up RGW server by setting to osd_pool_default_size. Hence closing this bug.<br />Thanks for the help.<br />--<br />Jiffin</p> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=1991102021-07-14T14:19:58ZJiffin Tony Thottan
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li></ul> Orchestrator - Bug #51642: cephadm/rgw : RGW server is not coming up: Initialization timeout, failed to initializehttps://tracker.ceph.com/issues/51642?journal_id=2281992022-11-14T15:51:32ZChristian Rohmann
<ul></ul><p>I just ran into this issue and would like to propose reopening this.</p>
<p>radosgw should clearly log something when starting to set up rados pools <br />and then an the actual error if not successful.</p>