Project

General

Profile

Fix #10927

init script doesn't start daemon - errors silently

Added by Alfredo Deza over 3 years ago. Updated almost 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
Start date:
02/23/2015
Due date:
% Done:

0%

Source:
other
Tags:
Backport:
hammer
Reviewed:
Affected Versions:
ceph-qa-suite:

Description

Trying to start the daemon in a RHEL box:

[ubuntu@magna123 configs]$ sudo service radosgw start
Redirecting to /bin/systemctl start  radosgw.service
Job for ceph-radosgw.service failed. See 'systemctl status ceph-radosgw.service' and 'journalctl -xn' for details.
[ubuntu@magna123 configs]$ echo $?
1

systemctl has no idea why it failed.

[ubuntu@magna123 configs]$ sudo systemctl status ceph-radosgw.service
ceph-radosgw.service - LSB: radosgw RESTful rados gateway
   Loaded: loaded (/etc/rc.d/init.d/ceph-radosgw)
   Active: failed (Result: exit-code) since Wed 2015-02-18 12:57:19 PST; 12s ago
  Process: 16160 ExecStart=/etc/rc.d/init.d/ceph-radosgw start (code=exited, status=1/FAILURE)

Feb 18 12:57:18 magna123 ceph-radosgw[16160]: Starting radosgw instance(s)...
Feb 18 12:57:19 magna123 ceph-radosgw[16160]: /bin/radosgw is not running.
Feb 18 12:57:19 magna123 systemd[1]: ceph-radosgw.service: control process exited, code=exited status=1
Feb 18 12:57:19 magna123 systemd[1]: Failed to start LSB: radosgw RESTful rados gateway.
Feb 18 12:57:19 magna123 systemd[1]: Unit ceph-radosgw.service entered failed state.

After enabling the log file location in ceph.conf it fails to write anything to it:

[ubuntu@magna123 configs]$ tail /var/log/radosgw/client.radosgw.magna.log
[ubuntu@magna123 configs]$

Adding a `-x` to the init script yielded the following in `/var/log/messages`:

Feb 20 13:42:25 magna123 ceph-radosgw: ++ ceph-conf -n client.radosgw.magna host
Feb 20 13:42:25 magna123 ceph-radosgw: + host=magna123.ceph.redhat.com
Feb 20 13:42:25 magna123 ceph-radosgw: ++ hostname -s
Feb 20 13:42:25 magna123 ceph-radosgw: + hostname=magna123
Feb 20 13:42:25 magna123 ceph-radosgw: + '[' magna123.ceph.redhat.com '!=' magna123 ']'

A few things that should be noted/fixed for this problem:

  • The init script should write errors to the log file if configued otherwise to syslog (whatever location is the default for the distro)
  • A user should not need to slap `-x` to the script to understand what is going on.
  • Errors should be reported to systemctl (for systemctl distros) or find any other means to report back to the user

Related issues

Related to rgw - Fix #13708: init script doesn't start daemon - errors silently (part 2) New 02/23/2015
Copied to rgw - Backport #13225: init script doesn't start daemon - errors silently Resolved 02/23/2015

Associated revisions

Revision c8354229 (diff)
Added by Vikhyat Umrao about 3 years ago

rgw: fix radosgw start-up script.

radosgw init script is unable to start radosgw daemon.
as it is relies on requiretty being disabled.
once init script start this daemon with sudo it fails
to start the daemon.

changing 'sudo' to 'su', it will fix this issue and
will also help running radosgw daemon with our new
"ceph" UID project.

Fixes: #10927

Signed-off-by: Vikhyat Umrao <>

Revision f51ab26d (diff)
Added by Vikhyat Umrao almost 3 years ago

rgw: fix radosgw start-up script.

radosgw init script is unable to start radosgw daemon.
as it is relies on requiretty being disabled.
once init script start this daemon with sudo it fails
to start the daemon.

changing 'sudo' to 'su', it will fix this issue and
will also help running radosgw daemon with our new
"ceph" UID project.

Fixes: #10927

Signed-off-by: Vikhyat Umrao <>
(cherry picked from commit c83542296aeb8f5dd96f5d9e09327e7a89370742)

History

#1 Updated by Vikhyat Umrao about 3 years ago

  • Status changed from New to In Progress
  • Assignee set to Vikhyat Umrao

#3 Updated by Vikhyat Umrao about 3 years ago

  • Status changed from In Progress to Resolved

Merged!

#4 Updated by Alfredo Deza about 3 years ago

  • Status changed from Resolved to New

I don't see how that PR addresses the problems described in this ticket.

The problem in this ticket was that there was an error in the configuration that required the user to add a `-x` to the script,
there was no output, no information in the logs, and systemctl didn't know why it failed.

I am re-opening.

#5 Updated by Ken Dreyer about 3 years ago

Yeah, the big problem that Vikhyat and I wanted to solve was the elimination of "sudo" from the init script, and https://github.com/ceph/ceph/pull/5325 handles this... but it doesn't fix all the issues Vikhyat has enumerated above.

(I've always wondered why we have to execute radosgw through $SYSTEMD_RUN, instead of simply running it directly in the script. it's yet more indirection for a script that's already really complicated.)

#6 Updated by Ken Dreyer almost 3 years ago

  • Status changed from New to Pending Backport
  • Backport set to hammer

This affects hammer and we should get the patch (https://github.com/ceph/ceph/pull/5325) into a future hammer release.

#7 Updated by Brad Hubbard almost 3 years ago

Increase verbosity via /etc/systemd/system.conf LogLevel=debug and you should see why the script failed. That's not really the scripts fault i don't think?

#8 Updated by Alfredo Deza almost 3 years ago

Raising the log level of SystemD as a whole doesn't seem the right approach here. The init script should be able to report on its failures with simple checks. As described in the ticket, this particular issue was the expectation of a host short name that wasn't met.

#9 Updated by Loic Dachary almost 3 years ago

  • Status changed from Pending Backport to Resolved

#10 Updated by Ken Dreyer almost 3 years ago

  • Status changed from Resolved to New
  • Release set to hammer

Given the discussion in the PR on GitHub, some of the issues Alfredo described in the initial ticket still remain with the RGW init script. re-opening

#11 Updated by Loic Dachary almost 3 years ago

  • Copied to Fix #13708: init script doesn't start daemon - errors silently (part 2) added

#12 Updated by Loic Dachary almost 3 years ago

  • Copied to deleted (Fix #13708: init script doesn't start daemon - errors silently (part 2))

#13 Updated by Loic Dachary almost 3 years ago

  • Related to Fix #13708: init script doesn't start daemon - errors silently (part 2) added

#14 Updated by Loic Dachary almost 3 years ago

  • Status changed from New to Resolved

Opening http://tracker.ceph.com/issues/13708 instead of re-opening this ticket, for the purpose of tracking the associated backport and resolution separately. Having a single ticket is going to be confusing since it will appear that the problem was resolved twice in different ways. A link between this ticket and http://tracker.ceph.com/issues/13708 was set.

Also available in: Atom PDF