Project

General

Profile

Bug #12972

SELinux denial: syslogd on sda1?

Added by Sage Weil over 8 years ago. Updated over 7 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

type=SYSCALL msg=audit(1441543381.487:2345): arch=c000003e syscall=54 success=yes exit=0 a0=4 a1=29 a2=40 a3=203e720 items=0 ppid=757 pid=10248 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="ip6tables" exe="/usr/sbin/xtables-multi" subj=system_u:system_r:ipta
bles_t:s0 key=(null)
type=PROCTITLE msg=audit(1441543381.487:2345): proctitle=2F7362696E2F6970367461626C6573002D740066696C746572002D5000464F525741524400414343455054
type=AVC msg=audit(1441543382.391:2346): avc:  denied  { search } for  pid=6044 comm=72733A6D61696E20513A526567 name="cephtest" dev="sda1" ino=21758084 scontext=system_u:system_r:syslogd_t:s0 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=dir permissive=1
type=SYSCALL msg=audit(1441543382.391:2346): arch=c000003e syscall=21 success=yes exit=0 a0=56351ca63820 a1=0 a2=56351ca64c00 a3=0 items=0 ppid=1 pid=6044 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm=72733A6D61696E20513A526567 exe="/usr/sbin/rsyslogd" subj=system_u:system_r:syslogd_t:s0 key=(null)
type=PROCTITLE msg=audit(1441543382.391:2346): proctitle=2F7573722F7362696E2F727379736C6F6764002D6E
type=SERVICE_STOP msg=audit(1441543382.756:2347): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=' comm="firewalld" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'

/a/sage-2015-09-06_05:36:07-rados-wip-sage-testing---basic-multi/1044674

History

#1 Updated by Loïc Dachary over 8 years ago

  • Description updated (diff)

#2 Updated by Loïc Dachary over 8 years ago

$ git --no-pager log --stat --oneline --graph ceph/infernalis..9b9c55a43323085f82e1a535261a848b811fde99
*   9b9c55a Merge remote-tracking branch 'gh/wip-post-file' into wip-sage-testing
|\  
| * 06824d7 debian/rules: fix /usr/share/ceph drop.ceph.com key mode
| |  debian/rules | 6 ++++++
| |  1 file changed, 6 insertions(+)
| * c111858 qa/workunits/post-file.sh: sudo
|    qa/workunits/post-file.sh | 2 +-
|    1 file changed, 1 insertion(+), 1 deletion(-)
*   210a4b7 Merge remote-tracking branch 'me/wip-scrub-stats' into wip-sage-testing
|\  
| * 75d9f58 osd/ReplicatedPG: use apply_ctx_stats() everywhere
| |  src/osd/ReplicatedPG.cc | 14 ++++++--------
| |  1 file changed, 6 insertions(+), 8 deletions(-)
| * eb2993a osd/ReplicatedPG: create apply_ctx_stats() helper
| |  src/osd/ReplicatedPG.cc | 6 ++++++
| |  src/osd/ReplicatedPG.h  | 2 ++
| |  2 files changed, 8 insertions(+)
| * 9bf103c osd/ReplicatedPG: snaptimmer: adjust stats through ctx->delta_stats
|    src/osd/ReplicatedPG.cc | 20 ++++++++++----------
|    1 file changed, 10 insertions(+), 10 deletions(-)
* a818f9e Revert "osd/ReplicatedPG: snaptimmer: adjust stats through ctx->delta_stats" 
|  src/osd/ReplicatedPG.cc | 18 ++++++++++--------
|  1 file changed, 10 insertions(+), 8 deletions(-)
* ebbb7d1 Merge branch 'wip-12908' of git://github.com/yuyuyu101/ceph into wip-sage-testing
* bda06f9 Merge branch 'wip-outdata-set' of git://github.com/XinzeChi/ceph into wip-sage-testing
*   56daad8 Merge remote-tracking branch 'gh/wip-12551' into wip-sage-testing
|\  
| * 4b2f8aa ReplicatedPG,Objecter: copy_get should include truncate_seq and size
|    src/osd/ReplicatedPG.cc |  6 ++++++
|    src/osd/ReplicatedPG.h  |  5 ++++-
|    src/osd/osd_types.cc    | 10 ++++++++--
|    src/osd/osd_types.h     | 10 ++++++++--
|    src/osdc/Objecter.h     | 18 ++++++++++++++++--
|    5 files changed, 42 insertions(+), 7 deletions(-)
* 3da8a14 Merge branch 'wip-12809' of git://github.com/athanatos/ceph into wip-sage-testing
* 989aa62 Merge branch 'wip-scrub-stats' of https://github.com/liewegas/ceph into wip-sage-testing
* c35e186 osd/ReplicatedPG: snaptimmer: adjust stats through ctx->delta_stats
   src/osd/ReplicatedPG.cc | 18 ++++++++----------
   1 file changed, 8 insertions(+), 10 deletions(-)

#3 Updated by Sage Weil over 8 years ago

/a/sage-2015-09-07_18:07:23-rados-wip-sage-testing---basic-multi/1046316

#4 Updated by Boris Ranto over 8 years ago

what is cephtest? Is it by any chance a script in a directory like /var/lib/<stg> or is it being run by syslogd?

Either way, the executable is obviously mislabelled (syslogd_t). Fixing that shall hopefully fix the issue. I'd need more info (ideally an access to a machine where it is happening) to be able to tell more.

#5 Updated by Sage Weil over 8 years ago

<sage> branto: oh.. teuthology runs syslogd with a custom conf /etc/rsyslog.d/80-cephtest.conf that logs to /home/ubuntu/cephtest/archive/... could that be it?
<sage> i don't think cephtest is ever an executable.. just a directory.
<-> jbautista is now known as jbautista|brb
<branto> sage: that is almost definitely it
<sage> ok.  hmm... is there a simple way to whitelist it for the teuthology runs?  we capture all syslog but running our own instance that logs to the archive dir...
<sage> *by
<branto> well, i suppose the issue here is that the logs are stored in the home dir
<branto> hence, the log files are improperly labelled
<branto> as you can see from the avc, the target context is user_home_t which means a file in some user's home directory
<sage> ah, yep, makes sense.
<branto> and the default policy for syslogd does not like it very much
<-- dang (~dang@10.10.51.107) has quit (Ping timeout: 180 seconds)
<branto> you could either manually relabel the log files (I'd not recommend this, though -- you'd loose the context with each global system relabelling) or log under the (mostly) default location
<-> zz_avozza is now known as avozza
<branto> and then copy to the home dir
<branto> or link right away
<-- vikhyat (~vumrao@vpn1-4-204.sin2.redhat.com) has quit (Quit: Leaving)
<-> jbautista|brb is now known as jbautista
<branto> sage: another option is to use global custom labelling rules (i.e. the first option), that might also be the simplest option
--> shylesh (~shylesh@vpn1-4-127.sin2.redhat.com) has joined #rh-ceph
<branto> you'd essentially just need to run some semanage command once per system install
<-> rmc|brb is now known as rmc
<gregsfortytwo> we've got an issue from ktdreyer in the tracker where our custom logrotate file (for teuthology testing) is breaking selinux as well
<gregsfortytwo> I'm not sure if there's something general we can do to adapt the policies for our test environment or if it being a pain is just a consequence of label-based security
<branto> sage: something like 'semanage fcontext -a -t syslogd_var_lib_t /path/to/log_file' should hopefully do the trick
<branto> gregsfortytwo: the same goes for the logrotate file
<sage> could we do that to the target directory before we start and the file screated there would be labeled accordingly?
<branto> sage: yep, it should work that way, although, it might need some slightly different syntax for the command
<branto> sage, gregsfortytwo: that command creates permanent custom policy for the file contexts, you need to run it once per installation
<branto> it also automatically applies to any newly created files (it is a permanent policy change)
<branto> the files that existed previously can be relabelled with restorecon
--> Tamil (~Adium@ovpn-113-66.phx2.redhat.com) has joined #rh-ceph
<sage> the whole dir will get removed at the end of the test anyway
<-> icolle is now known as icolle-brb
<branto> sage: one more thing, if you want to add a (recursive) rule for whole dir, you can use stg like: semanage fcontext -a -t syslogd_var_lib_t "/home/ubuntu/logdir(/.*)?" 

#6 Updated by Boris Ranto over 8 years ago

nvm, pointless comment, edited.

#7 Updated by Zack Cerza over 8 years ago

We currently do this:
https://github.com/ceph/teuthology/blob/master/teuthology/task/internal.py#L626-L629

Do we really need to set the same context on all of /home/ubuntu/cephtest ?

#8 Updated by Boris Ranto over 8 years ago

Well, for starters, the cephtest dir itself is not labelled properly. Hence, the log file that is created there is mislabelled. You need to label properly any file that is used by any daemon with custom SELinux policy. (be it ceph, syslogd or logrotate)

I'd really recommend using the semanage command, here. It was created exactly for these custom path scenarios. Additionally, it is persistent across reboots, etc. (the chcon does not need to persist across reboots, etc -- it is just a temporary rule, not a persistent one)

It is also simple to use, you just run it before you start running any tests/creating any files and it will sort all these things on its own/in the background. You need to know the right contexts for the files to create the correct persistent rules, though.

#9 Updated by Zack Cerza over 8 years ago

If all of /home/ubuntu/cephtest carries that context, won't we hit denials for different reasons?

Also, semanage doesn't exist on RHEL6 and presumably several other OSes. What's the solution there?

Edit: semanage is part of policycoreutils-python. First question still stands though.

#10 Updated by Zack Cerza over 8 years ago

The example in the IRC paste doesn't seem to work.

# semanage fcontext --add --type syslogd_var_lib_t ./setest
# ls -lZ ./setest
-rw-r--r--. root root unconfined_u:object_r:admin_home_t:s0 ./setest
# restorecon -v ./setest
# ls -lZ ./setest
-rw-r--r--. root root unconfined_u:object_r:admin_home_t:s0 ./setest

#11 Updated by Milan Broz over 8 years ago

Probably missing directory flag, see this example (/home/ceph_log):

  1. ls -lZ
    drwxr-xr-x. 2 root root unconfined_u:object_r:home_root_t:s0 4096 Sep 9 02:27 ceph_log
(note var_log_t context a -f d flag - applies to directory, default is all files)
  1. semanage fcontext --add --type var_log_t -f d '/home/ceph_log'
(check that it applies, but not changes yet)
  1. matchpathcon ceph_log
    ceph_log system_u:object_r:var_log_t:s0
(and fix it...)
  1. restorecon -v ceph_log
    restorecon reset /home/ceph_log context unconfined_u:object_r:home_root_t:s0->unconfined_u:object_r:var_log_t:s0

[root@ceph-node1 home]# ls -lZ
drwxr-xr-x. 2 root root unconfined_u:object_r:var_log_t:s0 4096 Sep 9 02:27 ceph_log

It could be possible that you have to change context of existing files as well.

#12 Updated by Milan Broz over 8 years ago

Sigh, this is really stupid tracker... here with fixed syntax, seems I cannot edit my comments.

Probably missing directory flag, see this example (/home/ceph_log):

# mkdir ceph_log
# ls -lZ
drwxr-xr-x. 2 root root unconfined_u:object_r:home_root_t:s0 4096 Sep  9 02:27 ceph_log

Add type to local list and check that context will apply (no changes yet):

# semanage fcontext --add --type var_log_t -f d '/home/ceph_log'
# matchpathcon ceph_log
ceph_log        system_u:object_r:var_log_t:s0

And fix it

# restorecon -v ceph_log
restorecon reset /home/ceph_log context unconfined_u:object_r:home_root_t:s0->unconfined_u:object_r:var_log_t:s0
# ls -lZ
drwxr-xr-x. 2 root root unconfined_u:object_r:var_log_t:s0 4096 Sep  9 02:27 ceph_log

It could be possible that you have to change context of existing files as well. Just try it.

#13 Updated by Boris Ranto over 8 years ago

@zack: The commands that I posted to the IRC create persistent rules, i.e. any newly created files will inherit those contexts after you apply the rules. They do not change the existing contexts of files. You need to run restorecon to do that.

Also, please use full paths. These commands are not designed for relative paths. With these commands you are creating a custom SELinux policy rules and they need full path spec for it to work properly.

Look at it this way: these rules are being stored in kernel, how is kernel supposed to know relative to what directory is the file supposed to be labelled that way?

I.e. the rule says: from now on, any file/directory that matches this (full) file spec shall be labelled this way.

e.g.:

[root@intel-s3e37-01 ~]# semanage fcontext --add --type syslogd_var_lib_t /root/setest2
[root@intel-s3e37-01 ~]# touch setest2
[root@intel-s3e37-01 ~]# restorecon setest2
[root@intel-s3e37-01 ~]# ll -Z setest2
-rw-r--r--. root root unconfined_u:object_r:syslogd_var_lib_t:s0 setest2
[root@intel-s3e37-01 ~]# 

PS: Unfortunately, it looks like you need to run the restorecon command to get the proper context for the files. You can run it recursively (which I recommend), though.

Also, I'm not saying you should label all of /home/ubuntu/cephtest with that single label, you should look for the set of rules that make sense based on the executables that write there/access those files.

#14 Updated by Zack Cerza over 8 years ago

Okay, I think I have this working the way we want:

2015-09-11 16:26:58,251.251 INFO:teuthology.orchestra.run.magna066:Running: 'mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog'
2015-09-11 16:26:58,294.294 INFO:teuthology.orchestra.run.magna066:Running: 'touch /home/ubuntu/cephtest/archive/syslog/kern.log'
2015-09-11 16:26:58,402.402 INFO:teuthology.orchestra.run.magna066:Running: 'touch /home/ubuntu/cephtest/archive/syslog/misc.log'
2015-09-11 16:26:58,510.510 INFO:teuthology.orchestra.run.magna066:Running: "sudo semanage fcontext --add --type var_log_t '/home/ubuntu/cephtest/archive/syslog/.*\\.log'" 
2015-09-11 16:27:00,294.294 INFO:teuthology.orchestra.run.magna066:Running: 'sudo restorecon -vR /home/ubuntu/cephtest/archive/syslog'
2015-09-11 16:27:00,360.360 INFO:teuthology.orchestra.run.magna066.stdout:restorecon reset /home/ubuntu/cephtest/archive/syslog/misc.log context unconfined_u:object_r:user_home_t:s0
->unconfined_u:object_r:var_log_t:s0
2015-09-11 16:27:00,360.360 INFO:teuthology.orchestra.run.magna066.stdout:restorecon reset /home/ubuntu/cephtest/archive/syslog/kern.log context unconfined_u:object_r:user_home_t:s0
->unconfined_u:object_r:var_log_t:s0

Does this look right?

#15 Updated by Zack Cerza over 8 years ago

  • Project changed from Ceph to teuthology
  • Assignee set to Zack Cerza

#16 Updated by Boris Ranto over 8 years ago

That will probably not be enough. The denial mentioned in this bz states that SELinux forbids 'search' on the cephtest directory itself so you need to change the context of that directory as well. Also, I'm not sure the var_log_t context is enough, here.

Anyway, apart from that, it lgtm.

#17 Updated by Vasu Kulkarni about 8 years ago

Where is the PR changes related to this? I dont see in the tracker

#18 Updated by Zack Cerza over 7 years ago

  • Assignee deleted (Zack Cerza)
  • Priority changed from Urgent to Normal

Also available in: Atom PDF