Bug #16800: SELinux denials found - Ceph - Ceph

Custom queries

Backports: mimic
Backports: missing release
Backports: nautilus
Bluestore
Bug queue
Bug queue - no subprojects
Bug triage
Ceph backlog
Crash queue
Crash triage
Feature Requests
Feedback
My issues
Need Review
Pending backports
Priority queue
Product Backlog Scrub
Project Triage
Test Failures

Actions

Copy link

Bug #16800

closed

SELinux denials found

Added by Kefu Chai almost 8 years ago. Updated over 7 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Kefu Chai

Category:

Target version:

% Done:

Source:

Development

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

reformatted output:

SELinux denials found on ubuntu@mira101.front.sepia.ceph.com: ['type=AVC msg=audit(1469366530.087:50415):

avc: denied { read } for pid=13501 comm="ceph-osd" name="magic" dev="sdf" ino=26 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1469366530.575:50428):

avc: denied { open } for pid=13538 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-1/magic" dev="sdf" ino=26 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1469366530.575:50428):

avc: denied { read } for pid=13538 comm="ceph-osd" name="magic" dev="sdf" ino=26 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1469366530.087:50415): avc: denied { open } for pid=13501 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-1/magic" dev="sdf" ino=26 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1']

see

http://pulpito.ceph.com/kchai-2016-07-24_02:17:13-rados-wip-kefu-testing---basic-mira/331090/
http://pulpito.ceph.com/kchai-2016-07-24_02:17:13-rados-wip-kefu-testing---basic-mira/331311/

History
Notes
Property changes

Actions

Copy link

Updated by Kefu Chai almost 8 years ago

Assignee set to Boris Ranto

Hi Boris, could you help to shed some lights on this issue?

all of them has "unlabeled_t" domain in tcontext.

Boris, shall/do we set a label on the mounted osd data dir, in this case it's /var/lib/ceph/osd/ceph-1? i found we do call "restorecon" when preparing a disk in ceph-disk. so it's a bug in ceph-disk or an environmental issue?

Actions

Copy link

Updated by Kefu Chai almost 8 years ago

Description updated (diff)

Actions

Copy link

Updated by Boris Ranto almost 8 years ago

Hi Kefu,

we only need to call restorecon on the root of a mount point, the rest of the mount point will then inherit the proper context according to the rules that we defined. That is why we call it in the ceph-disk when we are preparing the device.

These all point to a single file -- /var/lib/ceph/<osd>/magic. What does it contain? Is there anything special about the way it was created? Was it by any chance created outside the /var/lib/ceph directory and then moved? (mv/rename does not change a label)

If all of /var/lib/ceph/ was mislabelled we would definitely hit much more denials.

Actions

Copy link

Updated by Kefu Chai almost 8 years ago

Assignee changed from Boris Ranto to Loïc Dachary

What does it contain?

a magic string, for example, 'ceph osd volume v026', noting that the osd already exists. it is written when preparing the osd data dir.

Is there anything special about the way it was created?

it is created by ceph-disk. after it is created, we call "restorecon -R"

Was it by any chance created outside the /var/lib/ceph directory and then moved? (mv/rename does not change a label)

i have no idea, but if the osd is prepared by ceph-disk. the answer is no.

Loïc, is there anything standing up to you?

Actions

Copy link

Updated by Loïc Dachary over 7 years ago

Assignee deleted (~~Loïc Dachary~~)

I don't have any answers right now :-(

Actions

Copy link

Updated by Kefu Chai over 7 years ago

SELinux denials found on ubuntu@mira061.front.sepia.ceph.com: ['type=AVC msg=audit(1481273299.192:4526):

avc: denied { read } for pid=25177 comm="ceph-osd" name="type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273299.192:4527):

avc: denied { read } for pid=25174 comm="ceph-osd" name="type" dev="sdf" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273420.712:5855):

avc: denied { read } for pid=32809 comm="ceph-osd" name="type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273299.192:4527):

avc: denied { open } for pid=25174 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-1/type" dev="sdf" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273420.712:5854):

avc: denied { read } for pid=32812 comm="ceph-osd" name="type" dev="sdf" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273237.957:3993):

avc: denied { open } for pid=21766 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-0/type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273704.447:10186):

avc: denied { read } for pid=59348 comm="ceph-osd" name="type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273562.696:7871):

avc: denied { open } for pid=44979 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-1/type" dev="sdf" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273704.447:10186):

avc: denied { open } for pid=59348 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-0/type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273562.696:7871):

avc: denied { read } for pid=44979 comm="ceph-osd" name="type" dev="sdf" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273237.957:3992):

avc: denied { read } for pid=21768 comm="ceph-osd" name="type" dev="sdf" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273299.192:4526):

avc: denied { open } for pid=25177 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-0/type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273744.927:10826):

avc: denied { open } for pid=63134 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-0/type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273420.712:5854):

avc: denied { open } for pid=32812 comm="ceph-osd" path="/var/lib/ceph/osd/ceph-1/type" dev="sdf" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273744.927:10826):

avc: denied { read } for pid=63134 comm="ceph-osd" name="type" dev="sdh" ino=19 scontext=system_u:system_r:ceph_t:s0 tcontext=unconfined_u:object_r:unlabeled_t:s0 tclass=file permissive=1', 'type=AVC msg=audit(1481273238.047:3994): avc: denied { read } for pid=21766 comm="ceph-osd" name="/" dev="sdh" ino=16 scontext=system_u:system_r:ceph_t:s0 tcontext=system_u:object_r:unlabeled_t:s0 tclass=dir permissive=1']

see http://pulpito.ceph.com/kchai-2016-12-09_06:41:31-rados-wip-kefu-testing---basic-mira/

Actions

Copy link

Updated by Kefu Chai over 7 years ago

Status changed from New to 12
Assignee set to Kefu Chai

Actions

Copy link

Updated by Kefu Chai over 7 years ago

Status changed from 12 to Fix Under Review

~~https://github.com/ceph/ceph-qa-suite/pull/1303~~

~~an alternative fix at https://github.com/ceph/ceph/pull/12430~~

Actions

Copy link

Updated by Kefu Chai over 7 years ago

in http://pulpito.ceph.com/kchai-2016-12-13_16:06:03-rados-master---basic-mira/630560/

2016-12-13T16:23:08.203 INFO:teuthology.orchestra.run.mira095:Running: 'sudo mkdir -p /var/lib/ceph/osd/ceph-0'
...
2016-12-13T16:23:08.376 INFO:teuthology.orchestra.run.mira095:Running: 'yes | sudo mkfs.btrfs -m single -l 32768 -n 32768 /dev/sdh'
...
2016-12-13T16:23:09.407 INFO:teuthology.orchestra.run.mira095:Running: 'sudo mount -t btrfs -o noatime,user_subvol_rm_allowed /dev/sdh /var/lib/ceph/osd/ceph-0'
2016-12-13T16:23:11.393 INFO:teuthology.orchestra.run.mira095:Running: 'sudo ls -Z /var/lib/ceph/osd/ceph-0'
// no thing returned
2016-12-13T16:23:11.596 INFO:teuthology.orchestra.run.mira095:Running: 'sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap'
...
2016-12-13T16:23:13.446 INFO:teuthology.orchestra.run.mira095.stderr:2016-12-13 16:23:13.444052 7f7ce773a940 -1 created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 1350d6c5-626c-4a62-956e-ae9d1c9f604d
2016-12-13T16:23:13.446 INFO:teuthology.orchestra.run.mira095.stderr:2016-12-13 16:23:13.444116 7f7ce773a940 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory
2016-12-13T16:23:13.446 INFO:teuthology.orchestra.run.mira095.stderr:2016-12-13 16:23:13.444388 7f7ce773a940 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring

2016-12-13T16:23:13.456 INFO:teuthology.orchestra.run.mira095:Running: 'sudo ls -Z /var/lib/ceph/osd/ceph-0'
2016-12-13T16:23:13.654 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 ceph_fsid
2016-12-13T16:23:13.654 INFO:teuthology.orchestra.run.mira095.stdout:drwxr-xr-x. root root system_u:object_r:unlabeled_t:s0 current
2016-12-13T16:23:13.654 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 fsid
2016-12-13T16:23:13.654 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 journal
2016-12-13T16:23:13.654 INFO:teuthology.orchestra.run.mira095.stdout:-rw-------. root root unconfined_u:object_r:unlabeled_t:s0 keyring
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 magic
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 ready
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:drwxr-xr-x. root root system_u:object_r:unlabeled_t:s0 snap_1
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:drwxr-xr-x. root root system_u:object_r:unlabeled_t:s0 snap_2
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 store_version
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 superblock
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 type
2016-12-13T16:23:13.655 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:unlabeled_t:s0 whoami
2016-12-13T16:23:13.658 INFO:teuthology.orchestra.run.mira095:Running: 'sudo /sbin/restorecon -R /var/lib/ceph/osd/ceph-0'
2016-12-13T16:23:13.880 INFO:teuthology.orchestra.run.mira095:Running: 'sudo ls -Z /var/lib/ceph/osd/ceph-0'
2016-12-13T16:23:14.067 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 ceph_fsid
2016-12-13T16:23:14.067 INFO:teuthology.orchestra.run.mira095.stdout:drwxr-xr-x. root root system_u:object_r:ceph_var_lib_t:s0 current
2016-12-13T16:23:14.067 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 fsid
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 journal
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-------. root root unconfined_u:object_r:ceph_var_lib_t:s0 keyring
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 magic
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 ready
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:drwxr-xr-x. root root system_u:object_r:ceph_var_lib_t:s0 snap_1
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:drwxr-xr-x. root root system_u:object_r:ceph_var_lib_t:s0 snap_2
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 store_version
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 superblock
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 type
2016-12-13T16:23:14.068 INFO:teuthology.orchestra.run.mira095.stdout:-rw-r--r--. root root unconfined_u:object_r:ceph_var_lib_t:s0 whoami

please note that the label of the SELinux context of all files under "/var/lib/ceph/osd/ceph-0" was changed from "unlabeled_t" to "ceph_var_lib_t" after running "sudo /sbin/restorecon -R /var/lib/ceph/osd/ceph-0". and apparently, these files were created by "ceph-osd --mkfs".

Actions

Copy link

#10