Bug #44242
shaman container build fails
0%
Description
The failure looks like so:
Running scriptlet: trousers-0.3.14-4.el8.x86_64 100/238 groupadd: /etc/group.373: lock file already used groupadd: cannot lock /etc/group; try again later. useradd: group 'tss' does not exist Installing : trousers-0.3.14-4.el8.x86_64 100/238 Running scriptlet: trousers-0.3.14-4.el8.x86_64 100/238 warning: user tss does not exist - using root warning: group tss does not exist - using root warning: user tss does not exist - using root warning: group tss does not exist - using root ... Running scriptlet: unbound-libs-1.7.3-8.el8.x86_64 124/238 groupadd: /etc/group.387: lock file already used groupadd: cannot lock /etc/group; try again later. useradd: group 'unbound' does not exist error: %prein(unbound-libs-1.7.3-8.el8.x86_64) scriptlet failed, exit status 6 Error in PREIN scriptlet in rpm package unbound-libs Installing : python3-unbound-1.7.3-8.el8.x86_64 125/238 error: unbound-libs-1.7.3-8.el8.x86_64: install failed ... Error: Transaction failed Error: error building at STEP "RUN yum install -y epel-release && yum install -y jq && bash -c ' if [ -n "nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls" ]; then echo "[ganesha]" > /etc/yum.repos.d/ganesha.repo ; echo "name=ganesha" >> /etc/yum.repos.d/ganesha.repo ; if [[ "${CEPH_VERSION}" =~ master|^wip* ]]; then echo "baseurl=https://download.nfs-ganesha.org/3/LATEST/CentOS/el-\$releasever/\$basearch/" >> /etc/yum.repos.d/ganesha.repo ; echo "gpgcheck=0" >> /etc/yum.repos.d/ganesha.repo ; echo "enabled=1" >> /etc/yum.repos.d/ganesha.repo ; echo "[ganesha-noarch]" >> /etc/yum.repos.d/ganesha.repo ; echo "name=ganesha-noarch" >> /etc/yum.repos.d/ganesha.repo ; echo "baseurl=https://download.nfs-ganesha.org/3/LATEST/CentOS/el-\$releasever/noarch" >> /etc/yum.repos.d/ganesha.repo ; elif [[ "${CEPH_VERSION}" == nautilus ]]; then echo "baseurl=http://download.ceph.com/nfs-ganesha/rpm-V2.8-stable/$CEPH_VERSION/\$basearch/" >> /etc/yum.repos.d/ganesha.repo ; else echo "baseurl=http://download.ceph.com/nfs-ganesha/rpm-V2.7-stable/$CEPH_VERSION/\$basearch/" >> /etc/yum.repos.d/ganesha.repo ; fi ; echo "gpgcheck=0" >> /etc/yum.repos.d/ganesha.repo ; echo "enabled=1" >> /etc/yum.repos.d/ganesha.repo ; fi ; if [ -n "tcmu-runner ceph-iscsi python3-rtslib" ]; then curl -s -L https://shaman.ceph.com/api/repos/tcmu-runner/master/latest/centos/8/repo > /etc/yum.repos.d/tcmu-runner.repo ; if [[ "${CEPH_VERSION}" =~ master|^wip* ]]; then curl -s -L https://shaman.ceph.com/api/repos/ceph-iscsi/master/latest/centos/8/repo > /etc/yum.repos.d/ceph-iscsi.repo ; elif [[ "${CEPH_VERSION}" == nautilus ]]; then curl -s -L https://download.ceph.com/ceph-iscsi/3/rpm/el8/ceph-iscsi.repo -o /etc/yum.repos.d/ceph-iscsi.repo ; else curl -s -L https://download.ceph.com/ceph-iscsi/2/rpm/el8/ceph-iscsi.repo -o /etc/yum.repos.d/ceph-iscsi.repo ; fi ; fi' && yum update -y && rpm --import 'https://download.ceph.com/keys/release.asc' && if [[ "${CEPH_VERSION}" == nautilus ]]; then CEPH_MGR_K8SEVENTS="ceph-mgr-k8sevents"; if [[ -n "" ]]; then CPR= ; if [[ ${CPR:1:2} -eq 14 ]] && [[ ${CPR:4:1} -eq 2 ]] && [[ ${CPR:6} -lt 5 ]]; then CEPH_MGR_K8SEVENTS="" ; fi ; fi ; fi && bash -c ' if [[ 8 -eq 8 ]]; then yum install -y dnf-plugins-core ; yum copr enable -y ktdreyer/ceph-el8 ; echo "[lab-extras]" > /etc/yum.repos.d/lab-extras.repo ; echo "name=labextras" >> /etc/yum.repos.d/lab-extras.repo ; echo "baseurl=http://apt-mirror.front.sepia.ceph.com/lab-extras/8/" >> /etc/yum.repos.d/lab-extras.repo ; echo "enabled=1" >> /etc/yum.repos.d/lab-extras.repo ; echo "gpgcheck=0" >> /etc/yum.repos.d/lab-extras.repo ; fi && if [[ "${CEPH_VERSION}" =~ master|^wip* ]] || ${CEPH_DEVEL}; then REPO_URL=$(curl -s "https://shaman.ceph.com/api/search/?project=ceph&distros=centos/8&flavor=default&ref=${CEPH_VERSION}&sha1=latest" | jq -a ".[0] | .url"); RELEASE_VER=0 ; else RELEASE_VER=1 ; REPO_URL="http://download.ceph.com/rpm-${CEPH_VERSION}/el8/"; fi && rpm -Uvh "$REPO_URL/noarch/ceph-release-1-${RELEASE_VER}.el8.noarch.rpm" ' && yum install -y ca-certificates e2fsprogs ceph-common ceph-mon ceph-osd ceph-mds rbd-mirror ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-rook ceph-grafana-dashboards kmod lvm2 gdisk smartmontools nvme-cli libstoragemgmt ceph-radosgw libradosstriper1 nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls tcmu-runner ceph-iscsi python3-rtslib attr ceph-fuse rbd-nbd && INITIAL_SIZE="$(bash -c 'sz="$(du -sm --exclude=/proc /)" ; echo "${sz%*/}"')" && echo 'Postinstall cleanup' && (rm -rf "/usr/bin/hyperkube /usr/bin/etcd /usr/bin/systemd-analyze /usr/share/hwdata/{iab.txt,oui.txt} /etc/profile.d/lang.sh" && yum clean all && rpm -q ca-certificates e2fsprogs ceph-common ceph-mon ceph-osd ceph-mds rbd-mirror ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-rook ceph-grafana-dashboards kmod lvm2 gdisk smartmontools nvme-cli libstoragemgmt ceph-radosgw libradosstriper1 nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls tcmu-runner ceph-iscsi python3-rtslib attr ceph-fuse rbd-nbd ) && sed -i -e 's/udev_rules = 1/udev_rules = 0/' -e 's/udev_sync = 1/udev_sync = 0/' -e 's/obtain_device_list_from_udev = 1/obtain_device_list_from_udev = 0/' /etc/lvm/lvm.conf && grep -sqo "udev_sync = 0" /etc/lvm/lvm.conf && grep -sqo "udev_rules = 0" /etc/lvm/lvm.conf && grep -sqo "obtain_device_list_from_udev = 0" /etc/lvm/lvm.conf && rm -rf /etc/{selinux,systemd,udev} /lib/{lsb,udev} /tmp/* /usr/lib{,64}/{locale,udev,dracut} /usr/share/{doc,info,locale,man} /usr/share/{bash-completion,pkgconfig/bash-completion.pc} /var/log/* /var/tmp/* && find / -xdev -name "*.pyc" -o -name "*.pyo" -exec rm -f {} \; && if [ -f /usr/bin/ceph-dencoder ]; then gzip -9 /usr/bin/ceph-dencoder; fi && bash -c ' function ifstrip () { if compgen -g "$1"; then strip -s "$1"; fi } && ifstrip /usr/lib{,64}/ceph/erasure-code/* && ifstrip /usr/lib{,64}/rados-classes/* && ifstrip /usr/lib{,64}/python*/{dist,site}-packages/{rados,rbd,rgw}.*.so && ifstrip /usr/bin/{crushtool,monmaptool,osdmaptool}' && rm -f /usr/lib/ceph/mgr/dashboard/static/AdminLTE-*/plugins/datatables/extensions/TableTools/images/psd/* && find /var/log/ -type f -exec truncate -s 0 {} \; && FINAL_SIZE="$(bash -c 'sz="$(du -sm --exclude=/proc /)" ; echo "${sz%*/}"')" && REMOVED_SIZE=$((INITIAL_SIZE - FINAL_SIZE)) && echo "Cleaning process removed ${REMOVED_SIZE}MB" && echo "Dropped container size from ${INITIAL_SIZE}MB to ${FINAL_SIZE}MB" && rpm -q ca-certificates e2fsprogs ceph-common ceph-mon ceph-osd ceph-mds rbd-mirror ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-rook ceph-grafana-dashboards kmod lvm2 gdisk smartmontools nvme-cli libstoragemgmt ceph-radosgw libradosstriper1 nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls tcmu-runner ceph-iscsi python3-rtslib attr ceph-fuse rbd-nbd && echo 'Packages verified successfully'": error while running runtime: exit status 1
History
#1 Updated by Sage Weil about 4 years ago
Recent instances:
- wip-sage-testing-2020-02-21-0633 ... 51.91.144.35 ... https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/37145//consoleFull
- wip-swagner2-testing-2020-02-21-1038 .... 51.68.91.254 ... https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/37137//consoleFull
#2 Updated by Sage Weil about 4 years ago
wip-sage4-testing-2020-02-21-1319 .... 51.68.93.161 ... https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/37172//consoleFull
#3 Updated by Sage Weil about 4 years ago
wip-sage2-testing-2020-02-21-1459 51.68.84.74 https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/37178//consoleFull
#4 Updated by Sage Weil about 4 years ago
AAAHAHHHHHAHHHHHH
wip-sage-testing-2020-02-21-1528
51.68.95.182
https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/37182//consoleFull
#5 Updated by Sage Weil about 4 years ago
motherfucker
wip-sage2-testing-2020-02-21-1459 51.68.95.182 https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=huge/37182//consoleFull
#6 Updated by Kefu Chai about 4 years ago
#7 Updated by Sage Weil about 4 years ago
- Status changed from New to Resolved
#8 Updated by Sebastian Wagner about 4 years ago
I'm still getting this error:
based on master from today: https://github.com/ceph/ceph-ci/commits/wip-swagner2-testing-2020-02-26-1036
#9 Updated by Sebastian Wagner about 4 years ago
- Status changed from Resolved to New
- Priority changed from Immediate to High
reopened.
groupadd: /etc/group.388: lock file already used groupadd: cannot lock /etc/group; try again later. useradd: group 'unbound' does not exist error: %prein(unbound-libs-1.7.3-8.el8.x86_64) scriptlet failed, exit status 6 Error in PREIN scriptlet in rpm package unbound-libs Installing : python3-unbound-1.7.3-8.el8.x86_64 125/238 error: unbound-libs-1.7.3-8.el8.x86_64: install failed
#11 Updated by Sage Weil about 4 years ago
For now dgalloway has disabled el8 builds on ephemeral builds. That should make this problem go away for now, but it also means builds are very slow (not enough builders).
#12 Updated by Kefu Chai about 4 years ago
i just built the patched fuse-overlayfs packages and uploaded them to teuthology:~kchai
. once David is back, we can install it on the builder host where we use podman for building the images.
the source is 0.7.6 (https://github.com/containers/fuse-overlayfs/tree/v0.7.6) + https://github.com/containers/fuse-overlayfs/pull/184. the patch was proposed at https://bugzilla.redhat.com/show_bug.cgi?id=1802907#c5
the spec recipe comes from https://src.fedoraproject.org/rpms/fuse-overlayfs/blob/master/f/fuse-overlayfs.spec.
i think the built package should be equivalent to the one tested at https://bugzilla.redhat.com/show_bug.cgi?id=1802907#c13
#13 Updated by David Galloway about 4 years ago
I pushed the fuse-overlayfs RPM to http://apt-mirror.front.sepia.ceph.com/lab-extras/8/ and just re-enabled CentOS8 Ephemeral slaves. Please let me know if you see any more dnf installation failures.
#14 Updated by David Galloway about 4 years ago
- Status changed from New to In Progress
- Assignee set to David Galloway
#15 Updated by David Galloway about 4 years ago
David Galloway wrote:
I pushed the fuse-overlayfs RPM to http://apt-mirror.front.sepia.ceph.com/lab-extras/8/ and just re-enabled CentOS8 Ephemeral slaves. Please let me know if you see any more dnf installation failures.
The podman spec file must require >= a specific version so it won't pull in the latest version that's in lab-extras.
https://github.com/ceph/ceph-container/pull/1599 should take care of it.
#16 Updated by Sage Weil about 4 years ago
- Status changed from In Progress to Resolved
worked around this with https://github.com/ceph/ceph-container/pull/1599