Project

General

Profile

Bug #44242

shaman container build fails

Added by Sage Weil about 4 years ago. Updated about 4 years ago.

Status:
Resolved
Priority:
High
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The failure looks like so:

  Running scriptlet: trousers-0.3.14-4.el8.x86_64                       100/238 
groupadd: /etc/group.373: lock file already used
groupadd: cannot lock /etc/group; try again later.
useradd: group 'tss' does not exist

  Installing       : trousers-0.3.14-4.el8.x86_64                       100/238 
  Running scriptlet: trousers-0.3.14-4.el8.x86_64                       100/238 
warning: user tss does not exist - using root
warning: group tss does not exist - using root
warning: user tss does not exist - using root
warning: group tss does not exist - using root
...
  Running scriptlet: unbound-libs-1.7.3-8.el8.x86_64                    124/238 
groupadd: /etc/group.387: lock file already used
groupadd: cannot lock /etc/group; try again later.
useradd: group 'unbound' does not exist
error: %prein(unbound-libs-1.7.3-8.el8.x86_64) scriptlet failed, exit status 6

Error in PREIN scriptlet in rpm package unbound-libs
  Installing       : python3-unbound-1.7.3-8.el8.x86_64                 125/238 
error: unbound-libs-1.7.3-8.el8.x86_64: install failed
...
Error: Transaction failed
Error: error building at STEP "RUN yum install -y epel-release && yum install -y jq && bash -c '   if [ -n "nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls" ]; then     echo "[ganesha]" > /etc/yum.repos.d/ganesha.repo ;     echo "name=ganesha" >> /etc/yum.repos.d/ganesha.repo ;     if [[ "${CEPH_VERSION}" =~ master|^wip* ]]; then       echo "baseurl=https://download.nfs-ganesha.org/3/LATEST/CentOS/el-\$releasever/\$basearch/" >> /etc/yum.repos.d/ganesha.repo ;       echo "gpgcheck=0" >> /etc/yum.repos.d/ganesha.repo ;       echo "enabled=1" >> /etc/yum.repos.d/ganesha.repo  ;       echo "[ganesha-noarch]" >> /etc/yum.repos.d/ganesha.repo ;       echo "name=ganesha-noarch" >> /etc/yum.repos.d/ganesha.repo ;       echo "baseurl=https://download.nfs-ganesha.org/3/LATEST/CentOS/el-\$releasever/noarch" >> /etc/yum.repos.d/ganesha.repo ;     elif [[ "${CEPH_VERSION}" == nautilus ]]; then       echo "baseurl=http://download.ceph.com/nfs-ganesha/rpm-V2.8-stable/$CEPH_VERSION/\$basearch/" >> /etc/yum.repos.d/ganesha.repo ;     else       echo "baseurl=http://download.ceph.com/nfs-ganesha/rpm-V2.7-stable/$CEPH_VERSION/\$basearch/" >> /etc/yum.repos.d/ganesha.repo ;     fi ;     echo "gpgcheck=0" >> /etc/yum.repos.d/ganesha.repo ;     echo "enabled=1" >> /etc/yum.repos.d/ganesha.repo  ;   fi ;   if [ -n "tcmu-runner ceph-iscsi python3-rtslib" ]; then     curl -s -L https://shaman.ceph.com/api/repos/tcmu-runner/master/latest/centos/8/repo > /etc/yum.repos.d/tcmu-runner.repo ;     if [[ "${CEPH_VERSION}" =~ master|^wip* ]]; then       curl -s -L https://shaman.ceph.com/api/repos/ceph-iscsi/master/latest/centos/8/repo > /etc/yum.repos.d/ceph-iscsi.repo ;     elif [[ "${CEPH_VERSION}" == nautilus ]]; then       curl -s -L https://download.ceph.com/ceph-iscsi/3/rpm/el8/ceph-iscsi.repo -o /etc/yum.repos.d/ceph-iscsi.repo ;     else       curl -s -L https://download.ceph.com/ceph-iscsi/2/rpm/el8/ceph-iscsi.repo -o /etc/yum.repos.d/ceph-iscsi.repo ;     fi ;   fi' && yum update -y && rpm --import 'https://download.ceph.com/keys/release.asc' && if [[ "${CEPH_VERSION}" == nautilus ]]; then   CEPH_MGR_K8SEVENTS="ceph-mgr-k8sevents";   if [[ -n "" ]]; then     CPR= ;     if [[ ${CPR:1:2} -eq 14 ]] && [[ ${CPR:4:1} -eq 2 ]] && [[ ${CPR:6} -lt 5 ]]; then       CEPH_MGR_K8SEVENTS="" ;     fi ;   fi ; fi && bash -c '   if [[ 8 -eq 8 ]]; then     yum install -y dnf-plugins-core ;    yum copr enable -y ktdreyer/ceph-el8 ;    echo "[lab-extras]" > /etc/yum.repos.d/lab-extras.repo ;    echo "name=labextras" >> /etc/yum.repos.d/lab-extras.repo ;    echo "baseurl=http://apt-mirror.front.sepia.ceph.com/lab-extras/8/" >> /etc/yum.repos.d/lab-extras.repo ;    echo "enabled=1" >> /etc/yum.repos.d/lab-extras.repo ;    echo "gpgcheck=0" >> /etc/yum.repos.d/lab-extras.repo ;  fi &&   if [[ "${CEPH_VERSION}" =~ master|^wip* ]] || ${CEPH_DEVEL}; then     REPO_URL=$(curl -s "https://shaman.ceph.com/api/search/?project=ceph&distros=centos/8&flavor=default&ref=${CEPH_VERSION}&sha1=latest" | jq -a ".[0] | .url");     RELEASE_VER=0 ;  else     RELEASE_VER=1 ;    REPO_URL="http://download.ceph.com/rpm-${CEPH_VERSION}/el8/";   fi &&   rpm -Uvh "$REPO_URL/noarch/ceph-release-1-${RELEASE_VER}.el8.noarch.rpm" ' && yum install -y         ca-certificates         e2fsprogs         ceph-common          ceph-mon          ceph-osd         ceph-mds         rbd-mirror          ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-rook        ceph-grafana-dashboards         kmod         lvm2         gdisk     smartmontools     nvme-cli     libstoragemgmt         ceph-radosgw libradosstriper1         nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls         tcmu-runner ceph-iscsi python3-rtslib         attr ceph-fuse rbd-nbd &&     INITIAL_SIZE="$(bash -c 'sz="$(du -sm --exclude=/proc /)" ; echo "${sz%*/}"')" &&     echo 'Postinstall cleanup' &&  (rm -rf "/usr/bin/hyperkube /usr/bin/etcd /usr/bin/systemd-analyze /usr/share/hwdata/{iab.txt,oui.txt} /etc/profile.d/lang.sh" &&    yum clean all &&    rpm -q         ca-certificates         e2fsprogs         ceph-common          ceph-mon          ceph-osd         ceph-mds         rbd-mirror          ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-rook        ceph-grafana-dashboards         kmod         lvm2         gdisk     smartmontools     nvme-cli     libstoragemgmt         ceph-radosgw libradosstriper1         nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls         tcmu-runner ceph-iscsi python3-rtslib         attr ceph-fuse rbd-nbd ) && sed -i -e 's/udev_rules = 1/udev_rules = 0/' -e 's/udev_sync = 1/udev_sync = 0/' -e 's/obtain_device_list_from_udev = 1/obtain_device_list_from_udev = 0/' /etc/lvm/lvm.conf && grep -sqo "udev_sync = 0" /etc/lvm/lvm.conf && grep -sqo "udev_rules = 0" /etc/lvm/lvm.conf && grep -sqo "obtain_device_list_from_udev = 0" /etc/lvm/lvm.conf &&     rm -rf         /etc/{selinux,systemd,udev}         /lib/{lsb,udev}         /tmp/*         /usr/lib{,64}/{locale,udev,dracut}         /usr/share/{doc,info,locale,man}         /usr/share/{bash-completion,pkgconfig/bash-completion.pc}         /var/log/*         /var/tmp/* &&     find  / -xdev -name "*.pyc" -o -name "*.pyo" -exec rm -f {} \; &&     if [ -f /usr/bin/ceph-dencoder ]; then gzip -9 /usr/bin/ceph-dencoder; fi &&     bash -c '       function ifstrip () { if compgen -g "$1"; then strip -s "$1"; fi } &&       ifstrip /usr/lib{,64}/ceph/erasure-code/* &&       ifstrip /usr/lib{,64}/rados-classes/* &&       ifstrip /usr/lib{,64}/python*/{dist,site}-packages/{rados,rbd,rgw}.*.so &&       ifstrip /usr/bin/{crushtool,monmaptool,osdmaptool}' &&     rm -f /usr/lib/ceph/mgr/dashboard/static/AdminLTE-*/plugins/datatables/extensions/TableTools/images/psd/* &&     find /var/log/ -type f -exec truncate -s 0 {} \; &&     FINAL_SIZE="$(bash -c 'sz="$(du -sm --exclude=/proc /)" ; echo "${sz%*/}"')" &&     REMOVED_SIZE=$((INITIAL_SIZE - FINAL_SIZE)) &&     echo "Cleaning process removed ${REMOVED_SIZE}MB" &&     echo "Dropped container size from ${INITIAL_SIZE}MB to ${FINAL_SIZE}MB" &&     rpm -q         ca-certificates         e2fsprogs         ceph-common          ceph-mon          ceph-osd         ceph-mds         rbd-mirror          ceph-mgr ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-rook        ceph-grafana-dashboards         kmod         lvm2         gdisk     smartmontools     nvme-cli     libstoragemgmt         ceph-radosgw libradosstriper1         nfs-ganesha nfs-ganesha-ceph nfs-ganesha-rgw nfs-ganesha-rados-grace nfs-ganesha-rados-urls         tcmu-runner ceph-iscsi python3-rtslib         attr ceph-fuse rbd-nbd && echo 'Packages verified successfully'": error while running runtime: exit status 1

History

#7 Updated by Sage Weil about 4 years ago

  • Status changed from New to Resolved

#9 Updated by Sebastian Wagner about 4 years ago

  • Status changed from Resolved to New
  • Priority changed from Immediate to High

reopened.

groupadd: /etc/group.388: lock file already used
groupadd: cannot lock /etc/group; try again later.
useradd: group 'unbound' does not exist
error: %prein(unbound-libs-1.7.3-8.el8.x86_64) scriptlet failed, exit status 6

Error in PREIN scriptlet in rpm package unbound-libs
  Installing       : python3-unbound-1.7.3-8.el8.x86_64                 125/238 
error: unbound-libs-1.7.3-8.el8.x86_64: install failed

https://shaman.ceph.com/builds/ceph/wip-swagner2-testing-2020-02-26-1822/678b5b126e264e6f427c96322c7ba1724ebfc92f/default/193667/

#11 Updated by Sage Weil about 4 years ago

For now dgalloway has disabled el8 builds on ephemeral builds. That should make this problem go away for now, but it also means builds are very slow (not enough builders).

#12 Updated by Kefu Chai about 4 years ago

i just built the patched fuse-overlayfs packages and uploaded them to teuthology:~kchai . once David is back, we can install it on the builder host where we use podman for building the images.

the source is 0.7.6 (https://github.com/containers/fuse-overlayfs/tree/v0.7.6) + https://github.com/containers/fuse-overlayfs/pull/184. the patch was proposed at https://bugzilla.redhat.com/show_bug.cgi?id=1802907#c5

the spec recipe comes from https://src.fedoraproject.org/rpms/fuse-overlayfs/blob/master/f/fuse-overlayfs.spec.

i think the built package should be equivalent to the one tested at https://bugzilla.redhat.com/show_bug.cgi?id=1802907#c13

#13 Updated by David Galloway about 4 years ago

I pushed the fuse-overlayfs RPM to http://apt-mirror.front.sepia.ceph.com/lab-extras/8/ and just re-enabled CentOS8 Ephemeral slaves. Please let me know if you see any more dnf installation failures.

#14 Updated by David Galloway about 4 years ago

  • Status changed from New to In Progress
  • Assignee set to David Galloway

#15 Updated by David Galloway about 4 years ago

David Galloway wrote:

I pushed the fuse-overlayfs RPM to http://apt-mirror.front.sepia.ceph.com/lab-extras/8/ and just re-enabled CentOS8 Ephemeral slaves. Please let me know if you see any more dnf installation failures.

The podman spec file must require >= a specific version so it won't pull in the latest version that's in lab-extras.

https://github.com/ceph/ceph-container/pull/1599 should take care of it.

#16 Updated by Sage Weil about 4 years ago

  • Status changed from In Progress to Resolved

Also available in: Atom PDF