Project

General

Profile

Actions

Bug #55079

open

rpm: remove contents of build directory at end of %install section

Added by Tim Serong about 2 years ago. Updated over 1 year ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
backport_processed
Backport:
quincy,pacific,octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I've been doing some measurements of disk usage during SUSE RPM builds (of Pacific, but this should roughly apply for newer Cephs too). In our particular build environment, which builds everything in VMs, we see something like this:

                      Filesystem      Size  Used Avail Use% Mounted on
  df start of build:  /dev/vda         53G   14G   40G  25% /
  df end of build:    /dev/vda         53G   31G   23G  58% /
  df end of install:  /dev/vda         53G   39G   15G  74% /
  df before clamscan: /dev/vda         53G   41G   13G  78% /
  df after clamscan:  /dev/vda         53G   50G  3.9G  93% /

So after compiling everything, we've consumed about 17GB (that's all the binaries and object files and whatnot that end up in the "build" directory in the source tree). Then, after %install (which installs everything in the build root, ready to be turned into actual RPMs), we've used another 8GB. The next part - the clamscan bit - is one of the rpmlint checks SUSE runs, which takes another 9G when it extracts all the built RPMs (including debuginfo RPMs), in order to scan them.

In summary, our build worker VMs currently need a bit over 50G disk to build Ceph.

If I add `rm -rf build` to the very end of the install section, to get rid of the 17GB of built binaries, we go into clamscan with 24G free, rather than 41G free, and when clamscan finishes we're using 32G. This means the peak build disk usage with that change is about 39G, so we reduce our build worker's disk space requirements by about 11G (or 20%).


Related issues 3 (0 open3 closed)

Copied to Ceph - Backport #55097: pacific: rpm: remove contents of build directory at end of %install sectionResolvedTim SerongActions
Copied to Ceph - Backport #55098: quincy: rpm: remove contents of build directory at end of %install sectionResolvedTim SerongActions
Copied to Ceph - Backport #55099: octopus: rpm: remove contents of build directory at end of %install sectionRejectedTim SerongActions
Actions #1

Updated by Tim Serong about 2 years ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 45664
Actions #2

Updated by Tim Serong about 2 years ago

Note for backports:
- Master and Quincy use %{_vpath_builddir}
- Pacific and Octopus use build

Actions #3

Updated by Tim Serong about 2 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions #4

Updated by Backport Bot about 2 years ago

  • Copied to Backport #55097: pacific: rpm: remove contents of build directory at end of %install section added
Actions #5

Updated by Backport Bot about 2 years ago

  • Copied to Backport #55098: quincy: rpm: remove contents of build directory at end of %install section added
Actions #6

Updated by Backport Bot about 2 years ago

  • Copied to Backport #55099: octopus: rpm: remove contents of build directory at end of %install section added
Actions #7

Updated by Tim Serong about 2 years ago

I've just discovered that blanket removal of everything under "build" at the end of %install means we lose some data necessary for debugsource packages, so I'll have to try to rework this a bit (>sigh<). It's hopefully enough to just shift it to the %clean section, just gotta wait for a test build to confirm.

Actions #8

Updated by Tim Serong about 2 years ago

Yep, it works properly in the %clean section, so I've gone ahead and opened https://github.com/ceph/ceph/pull/45705 for that tweak.

Actions #9

Updated by Backport Bot over 1 year ago

  • Tags set to backport_processed
Actions

Also available in: Atom PDF