Infrastructure

For context, here is the list of packages providing promtool.

In the specific case of Ubuntu, the versions provided for Ubuntu <20 "don't support the test rules command":http://manpages.ubuntu.com/manpages/focal/en/man1/promtool.1.html

So I started to try to install a system via Cobbler but Canonical made kickstart/automated installs very difficult beginning with Focal.

Okay, braggi18 is running Focal now. You can force a job to run on it by setting its required labels to: gigantic&&x86_64&&focal&&!bionic&&!xenial

Thank you for your efforts, David! i see we have at least 4 focal test nodes.

i will work with Ernesto to

1. verify if focal works the updated change. and
2. if the 4 nodess are enough for serving the "make check" runs
3. merge the change to use the updated test

i see we have at least 4 focal test nodes

Errr wait. AFAIK there is only one. If another builder has the 'focal' label but also has the 'xenial' or 'bionic' labels, it may not actually be running Focal.

ack. checked. indeed, there is only a single focal node available.

i am working on https://github.com/ceph/ceph/pull/40205 .

rebuilding boost for focal.

and taking this opportunity to upgrade to boost v1.75.

i've

• rebuilt boost 1.75 for focal and bionic
• rebuilt libzbd for focal
• tested follow PRs which enable us to build on focal + boost 1.75
  - https://github.com/ceph/ceph/pull/40230
  - https://github.com/ceph/ceph/pull/40229
  - https://github.com/ceph/ceph/pull/40205
  - https://github.com/ceph/ceph/pull/40272

after these PRs above get merged. can we have more focal test nodes for serving the "make check" load from master? i can help backport some changes to enable octopus and pacific to build on focal + boost 1.75 if we want to share the load with a bigger pool of jenkins test nodes.

Kefu Chai wrote:

after these PRs above get merged. can we have more focal test nodes for serving the "make check" load from master? i can help backport some changes to enable octopus and pacific to build on focal + boost 1.75 if we want to share the load with a bigger pool of jenkins test nodes.

Yes, please. I'd have to create a separate job to run only master make check on Focal nodes.

David Galloway wrote:

Kefu Chai wrote:

after these PRs above get merged. can we have more focal test nodes for serving the "make check" load from master? i can help backport some changes to enable octopus and pacific to build on focal + boost 1.75 if we want to share the load with a bigger pool of jenkins test nodes.

Yes, please. I'd have to create a separate job to run only master make check on Focal nodes.

sure. i am now testing the builds on LTS branches. and the progress is tracked here:

• nautilus:
  • https://github.com/ceph/ceph/pull/40287
  • https://github.com/ceph/ceph/pull/40333
  • https://github.com/ceph/ceph/pull/40335
• ocotpus: https://github.com/ceph/ceph/pull/40286
• pacific: https://github.com/ceph/ceph/pull/40285

At least some of this, such as https://github.com/ceph/ceph/pull/40205/commits/f381aa8bf0e175940153975fa1534ef0559ecadd, needs to be backported to pacific since be7f9e704c8 was backported.

[edit] We've decided in today's rados suite review meeting to revert be7f9e704c8 from pacific since it's blocking tests and we are on the eve of a release. https://github.com/ceph/ceph/pull/40419 [/edit]

@David, all PRs have been merged. so I think we're ready to switch to focal. BTW, I don't think we need gigantic nodes for running "make check", "huge" ones are good enough.

could you kindly let me know what else I can do?

This is complete but we're seeing build failures of Nautilus on Xenial now.

https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=xenial,DIST=xenial,MACHINE_SIZE=gigantic/53366//consoleFull

...

PBUILDER_OPERATION=build
PBUILDER_PKGDATADIR=/usr/share/pbuilder
PBUILDER_PKGLIBDIR=/usr/lib/pbuilder
PBUILDER_SYSCONFDIR=/etc
PWD=/build/ceph-14.2.18-43-g81849f16
SHELL=/bin/bash
SHLVL=1
SOURCE_DATE_EPOCH=1617041584
SUDO_COMMAND=/usr/sbin/pbuilder build --distribution xenial --basetgz /srv/debian-base/xenial.tgz --buildresult ./release/14.2.18-43-g81849f16 --profiles nocheck --use-network yes ./release/14.2.18-43-g81849f16/ceph_14.2.18-43-g81849f16-1xenial.dsc
SUDO_GID=1108
SUDO_UID=1108
SUDO_USER=jenkins-build
TERM=unknown
USER=pbuilder
_=/usr/bin/systemd-run
dh_auto_configure --buildsystem=cmake -- -DWITH_OCF=ON -DWITH_LTTNG=ON -DWITH_PYTHON3=ON -DWITH_MGR_DASHBOARD_FRONTEND=OFF -DWITH_CEPHFS_JAVA=ON -DWITH_CEPHFS_SHELL=ON -DWITH_SYSTEMD=ON -DCEPH_SYSTEMD_ENV_DIR=/etc/default -DCMAKE_INSTALL_LIBDIR=/usr/lib -DCMAKE_INSTALL_LIBEXECDIR=/usr/lib -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_SYSTEMD_SERVICEDIR=/lib/systemd/system -DBOOST_J=48 -DWITH_BOOST_CONTEXT=ON -DALLOCATOR=tcmalloc -DWITH_STATIC_LIBSTDCXX=ON
    mkdir -p obj-x86_64-linux-gnu
    cd obj-x86_64-linux-gnu
    cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_LOCALSTATEDIR=/var -DWITH_OCF=ON -DWITH_LTTNG=ON -DWITH_PYTHON3=ON -DWITH_MGR_DASHBOARD_FRONTEND=OFF -DWITH_CEPHFS_JAVA=ON -DWITH_CEPHFS_SHELL=ON -DWITH_SYSTEMD=ON -DCEPH_SYSTEMD_ENV_DIR=/etc/default -DCMAKE_INSTALL_LIBDIR=/usr/lib -DCMAKE_INSTALL_LIBEXECDIR=/usr/lib -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_SYSTEMD_SERVICEDIR=/lib/systemd/system -DBOOST_J=48 -DWITH_BOOST_CONTEXT=ON -DALLOCATOR=tcmalloc -DWITH_STATIC_LIBSTDCXX=ON
-- The CXX compiler identification is GNU 5.3.1
-- The C compiler identification is GNU 5.3.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/x86_64-linux-gnu-gcc
-- Check for working CXX compiler: /usr/bin/x86_64-linux-gnu-g++
-- Check for working CXX compiler: /usr/bin/x86_64-linux-gnu-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/x86_64-linux-gnu-gcc
-- Check for working C compiler: /usr/bin/x86_64-linux-gnu-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
CMake Error at CMakeLists.txt:42 (message):
  GCC 7+ required due to C++17 requirements

-- Configuring incomplete, errors occurred!
See also "/build/ceph-14.2.18-43-g81849f16/obj-x86_64-linux-gnu/CMakeFiles/CMakeOutput.log".
See also "/build/ceph-14.2.18-43-g81849f16/obj-x86_64-linux-gnu/CMakeFiles/CMakeError.log".
    cd /build/ceph-14.2.18-43-g81849f16
    cd obj-x86_64-linux-gnu
    "tail -v -n +0 CMakeCache.txt" 
==> CMakeCache.txt <==
# This is the CMakeCache file.
# For build in directory: /build/ceph-14.2.18-43-g81849f16/obj-x86_64-linux-gnu
# It was generated by CMake: /usr/bin/cmake
# You can edit this file to change values found and used by cmake.
# If you do not want to change any of the values, simply exit the editor.
# If you do want to change a value, simply edit, save, and exit the editor.
# The syntax for the file is as follows:
# KEY:TYPE=VALUE
# KEY is the name of a variable in the cache.
# TYPE is a hint to GUIs for the type of VALUE, DO NOT EDIT TYPE!.
# VALUE is the current value for the KEY.

...

the xenial build failure was addressed by https://github.com/ceph/ceph-build/pull/1785.

there is still a test failure issue on nautilus. it should be fixed by https://github.com/ceph/ceph/pull/40516. which includes a backport from https://github.com/ceph/ceph/pull/40515.