Ceph

API test sighting: https://jenkins.ceph.com/job/ceph-api/43159/

Laura Flores wrote:

Affects make check on Pacific PRs.

ImportError: cannot import name 'sysconfig' from 'distutils'
[...]

Forgot to link the example here: https://jenkins.ceph.com/job/ceph-pull-requests/102059/

So far, both examples involve braggi15.

https://jenkins.ceph.com/job/ceph-pull-requests/101834/

adami07

Thinking this is related to Jammy Jellyfish.

https://jenkins.ceph.com/job/ceph-api/43159/

braggi15

https://jenkins.ceph.com/job/ceph-pull-requests/102096/

braggi09

Turns out, this is not a dupe of #57230. The issue with this failure is the same in that affected nodes are Ubuntu 22.04, which require Python 3.10.

But for this failure, cmake is successfully finding Python 3.8:

Found Python3: /usr/bin/python3.8 (found suitable exact version "3.8.10") found components: Interpreter Development 

And then later experiencing a problem with python-distutils since it's using 3.8.

We somehow need to further enforce the use of Python 3.10 on Ubuntu 22.04.

Other Pacific build errors found while working on this:

Item 1 (build failure):

/usr/include/fmt/core.h:1728:7: error: static assertion failed: Cannot format an argument. To make type T formattable provide a formatter<T> specialization: https://fmt.dev/latest/api.html#udt

This is fixed in main by https://github.com/ceph/ceph/commit/bb9d099ede8b9df36dba735a95bc1600fdd11408, but backporting that to Pacific looks complicated, and in any case doesn't make sense because Crimson is still under active development. Better to just turn off WITH_SEASTAR for Pacific builds.

Item 2 (build failure):

CMake Error at src/pybind/CMakeLists.txt:74 (message):
  Python and Ceph link to different OpenSSL versions: 1.1.1

   vs 3.0.2

Seems to only occur when building on python 3.8 workers. Uncertain of fix, but maybe https://github.com/ceph/ceph/pull/47741 will help?

Item 3 (build failure):

ImportError: cannot import name 'Sequence' from 'collections' (/usr/lib/python3.10/collections/__init__.py)

This is fixed by backporting https://github.com/ceph/ceph/pull/47696/commits/aef8f5c0be3bb0d00d663f96595d2038452467fa and https://github.com/ceph/ceph/pull/47696/commits/afd724b8f7a0a6c77e88435230e51d0947e82dc6

Item 4 (can't load dashboard mgr module):

2022-08-22T05:52:31.271+0000 7fe6fca7b500 -1 mgr[py] Module not found: 'dashboard'
2022-08-22T05:52:31.271+0000 7fe6fca7b500 -1 mgr[py] Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-api/src/pybind/mgr/dashboard/__init__.py", line 61, in <module>
    from .module import Module, StandbyModule  # noqa: F401
  File "/home/jenkins-build/build/workspace/ceph-api/src/pybind/mgr/dashboard/module.py", line 51, in <module>
    patch_cherrypy(cherrypy.__version__)
  File "/home/jenkins-build/build/workspace/ceph-api/src/pybind/mgr/dashboard/cherrypy_backports.py", line 197, in patch_cherrypy
    accept_socket_error_0(v)
  File "/home/jenkins-build/build/workspace/ceph-api/src/pybind/mgr/dashboard/cherrypy_backports.py", line 124, in accept_socket_error_0
    if v < StrictVersion("9.0.0") or cheroot_version < StrictVersion("6.5.5"):
  File "/usr/lib/python3.10/distutils/version.py", line 64, in __gt__
    c = self._cmp(other)
  File "/usr/lib/python3.10/distutils/version.py", line 168, in _cmp
    other = StrictVersion(other)
  File "/usr/lib/python3.10/distutils/version.py", line 40, in __init__
    self.parse(vstring)
  File "/usr/lib/python3.10/distutils/version.py", line 137, in parse
    raise ValueError("invalid version number '%s'" % vstring)
ValueError: invalid version number '8.5.2+ds1'

2022-08-22T05:52:31.271+0000 7fe6fca7b500 -1 mgr[py] Class not found in module 'dashboard'
2022-08-22T05:52:31.271+0000 7fe6fca7b500 -1 mgr[py] Error loading module 'dashboard': (2) No such file or directory

This is fixed by backporting https://github.com/ceph/ceph/pull/47500

Item 5 (problems running dashboard in mgr logs):

PY_SSIZE_T_CLEAN macro must be defined for '#' formats

Fixed by backporting https://github.com/tchaikov/ceph/commit/389054888f2aa782f73564125ec7a1ef0212d536