Project

General

Profile

Actions

Bug #61399

open

qa: build failure for ior

Added by Patrick Donnelly 11 months ago. Updated 7 months ago.

Status:
Pending Backport
Priority:
High
Assignee:
Category:
-
Target version:
% Done:

0%

Source:
Q/A
Tags:
backport_processed
Backport:
reef,quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
qa-suite
Labels (FS):
qa, qa-failure
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2023-05-24T02:32:38.769 INFO:teuthology.orchestra.run.smithi092.stdout:Making all in src
2023-05-24T02:32:38.771 INFO:teuthology.orchestra.run.smithi092.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/ior/src'
2023-05-24T02:32:38.772 INFO:teuthology.orchestra.run.smithi092.stdout:make  all-am
2023-05-24T02:32:38.775 INFO:teuthology.orchestra.run.smithi092.stdout:make[2]: Entering directory '/home/ubuntu/cephtest/ior/src'
2023-05-24T02:32:38.803 INFO:teuthology.orchestra.run.smithi092.stdout:mpicc -DHAVE_CONFIG_H -I.     -g -O2 -MT ior.o -MD -MP -MF .deps/ior.Tpo -c -o ior.o ior.c
2023-05-24T02:32:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:ior.c: In function 'CountTasksPerNode':
2023-05-24T02:32:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:ior.c:476:42: warning: '%s' directive writing up to 1023 bytes into a region of size between 1004 and 1014 [-Wformat-overflow=]
2023-05-24T02:32:39.027 INFO:teuthology.orchestra.run.smithi092.stderr:  476 |                 sprintf(tmp, "task %d on %s", rank, localhost);
2023-05-24T02:32:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:      |                                          ^~         ~~~~~~~~~
2023-05-24T02:32:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:In file included from /usr/include/stdio.h:894,
2023-05-24T02:32:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:                 from ior.c:15:
2023-05-24T02:32:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:38:10: note: '__builtin___sprintf_chk' output between 11 and 1044 bytes into a destination of size 1024
2023-05-24T02:32:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:   38 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
2023-05-24T02:32:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-05-24T02:32:39.028 INFO:teuthology.orchestra.run.smithi092.stderr:   39 |                                   __glibc_objsize (__s), __fmt,
2023-05-24T02:32:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:      |                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-05-24T02:32:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:   40 |                                   __va_arg_pack ());
2023-05-24T02:32:39.029 INFO:teuthology.orchestra.run.smithi092.stderr:      |                                   ~~~~~~~~~~~~~~~~~
2023-05-24T02:32:39.183 INFO:teuthology.orchestra.run.smithi172.stdout:mv -f .deps/ior.Tpo .deps/ior.Po
2023-05-24T02:32:39.186 INFO:teuthology.orchestra.run.smithi172.stdout:mpicc -DHAVE_CONFIG_H -I.     -g -O2 -MT utilities.o -MD -MP -MF .deps/utilities.Tpo -c -o utilities.o utilities.c
2023-05-24T02:32:39.383 INFO:teuthology.orchestra.run.smithi172.stdout:mv -f .deps/utilities.Tpo .deps/utilities.Po
2023-05-24T02:32:39.385 INFO:teuthology.orchestra.run.smithi172.stdout:mpicc -DHAVE_CONFIG_H -I.     -g -O2 -MT parse_options.o -MD -MP -MF .deps/parse_options.Tpo -c -o parse_options.o parse_options.c
2023-05-24T02:32:39.603 INFO:teuthology.orchestra.run.smithi172.stdout:mv -f .deps/parse_options.Tpo .deps/parse_options.Po
2023-05-24T02:32:39.607 INFO:teuthology.orchestra.run.smithi172.stdout:mpicc -DHAVE_CONFIG_H -I.     -g -O2 -MT aiori-POSIX.o -MD -MP -MF .deps/aiori-POSIX.Tpo -c -o aiori-POSIX.o aiori-POSIX.c
2023-05-24T02:32:39.752 INFO:teuthology.orchestra.run.smithi092.stdout:mv -f .deps/ior.Tpo .deps/ior.Po
2023-05-24T02:32:39.755 INFO:teuthology.orchestra.run.smithi092.stdout:mpicc -DHAVE_CONFIG_H -I.     -g -O2 -MT utilities.o -MD -MP -MF .deps/utilities.Tpo -c -o utilities.o utilities.c
2023-05-24T02:32:39.776 INFO:teuthology.orchestra.run.smithi172.stdout:mv -f .deps/aiori-POSIX.Tpo .deps/aiori-POSIX.Po
2023-05-24T02:32:39.778 INFO:teuthology.orchestra.run.smithi172.stdout:mpicc -DHAVE_CONFIG_H -I.     -g -O2 -MT aiori-MPIIO.o -MD -MP -MF .deps/aiori-MPIIO.Tpo -c -o aiori-MPIIO.o aiori-MPIIO.c
2023-05-24T02:32:39.848 INFO:teuthology.orchestra.run.smithi172.stderr:aiori-MPIIO.c: In function 'MPIIO_Xfer':
2023-05-24T02:32:39.849 INFO:teuthology.orchestra.run.smithi172.stderr:aiori-MPIIO.c:236:24: warning: assignment to 'int (*)(struct ADIOI_FileD *, void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, void *, int,  int,  MPI_Status *)'} from incompatible pointer type 'int (*)(struct ADIOI_FileD *, const void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, const void *, int,  int,  MPI_Status *)'} [-Wincompatible-pointer-types]
2023-05-24T02:32:39.849 INFO:teuthology.orchestra.run.smithi172.stderr:  236 |                 Access = MPI_File_write;
2023-05-24T02:32:39.849 INFO:teuthology.orchestra.run.smithi172.stderr:      |                        ^
2023-05-24T02:32:39.849 INFO:teuthology.orchestra.run.smithi172.stderr:aiori-MPIIO.c:237:27: warning: assignment to 'int (*)(struct ADIOI_FileD *, MPI_Offset,  void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, long int,  void *, int,  int,  MPI_Status *)'} from incompatible pointer type 'int (*)(struct ADIOI_FileD *, MPI_Offset,  const void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, long int,  const void *, int,  int,  MPI_Status *)'} [-Wincompatible-pointer-types]
2023-05-24T02:32:39.850 INFO:teuthology.orchestra.run.smithi172.stderr:  237 |                 Access_at = MPI_File_write_at;
2023-05-24T02:32:39.850 INFO:teuthology.orchestra.run.smithi172.stderr:      |                           ^
2023-05-24T02:32:39.850 INFO:teuthology.orchestra.run.smithi172.stderr:aiori-MPIIO.c:238:28: warning: assignment to 'int (*)(struct ADIOI_FileD *, void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, void *, int,  int,  MPI_Status *)'} from incompatible pointer type 'int (*)(struct ADIOI_FileD *, const void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, const void *, int,  int,  MPI_Status *)'} [-Wincompatible-pointer-types]
2023-05-24T02:32:39.851 INFO:teuthology.orchestra.run.smithi172.stderr:  238 |                 Access_all = MPI_File_write_all;
2023-05-24T02:32:39.851 INFO:teuthology.orchestra.run.smithi172.stderr:      |                            ^
2023-05-24T02:32:39.851 INFO:teuthology.orchestra.run.smithi172.stderr:aiori-MPIIO.c:239:31: warning: assignment to 'int (*)(struct ADIOI_FileD *, MPI_Offset,  void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, long int,  void *, int,  int,  MPI_Status *)'} from incompatible pointer type 'int (*)(struct ADIOI_FileD *, MPI_Offset,  const void *, int,  MPI_Datatype,  MPI_Status *)' {aka 'int (*)(struct ADIOI_FileD *, long int,  const void *, int,  int,  MPI_Status *)'} [-Wincompatible-pointer-types]
2023-05-24T02:32:39.851 INFO:teuthology.orchestra.run.smithi172.stderr:  239 |                 Access_at_all = MPI_File_write_at_all;
2023-05-24T02:32:39.852 INFO:teuthology.orchestra.run.smithi172.stderr:      |                               ^
2023-05-24T02:32:39.871 INFO:teuthology.orchestra.run.smithi172.stderr:aiori-MPIIO.c: In function 'MPIIO_SetVersion':
2023-05-24T02:32:39.871 INFO:teuthology.orchestra.run.smithi172.stderr:aiori-MPIIO.c:378:38: warning: ' (version=' directive writing 10 bytes into a region of size between 1 and 1024 [-Wformat-overflow=]
2023-05-24T02:32:39.871 INFO:teuthology.orchestra.run.smithi172.stderr:  378 |         sprintf(test->apiVersion, "%s (version=%d, subversion=%d)",
2023-05-24T02:32:39.872 INFO:teuthology.orchestra.run.smithi172.stderr:      |                                      ^~~~~~~~~~
2023-05-24T02:32:39.872 INFO:teuthology.orchestra.run.smithi172.stderr:In file included from /usr/include/stdio.h:894,
2023-05-24T02:32:39.872 INFO:teuthology.orchestra.run.smithi172.stderr:                 from aiori-MPIIO.c:19:
2023-05-24T02:32:39.872 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:38:10: note: '__builtin___sprintf_chk' output between 27 and 1070 bytes into a destination of size 1024
2023-05-24T02:32:39.873 INFO:teuthology.orchestra.run.smithi172.stderr:   38 |   return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1,
2023-05-24T02:32:39.873 INFO:teuthology.orchestra.run.smithi172.stderr:      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-05-24T02:32:39.874 INFO:teuthology.orchestra.run.smithi172.stderr:   39 |                                   __glibc_objsize (__s), __fmt,
2023-05-24T02:32:39.874 INFO:teuthology.orchestra.run.smithi172.stderr:      |                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2023-05-24T02:32:39.874 INFO:teuthology.orchestra.run.smithi172.stderr:   40 |                                   __va_arg_pack ());
2023-05-24T02:32:39.874 INFO:teuthology.orchestra.run.smithi172.stderr:      |                                   ~~~~~~~~~~~~~~~~~
2023-05-24T02:32:39.959 INFO:teuthology.orchestra.run.smithi092.stdout:mv -f .deps/utilities.Tpo .deps/utilities.Po
2023-05-24T02:32:39.963 INFO:teuthology.orchestra.run.smithi092.stdout:mpicc -DHAVE_CONFIG_H -I.     -g -O2 -MT parse_options.o -MD -MP -MF .deps/parse_options.Tpo -c -o parse_options.o parse_options.c
2023-05-24T02:32:39.963 INFO:teuthology.orchestra.run.smithi172.stdout:mv -f .deps/aiori-MPIIO.Tpo .deps/aiori-MPIIO.Po
2023-05-24T02:32:39.968 INFO:teuthology.orchestra.run.smithi172.stdout:mpicc  -g -O2   -o ior ior.o utilities.o parse_options.o aiori-POSIX.o aiori-MPIIO.o     -lm
2023-05-24T02:32:39.988 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: utilities.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: multiple definition of `ncmpi_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: first defined here
2023-05-24T02:32:39.988 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: utilities.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: multiple definition of `hdf5_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: first defined here
2023-05-24T02:32:39.988 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: utilities.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: multiple definition of `mpiio_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: first defined here
2023-05-24T02:32:39.988 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: utilities.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: multiple definition of `posix_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: first defined here
2023-05-24T02:32:39.990 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: parse_options.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: multiple definition of `ncmpi_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: first defined here
2023-05-24T02:32:39.990 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: parse_options.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: multiple definition of `hdf5_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: first defined here
2023-05-24T02:32:39.991 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: parse_options.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: multiple definition of `mpiio_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: first defined here
2023-05-24T02:32:39.991 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: parse_options.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: multiple definition of `posix_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: first defined here
2023-05-24T02:32:39.992 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-POSIX.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: multiple definition of `ncmpi_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: first defined here
2023-05-24T02:32:39.992 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-POSIX.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: multiple definition of `hdf5_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: first defined here
2023-05-24T02:32:39.992 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-POSIX.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: multiple definition of `mpiio_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: first defined here
2023-05-24T02:32:39.992 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-POSIX.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: multiple definition of `posix_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: first defined here
2023-05-24T02:32:39.993 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-MPIIO.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: multiple definition of `ncmpi_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:68: first defined here
2023-05-24T02:32:39.993 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-MPIIO.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: multiple definition of `hdf5_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:67: first defined here
2023-05-24T02:32:39.994 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-MPIIO.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: multiple definition of `mpiio_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:66: first defined here
2023-05-24T02:32:39.994 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: aiori-MPIIO.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: multiple definition of `posix_aiori'; ior.o:/home/ubuntu/cephtest/ior/src/aiori.h:65: first defined here
2023-05-24T02:32:40.186 INFO:teuthology.orchestra.run.smithi092.stdout:mv -f .deps/parse_options.Tpo .deps/parse_options.Po
...
2023-05-24T02:32:40.249 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: (.text+0x451): undefined reference to `fi_version'
2023-05-24T02:32:40.249 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: (.text+0x461): undefined reference to `fi_version'
2023-05-24T02:32:40.249 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: (.text+0x480): undefined reference to `fi_dupinfo'
2023-05-24T02:32:40.249 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: (.text+0x4c9): undefined reference to `fi_version'
2023-05-24T02:32:40.249 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmpich.a(lib_libmpich_la-init_provider.o): in function `MPIDI_OFI_find_provider_cleanup':
2023-05-24T02:32:40.250 INFO:teuthology.orchestra.run.smithi172.stderr:(.text+0x781): undefined reference to `fi_freeinfo'
2023-05-24T02:32:40.250 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmpich.a(lib_libmpich_la-init_provider.o): in function `MPIDI_OFI_get_required_version':
2023-05-24T02:32:40.250 INFO:teuthology.orchestra.run.smithi172.stderr:(.text+0x841): undefined reference to `fi_version'
2023-05-24T02:32:40.250 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmpich.a(lib_libmpich_la-init_addrxchg.o): in function `MPIDI_OFI_addr_exchange_root_ctx':
2023-05-24T02:32:40.250 INFO:teuthology.orchestra.run.smithi172.stderr:(.text+0x31c): undefined reference to `fi_strerror'
2023-05-24T02:32:40.251 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: (.text+0x3b5): undefined reference to `fi_strerror'
2023-05-24T02:32:40.251 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: (.text+0x45d): undefined reference to `fi_strerror'
2023-05-24T02:32:40.251 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: (.text+0x555): undefined reference to `fi_strerror'
2023-05-24T02:32:40.251 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmpich.a(lib_libmpich_la-init_addrxchg.o): in function `MPIDI_OFI_addr_exchange_all_ctx':
2023-05-24T02:32:40.251 INFO:teuthology.orchestra.run.smithi172.stderr:(.text+0xb35): undefined reference to `fi_strerror'
2023-05-24T02:32:40.251 INFO:teuthology.orchestra.run.smithi172.stderr:/usr/bin/ld: /usr/lib/x86_64-linux-gnu/libmpich.a(lib_libmpich_la-init_addrxchg.o):(.text+0xdf6): more undefined references to `fi_strerror' follow
2023-05-24T02:32:40.308 INFO:teuthology.orchestra.run.smithi172.stderr:collect2: error: ld returned 1 exit status
2023-05-24T02:32:40.309 INFO:teuthology.orchestra.run.smithi172.stderr:make[2]: *** [Makefile:299: ior] Error 1
2023-05-24T02:32:40.309 INFO:teuthology.orchestra.run.smithi172.stdout:make[2]: Leaving directory '/home/ubuntu/cephtest/ior/src'
2023-05-24T02:32:40.309 INFO:teuthology.orchestra.run.smithi172.stderr:make[1]: *** [Makefile:208: all] Error 2
2023-05-24T02:32:40.310 INFO:teuthology.orchestra.run.smithi172.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/ior/src'
2023-05-24T02:32:40.310 DEBUG:teuthology.orchestra.run:got remote process result: 2
2023-05-24T02:32:40.311 INFO:teuthology.orchestra.run.smithi172.stderr:make: *** [Makefile:261: all-recursive] Error 1
2023-05-24T02:32:40.311 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/run_tasks.py", line 105, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/run_tasks.py", line 84, in run_one_task
    return task(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/task/pexec.py", line 149, in task
    p.spawn(_exec_host, barrier, barrier_queue, remote[0], sudo, testdir, remote[1])
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/task/pexec.py", line 60, in _exec_host
    tor.wait([r])
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/orchestra/run.py", line 181, in _raise_for_status
    raise CommandFailedError(
teuthology.exceptions.CommandFailedError: Command failed on smithi172 with status 2: 'TESTDIR=/home/ubuntu/cephtest bash -s'
2023-05-24T02:32:40.383 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4da8bbc47d444b9c91169957e0eb1d99
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/run_tasks.py", line 105, in run_tasks
    manager = run_one_task(taskname, ctx=ctx, config=config)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/run_tasks.py", line 84, in run_one_task
    return task(**kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/task/pexec.py", line 149, in task
    p.spawn(_exec_host, barrier, barrier_queue, remote[0], sudo, testdir, remote[1])
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 84, in __exit__
    for result in self:
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 98, in __next__
    resurrect_traceback(result)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 30, in resurrect_traceback
    raise exc.exc_info[1]
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/parallel.py", line 23, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/task/pexec.py", line 60, in _exec_host
    tor.wait([r])
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/orchestra/run.py", line 479, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_teuthology_c6e0984e7e7716eedf6039f15c6a9d1043e0c5f2/teuthology/orchestra/run.py", line 181, in _raise_for_status
    raise CommandFailedError(
teuthology.exceptions.CommandFailedError: Command failed on smithi172 with status 2: 'TESTDIR=/home/ubuntu/cephtest bash -s'

From: /ceph/teuthology-archive/pdonnell-2023-05-23_18:20:18-fs-wip-pdonnell-testing-20230523.134409-distro-default-smithi/7284202/teuthology.log


Related issues 4 (4 open0 closed)

Related to CephFS - Bug #61574: qa: build failure for mdtest projectPending BackportRishabh Dave

Actions
Related to CephFS - Bug #63212: qa: failed to download ior.tbz2NewRishabh Dave

Actions
Copied to CephFS - Backport #63096: reef: qa: build failure for iorIn ProgressRishabh DaveActions
Copied to CephFS - Backport #63097: quincy: qa: build failure for iorIn ProgressRishabh DaveActions
Actions #1

Updated by Milind Changire 11 months ago

  • Assignee set to Rishabh Dave
Actions #4

Updated by Patrick Donnelly 11 months ago

  • Related to Bug #61574: qa: build failure for mdtest project added
Actions #7

Updated by Rishabh Dave 10 months ago

  • Pull request ID set to 52416
Actions #8

Updated by Rishabh Dave 10 months ago

  • Subject changed from qa: "[Makefile:299: ior] Error 1" to qa: build failure for ior
  • Status changed from New to In Progress
Actions #9

Updated by Venky Shankar 8 months ago

  • Status changed from In Progress to Resolved

Rishabh, does this need backporting?

Actions #10

Updated by Venky Shankar 8 months ago

  • Status changed from Resolved to In Progress

had to revert the changes - https://github.com/ceph/ceph/pull/53036

Actions #11

Updated by Rishabh Dave 8 months ago

  • Pull request ID changed from 52416 to 53079

https://github.com/ceph/ceph/pull/52416 was merged accidentally (and then reverted). I've opened new PR for same patch - https://github.com/ceph/ceph/pull/53079.

Actions #12

Updated by Rishabh Dave 8 months ago

  • Status changed from In Progress to Fix Under Review
Actions #13

Updated by Rishabh Dave 8 months ago

The PR has been merged just now, I'll check with Venky if this needs to be backported.

Actions #14

Updated by Rishabh Dave 8 months ago

What fixed this issue is using latest version of ior project as well as purging and then again installing mpich package. Either of these two alone are insufficient to fix this issue.

Using new version of ior project (but mpich is not purged and installed again) led the configure script to fail; see - https://github.com/ceph/ceph/pull/53079#issuecomment-1690479566. And purging plus installing mpich again with old ior project version leads to a build failure (specifically, make command fails); see - https://github.com/ceph/ceph/pull/53079#issuecomment-1690219155.

So the exact root cause of this issue is not known. It probably has to do something with mpich package's config files that come by default (which gets deleted and installed when purge+install is done) or it has something to do with internal detail of ior project -- how ior project uses mpich package.

I talked with Venky about this. We decided to not spend more time finding the exact cause since there are more tickets that are on a priority higher than this.

Actions #15

Updated by Venky Shankar 7 months ago

  • Status changed from Fix Under Review to Resolved

Rishabh, this change does not need backport, yes?

Actions #16

Updated by Venky Shankar 7 months ago

  • Status changed from Resolved to Pending Backport
  • Backport set to reef,quincy
Actions #17

Updated by Backport Bot 7 months ago

Actions #18

Updated by Backport Bot 7 months ago

Actions #19

Updated by Backport Bot 7 months ago

  • Tags set to backport_processed
Actions #20

Updated by Venky Shankar 7 months ago

This change drops in a new tarball with version number suffixed, so q/r release backports runs without the backport would fail (obviously, till the backport is merged).

Actions #21

Updated by Venky Shankar 6 months ago

  • Related to Bug #63212: qa: failed to download ior.tbz2 added
Actions

Also available in: Atom PDF