Project

General

Profile

Actions
Copy link

Bug #44587

open

failed to write <pid> to cgroup.procs:

Added by Sage Weil about 4 years ago. Updated 8 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
cephadm (binary)
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2020-03-13T01:55:00.607 INFO:ceph.prometheus.a.smithi166.stdout:Mar 13 01:55:00 smithi166 systemd[1]: Starting Ceph prometheus.a for eee3c39c-64cc-11ea-9a47-001a4aab830c...
2020-03-13T01:55:00.725 INFO:teuthology.orchestra.run.smithi166.stderr:Error: container_linux.go:346: starting container process caused "process_linux.go:297: applying cgroup configuration for process caused \"failed to write 34473 to cgroup.procs: open /sys/fs/cgroup/blkio/machine.slice/libpod-bfcc3e5be38bb3da79931ab84b072b089d34acc75a8b70bc882f473680ebf672.scope/cgroup.procs: no such file or directory\"": OCI runtime command not found error

/a/sage-2020-03-13_01:29:38-rados:cephadm-wip-sage3-testing-2020-03-12-1708-distro-basic-smithi/4851460

Related issues 1 (0 open — 1 closed)

Has duplicate Ceph - Bug #50280: cephadm: RuntimeError: uid/gid not foundDuplicate

Actions
Actions
Copy link
 #1

Updated by Sebastian Wagner almost 4 years ago

  • Source set to Q/A
Actions
Copy link
 #2

Updated by Sebastian Wagner over 3 years ago

  • Status changed from New to Can't reproduce
Actions
Copy link
 #3

Updated by Sebastian Wagner almost 3 years ago

  • Has duplicate Bug #50280: cephadm: RuntimeError: uid/gid not found added
Actions
Copy link
 #4

Updated by Sebastian Wagner almost 3 years ago

  • Status changed from Can't reproduce to New
Actions
Copy link
 #5

Updated by Sebastian Wagner almost 3 years ago

  • Category set to cephadm (binary)
Actions
Copy link
 #8

Updated by Sebastian Wagner over 2 years ago

caused by a downgrade of podman in sepia.

Actions
Copy link
 #9

Updated by Neha Ojha over 2 years ago 

2021-12-07T19:40:29.616 INFO:teuthology.orchestra.run.smithi174.stderr:Non-zero exit code 126 from /bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint stat --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph:02e20d0396c1e49054be17c5f0527ae529a28c91 -e NODE_NAME=smithi174 -e CEPH_USE_RANDOM_NONCE=1 quay.ceph.io/ceph-ci/ceph:02e20d0396c1e49054be17c5f0527ae529a28c91 -c %u %g /var/lib/ceph
2021-12-07T19:40:29.617 INFO:teuthology.orchestra.run.smithi174.stderr:stat: stderr Error: OCI runtime error: container_linux.go:370: starting container process caused: process_linux.go:326: applying cgroup configuration for process caused: failed to write 33700 to cgroup.procs: write /sys/fs/cgroup/devices/machine.slice/libpod-dc31f7868d43ce593f65a3f010c9ddf13a2ce45c47eab28e2a6f0d99f5a009b9.scope/cgroup.procs: no such device
2021-12-07T19:40:29.622 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last):
2021-12-07T19:40:29.623 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 8339, in <module>
2021-12-07T19:40:29.623 INFO:teuthology.orchestra.run.smithi174.stderr:    main()
2021-12-07T19:40:29.623 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 8327, in main
2021-12-07T19:40:29.623 INFO:teuthology.orchestra.run.smithi174.stderr:    r = ctx.func(ctx)
2021-12-07T19:40:29.623 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 1719, in _infer_config
2021-12-07T19:40:29.624 INFO:teuthology.orchestra.run.smithi174.stderr:    return func(ctx)
2021-12-07T19:40:29.624 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 1676, in _infer_fsid
2021-12-07T19:40:29.624 INFO:teuthology.orchestra.run.smithi174.stderr:    return func(ctx)
2021-12-07T19:40:29.624 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 1719, in _infer_config
2021-12-07T19:40:29.625 INFO:teuthology.orchestra.run.smithi174.stderr:    return func(ctx)
2021-12-07T19:40:29.625 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 1763, in _infer_image
2021-12-07T19:40:29.625 INFO:teuthology.orchestra.run.smithi174.stderr:    return func(ctx)
2021-12-07T19:40:29.625 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 1663, in _validate_fsid
2021-12-07T19:40:29.626 INFO:teuthology.orchestra.run.smithi174.stderr:    return func(ctx)
2021-12-07T19:40:29.626 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 5142, in command_shell
2021-12-07T19:40:29.626 INFO:teuthology.orchestra.run.smithi174.stderr:    make_log_dir(ctx, ctx.fsid)
2021-12-07T19:40:29.626 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 1865, in make_log_dir
2021-12-07T19:40:29.626 INFO:teuthology.orchestra.run.smithi174.stderr:    uid, gid = extract_uid_gid(ctx)
2021-12-07T19:40:29.627 INFO:teuthology.orchestra.run.smithi174.stderr:  File "/home/ubuntu/cephtest/cephadm", line 2619, in extract_uid_gid
2021-12-07T19:40:29.627 INFO:teuthology.orchestra.run.smithi174.stderr:    raise RuntimeError('uid/gid not found')
2021-12-07T19:40:29.627 INFO:teuthology.orchestra.run.smithi174.stderr:RuntimeError: uid/gid not found

/a/yuriw-2021-12-07_16:04:59-rados-wip-yuri5-testing-2021-12-06-1619-distro-default-smithi/6550593

Actions
Copy link
 #10

Updated by Aishwarya Mathuria over 2 years ago

/a/yuriw-2022-01-13_18:06:52-rados-wip-yuri3-testing-2022-01-13-0809-distro-default-smithi/6614461

Actions
Copy link
 #11

Updated by Redouane Kachach Elhichou about 2 years ago

  • Priority changed from High to Normal
Actions
Copy link
 #12

Updated by Kamoltat (Junior) Sirivadhna over 1 year ago

  • Subject changed from failed to write 34473 to cgroup.procs: to failed to write <pid> to cgroup.procs:

/a/yuriw-2022-07-22_03:30:40-rados-wip-yuri3-testing-2022-07-21-1604-distro-default-smithi/6944442/

2022-07-23T00:44:33.389 INFO:journalctl@ceph.mon.b.smithi170.stdout:Jul 23 00:44:33 smithi170 systemd[1]: Starting Ceph mon.b for 3911b92e-0a20-11ed-842f-001a4aab830c...
2022-07-23T00:44:33.684 INFO:teuthology.orchestra.run.smithi170.stderr:Error: OCI runtime error: container_linux.go:380: starting container process caused: process_linux.go:385: applying cgroup configuration for process caused: failed to write 90338 to cgroup.procs: write /sys/fs/cgroup/memory/machine.slice/libpod-a48e01ceb59134e5039749a70089fc9bd8cbded60f9a5ee9c7a2dcd45778ab66.scope/cgroup.procs: no such device
2022-07-23T00:44:33.711 DEBUG:teuthology.orchestra.run:got remote process result: 126
2022-07-23T00:44:33.713 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_8d598431210977f8caccec83230b4bfec7bd5d3f/teuthology/contextutil.py", line 31, in nested
    vars.append(enter())
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/cephadm.py", line 678, in ceph_mons
    stdout=StringIO(),
  File "/home/teuthworker/src/github.com_ceph_ceph-c_19acc2f7e3edb197f028ffa801e28f62f3698c79/qa/tasks/cephadm.py", line 49, in _shell
    **kwargs
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_8d598431210977f8caccec83230b4bfec7bd5d3f/teuthology/orchestra/remote.py", line 510, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_8d598431210977f8caccec83230b4bfec7bd5d3f/teuthology/orchestra/run.py", line 455, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_8d598431210977f8caccec83230b4bfec7bd5d3f/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_8d598431210977f8caccec83230b4bfec7bd5d3f/teuthology/orchestra/run.py", line 183, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi170 with status 126: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:19acc2f7e3edb197f028ffa801e28f62f3698c79 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3911b92e-0a20-11ed-842f-001a4aab830c -- ceph mon dump -f json'
2022-07-23T00:44:33.713 INFO:tasks.cephadm:Cleaning up testdir ceph.* files...
Actions
Copy link
 #13

Updated by Adam King over 1 year ago

  • Translation missing: en.field_tag_list set to test-failure
Actions
Copy link
 #14

Updated by Laura Flores about 1 year ago

/a/yuriw-2023-02-18_15:53:03-rados-wip-yuri5-testing-2023-02-17-1400-quincy-distro-default-smithi/7180663

Actions
Copy link
 #15

Updated by Laura Flores 8 months ago

/a/yuriw-2023-09-01_19:14:47-rados-wip-batrick-testing-20230831.124848-pacific-distro-default-smithi/7386507

Actions
Copy link

Also available in: Atom PDF