Project

General

Profile

Actions

Bug #58963

open

OSD is mute. No std::err, no log.

Added by Adam Kupczyk about 1 year ago. Updated about 1 year ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I encountered a problem of `ceph-osd` not printing any std::err and not putting anything to the --log-file.
Just starts and then exits.

Env:
- root user on virtual machine

Run 0:
`ceph-osd -f --cluster test --setuser ceph --setgroup ceph --id 0 --log-to-file=true --log-file=/tmp/log.txt --debug_deliberately_leak_memory=true --osd_data=/tmp/cephtmp`

Result:
Process runs for milisecond and exits.
No std::err output and empty /tmp/log.txt.

Modification 1: strace
Command line:
`strace -f -e execve ceph-osd -f --cluster test --setuser ceph --setgroup ceph --id 0 --log-to-file=true --log-file=/tmp/log.txt --debug_deliberately_leak_memory=true --osd_data=/tmp/cephtmp`
Result:
typical strace output+
`2023-03-10T11:33:31.286+0000 7ff07f95a640 -1 deliberately leaking some memory`
/tmp/log.txt:
```2023-03-10T11:33:31.286+0000 7ff07f95a640 0 set uid:gid to 167:167 (ceph:ceph)
2023-03-10T11:33:31.286+0000 7ff07f95a640 -1 deliberately leaking some memory
2023-03-10T11:33:31.286+0000 7ff07f95a640 0 ceph version 18.0.0-2859-g44e4dfd0 (44e4dfd0d8d31f3da4cb9cb96b8d7ba4f587b37c) reef (dev), process ceph-osd, pid 42895
2023-03-10T11:33:31.287+0000 7ff07f95a640 0 pidfile_write: ignore empty --pid-file
2023-03-10T11:33:31.287+0000 7ff07f95a640 -1 missing 'type' file and unable to infer osd type```

Modification 2: mkfifo
Preparation step, to force process to pause
mkfifo /tmp/cephtmp/type
Now ceph-osd will wait forever for someone to put data to the other side of the pipe.

Command line:
`ceph-osd -f --cluster test --setuser ceph --setgroup ceph --id 0 --log-to-file=true --log-file=/tmp/log.txt --debug_deliberately_leak_memory=true --osd_data=/tmp/cephtmp`

Result:
std::err:
`2023-03-10T13:16:31.590+0000 7ff9bcb54640 -1 deliberately leaking some memory`
/tmp/log.txt:
```2023-03-10T13:16:31.590+0000 7ff9bcb54640 0 set uid:gid to 167:167 (ceph:ceph)
2023-03-10T13:16:31.590+0000 7ff9bcb54640 -1 deliberately leaking some memory
2023-03-10T13:16:31.590+0000 7ff9bcb54640 0 ceph version 18.0.0-2859-g44e4dfd0 (44e4dfd0d8d31f3da4cb9cb96b8d7ba4f587b37c) reef (dev), process ceph-osd, pid 43061
2023-03-10T13:16:31.590+0000 7ff9bcb54640 0 pidfile_write: ignore empty --pid-file```


Files

no-std-err.txt.gz (641 KB) no-std-err.txt.gz Adam Kupczyk, 03/13/2023 09:10 AM
Actions #1

Updated by Adam Kupczyk about 1 year ago

A complete session when trying to get closer to why no output.

Actions #2

Updated by Adam Kupczyk about 1 year ago

NOTE

After `dnf update` problem went away.

Actions

Also available in: Atom PDF