Project

General

Profile

Bug #39713

"Ceph -s" execution consumes too much time

Added by Lei Liu 3 months ago. Updated about 1 month ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
Start date:
05/13/2019
Due date:
% Done:

0%

Source:
Tags:
ceph cli
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

ceph version: 14.2.0 update from luminous

  1. strace -c ceph -s
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 99.91    0.016000          74       216           select
  0.09    0.000015           0       385           read
  0.00    0.000000           0        19           write
  0.00    0.000000           0      1151       948 open
  0.00    0.000000           0       207           close
  0.00    0.000000           0       301       250 stat
  0.00    0.000000           0       348           fstat
  0.00    0.000000           0         6           lstat
  0.00    0.000000           0        62        58 lseek
  0.00    0.000000           0       138           mmap
  0.00    0.000000           0        70           mprotect
  0.00    0.000000           0        30           munmap
  0.00    0.000000           0        57           brk
  0.00    0.000000           0        68           rt_sigaction
  0.00    0.000000           0         1           rt_sigprocmask
  0.00    0.000000           0         6         1 ioctl
  0.00    0.000000           0        28        28 access
  0.00    0.000000           0         2           pipe
  0.00    0.000000           0         1           mremap
  0.00    0.000000           0         7           clone
  0.00    0.000000           0         1           execve
  0.00    0.000000           0         1           wait4
  0.00    0.000000           0         1           uname
  0.00    0.000000           0        11           fcntl
  0.00    0.000000           0         4           getdents
  0.00    0.000000           0         2         2 readlink
  0.00    0.000000           0         2           getrlimit
  0.00    0.000000           0         1           sysinfo
  0.00    0.000000           0         1           getuid
  0.00    0.000000           0         1           getgid
  0.00    0.000000           0         1           geteuid
  0.00    0.000000           0         1           getegid
  0.00    0.000000           0         1         1 statfs
  0.00    0.000000           0         1           arch_prctl
  0.00    0.000000           0        25         8 futex
  0.00    0.000000           0         1           set_tid_address
  0.00    0.000000           0         1           set_robust_list
  0.00    0.000000           0         1           membarrier
------ ----------- ----------- --------- --------- ----------------
100.00    0.016015                  3160      1296 total
mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f2fd4de4000
mprotect(0x7f2fd4de4000, 4096, PROT_NONE) = 0
clone(child_stack=0x7f2fd55e3ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f2fd55e49d0, tls=0x7f2fd55e4700, child_tidptr=0x7f2fd55e49d0) = 3768757
futex(0x14e4870, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff) = 0
futex(0x143e760, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff) = 0
select(0, NULL, NULL, NULL, {0, 1000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 2000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 4000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 8000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 32000}) = 0 (Timeout)
clone(child_stack=0x7f2fd55e3ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f2fd55e49d0, tls=0x7f2fd55e4700, child_tidptr=0x7f2fd55e49d0) = 3768759
futex(0x143e760, FUTEX_WAKE_PRIVATE, 1) = 0
futex(0x14907c0, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x143e760, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x143e760, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff) = 0
clone(child_stack=0x7f2fd55e3ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f2fd55e49d0, tls=0x7f2fd55e4700, child_tidptr=0x7f2fd55e49d0) = 3768760
futex(0x143e760, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x143e760, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff) = -1 EAGAIN (Resource temporarily unavailable)
select(0, NULL, NULL, NULL, {0, 1000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 2000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 4000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 8000})  = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 32000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)

lots of Timeout message

What system parameter settings are unreasonable for ceph 14.2.0 ?

History

#1 Updated by Casey Bodley 3 months ago

  • Project changed from rgw to Ceph

#2 Updated by Nathan Fish about 2 months ago

Has there been any progress on this? If not, can it be moved to 14.2.3? This is one of 2 issues blocking 14.2.2 and there are a lot of important bugs fixed in 14.2.2.

#3 Updated by Lei Liu about 2 months ago

Nathan Fish wrote:

Has there been any progress on this? If not, can it be moved to 14.2.3? This is one of 2 issues blocking 14.2.2 and there are a lot of important bugs fixed in 14.2.2.

Hi Nathan,

The problem still exists, cluster ceph version 14.2.1 (d555a9489eb35f84f2e1ef49b77e19da9d113972) nautilus (stable), I checked the official website of ceph. The latest release version is 14.2.1. I have not found the updated release version. What am I missing?

Thanks for your reply.

#4 Updated by Nathan Fish about 1 month ago

14.2.2 has indeed not been released yet, that's the problem. This is one of two issues which are listed as blocking the release of 14.2.2:

https://tracker.ceph.com/versions/574

14.2.2 has multiple important bug fixes which I would like to put into production soon. So I would like to know if this issue is nearly done, or if it can be moved to 14.2.3 so 14.2.2 can ship.

#5 Updated by Lei Liu about 1 month ago

Of course, but I don't have no permission to modify, I think this problem may be caused by the operating system settings, currently not resolved, the administrator can close this issue.

Will test while v14.2.2 released.

Also available in: Atom PDF