Project

General

Profile

Actions
Copy link

Bug #3491

closed

test_librbd_fsx: too many open files

Added by Sage Weil over 11 years ago. Updated over 11 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Samuel Just
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

this is probably something runaway in the msgr, but:

ubuntu@teuthology:/a/teuthology-2012-11-13_19:00:04-regression-next-testing-basic/15008$ cat config.yaml 
kernel: &id001
  kdb: true
  sha1: 22cddde104d715600a4c218bf9224923208afe90
nuke-on-error: true
overrides:
  ceph:
    conf:
      global:
        ms inject socket failures: 5000
    fs: btrfs
    log-whitelist:
    - slow request
    sha1: 7926ef53935313501d4a7fe0e587f3e3b00b313c
  s3tests:
    branch: next
  workunit:
    sha1: 7926ef53935313501d4a7fe0e587f3e3b00b313c
roles:
- - mon.a
  - mon.c
  - osd.0
  - osd.1
  - osd.2
- - mon.b
  - mds.a
  - osd.3
  - osd.4
  - osd.5
- - client.0
targets:
  ubuntu@plana42.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzQfmtpfECJ+NZaaiSH/R8X+dGXHH+aDTCKGLLiHhW9fttxzfzcJJaBx1b664D3ynZAC7NiaegfLDTCMW7FFVDUltMQcWjsM4BqfFipIquDP4KOclCc6EwG5aYG/MLCJwL6sovt1uKg00bSkVQsUSHBgZbMJKCjCbBb0XPxfuS4dppA3diEZBOMt1YHr+NdV7sace/Gc7YBlGsNOinnqkKfVWIpfYCiTQ18cvaisSEHsQR6zhKqrX4afQk13cTjdvZeQp9AXxRIf1g9fq2zHVWMdJdVNR8D0BSBtfAzMqIqZ8qcJqmzQN0Zq9Wk9Y021vMFORZy2SFI6c7yBWDJLdT
  ubuntu@plana44.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYE0eu9E8TQwtUy89Wldp54VbNBEoO9XQf77eXXzzmNwYUFRrNX0mZV/I8GqyRJuMrPG8V4aZBthBHTtnEmQ6RAS7fVdthi/hEgwnM9cAqY3KX9mR5xJnHBc/fa5KLrnSr3Wrztf42PpQNEN5Tk55K6wWUlZOTHU3vE0j3kF+YQ5FeBhQbghztHPKFR8bOmZJp9TpbXgbvEM2RWr9bYtro1KuQOgrairyVVNWdAuwZuxSQT4soyHoSkY9JmeXKsNRAOamxH9w57mDC3PXui7r6Fp8OCWSK+GmlLTtPaZtulSCcucaZtpVae7F4s9JNxaRl5RxuUtwMRfgAHGlL2BZv
  ubuntu@plana55.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdrzGTR0Fbl6sedYlwlX+FlmF6fuE3l/RTu2kzOkmG47rPEn5CI37Injb7Epc50RXCbUIfzmDqtEY6uZT3YssYrE4jvhQlynPndbn1KmiTbgxTyuumGXv7O4OOntezighA1W49phUNZys1DhdEEO8VSQAIdHrBgBLhY9DDgC4LAhrP4BSbDTN0rUXtYYHBj4aa3sJV0o3sKjpsyjjlieEQnto6JkjK6EGZCSuY+AyMZyLJjFTgMwJ9i4aC5eZoWZAWSDfDsxo8PtFR+kjUmz5uiheyn5lAzKBxmd4ZNojf7wOhSGia0ghbtUeQkdoRZXZhP2ourNn3uAguf1xt43kX
task:
- ceph:
    conf:
      client:
        rbd cache: true
        rbd cache max dirty: 0
tasks:
- internal.lock_machines: 3
- internal.save_config: null
- internal.check_lock: null
- internal.connect: null
- internal.check_conflict: null
- kernel: *id001
- internal.base: null
- internal.archive: null
- internal.coredump: null
- internal.syslog: null
- internal.timer: null
- chef: null
- clock: null
- ceph: null
- rbd_fsx:
    clients:
    - client.0
    ops: 5000

tail of log:
2012-11-14 09:10:46.378617 7f4ccbc81700 -1 -- 10.214.132.23:0/1030198 >> 10.214.132.34:6803/21803 pipe(0x7f4f100f7920 sd=-1 :0 pgs=0 cs=0 l=1).connect couldn't created socket Too many open files
2012-11-14 09:10:46.378640 7f4ccb77c700 -1 -- 10.214.132.23:0/1030198 >> 10.214.132.34:6806/21804 pipe(0x7f4f103151a0 sd=-1 :0 pgs=0 cs=0 l=1).connect couldn't created socket Too many open files
2012-11-14 09:10:46.378647 7f4ccb87d700 -1 -- 10.214.132.23:0/1030198 >> 10.214.132.36:6800/10787 pipe(0x7f4f10064f90 sd=-1 :0 pgs=0 cs=0 l=1).connect couldn't created socket Too many open files
2012-11-14 09:10:47.088595 7f4ccb67b700 -1 -- 10.214.132.23:0/1030198 >> 10.214.132.34:6789/0 pipe(0x7f4f0c0aa300 sd=-1 :0 pgs=0 cs=0 l=1).connect couldn't created socket Too many open files
2012-11-14 09:10:47.088642 7f4ccb67b700  0 -- 10.214.132.23:0/1030198 >> 10.214.132.34:6789/0 pipe(0x7f4f0c0aa300 sd=-1 :0 pgs=0 cs=0 l=1).fault
2012-11-14 09:10:47.088660 7f4ccb67b700 -1 -- 10.214.132.23:0/1030198 >> 10.214.132.34:6789/0 pipe(0x7f4f0c0aa300 sd=-1 :0 pgs=0 cs=0 l=1).connect couldn't created socket Too many open files
2012-11-14 09:10:47.288840 7f4ccb67b700 -1 -- 10.214.132.23:0/1030198 >> 10.214.132.34:6789/0 pipe(0x7f4f0c0aa300 sd=-1 :0 pgs=0 cs=0 l=1).connect couldn't created socket Too many open files
2012-11-14 09:10:47.689041 7f4ccb67b700 -1 -- 10.214.132.23:0/1030198 >> 10.214.132.34:6789/0 pipe(0x7f4f0c0aa300 sd=-1 :0 pgs=0 cs=0 l=1).connect couldn't created socket Too many open files
Actions
Copy link

Also available in: Atom PDF