Project

General

Profile

Bug #14107

librbd: fsx crash during clone

Added by Josh Durgin about 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Urgent
Target version:
-
Start date:
12/17/2015
Due date:
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

From http://teuthology.ovh.sepia.ceph.com/teuthology/teuthology-2015-12-16_23:00:03-rbd-jewel-distro-basic-openstack/40699/teuthology.log

2015-12-17T11:05:11.577 INFO:teuthology.orchestra.run.target079054:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 5000 pool_client.0 image_client.0'
2015-12-17T11:05:11.868 INFO:teuthology.orchestra.run.target079054.stdout:mapped writes DISABLED
2015-12-17T11:05:11.869 INFO:teuthology.orchestra.run.target079054.stdout:mapped reads DISABLED
2015-12-17T11:05:11.869 INFO:teuthology.orchestra.run.target079054.stdout:Seed set to 311
2015-12-17T11:05:17.828 INFO:teuthology.orchestra.run.target079054.stdout:1 write    0x50bfeeb thru    0x50c1c2e    (0x1d44 bytes)
2015-12-17T11:05:17.839 INFO:teuthology.orchestra.run.target079054.stdout:3 punch    from 0x2c07a91 to 0x2c0b337, (0x38a6 bytes)
2015-12-17T11:05:17.840 INFO:teuthology.orchestra.run.target079054.stdout:truncating to largest ever: 0xccbf796
2015-12-17T11:05:17.859 INFO:teuthology.orchestra.run.target079054.stdout:4 trunc    from 0x50c1c2f to 0xccbf796
2015-12-17T11:05:17.888 INFO:teuthology.orchestra.run.target079054.stdout:5 trunc    from 0xccbf796 to 0xaf63f1b
2015-12-17T11:05:17.926 INFO:teuthology.orchestra.run.target079054.stdout:7 read    0x1b61455 thru    0x1b6ade9    (0x9995 bytes)
2015-12-17T11:05:17.942 INFO:teuthology.orchestra.run.target079054.stdout:8 trunc    from 0xaf63f1b to 0x521bc4f
2015-12-17T11:05:17.977 INFO:teuthology.orchestra.run.target079054.stdout:9 read    0x58b65e thru    0x58d7ca    (0x216d bytes)
2015-12-17T11:05:17.979 INFO:teuthology.orchestra.run.target079054.stdout:11 read    0x2518619 thru    0x2524471    (0xbe59 bytes)
2015-12-17T11:05:17.980 INFO:teuthology.orchestra.run.target079054.stdout:13 write    0x5f6b61 thru    0x5fc3ef    (0x588f bytes)
2015-12-17T11:05:17.988 INFO:teuthology.orchestra.run.target079054.stdout:15 read    0x2d4a5ff thru    0x2d51d99    (0x779b bytes)
2015-12-17T11:05:17.990 INFO:teuthology.orchestra.run.target079054.stdout:16 read    0x49381c6 thru    0x4943bb0    (0xb9eb bytes)
2015-12-17T11:05:18.014 INFO:teuthology.orchestra.run.target079054.stdout:17 write    0x666e004 thru    0x6677c41    (0x9c3e bytes)
2015-12-17T11:05:18.065 INFO:teuthology.orchestra.run.target079054.stdout:18 write    0xbfae924 thru    0xbfb8f7b    (0xa658 bytes)
2015-12-17T11:05:18.109 INFO:teuthology.orchestra.run.target079054.stdout:20 trunc    from 0xbfb8f7c to 0x1c78ef
2015-12-17T11:05:18.155 INFO:teuthology.orchestra.run.target079054.stdout:23 punch    from 0x9ec39 to 0xad975, (0xed3c bytes)
2015-12-17T11:05:18.158 INFO:teuthology.orchestra.run.target079054.stdout:24 read    0x10883d thru    0x10e5e4    (0x5da8 bytes)
2015-12-17T11:05:18.159 INFO:teuthology.orchestra.run.target079054.stdout:26 read    0x3a100 thru    0x3abaa    (0xaab bytes)
2015-12-17T11:05:18.177 INFO:teuthology.orchestra.run.target079054.stdout:27 trunc    from 0x1c78ef to 0x7df1698
2015-12-17T11:05:18.196 INFO:teuthology.orchestra.run.target079054.stdout:28 punch    from 0x1d44662 to 0x1d4bb16, (0x74b4 bytes)
2015-12-17T11:05:18.209 INFO:teuthology.orchestra.run.target079054.stdout:29 trunc    from 0x7df1698 to 0x4366d2f
2015-12-17T11:05:18.289 INFO:teuthology.orchestra.run.target079054.stdout:30 write    0xde6120d thru    0xde6bf58    (0xad4c bytes)
2015-12-17T11:05:18.303 INFO:teuthology.orchestra.run.target079054.stdout:31 write    0xd272fc8 thru    0xd277270    (0x42a9 bytes)
2015-12-17T11:05:18.338 INFO:teuthology.orchestra.run.target079054.stdout:32 write    0xe75214f thru    0xe759b1d    (0x79cf bytes)
2015-12-17T11:05:18.347 INFO:teuthology.orchestra.run.target079054.stdout:33 write    0xd32a0ba thru    0xd32ba39    (0x1980 bytes)
2015-12-17T11:05:18.353 INFO:teuthology.orchestra.run.target079054.stdout:35 punch    from 0xc658ce0 to 0xc65e097, (0x53b7 bytes)
2015-12-17T11:05:18.381 INFO:teuthology.orchestra.run.target079054.stdout:36 trunc    from 0xe759b1e to 0x4b95cfc
2015-12-17T11:05:18.430 INFO:teuthology.orchestra.run.target079054.stdout:38 trunc    from 0x4b95cfc to 0x4ab698c
2015-12-17T11:05:18.452 INFO:teuthology.orchestra.run.target079054.stdout:43 read    0x3aaf39 thru    0x3ae95d    (0x3a25 bytes)
2015-12-17T11:05:18.499 INFO:teuthology.orchestra.run.target079054.stdout:44 write    0xd690b28 thru    0xd697e5d    (0x7336 bytes)
2015-12-17T11:05:18.511 INFO:teuthology.orchestra.run.target079054.stdout:45 read    0xc23fbad thru    0xc24e87e    (0xecd2 bytes)
2015-12-17T11:05:18.511 INFO:teuthology.orchestra.run.target079054.stdout:46 punch    from 0x4cb40dd to 0x4cb48ce, (0x7f1 bytes)
2015-12-17T11:05:18.525 INFO:teuthology.orchestra.run.target079054.stdout:48 trunc    from 0xd697e5e to 0x98b77e4
2015-12-17T11:05:18.558 INFO:teuthology.orchestra.run.target079054.stdout:49 write    0x8393710 thru    0x839a890    (0x7181 bytes)
2015-12-17T11:05:18.579 INFO:teuthology.orchestra.run.target079054.stdout:50 write    0x3f053bc thru    0x3f0f2ee    (0x9f33 bytes)
2015-12-17T11:05:18.593 INFO:teuthology.orchestra.run.target079054.stdout:51 read    0x8cb3cff thru    0x8cbb26e    (0x7570 bytes)
2015-12-17T11:05:18.595 INFO:teuthology.orchestra.run.target079054.stdout:55 read    0xd166fb thru    0xd1cb6b    (0x6471 bytes)
2015-12-17T11:05:18.598 INFO:teuthology.orchestra.run.target079054.stdout:57 punch    from 0x963225f to 0x9632de1, (0xb82 bytes)
2015-12-17T11:05:18.599 INFO:teuthology.orchestra.run.target079054.stdout:59 read    0xc30527 thru    0xc34c71    (0x474b bytes)
2015-12-17T11:05:18.600 INFO:teuthology.orchestra.run.target079054.stdout:61 write    0x4d77619 thru    0x4d86914    (0xf2fc bytes)
2015-12-17T11:05:18.623 INFO:teuthology.orchestra.run.target079054.stdout:62 punch    from 0x29c4378 to 0x29d0035, (0xbcbd bytes)
2015-12-17T11:05:18.625 INFO:teuthology.orchestra.run.target079054.stdout:64 write    0x56f53e9 thru    0x56fe573    (0x918b bytes)
2015-12-17T11:05:18.642 INFO:teuthology.orchestra.run.target079054.stdout:65 read    0x876c1e6 thru    0x87797a5    (0xd5c0 bytes)
2015-12-17T11:05:18.651 INFO:teuthology.orchestra.run.target079054.stdout:66 trunc    from 0x98b77e4 to 0x7a8d7c8
2015-12-17T11:05:18.681 INFO:teuthology.orchestra.run.target079054.stdout:68 read    0x2d88c02 thru    0x2d8dace    (0x4ecd bytes)
2015-12-17T11:05:18.682 INFO:teuthology.orchestra.run.target079054.stdout:truncating to largest ever: 0xebbd61c
2015-12-17T11:05:18.701 INFO:teuthology.orchestra.run.target079054.stdout:69 trunc    from 0x7a8d7c8 to 0xebbd61c
2015-12-17T11:05:18.755 INFO:teuthology.orchestra.run.target079054.stdout:70 trunc    from 0xebbd61c to 0x86df61b
2015-12-17T11:05:18.827 INFO:teuthology.orchestra.run.target079054.stdout:71 trunc    from 0x86df61b to 0x49fd31f
2015-12-17T11:05:18.879 INFO:teuthology.orchestra.run.target079054.stdout:72 punch    from 0x45c9c79 to 0x45d7a66, (0xdded bytes)
2015-12-17T11:05:18.889 INFO:teuthology.orchestra.run.target079054.stdout:75 trunc    from 0x49fd31f to 0x1e065b2
2015-12-17T11:05:18.960 INFO:teuthology.orchestra.run.target079054.stdout:76 write    0xcab04e3 thru    0xcabaee9    (0xaa07 bytes)
2015-12-17T11:05:18.972 INFO:teuthology.orchestra.run.target079054.stdout:77 write    0x5b2f627 thru    0x5b32de5    (0x37bf bytes)
2015-12-17T11:05:18.981 INFO:teuthology.orchestra.run.target079054.stdout:78 write    0x8caf1a8 thru    0x8cb3e18    (0x4c71 bytes)
2015-12-17T11:05:18.993 INFO:teuthology.orchestra.run.target079054.stdout:79 punch    from 0x1957b96 to 0x195a1d0, (0x263a bytes)
2015-12-17T11:05:18.994 INFO:teuthology.orchestra.run.target079054.stdout:80 read    0x15efa80 thru    0x15fe420    (0xe9a1 bytes)
2015-12-17T11:05:18.995 INFO:teuthology.orchestra.run.target079054.stdout:82 read    0x83e8b7e thru    0x83eeb9c    (0x601f bytes)
2015-12-17T11:05:19.004 INFO:teuthology.orchestra.run.target079054.stdout:83 trunc    from 0xcabaeea to 0xa150119
2015-12-17T11:05:19.036 INFO:teuthology.orchestra.run.target079054.stdout:84 read    0x57ad6b2 thru    0x57b9fd8    (0xc927 bytes)
2015-12-17T11:05:19.037 INFO:teuthology.orchestra.run.target079054.stdout:85 punch    from 0x57fd27c to 0x58052d2, (0x8056 bytes)
2015-12-17T11:05:19.053 INFO:teuthology.orchestra.run.target079054.stdout:86 trunc    from 0xa150119 to 0xd881c09
2015-12-17T11:05:19.075 INFO:teuthology.orchestra.run.target079054.stdout:89 punch    from 0x2e96617 to 0x2ea2a42, (0xc42b bytes)
2015-12-17T11:05:19.076 INFO:teuthology.orchestra.run.target079054.stdout:90 read    0x8fa6f1b thru    0x8fb5da3    (0xee89 bytes)
2015-12-17T11:05:19.079 INFO:teuthology.orchestra.run.target079054.stdout:93 punch    from 0x99977c1 to 0x999f7fe, (0x803d bytes)
2015-12-17T11:05:19.080 INFO:teuthology.orchestra.run.target079054.stdout:94 read    0x1f7bc08 thru    0x1f81a9e    (0x5e97 bytes)
2015-12-17T11:05:19.109 INFO:teuthology.orchestra.run.target079054.stdout:95 trunc    from 0xd881c09 to 0x250ce0e
2015-12-17T11:05:19.180 INFO:teuthology.orchestra.run.target079054.stdout:96 write    0x5be37fa thru    0x5bf0cc0    (0xd4c7 bytes)
2015-12-17T11:05:19.214 INFO:teuthology.orchestra.run.target079054.stdout:98 trunc    from 0x5bf0cc1 to 0xe312576
2015-12-17T11:05:19.234 INFO:teuthology.orchestra.run.target079054.stdout:104 punch    from 0x674af8b to 0x6750600, (0x5675 bytes)
2015-12-17T11:05:19.236 INFO:teuthology.orchestra.run.target079054.stdout:105 clone    1 order 18 su 16384 sc 2
2015-12-17T11:05:21.677 ERROR:teuthology.parallel:Exception in parallel execution
Traceback (most recent call last):
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 82, in __exit__
    for result in self:
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 101, in next
    resurrect_traceback(result)
  File "/home/teuthworker/src/teuthology_master/teuthology/parallel.py", line 19, in capture_traceback
    return func(*args, **kwargs)
  File "/home/teuthworker/src/ceph-qa-suite_jewel/tasks/rbd_fsx.py", line 93, in _run_one_client
    remote.run(args=args)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/remote.py", line 156, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 378, in run
    r.wait()
  File "/home/teuthworker/src/teuthology_master/teuthology/orchestra/run.py", line 110, in wait
    raise CommandCrashedError(command=self.command)
CommandCrashedError: Command crashed: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 5000 pool_client.0 image_client.0'

Associated revisions

Revision bab16bbb (diff)
Added by Jason Dillaman about 3 years ago

librbd: stop the copyup thread during shutdown

Fixes: #14107, #14108
Signed-off-by: Jason Dillaman <>

History

#1 Updated by Jason Dillaman about 3 years ago

  • Status changed from New to In Progress
  • Assignee set to Jason Dillaman

#2 Updated by Jason Dillaman about 3 years ago

  • Status changed from In Progress to Need Review

#3 Updated by Jason Dillaman about 3 years ago

  • Status changed from Need Review to Resolved

Also available in: Atom PDF