Project

General

Profile

Actions
Copy link

Bug #43274

open

unittest_rbd_mirror: Exception: SegFault

Added by Sebastian Wagner over 4 years ago. Updated over 2 years ago.

Status:
Need More Info
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Unfortunately, I don't know what exactly went wrong:

185/191 Test #184: unittest_rbd_mirror .....................***Exception: SegFault 11.74 sec
[==========] Running 279 tests from 34 test suites.
[----------] Global test environment set-up.
[----------] 13 tests from TestMockImageMap
[ RUN      ] TestMockImageMap.SetLocalImages
seed 1526
[       OK ] TestMockImageMap.SetLocalImages (8 ms)
[ RUN      ] TestMockImageMap.AddRemoveLocalImage
[       OK ] TestMockImageMap.AddRemoveLocalImage (25 ms)
[ RUN      ] TestMockImageMap.AddRemoveRemoteImage
[       OK ] TestMockImageMap.AddRemoveRemoteImage (15 ms)
[ RUN      ] TestMockImageMap.AddRemoveRemoteImageDuplicateNotification
[       OK ] TestMockImageMap.AddRemoveRemoteImageDuplicateNotification (5 ms)
[ RUN      ] TestMockImageMap.AcquireImageErrorRetry
[       OK ] TestMockImageMap.AcquireImageErrorRetry (2 ms)
[ RUN      ] TestMockImageMap.RemoveRemoteAndLocalImage
[       OK ] TestMockImageMap.RemoveRemoteAndLocalImage (2 ms)
[ RUN      ] TestMockImageMap.AddInstance
[       OK ] TestMockImageMap.AddInstance (4 ms)
[ RUN      ] TestMockImageMap.RemoveInstance
[       OK ] TestMockImageMap.RemoveInstance (7 ms)
[ RUN      ] TestMockImageMap.AddInstancePingPongImageTest
[       OK ] TestMockImageMap.AddInstancePingPongImageTest (34 ms)
[ RUN      ] TestMockImageMap.RemoveInstanceWithRemoveImage
[       OK ] TestMockImageMap.RemoveInstanceWithRemoveImage (23 ms)
[ RUN      ] TestMockImageMap.AddErrorAndRemoveImage
[       OK ] TestMockImageMap.AddErrorAndRemoveImage (35 ms)
[ RUN      ] TestMockImageMap.MirrorUUIDUpdated
[       OK ] TestMockImageMap.MirrorUUIDUpdated (44 ms)
[ RUN      ] TestMockImageMap.RebalanceImageMap
[       OK ] TestMockImageMap.RebalanceImageMap (40 ms)
[----------] 13 tests from TestMockImageMap (244 ms total)

[----------] 14 tests from TestMockImageReplayer
[ RUN      ] TestMockImageReplayer.StartStop
Failed to load class: cas (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_cas.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_cas.so: undefined symbol: _Z13cls_has_chunkPvNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
Failed to load class: log (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_log.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_log.so: undefined symbol: _Z24cls_cxx_map_write_headerPvPN4ceph6buffer7v14_2_04listE
Failed to load class: rgw (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_rgw.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_rgw.so: undefined symbol: _Z19cls_current_versionPv
Failed to load class: user (/home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_user.so): /home/jenkins-build/build/workspace/ceph-pull-requests/build/lib/libcls_user.so: undefined symbol: _Z24cls_cxx_map_write_headerPvPN4ceph6buffer7v14_2_04listE
[       OK ] TestMockImageReplayer.StartStop (317 ms)
[ RUN      ] TestMockImageReplayer.LocalImagePrimary
[       OK ] TestMockImageReplayer.LocalImagePrimary (146 ms)
[ RUN      ] TestMockImageReplayer.LocalImageDNE
[       OK ] TestMockImageReplayer.LocalImageDNE (196 ms)
[ RUN      ] TestMockImageReplayer.PrepareLocalImageError
[       OK ] TestMockImageReplayer.PrepareLocalImageError (194 ms)
[ RUN      ] TestMockImageReplayer.GetRemoteImageIdDNE
[       OK ] TestMockImageReplayer.GetRemoteImageIdDNE (174 ms)
[ RUN      ] TestMockImageReplayer.GetRemoteImageIdNonLinkedDNE
[       OK ] TestMockImageReplayer.GetRemoteImageIdNonLinkedDNE (224 ms)
[ RUN      ] TestMockImageReplayer.GetRemoteImageIdError
[       OK ] TestMockImageReplayer.GetRemoteImageIdError (228 ms)
[ RUN      ] TestMockImageReplayer.BootstrapError
[       OK ] TestMockImageReplayer.BootstrapError (154 ms)
[ RUN      ] TestMockImageReplayer.StopBeforeBootstrap
[       OK ] TestMockImageReplayer.StopBeforeBootstrap (215 ms)
[ RUN      ] TestMockImageReplayer.StartExternalReplayError
[       OK ] TestMockImageReplayer.StartExternalReplayError (152 ms)
[ RUN      ] TestMockImageReplayer.StopError
[       OK ] TestMockImageReplayer.StopError (169 ms)
[ RUN      ] TestMockImageReplayer.Replay
[       OK ] TestMockImageReplayer.Replay (177 ms)
[ RUN      ] TestMockImageReplayer.DecodeError
[       OK ] TestMockImageReplayer.DecodeError (157 ms)
[ RUN      ] TestMockImageReplayer.DelayedReplay
[       OK ] TestMockImageReplayer.DelayedReplay (2153 ms)
[----------] 14 tests from TestMockImageReplayer (4663 ms total)

[----------] 5 tests from TestMockImageSync
[ RUN      ] TestMockImageSync.SimpleSync
[       OK ] TestMockImageSync.SimpleSync (198 ms)
[ RUN      ] TestMockImageSync.RestartSync
[       OK ] TestMockImageSync.RestartSync (173 ms)
[ RUN      ] TestMockImageSync.CancelNotifySyncRequest
[       OK ] TestMockImageSync.CancelNotifySyncRequest (159 ms)
[ RUN      ] TestMockImageSync.CancelImageCopy
[       OK ] TestMockImageSync.CancelImageCopy (195 ms)
[ RUN      ] TestMockImageSync.CancelAfterCopyImage
[       OK ] TestMockImageSync.CancelAfterCopyImage (166 ms)
[----------] 5 tests from TestMockImageSync (898 ms total)

[----------] 3 tests from TestMockInstanceReplayer
[ RUN      ] TestMockInstanceReplayer.AcquireReleaseImage
[       OK ] TestMockInstanceReplayer.AcquireReleaseImage (16 ms)
[ RUN      ] TestMockInstanceReplayer.RemoveFinishedImage
[       OK ] TestMockInstanceReplayer.RemoveFinishedImage (24 ms)
[ RUN      ] TestMockInstanceReplayer.Reacquire
[       OK ] TestMockInstanceReplayer.Reacquire (2 ms)
[----------] 3 tests from TestMockInstanceReplayer (42 ms total)

[----------] 11 tests from TestMockInstanceWatcher
[ RUN      ] TestMockInstanceWatcher.InitShutdown
[       OK ] TestMockInstanceWatcher.InitShutdown (23 ms)
[ RUN      ] TestMockInstanceWatcher.InitError
[       OK ] TestMockInstanceWatcher.InitError (18 ms)
[ RUN      ] TestMockInstanceWatcher.ShutdownError
[       OK ] TestMockInstanceWatcher.ShutdownError (15 ms)
[ RUN      ] TestMockInstanceWatcher.Remove
[       OK ] TestMockInstanceWatcher.Remove (16 ms)
[ RUN      ] TestMockInstanceWatcher.RemoveNoent
[       OK ] TestMockInstanceWatcher.RemoveNoent (12 ms)
[ RUN      ] TestMockInstanceWatcher.ImageAcquireRelease
[       OK ] TestMockInstanceWatcher.ImageAcquireRelease (36 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageRemoved
[       OK ] TestMockInstanceWatcher.PeerImageRemoved (36 ms)
[ RUN      ] TestMockInstanceWatcher.ImageAcquireReleaseCancel
[       OK ] TestMockInstanceWatcher.ImageAcquireReleaseCancel (31 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageAcquireWatchDNE
[       OK ] TestMockInstanceWatcher.PeerImageAcquireWatchDNE (17 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageReleaseWatchDNE
[       OK ] TestMockInstanceWatcher.PeerImageReleaseWatchDNE (32 ms)
[ RUN      ] TestMockInstanceWatcher.PeerImageRemovedCancel
[       OK ] TestMockInstanceWatcher.PeerImageRemovedCancel (12 ms)
[----------] 11 tests from TestMockInstanceWatcher (250 ms total)

[----------] 11 tests from TestMockInstanceWatcher_NotifySync
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartStopOnLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartStopOnLeader (48 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.CancelStartedOnLeader
[       OK ] TestMockInstanceWatcher_NotifySync.CancelStartedOnLeader (49 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartStopOnNonLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartStopOnNonLeader (36 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.CancelStartedOnNonLeader
[       OK ] TestMockInstanceWatcher_NotifySync.CancelStartedOnNonLeader (41 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.CancelWaitingOnNonLeader
[       OK ] TestMockInstanceWatcher_NotifySync.CancelWaitingOnNonLeader (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.InFlightPrevNotification
[       OK ] TestMockInstanceWatcher_NotifySync.InFlightPrevNotification (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.NoInFlightReleaseAcquireLeader
[       OK ] TestMockInstanceWatcher_NotifySync.NoInFlightReleaseAcquireLeader (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartedOnLeaderReleaseLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartedOnLeaderReleaseLeader (34 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.WaitingOnLeaderReleaseLeader
[       OK ] TestMockInstanceWatcher_NotifySync.WaitingOnLeaderReleaseLeader (46 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.StartedOnNonLeaderAcquireLeader
[       OK ] TestMockInstanceWatcher_NotifySync.StartedOnNonLeaderAcquireLeader (29 ms)
[ RUN      ] TestMockInstanceWatcher_NotifySync.WaitingOnNonLeaderAcquireLeader
[       OK ] TestMockInstanceWatcher_NotifySync.WaitingOnNonLeaderAcquireLeader (34 ms)
[----------] 11 tests from TestMockInstanceWatcher_NotifySync (456 ms total)

[----------] 4 tests from TestMockLeaderWatcher
[ RUN      ] TestMockLeaderWatcher.InitShutdown
[       OK ] TestMockLeaderWatcher.InitShutdown (33 ms)
[ RUN      ] TestMockLeaderWatcher.InitReleaseShutdown
[       OK ] TestMockLeaderWatcher.InitReleaseShutdown (19 ms)
[ RUN      ] TestMockLeaderWatcher.AcquireError
[       OK ] TestMockLeaderWatcher.AcquireError (12 ms)
[ RUN      ] TestMockLeaderWatcher.Break
[       OK ] TestMockLeaderWatcher.Break (2012 ms)
[----------] 4 tests from TestMockLeaderWatcher (2076 ms total)

[----------] 12 tests from TestMockMirrorStatusUpdater
[ RUN      ] TestMockMirrorStatusUpdater.InitShutDown
[       OK ] TestMockMirrorStatusUpdater.InitShutDown (13 ms)
[ RUN      ] TestMockMirrorStatusUpdater.InitStatusWatcherError
[       OK ] TestMockMirrorStatusUpdater.InitStatusWatcherError (26 ms)
[ RUN      ] TestMockMirrorStatusUpdater.ShutDownStatusWatcherError
[       OK ] TestMockMirrorStatusUpdater.ShutDownStatusWatcherError (14 ms)
[ RUN      ] TestMockMirrorStatusUpdater.SmallBatch
[       OK ] TestMockMirrorStatusUpdater.SmallBatch (24 ms)
[ RUN      ] TestMockMirrorStatusUpdater.LargeBatch
[       OK ] TestMockMirrorStatusUpdater.LargeBatch (30 ms)
[ RUN      ] TestMockMirrorStatusUpdater.OverwriteStatus
[       OK ] TestMockMirrorStatusUpdater.OverwriteStatus (11 ms)
[ RUN      ] TestMockMirrorStatusUpdater.OverwriteStatusInFlight
[       OK ] TestMockMirrorStatusUpdater.OverwriteStatusInFlight (7 ms)
[ RUN      ] TestMockMirrorStatusUpdater.ImmediateUpdate
[       OK ] TestMockMirrorStatusUpdater.ImmediateUpdate (9 ms)
[ RUN      ] TestMockMirrorStatusUpdater.RemoveIdleStatus
[       OK ] TestMockMirrorStatusUpdater.RemoveIdleStatus (20 ms)
[ RUN      ] TestMockMirrorStatusUpdater.RemoveInFlightStatus
[       OK ] TestMockMirrorStatusUpdater.RemoveInFlightStatus (9 ms)
[ RUN      ] TestMockMirrorStatusUpdater.ShutDownWhileUpdating
[       OK ] TestMockMirrorStatusUpdater.ShutDownWhileUpdating (14 ms)
[ RUN      ] TestMockMirrorStatusUpdater.MirrorPeerSitePing
[       OK ] TestMockMirrorStatusUpdater.MirrorPeerSitePing (24 ms)
[----------] 12 tests from TestMockMirrorStatusUpdater (201 ms total)

[----------] 6 tests from TestMockNamespaceReplayer
[ RUN      ] TestMockNamespaceReplayer.Init_LocalMirrorStatusUpdaterError
[       OK ] TestMockNamespaceReplayer.Init_LocalMirrorStatusUpdaterError (55 ms)
[ RUN      ] TestMockNamespaceReplayer.Init_RemoteMirrorStatusUpdaterError
[       OK ] TestMockNamespaceReplayer.Init_RemoteMirrorStatusUpdaterError (32 ms)
[ RUN      ] TestMockNamespaceReplayer.Init_InstanceReplayerError
[       OK ] TestMockNamespaceReplayer.Init_InstanceReplayerError (12 ms)
[ RUN      ] TestMockNamespaceReplayer.Init_InstanceWatcherError
[       OK ] TestMockNamespaceReplayer.Init_InstanceWatcherError (20 ms)
[ RUN      ] TestMockNamespaceReplayer.Init
[       OK ] TestMockNamespaceReplayer.Init (16 ms)
[ RUN      ] TestMockNamespaceReplayer.AcuqireLeader
[       OK ] TestMockNamespaceReplayer.AcuqireLeader (9 ms)
[----------] 6 tests from TestMockNamespaceReplayer (144 ms total)

[----------] 4 tests from TestMockPoolReplayer
[ RUN      ] TestMockPoolReplayer.ConfigKeyOverride
[       OK ] TestMockPoolReplayer.ConfigKeyOverride (47 ms)
[ RUN      ] TestMockPoolReplayer.AcquireReleaseLeader
[       OK ] TestMockPoolReplayer.AcquireReleaseLeader (55 ms)
[ RUN      ] TestMockPoolReplayer.Namespaces
[       OK ] TestMockPoolReplayer.Namespaces (2075 ms)
[ RUN      ] TestMockPoolReplayer.NamespacesError

https://jenkins.ceph.com/job/ceph-pull-requests/40443/console

https://github.com/ceph/ceph/pull/32182

Related issues 2 (0 open — 2 closed)

Related to CI - Bug #43566: jenkins: ctest output size for failed test is too shortResolvedDavid Galloway

Actions
Related to rbd - Bug #46679: make check: run-rbd-unit-tests-127.sh: TestLibRBD.QuiesceWatchError: pure virtual method calledResolvedJason Dillaman

Actions
Actions
Copy link
 #1

Updated by Jason Dillaman over 4 years ago

  • Status changed from New to Need More Info
Actions
Copy link
 #2

Updated by Mykola Golub over 4 years ago

Reported about the issue with truncated failed test output to CI project [1].

[1] https://tracker.ceph.com/issues/43566

Actions
Copy link
 #3

Updated by Sebastian Wagner over 3 years ago

  • Related to Bug #43566: jenkins: ctest output size for failed test is too short added
Actions
Copy link
 #4

Updated by Sebastian Wagner over 3 years ago

  • Related to Bug #46679: make check: run-rbd-unit-tests-127.sh: TestLibRBD.QuiesceWatchError: pure virtual method called added
Actions
Copy link
 #5

Updated by Kefu Chai about 3 years ago 

[ RUN      ] TestImageReplayer/4.SnapshotLimit
remote_snap_id=4
local_snap_id=3, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=4, last_copied_object_number=0, snap_seqs={4=18446744073709551614}]
flushed
remote_snap_id=5
local_snap_id=4, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=5, last_copied_object_number=0, snap_seqs={5=18446744073709551614}]
flushed
remote_snap_id=6
local_snap_id=5, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=6, last_copied_object_number=0, snap_seqs={6=18446744073709551614}]

....
[ RUN      ] TestImageReplayer/4.SnapshotLimit
remote_snap_id=4
local_snap_id=3, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=4, last_copied_object_number=0, snap_seqs={4=18446744073709551614}]
flushed
remote_snap_id=5
local_snap_id=4, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=5, last_copied_object_number=0, snap_seqs={5=18446744073709551614}]
flushed
remote_snap_id=6
local_snap_id=5, local_snap_ns=[mirror state=non-primary, complete=1, mirror_peer_uuids=, primary_mirror_uuid=e18aeb26-ab6d-4b25-81af-6b6ffc6fa059, primary_snap_id=6, last_copied_object_number=0, snap_seqs={6=18446744073709551614}]
...
The following tests FAILED:
    205 - unittest_rbd_mirror (SEGFAULT)
Actions
Copy link
 #6

Updated by Deepika Upadhyay over 2 years ago 

 [----------] 4 tests from TestMockPoolReplayer
  3287 [ RUN      ] TestMockPoolReplayer.ConfigKeyOverride
  3288 terminate called after throwing an instance of 'std::out_of_range'
  3289   what():  bitset::test: __position (which is 7955819) >= _Nb (which is 131072)

locally rep

Actions
Copy link

Also available in: Atom PDF