Project

General

Profile

Actions

Bug #46455

open

Attempting to sync bucket results in segfault

Added by Jonathon Beauregard almost 4 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
ceph-ansible
Pull request ID:
Crash signature (v1):

167a6013664efdcf74a07a0cf91dc5dd91c6a613cdac6f7b0a8acff6ce29812e

Crash signature (v2):

Description

Attempting to sync a bucket to a cloud zone results in segfault

[root@ovbh-vtest-cephmon01 fpcuser]# ceph crash info 2020-07-10T15:40:51.138417Z_69e749ac-6dc3-499c-96f8-7e949a943c69 {
"backtrace": [
"(()+0xf630) [0x7f7cd4ffa630]",
"(rgw_sync_bucket_entities::set_bucket(std::optional<std::string>, std::optional<std::string>, std::optional<std::string>)+0x9f) [0x55b93f8bc6cf]",
"(main()+0x192cf) [0x55b93f64bb9f]",
"(__libc_start_main()+0xf5) [0x7f7cd2ee5555]",
"(()+0x27cb50) [0x55b93f678b50]"
],
"ceph_version": "15.2.4",
"crash_id": "2020-07-10T15:40:51.138417Z_69e749ac-6dc3-499c-96f8-7e949a943c69",
"entity_name": "client.admin",
"os_id": "centos",
"os_name": "CentOS Linux",
"os_version": "7 (Core)",
"os_version_id": "7",
"process_name": "radosgw-admin",
"stack_sig": "167a6013664efdcf74a07a0cf91dc5dd91c6a613cdac6f7b0a8acff6ce29812e",
"timestamp": "2020-07-10T15:40:51.138417Z",
"utsname_hostname": "ovbh-vtest-cephmon01.example.com",
"utsname_machine": "x86_64",
"utsname_release": "3.10.0-1127.13.1.el7.x86_64",
"utsname_sysname": "Linux",
"utsname_version": "#1 SMP Tue Jun 23 15:46:38 UTC 2020"
}

Additional information. Full logs attached.

[root@ovbh-vtest-cephmon01 fpcuser]# ceph -v
ceph version 15.2.4 (7447c15c6ff58d7fce91843b705a268a1917325c) octopus (stable)

[root@ovbh-vtest-cephmon01 fpcuser]# radosgw-admin sync group get --rgw-zone=default
[ {
"key": "group1",
"val": {
"id": "group1",
"data_flow": {
"directional": [ {
"source_zone": "default",
"dest_zone": "ceph-s3"
}
]
},
"pipes": [],
"status": "allowed"
}
}
]

[root@ovbh-vtest-cephmon01 fpcuser]# radosgw-admin zone get --rgw-zone=default {
"id": "231ef5f2-0a08-4a46-a7e7-7d42391ee29d",
"name": "default",
"domain_root": "default.rgw.meta:root",
"control_pool": "default.rgw.control",
"gc_pool": "default.rgw.log:gc",
"lc_pool": "default.rgw.log:lc",
"log_pool": "default.rgw.log",
"intent_log_pool": "default.rgw.log:intent",
"usage_log_pool": "default.rgw.log:usage",
"roles_pool": "default.rgw.meta:roles",
"reshard_pool": "default.rgw.log:reshard",
"user_keys_pool": "default.rgw.meta:users.keys",
"user_email_pool": "default.rgw.meta:users.email",
"user_swift_pool": "default.rgw.meta:users.swift",
"user_uid_pool": "default.rgw.meta:users.uid",
"otp_pool": "default.rgw.otp",
"system_key": {
"access_key": "XXXX",
"secret_key": "XXXX"
},
"placement_pools": [ {
"key": "default-placement",
"val": {
"index_pool": "default.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "default.rgw.buckets.data"
}
},
"data_extra_pool": "default.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "8eb52879-ee78-45bb-ae72-a1de718bac81"
}

[root@ovbh-vtest-cephmon01 fpcuser]# radosgw-admin zone get --rgw-zone=ceph-s3 {
"id": "8f7b83e6-5bcd-4e00-9d2c-38c7fb255cc4",
"name": "ceph-s3",
"domain_root": "ceph-s3.rgw.meta:root",
"control_pool": "ceph-s3.rgw.control",
"gc_pool": "ceph-s3.rgw.log:gc",
"lc_pool": "ceph-s3.rgw.log:lc",
"log_pool": "ceph-s3.rgw.log",
"intent_log_pool": "ceph-s3.rgw.log:intent",
"usage_log_pool": "ceph-s3.rgw.log:usage",
"roles_pool": "ceph-s3.rgw.meta:roles",
"reshard_pool": "ceph-s3.rgw.log:reshard",
"user_keys_pool": "ceph-s3.rgw.meta:users.keys",
"user_email_pool": "ceph-s3.rgw.meta:users.email",
"user_swift_pool": "ceph-s3.rgw.meta:users.swift",
"user_uid_pool": "ceph-s3.rgw.meta:users.uid",
"otp_pool": "ceph-s3.rgw.otp",
"system_key": {
"access_key": "XXXX",
"secret_key": "XXXX"
},
"placement_pools": [ {
"key": "default-placement",
"val": {
"index_pool": "ceph-s3.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "ceph-s3.rgw.buckets.data"
}
},
"data_extra_pool": "ceph-s3.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"tier_config": {
"connection": {
"access_key": "XXXX",
"endpoint": "https://s3.wasabisys.com",
"secret": "XXXX"
},
"target_path": "ceph-s3-bucket/${bucket}"
},
"realm_id": "8eb52879-ee78-45bb-ae72-a1de718bac81"
}


Files

full_debug_stackdump2.txt (449 KB) full_debug_stackdump2.txt Jonathon Beauregard, 07/10/2020 04:10 PM
Actions #1

Updated by Jonathon Beauregard almost 4 years ago

command that results in segfault

radosgw-admin sync group pipe create --bucket='test2' --group-id=anothertest --pipe-id=pipe2 --source-zones='default' --dest-zones='ceph-s3'

Actions #2

Updated by Greg Farnum almost 3 years ago

  • Project changed from Ceph to rgw
Actions #3

Updated by Yaarit Hatuka over 2 years ago

  • Crash signature (v1) updated (diff)

Moving the crash data into a note as 'Crash signature (v1)' should hold the value of 'stack_sig' key:

 "backtrace": [
        "(()+0xf630) [0x7f7cd4ffa630]",
        "(rgw_sync_bucket_entities::set_bucket(std::optional<std::string>, std::optional<std::string>, std::optional<std::string>)+0x9f) [0x55b93f8bc6cf]",
        "(main()+0x192cf) [0x55b93f64bb9f]",
        "(__libc_start_main()+0xf5) [0x7f7cd2ee5555]",
        "(()+0x27cb50) [0x55b93f678b50]" 
    ],
    "ceph_version": "15.2.4",
    "crash_id": "2020-07-10T15:40:51.138417Z_69e749ac-6dc3-499c-96f8-7e949a943c69",
    "entity_name": "client.admin",
    "os_id": "centos",
    "os_name": "CentOS Linux",
    "os_version": "7 (Core)",
    "os_version_id": "7",
    "process_name": "radosgw-admin",
    "stack_sig": "167a6013664efdcf74a07a0cf91dc5dd91c6a613cdac6f7b0a8acff6ce29812e",
    "timestamp": "2020-07-10T15:40:51.138417Z",
    "utsname_hostname": "ovbh-vtest-cephmon01.example.com",
    "utsname_machine": "x86_64",
    "utsname_release": "3.10.0-1127.13.1.el7.x86_64",
    "utsname_sysname": "Linux",
    "utsname_version": "#1 SMP Tue Jun 23 15:46:38 UTC 2020" 
Actions

Also available in: Atom PDF