Project

General

Profile

Bug #15935

Actvate OSD fail due to the fsid mismatched

Added by Anonymous almost 8 years ago. Updated about 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature (v1):
Crash signature (v2):

Description

The ceph enviornment: 1 ceph deploy node, 1 ceph monitor nod, 3 ceph storage nodes.
OS: centos 7
Ceph: Jewel 10.2.1
It is fresh installation. After the first installation, all the OSD nodes are in "down" status.
Perform the "purge", "purgedata" and "forgetkeys" operations, then re-install ceph to all the nodes.
When try to activate the OSD node with command "ceph-deploy osd activate centos3:/var/local/osd0 centos4:/var/local/osd1 centos5:/var/local/osd2", the error is reported because the fsid is wrong.

The Log is as below:
[2016-05-19 06:32:20,176][ceph_deploy.conf][DEBUG ] found configuration file at: /home/toneceph/.cephdeploy.conf
[2016-05-19 06:32:20,182][ceph_deploy.cli][INFO ] Invoked (1.5.33): /usr/bin/ceph-deploy osd activate centos3:/var/local/osd0 centos4:/var/local/osd1 centos5:/var/local/osd2
[2016-05-19 06:32:20,182][ceph_deploy.cli][INFO ] ceph-deploy options:
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] username : None
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] verbose : False
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] overwrite_conf : False
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] subcommand : activate
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] quiet : False
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0x1c26290>
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] cluster : ceph
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] func : <function osd at 0x1c14050>
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] ceph_conf : None
[2016-05-19 06:32:20,183][ceph_deploy.cli][INFO ] default_release : False
[2016-05-19 06:32:20,184][ceph_deploy.cli][INFO ] disk : [('centos3', '/var/local/osd0', None), ('centos4', '/var/local/osd1', None), ('centos5', '/var/local/osd2', None)]
[2016-05-19 06:32:20,184][ceph_deploy.osd][DEBUG ] Activating cluster ceph disks centos3:/var/local/osd0: centos4:/var/local/osd1: centos5:/var/local/osd2:
[2016-05-19 06:32:40,921][centos3][DEBUG ] connection detected need for sudo
[2016-05-19 06:32:41,563][centos3][DEBUG ] connected to host: centos3
[2016-05-19 06:32:41,566][centos3][DEBUG ] detect platform information from remote host
[2016-05-19 06:32:41,594][centos3][DEBUG ] detect machine type
[2016-05-19 06:32:41,619][centos3][DEBUG ] find the location of an executable
[2016-05-19 06:32:41,625][ceph_deploy.osd][INFO ] Distro info: CentOS Linux 7.2.1511 Core
[2016-05-19 06:32:41,625][ceph_deploy.osd][DEBUG ] activating host centos3 disk /var/local/osd0
[2016-05-19 06:32:41,625][ceph_deploy.osd][DEBUG ] will use init type: systemd
[2016-05-19 06:32:41,625][centos3][DEBUG ] find the location of an executable
[2016-05-19 06:32:41,639][centos3][INFO ] Running command: sudo /usr/sbin/ceph-disk -v activate --mark-init systemd --mount /var/local/osd0
[2016-05-19 06:32:41,838][centos3][WARNING] main_activate: path = /var/local/osd0
[2016-05-19 06:32:41,839][centos3][WARNING] activate: Cluster uuid is 49a2bef6-313b-447f-9ab1-d6f5bab2c882
[2016-05-19 06:32:41,839][centos3][WARNING] command: Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid
[2016-05-19 06:32:42,162][centos3][WARNING] Traceback (most recent call last):
[2016-05-19 06:32:42,162][centos3][WARNING] File "/usr/sbin/ceph-disk", line 9, in <module>
[2016-05-19 06:32:42,162][centos3][WARNING] load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')()
[2016-05-19 06:32:42,162][centos3][WARNING] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 4964, in run
[2016-05-19 06:32:42,163][centos3][WARNING] main(sys.argv[1:])
[2016-05-19 06:32:42,163][centos3][WARNING] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 4915, in main
[2016-05-19 06:32:42,163][centos3][WARNING] args.func(args)
[2016-05-19 06:32:42,163][centos3][WARNING] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3277, in main_activate
[2016-05-19 06:32:42,163][centos3][WARNING] init=args.mark_init,
[2016-05-19 06:32:42,163][centos3][WARNING] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3097, in activate_dir
[2016-05-19 06:32:42,163][centos3][WARNING] (osd_id, cluster) = activate(path, activate_key_template, init)
[2016-05-19 06:32:42,163][centos3][WARNING] File "/usr/lib/python2.7/site-packages/ceph_disk/main.py", line 3173, in activate
[2016-05-19 06:32:42,164][centos3][WARNING] ' with fsid %s' % ceph_fsid)
[2016-05-19 06:32:42,164][centos3][WARNING] ceph_disk.main.Error: Error: No cluster conf found in /etc/ceph with fsid 49a2bef6-313b-447f-9ab1-d6f5bab2c882
[2016-05-19 06:32:42,168][centos3][ERROR ] RuntimeError: command returned non-zero exit status: 1
[2016-05-19 06:32:42,168][ceph_deploy][ERROR ] RuntimeError: Failed to execute command: /usr/sbin/ceph-disk -v activate --mark-init systemd --mount /var/local/osd0

The context of the ceph.conf is as below. Obviously the fsid is different from the log.
[toneceph@centos1 my-cluster]$ cat ceph.conf
[global]
fsid = 8031564c-fb1e-410d-ad51-ba0e3309753c
mon_initial_members = centos2
mon_host = 192.168.122.51
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
public network = 192.168.122.34/255.255.255.0

History

#1 Updated by Anonymous almost 8 years ago

Run "ceph-deploy disk zap ...." after the command "ceph-deploy install ceph-node1 ceph-node2 ceph-node3 ceph-node4", the issue is disappered. Is it a workaround?

#2 Updated by Anonymous almost 8 years ago

Is it a function fault, or else?

#3 Updated by Alfredo Deza about 6 years ago

  • Status changed from New to Closed

Clusters with mismatched FSID indicates a possible leftover ceph.conf file, or mixing previously deployed nodes with new ones.

Also available in: Atom PDF