Actions
Bug #38221
closedLC_NUMERIC can make lvm.py fail
Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Target version:
-
% Done:
0%
Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
ceph-ansible
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
(I initialy filed this in the wrong place, https://github.com/ceph/ceph-ansible/issues/3579 )
Bug Report
What happened:
As a European, I am used to writing e.g. "two and a half" as 2,5 (not 2.5). As such I tend to set `LC_NUMERIC=en_DK.UTF-8` on my Linux workstation1.
When I ssh to a RHEL7 box, these get then passed as per the default `AcceptEnv LANG LC_CTYPE LC_NUMERIC`… etc in `/etc/ssh/sshd_config` of RHEL7 (also affects Fedora 28).
So, with this setting, the OSDs will get the var as per the default sshd_config on the OSDs and (as expected) have the comma separator in `vgs` output.
[root@odroid-hc2-00 ~]# env|grep LC LC_MEASUREMENT=en_DK.UTF-8 LC_MONETARY=de_DE.UTF-8 LC_COLLATE=en_US.UTF-8 LC_NUMERIC=en_DK.UTF-8 LC_TIME=de_DE.UTF-8 [root@odroid-hc2-00 ~]# vgs --noheadings --readonly --units=g --separator=";" ceph-64ff50de-28b8-4bb6-bf76-601df9dc1e8e;1;1;0;wz--n-;931,48g;0g [root@odroid-hc2-00 ~]# unset LC_NUMERIC [root@odroid-hc2-00 ~]# vgs --noheadings --readonly --units=g --separator=";" ceph-64ff50de-28b8-4bb6-bf76-601df9dc1e8e;1;1;0;wz--n-;931.48g;0g
It seems that `/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py` on the OSDs chokes on the comma, expecting a dot. (my OSDs are Fedora 28 ARM based, do tell if you need me to reproduce with RHEL7 x86_64 based OSDs)
If I unset the LC_NUMERIC on the host running ceph-ansible, `ansible-playbook site.yml` (copy of site.yml.samle) works just fine.
If `LC_NUMERIC=en_DK.UTF-8`, then I get
2019-02-06 15:49:45,682 p=27999 u=ansible | TASK [ceph-osd : use ceph-volume lvm batch to create bluestore osds] **************************************************
2019-02-06 15:49:45,683 p=27999 u=ansible | Wednesday 06 February 2019 15:49:45 +0100 (0:00:00.508) 0:06:06.035 ****
2019-02-06 15:49:49,216 p=27999 u=ansible | fatal: [odroid-hc2-03]: FAILED! => {"changed": true, "cmd": ["ceph-volume", "--cluster", "ceph", "lvm", "batch", "--bluestore", "--yes", "--dmcrypt", "/dev/sda"], "delta": "0:00:02.467296", "end": "2019-02-06 15:49:49.161515", "msg": "non-zero return code", "rc": 1, "start": "2019-02-06 15:49:46.694219", "stderr": "Traceback (most recent call last):\n File \"/usr/sbin/ceph-volume\", line 11, in <module>\n load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')()\n File \"/usr/lib/python2.7/site-packages/ceph_volume/main.py\", line 38, in __init__\n self.main(self.argv)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/decorators.py\", line 59, in newfunc\n return f(*a, **kw)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/main.py\", line 148, in main\n terminal.dispatch(self.mapper, subcommand_args)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/terminal.py\", line 182, in dispatch\n instance.main()\n File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/main.py\", line 40, in main\n terminal.dispatch(self.mapper, self.argv)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/terminal.py\", line 182, in dispatch\n instance.main()\n File \"/usr/lib/python2.7/site-packages/ceph_volume/decorators.py\", line 16, in is_root\n return func(*a, **kw)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/batch.py\", line 284, in main\n self.execute(args)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/batch.py\", line 175, in execute\n strategy.execute()\n File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/strategies/bluestore.py\", line 126, in execute\n lvs = lvm.create_lvs(create['vg'], parts=create['parts'], name_prefix='osd-data')\n File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 643, in create_lvs\n sizing = volume_group.sizing(parts=parts, size=size)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 1067, in sizing\n size = int(self.free / parts)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 1008, in free\n return self._parse_size(self.vg_free)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 994, in _parse_size\n return util.str_to_int(integer)\n File \"/usr/lib/python2.7/site-packages/ceph_volume/util/__init__.py\", line 40, in str_to_int\n raise RuntimeError(error_msg)\nRuntimeError: Unable to convert to integer: '1862,98'", "stderr_lines": ["Traceback (most recent call last):", " File \"/usr/sbin/ceph-volume\", line 11, in <module>", " load_entry_point('ceph-volume==1.0.0', 'console_scripts', 'ceph-volume')()", " File \"/usr/lib/python2.7/site-packages/ceph_volume/main.py\", line 38, in __init__", " self.main(self.argv)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/decorators.py\", line 59, in newfunc", " return f(*a, **kw)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/main.py\", line 148, in main", " terminal.dispatch(self.mapper, subcommand_args)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/terminal.py\", line 182, in dispatch", " instance.main()", " File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/main.py\", line 40, in main", " terminal.dispatch(self.mapper, self.argv)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/terminal.py\", line 182, in dispatch", " instance.main()", " File \"/usr/lib/python2.7/site-packages/ceph_volume/decorators.py\", line 16, in is_root", " return func(*a, **kw)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/batch.py\", line 284, in main", " self.execute(args)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/batch.py\", line 175, in execute", " strategy.execute()", " File \"/usr/lib/python2.7/site-packages/ceph_volume/devices/lvm/strategies/bluestore.py\", line 126, in execute", " lvs = lvm.create_lvs(create['vg'], parts=create['parts'], name_prefix='osd-data')", " File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 643, in create_lvs", " sizing = volume_group.sizing(parts=parts, size=size)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 1067, in sizing", " size = int(self.free / parts)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 1008, in free", " return self._parse_size(self.vg_free)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/api/lvm.py\", line 994, in _parse_size", " return util.str_to_int(integer)", " File \"/usr/lib/python2.7/site-packages/ceph_volume/util/__init__.py\", line 40, in str_to_int", " raise RuntimeError(error_msg)", "RuntimeError: Unable to convert to integer: '1862,98'"], "stdout": "Running command: vgcreate --force --yes ceph-8e748fdb-e322-4be0-b3c1-6bc0f1120351 /dev/sda\n stdout: Physical volume \"/dev/sda\" successfully created.\n stdout: Volume group \"ceph-8e748fdb-e322-4be0-b3c1-6bc0f1120351\" successfully created", "stdout_lines": ["Running command: vgcreate --force --yes ceph-8e748fdb-e322-4be0-b3c1-6bc0f1120351 /dev/sda", " stdout: Physical volume \"/dev/sda\" successfully created.", " stdout: Volume group \"ceph-8e748fdb-e322-4be0-b3c1-6bc0f1120351\" successfully created"]}
When this happens, the OSD node have a VG defined but no LV in that VG after the playbook errored out. I'll attach a full log of the ansible-playbook run.
What you expected to happen:
If ceph-ansible has requirements for LANG and/or LC_NUMERIC, it should enforce their values.
How to reproduce it (minimal and precise):
`LC_NUMERIC=en_DK.UTF-8 ansible-playbook site.yml` with the following;
[ansible@ceph-ansible ceph-ansible]$ grep "^[^#;]" group_vars/osds.yml
---
dummy:
copy_admin_key: true
devices:
- /dev/sda
dmcrypt: True
osd_scenario: lvm
[ansible@ceph-ansible ceph-ansible]$ grep "^[^#;]" group_vars/all.yml
---
dummy:
fetch_directory: ~/ceph-ansible-keys
cluster: ceph
configure_firewall: False
ntp_service_enabled: true
ntp_daemon_type: chronyd
ceph_origin: distro
ceph_repository: rhcs
ceph_rhcs_version: 3
ceph_repository_type: cdn
fsid: "{{ cluster_uuid.stdout }}"
generate_fsid: True
rbd_cache_writethrough_until_flush: "false"
rbd_client_directories: false # as per CEPH125-RHCS3.0-en-1-20180517 pages 45 and 60
monitor_interface: eth0
journal_size: 5120 # OSD journal size in MB
public_network: 192.168.50.0/24 # HouseNet
cluster_network: "{{ public_network | regex_replace(' ', '') }}"
ceph_conf_overrides:
global:
mon_allow_pool_delete: true
client:
rbd_default_features: 1
ceph_docker_image: "rhceph-3-rhel7"
ceph_docker_image_tag: "latest"
ceph_docker_registry: "registry.access.redhat.com/rhceph/"
[ansible@ceph-ansible ceph-ansible]$ cat /etc/ansible/hosts
[ceph-arm-nodes]
odroid-hc2-[00:04]
[ceph-x86-nodes]
ceph-ansible
[ceph-housenet:children]
ceph-arm-nodes
ceph-x86-nodes
[ceph-housenet:vars]
ansible_user=ansible
[mons]
odroid-hc2-[00:02]
[mgrs]
odroid-hc2-[00:02]
[osds]
odroid-hc2-[00:04]
[clients]
odroid-hc2-00
[ansible@ceph-ansible ceph-ansible]$
Environment:
- OS (e.g. from /etc/os-release): Red Hat Enterprise Linux Server release 7.6 (Maipo)
- Kernel (e.g. `uname -a`): Linux ceph-ansible.internal.pcfe.net 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux
- Docker version if applicable (e.g. `docker version`):
- Ansible version (e.g. `ansible-playbook --version`): ansible-playbook 2.6.12
- ceph-ansible version (e.g. `git head or tag or stable branch`): ceph-ansible-3.2.4-1.el7cp.noarch
- Ceph version (e.g. `ceph -v`): ceph version 12.2.10 (177915764b752804194937482a39e95e0ca3de94) luminous (stable)
note: all except ceph -v collected on the RHEL7 x86_64 machine running ceph-ansible. ceph -v was collected on an affected OSD running Fedora 28 ARM
[1] JFYI; the complete list of language related varoables I normally set is;
LANG=en_US.UTF-8 GDM_LANG=en_GB.UTF-8 LANGUAGE=en_GB:en_US LC_MONETARY=de_DE.UTF-8 LC_NUMERIC=en_DK.UTF-8 LC_COLLATE=en_US.UTF-8 LC_MEASUREMENT=en_DK.UTF-8 LC_TIME=de_DE.UTF-8
Files
Actions