Support #8915
Ceph Firefly 0.80.4 : health HEALTH_WARN pool volumes has too few pgs; crush map has legacy tunables
0%
Description
Hello Ceph Developers
Recently i have upgraded from dumpling to Firefly stable release 0.80.4 . As soon as upgrade completed i am getting the following health warnings , however these were not there when the cluster was running dumpling.
health HEALTH_WARN pool volumes has too few pgs;crush map has legacy tunables
# ceph -s cluster 009d3518-e60d-4f74-a26d-c08c1976263c health HEALTH_WARN pool volumes has too few pgs;crush map has legacy tunables monmap e3: 3 mons at {storage0111-ib=192.168.100.111:6789/0,storage0112-ib=192.168.100.112:6789/0,storage0113-ib=192.168.100.113:6789/0}, election epoch 274, quorum 0,1,2 storage0111-ib,storage0112-ib,storage0113-ib mdsmap e4: 1/1/1 up {0=storage0101-ib=up:active} osdmap e27393: 99 osds: 99 up, 99 in pgmap v294182: 23744 pgs, 17 pools, 2595 GB data, 649 kobjects 2268 GB used, 307 TB / 309 TB avail 23744 active+clean
Warning 1 : crush map has legacy tunables¶
I have tried changing crush profiles to optimal, firefly but getting this error.
# ceph osd crush tunables optimal Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.45, osd.46, osd.48, osd.50, osd.51, osd.53, osd.55, osd.57, osd.59, osd.60, osd.62 # ceph osd getcrushmap -o /tmp/crush got crush map from osdmap epoch 25881 # crushtool -i /tmp/crush --set-choose-local-tries 0 --set-choose-local-fallback-tries 0 --set-choose-total-tries 50 -o /tmp/crush.new # ceph osd setcrushmap -i /tmp/crush.new Error EINVAL: Could not change CRUSH: features 262144 unsupported by: osd.45, osd.46, osd.48, osd.50, osd.51, osd.53, osd.55, osd.57, osd.59, osd.60, osd.62 # ceph osd crush tunables firefly Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.45, osd.46, osd.48, osd.50, osd.51, osd.53, osd.55, osd.57, osd.59, osd.60, osd.62
Next day i again tried to change crush tunables , but this time i am getting unsupported version for OTHER SET OF OSDS.
# ceph osd crush tunables firefly Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.55, osd.57, osd.59, osd.60, osd.62 # ceph osd crush tunables default Error EINVAL: Could not change CRUSH: features 33816576 unsupported by: osd.55, osd.57, osd.59, osd.60, osd.62 # ceph osd crush tunables optimal Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.55, osd.57, osd.59, osd.60, osd.62
Warning 2 : Pool volumes has too few pgs¶
For this warning i tried changing PG size for volumes poll , but NO LUCK . The interesting part is out of 16 pools , Ceph is showing warning only for 1 pool.
The other 15 pools actually has less PG numbers then volumes pool. If PG number is really a problem , Ceph should throw warning for ALL the POOLS of my cluster.
# ceph osd dump | grep -i pool pool 0 'data' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 crash_replay_interval 45 stripe_width 0 pool 1 'metadata' replicated size 2 min_size 1 crush_ruleset 1 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 stripe_width 0 pool 2 'rbd' replicated size 2 min_size 1 crush_ruleset 2 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 stripe_width 0 pool 3 '.rgw' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 112 stripe_width 0 pool 4 '.rgw.root' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 114 stripe_width 0 pool 5 '.rgw.control' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 115 stripe_width 0 pool 6 '.rgw.buckets' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 117 stripe_width 0 pool 7 '.rgw.buckets.index' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 118 stripe_width 0 pool 8 '.users.swift' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 120 stripe_width 0 pool 9 'volumes' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2048 pgp_num 2048 last_change 23486 stripe_width 0 pool 10 'images' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 284 stripe_width 0 pool 11 'backups' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 286 stripe_width 0 pool 12 '.rgw.gc' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 398 stripe_width 0 pool 13 '.users.uid' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 399 stripe_width 0 pool 14 '.users' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 403 stripe_width 0 pool 15 '.log' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 13260 stripe_width 0 pool 16 '' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 23285 stripe_width 0
pool 9 'volumes' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2048 pgp_num 2048 last_change 23486 stripe_width 0
It would be great if you can provide solutions to both of these problems.
PS : Cluster upgrade process was successful , all the nodes (MON,OSD) are running same version of Ceph i.e. 0.80.4
- Regards
Karan Singh
History
#1 Updated by Greg Farnum over 9 years ago
- Tracker changed from Bug to Support
- Status changed from New to Closed
- Priority changed from Immediate to Normal
These topics are pretty well-covered in the release notes. Please refer to those, and if you have any further questions bring them up on irc or the mailing list. :)
#2 Updated by Swami Reddy about 8 years ago
Is this problem fixed??