Project

General

Profile

Support #8915

Ceph Firefly 0.80.4 : health HEALTH_WARN pool volumes has too few pgs; crush map has legacy tunables

Added by karan singh over 5 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
Monitor
Target version:
% Done:

0%

Tags:
crush map has legacy tunables
Reviewed:
Affected Versions:
Pull request ID:

Description

Hello Ceph Developers

Recently i have upgraded from dumpling to Firefly stable release 0.80.4 . As soon as upgrade completed i am getting the following health warnings , however these were not there when the cluster was running dumpling.

health HEALTH_WARN pool volumes has too few pgs;crush map has legacy tunables

# ceph -s
    cluster 009d3518-e60d-4f74-a26d-c08c1976263c
     health HEALTH_WARN pool volumes has too few pgs;crush map has legacy tunables
     monmap e3: 3 mons at {storage0111-ib=192.168.100.111:6789/0,storage0112-ib=192.168.100.112:6789/0,storage0113-ib=192.168.100.113:6789/0}, election epoch 274, quorum 0,1,2 storage0111-ib,storage0112-ib,storage0113-ib
     mdsmap e4: 1/1/1 up {0=storage0101-ib=up:active}
     osdmap e27393: 99 osds: 99 up, 99 in
     pgmap v294182: 23744 pgs, 17 pools, 2595 GB data, 649 kobjects
            2268 GB used, 307 TB / 309 TB avail
               23744 active+clean

Warning 1 : crush map has legacy tunables

I have tried changing crush profiles to optimal, firefly but getting this error.

 # ceph osd crush tunables optimal
Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.45, osd.46, osd.48, osd.50, osd.51, osd.53, osd.55, osd.57, osd.59, osd.60, osd.62

 # ceph osd getcrushmap -o /tmp/crush
got crush map from osdmap epoch 25881

 # crushtool -i /tmp/crush --set-choose-local-tries 0 --set-choose-local-fallback-tries 0 --set-choose-total-tries 50 -o /tmp/crush.new

 # ceph osd setcrushmap -i /tmp/crush.new
Error EINVAL: Could not change CRUSH: features 262144 unsupported by: osd.45, osd.46, osd.48, osd.50, osd.51, osd.53, osd.55, osd.57, osd.59, osd.60, osd.62

 # ceph osd crush tunables firefly
Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.45, osd.46, osd.48, osd.50, osd.51, osd.53, osd.55, osd.57, osd.59, osd.60, osd.62
 

Next day i again tried to change crush tunables , but this time i am getting unsupported version for OTHER SET OF OSDS.
 # ceph osd crush tunables firefly
Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.55, osd.57, osd.59, osd.60, osd.62

 # ceph osd crush tunables default
Error EINVAL: Could not change CRUSH: features 33816576 unsupported by: osd.55, osd.57, osd.59, osd.60, osd.62

 # ceph osd crush tunables optimal
Error EINVAL: Could not change CRUSH: features 2199057072128 unsupported by: osd.55, osd.57, osd.59, osd.60, osd.62

Warning 2 : Pool volumes has too few pgs

For this warning i tried changing PG size for volumes poll , but NO LUCK . The interesting part is out of 16 pools , Ceph is showing warning only for 1 pool.
The other 15 pools actually has less PG numbers then volumes pool. If PG number is really a problem , Ceph should throw warning for ALL the POOLS of my cluster.

# ceph osd dump | grep -i pool
pool 0 'data' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 crash_replay_interval 45 stripe_width 0
pool 1 'metadata' replicated size 2 min_size 1 crush_ruleset 1 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 stripe_width 0
pool 2 'rbd' replicated size 2 min_size 1 crush_ruleset 2 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 stripe_width 0
pool 3 '.rgw' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 112 stripe_width 0
pool 4 '.rgw.root' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 114 stripe_width 0
pool 5 '.rgw.control' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 115 stripe_width 0
pool 6 '.rgw.buckets' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 117 stripe_width 0
pool 7 '.rgw.buckets.index' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 118 stripe_width 0
pool 8 '.users.swift' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 120 stripe_width 0
pool 9 'volumes' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2048 pgp_num 2048 last_change 23486 stripe_width 0
pool 10 'images' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 284 stripe_width 0
pool 11 'backups' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 128 pgp_num 128 last_change 286 stripe_width 0
pool 12 '.rgw.gc' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 398 stripe_width 0
pool 13 '.users.uid' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 399 stripe_width 0
pool 14 '.users' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 403 stripe_width 0
pool 15 '.log' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 13260 stripe_width 0
pool 16 '' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 4096 pgp_num 4096 last_change 23285 stripe_width 0

pool 9 'volumes' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 2048 pgp_num 2048 last_change 23486 stripe_width 0

It would be great if you can provide solutions to both of these problems.
PS : Cluster upgrade process was successful , all the nodes (MON,OSD) are running same version of Ceph i.e. 0.80.4

- Regards
Karan Singh

History

#1 Updated by Greg Farnum over 5 years ago

  • Tracker changed from Bug to Support
  • Status changed from New to Closed
  • Priority changed from Immediate to Normal

These topics are pretty well-covered in the release notes. Please refer to those, and if you have any further questions bring them up on irc or the mailing list. :)

#2 Updated by Swami Reddy about 4 years ago

Is this problem fixed??

Also available in: Atom PDF