Project

General

Profile

Bug #42370

mgr/balancer: 'dict_keys' object does not support indexing

Added by Mykola Golub almost 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,mimic,luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

There is a python3 compatibility issue for balancer module running in crush-compat mode:

# ceph config-key dump | grep balancer
  "config-history/4/+mgr/mgr/balancer/mode": "crush-compat",
  "config-history/5/+mgr/mgr/balancer/active": "true",
  "config/mgr/mgr/balancer/active": "true",
  "config/mgr/mgr/balancer/mode": "crush-compat",

# ceph balancer status
  Error EIO: Module 'balancer' has experienced an error and cannot handle commands: 'dict_keys' object

It has been reported for 14.2.2 but I believe it is present in the master too.


Related issues

Copied to mgr - Backport #42390: luminous: mgr/balancer: 'dict_keys' object does not support indexing Resolved
Copied to mgr - Backport #42391: mimic: mgr/balancer: 'dict_keys' object does not support indexing Resolved
Copied to mgr - Backport #42392: nautilus: mgr/balancer: 'dict_keys' object does not support indexing Resolved

History

#1 Updated by Mykola Golub almost 2 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 30987

#2 Updated by Kefu Chai almost 2 years ago

  • Status changed from Fix Under Review to Pending Backport

#3 Updated by Eugen Block almost 2 years ago

Is there anything I can temporarily do to get rid of the HEALTH_ERR status? I noticed that our crushmap still had some straw entries and executed ceph osd crush set-all-straw-buckets-to-straw2

After that the cluster went in error status:

ceph01:~ #  ceph -s
  cluster:
    id:     655cb05a-435a-41ba-83d9-8549f7c36167
    health: HEALTH_ERR
            Module 'balancer' has failed: 'dict_keys' object does not support indexing

Looking into MGR logs I find this message occuring every minute:

mgr[balancer] Error creating compat weight-set

I hadn't noticed that message before running set-all-straw-buckets-to-straw2. Obviously, balancer module can't be disabled, so I'm stuck with the error status.

#4 Updated by Nathan Cutler almost 2 years ago

  • Copied to Backport #42390: luminous: mgr/balancer: 'dict_keys' object does not support indexing added

#5 Updated by Nathan Cutler almost 2 years ago

  • Copied to Backport #42391: mimic: mgr/balancer: 'dict_keys' object does not support indexing added

#6 Updated by Nathan Cutler almost 2 years ago

  • Copied to Backport #42392: nautilus: mgr/balancer: 'dict_keys' object does not support indexing added

#7 Updated by Eugen Block almost 2 years ago

Eugen Block wrote:

[...]
Is there anything I can temporarily do to get rid of the HEALTH_ERR status?

For a temporary solution I applied the proposed change to /usr/share/ceph/mgr/balancer/module.py on the active MGR and restarted it. The balancer is currently remapping PGs, so it seems to work now. Also the set-all-straw-buckets-to-straw2 was successful.

#8 Updated by Nathan Cutler over 1 year ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Also available in: Atom PDF