Project

General

Profile

Actions
Copy link

Bug #42370

closed

mgr/balancer: 'dict_keys' object does not support indexing

Added by Mykola Golub over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Mykola Golub
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,mimic,luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
30987
Crash signature (v1):
Crash signature (v2):

Description

There is a python3 compatibility issue for balancer module running in crush-compat mode:

# ceph config-key dump | grep balancer
  "config-history/4/+mgr/mgr/balancer/mode": "crush-compat",
  "config-history/5/+mgr/mgr/balancer/active": "true",
  "config/mgr/mgr/balancer/active": "true",
  "config/mgr/mgr/balancer/mode": "crush-compat",

# ceph balancer status
  Error EIO: Module 'balancer' has experienced an error and cannot handle commands: 'dict_keys' object

It has been reported for 14.2.2 but I believe it is present in the master too.

Related issues 3 (0 open — 3 closed)

Copied to mgr - Backport #42390: luminous: mgr/balancer: 'dict_keys' object does not support indexingResolvedKonstantin ShalyginActions
Copied to mgr - Backport #42391: mimic: mgr/balancer: 'dict_keys' object does not support indexingResolvedNathan CutlerActions
Copied to mgr - Backport #42392: nautilus: mgr/balancer: 'dict_keys' object does not support indexingResolvedNathan CutlerActions
Actions
Copy link
 #1

Updated by Mykola Golub over 4 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 30987
Actions
Copy link
 #2

Updated by Kefu Chai over 4 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #3

Updated by Eugen Block over 4 years ago 

Is there anything I can temporarily do to get rid of the HEALTH_ERR status? I noticed that our crushmap still had some straw entries and executed ceph osd crush set-all-straw-buckets-to-straw2

After that the cluster went in error status:

ceph01:~ #  ceph -s
  cluster:
    id:     655cb05a-435a-41ba-83d9-8549f7c36167
    health: HEALTH_ERR
            Module 'balancer' has failed: 'dict_keys' object does not support indexing

Looking into MGR logs I find this message occuring every minute:

mgr[balancer] Error creating compat weight-set

I hadn't noticed that message before running set-all-straw-buckets-to-straw2. Obviously, balancer module can't be disabled, so I'm stuck with the error status.

Actions
Copy link
 #4

Updated by Nathan Cutler over 4 years ago

  • Copied to Backport #42390: luminous: mgr/balancer: 'dict_keys' object does not support indexing added
Actions
Copy link
 #5

Updated by Nathan Cutler over 4 years ago

  • Copied to Backport #42391: mimic: mgr/balancer: 'dict_keys' object does not support indexing added
Actions
Copy link
 #6

Updated by Nathan Cutler over 4 years ago

  • Copied to Backport #42392: nautilus: mgr/balancer: 'dict_keys' object does not support indexing added
Actions
Copy link
 #7

Updated by Eugen Block over 4 years ago

Eugen Block wrote:

[...]
Is there anything I can temporarily do to get rid of the HEALTH_ERR status?

For a temporary solution I applied the proposed change to /usr/share/ceph/mgr/balancer/module.py on the active MGR and restarted it. The balancer is currently remapping PGs, so it seems to work now. Also the set-all-straw-buckets-to-straw2 was successful.

Actions
Copy link
 #8

Updated by Nathan Cutler over 4 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions
Copy link

Also available in: Atom PDF