Bug #39136: ceph balancer upmap unable to optimize mixed sized osds. - mgr - Ceph

Actions

Copy link

Bug #39136

open

ceph balancer upmap unable to optimize mixed sized osds.

Added by Iain Buclaw about 5 years ago. Updated almost 5 years ago.

Status:

New

Priority:

Normal

Assignee:

Category:

balancer module

Target version:

Ceph - v12.2.12

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

ceph-qa-suite:

rados

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

One OSD (osd.3) was formatted with the wrong filesystem, resulting in it being 14GiB larger than all other disks.

Despite this, it has 50GiB more data than all other OSDs, and ceph balancer claims that the distribution is perfect.

There is only one pool in this cluster, only storing with small (KB) files, and there's no reference of locality in how data is distributed.

All disks should have equal (within a couple of GB) utilisation, they currently do not.

ID  CLASS WEIGHT   REWEIGHT SIZE    USE     AVAIL   %USE  VAR  PGS TYPE NAME
 -2        0.88199        -  883GiB  428GiB  455GiB 48.51 0.95   -     host 247
  0   ssd  0.44099  1.00000  442GiB  215GiB  227GiB 48.58 0.95  63         osd.0
  1   ssd  0.44099  1.00000  442GiB  214GiB  228GiB 48.44 0.95  64         osd.1
 -3        0.88199        -  897GiB  497GiB  400GiB 55.38 1.09   -     host 248 
  2   ssd  0.44099  1.00000  442GiB  224GiB  218GiB 50.66 0.99  63         osd.2
  3   ssd  0.44099  1.00000  456GiB  273GiB  182GiB 59.95 1.18  64         osd.3
 -7        0.88199        -  883GiB  455GiB  428GiB 51.48 1.01   -     host 252 
 10   ssd  0.44099  1.00000  442GiB  227GiB  215GiB 51.38 1.01  69         osd.10
 11   ssd  0.44099  1.00000  442GiB  228GiB  214GiB 51.58 1.01  69         osd.11
 -8        0.88199        -  883GiB  408GiB  475GiB 46.21 0.91   -     host 253 
 12   ssd  0.44099  1.00000  442GiB  204GiB  238GiB 46.09 0.90  62         osd.12
 13   ssd  0.44099  1.00000  442GiB  205GiB  237GiB 46.33 0.91  63         osd.13
 -9        0.88199        -  883GiB  446GiB  437GiB 50.52 0.99   -     host 254 
 14   ssd  0.44099  1.00000  442GiB  223GiB  219GiB 50.49 0.99  70         osd.14
 15   ssd  0.44099  1.00000  442GiB  223GiB  218GiB 50.55 0.99  62         osd.15
-10        0.88199        -  883GiB  450GiB  433GiB 50.97 1.00   -     host 255 
 16   ssd  0.44099  1.00000  442GiB  222GiB  219GiB 50.38 0.99  69         osd.16
 17   ssd  0.44099  1.00000  442GiB  228GiB  214GiB 51.55 1.01  67         osd.17
-11        0.88199        -  883GiB  451GiB  432GiB 51.06 1.00   -     host 256 
 18   ssd  0.44099  1.00000  442GiB  227GiB  214GiB 51.43 1.01  74         osd.18
 19   ssd  0.44099  1.00000  442GiB  224GiB  218GiB 50.69 0.99  70         osd.19
-12        0.88199        -  883GiB  451GiB  432GiB 51.04 1.00   -     host 257 
 20   ssd  0.44099  1.00000  442GiB  228GiB  214GiB 51.59 1.01  71         osd.20
 21   ssd  0.44099  1.00000  442GiB  223GiB  219GiB 50.50 0.99  68         osd.21
-13        0.88199        -  883GiB  451GiB  433GiB 51.02 1.00   -     host 258 
 22   ssd  0.44099  1.00000  442GiB  227GiB  214GiB 51.50 1.01  71         osd.22
 23   ssd  0.44099  1.00000  442GiB  223GiB  218GiB 50.54 0.99  64         osd.23
-14        0.88199        -  883GiB  451GiB  432GiB 51.07 1.00   -     host 259 
 24   ssd  0.44099  1.00000  442GiB  223GiB  218GiB 50.54 0.99  68         osd.24
 25   ssd  0.44099  1.00000  442GiB  228GiB  214GiB 51.60 1.01  76         osd.25
-15        0.88199        -  883GiB  452GiB  431GiB 51.14 1.00   -     host 260 
 26   ssd  0.44099  1.00000  442GiB  228GiB  213GiB 51.69 1.01  74         osd.26
 27   ssd  0.44099  1.00000  442GiB  223GiB  218GiB 50.60 0.99  65         osd.27
-16        0.88199        -  883GiB  451GiB  432GiB 51.08 1.00   -     host 261 
 28   ssd  0.44099  1.00000  442GiB  224GiB  218GiB 50.68 0.99  66         osd.28
 29   ssd  0.44099  1.00000  442GiB  227GiB  214GiB 51.47 1.01  68         osd.29
-17        0.88199        -  883GiB  451GiB  432GiB 51.12 1.00   -     host 262 
 30   ssd  0.44099  1.00000  442GiB  223GiB  218GiB 50.58 0.99  67         osd.30
 31   ssd  0.44099  1.00000  442GiB  228GiB  213GiB 51.67 1.01  67         osd.31
-18        0.88199        -  883GiB  456GiB  427GiB 51.65 1.01   -     host 263 
 32   ssd  0.44099  1.00000  442GiB  228GiB  214GiB 51.62 1.01  71         osd.32
 33   ssd  0.44099  1.00000  442GiB  228GiB  213GiB 51.67 1.01  73         osd.33
-23        0.88199        -  883GiB  451GiB  432GiB 51.07 1.00   -     host 349 
  4   ssd  0.44099  1.00000  442GiB  227GiB  214GiB 51.44 1.01  70         osd.4
  5   ssd  0.44099  1.00000  442GiB  224GiB  218GiB 50.69 0.99  63         osd.5
-21        0.88199        -  883GiB  452GiB  431GiB 51.20 1.00   -     host 350 
  6   ssd  0.44099  1.00000  442GiB  229GiB  213GiB 51.80 1.02  63         osd.6
  7   ssd  0.44099  1.00000  442GiB  223GiB  218GiB 50.60 0.99  67         osd.7
-19        0.88199        -  883GiB  450GiB  433GiB 51.01 1.00   -     host 351 
  8   ssd  0.44099  1.00000  442GiB  223GiB  219GiB 50.47 0.99  62         osd.8
  9   ssd  0.44099  1.00000  442GiB  228GiB  214GiB 51.55 1.01  66         osd.9
-25        0.88199        -  883GiB  452GiB  431GiB 51.21 1.00   -     host 352 
 34   ssd  0.44099  1.00000  442GiB  229GiB  213GiB 51.76 1.02  76         osd.34
 35   ssd  0.44099  1.00000  442GiB  224GiB  218GiB 50.67 0.99  65         osd.35
-38        0.88199        -  883GiB  455GiB  428GiB 51.56 1.01   -     host 353   
 36   ssd  0.44099  1.00000  442GiB  227GiB  214GiB 51.50 1.01  72         osd.36
 37   ssd  0.44099  1.00000  442GiB  228GiB  214GiB 51.62 1.01  67         osd.37