Project

General

Profile

Bug #13089

mon: check for store writeablility before participating in election

Added by Samuel Just about 5 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Sage Weil
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
Backport:
hammer,firefly
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Related issues

Duplicated by Ceph - Bug #13200: mon: does not check for IO errors on every transaction Duplicate 09/22/2015
Copied to Ceph - Backport #13339: mon: check for store writeablility before participating in election Resolved
Copied to Ceph - Backport #13252: mon: does not check for IO errors on every transaction Resolved 09/22/2015

Associated revisions

Revision 2fb7b1f0 (diff)
Added by Sage Weil about 5 years ago

mon/MonitorDBStore: assert/crash if there is a write error

Do this globally intead of relying on teh zillion mon callers to
check the error code. There are no cases where we want to
tolerate a commit failure.

Fixes: #13089
Signed-off-by: Sage Weil <>

Revision d2bfa190 (diff)
Added by Sage Weil about 5 years ago

mon/MonitorDBStore: assert/crash if there is a write error

Do this globally intead of relying on teh zillion mon callers to
check the error code. There are no cases where we want to
tolerate a commit failure.

Fixes: #13089
Signed-off-by: Sage Weil <>
(cherry picked from commit 2fb7b1f0e33ada7c9a1be3de2f7708eb0760fcef)

Revision 20a4c0c4 (diff)
Added by Sage Weil about 5 years ago

mon/MonitorDBStore: assert/crash if there is a write error

Do this globally intead of relying on teh zillion mon callers to
check the error code. There are no cases where we want to
tolerate a commit failure.

Fixes: #13089
Signed-off-by: Sage Weil <>
(cherry picked from commit 2fb7b1f0e33ada7c9a1be3de2f7708eb0760fcef)

History

#1 Updated by Greg Farnum about 5 years ago

Mmm, I'm not sure we want to do the naive thing (write a file and fsync) on every election as that will slow them down a lot in the common case. Probably if we just check store accessibility on bootup, flip a flag if we ever fail one, and check for it on every election?

#2 Updated by Sage Weil about 5 years ago

Greg Farnum wrote:

Mmm, I'm not sure we want to do the naive thing (write a file and fsync) on every election as that will slow them down a lot in the common case. Probably if we just check store accessibility on bootup, flip a flag if we ever fail one, and check for it on every election?

Yeah I think a check on boot is sufficient.

#3 Updated by Kefu Chai about 5 years ago

  • Assignee set to Kefu Chai

#4 Updated by Joao Eduardo Luis about 5 years ago

Can we have a bit more context please? What's the rationale behind this? The monitor should not start if leveldb is not open, and since it's not open it's not writeable. Is this in case the disk is full or the fs is read-only? Or something else I'm missing?

#5 Updated by Kefu Chai about 5 years ago

  • Status changed from New to Fix Under Review
  • Assignee changed from Kefu Chai to Sage Weil

#6 Updated by Kefu Chai about 5 years ago

  • Tags set to hammer

#7 Updated by Kefu Chai about 5 years ago

  • Tags deleted (hammer)
  • Backport set to hammer

#8 Updated by Sage Weil about 5 years ago

  • Status changed from Fix Under Review to Pending Backport

#9 Updated by Loic Dachary about 5 years ago

  • Backport changed from hammer to hammer,firefly

#10 Updated by Loic Dachary almost 5 years ago

  • Status changed from Pending Backport to Resolved

#11 Updated by Kefu Chai over 4 years ago

please note that the background compaction fails, if leveldb can not create new tables or remove merged tables.

so, if we "chmod -w store.db". and leveldb complains in this case, but it does not return error on Write() unless "leveldb_paranoid" is true, but this option is "false" by default. so "failed to write to db" assertion is not triggered, even if compaction fails with the default settings.

Also available in: Atom PDF