Feature #53050
closed
Support blocklisting a CIDR range
Added by Greg Farnum over 2 years ago.
Updated over 1 year ago.
Description
Disaster recovery use cases want to be able to fence off entire IP ranges, rather than needing to specify individual IPs. This is for several important reasons:
1) They may not know the individual IPs involved.
2) We do not want to inflate the OSD map with every IP in a potentially-large cluster
3) Nobody wants to have to issue 1000 or 10000 blocklist commands, and we don't want to have to commit that many OSDMap updates that quickly.
Extend the "osd blocklist" command (or add a similar one?) to work with CIDR ranges in addition to single IPs, and update the code that works with blocklists to deal with that change.
So we're going to put a huge asterisk here that the CIDR range of machines must be hard-rebooted, right? Otherwise whenever the unblocklist event comes in, any kernel mounts with dirty data will happily flush that out.
That was the rationale for the new generation number we discussed in various places including:
https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/3FTOBHPZQGTEY3RHMO2EXKOLSP3SJGNW/
Also, I think compound blocklist commands should be supported as there will probably be several CIDR ranges involved.
Patrick Donnelly wrote:
So we're going to put a huge asterisk here that the CIDR range of machines must be hard-rebooted, right? Otherwise whenever the unblocklist event comes in, any kernel mounts with dirty data will happily flush that out.
I mean that’s only if you have on the force reconnect, which is a mount option, right? So anybody who wants to plan to use range blocklisting can just not use bad remounts. Or, they can arrange to reboot, which I get the impression is going to happen anyway since other layers need to do updates and fallbacks.
That was the rationale for the new generation number we discussed in various places including:
https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/3FTOBHPZQGTEY3RHMO2EXKOLSP3SJGNW/
Yeah. That proposal isn’t gonna happen any time soon, if ever. I can knock this ticket out this or next week.
Also, I think compound blocklist commands should be supported as there will probably be several CIDR ranges involved.
Meh; we can handle three osdmap commits if needed but multiple ranges was specifically not part of the request — just a cluster with a single fixed range.
Greg Farnum wrote:
Patrick Donnelly wrote:
So we're going to put a huge asterisk here that the CIDR range of machines must be hard-rebooted, right? Otherwise whenever the unblocklist event comes in, any kernel mounts with dirty data will happily flush that out.
I mean that’s only if you have on the force reconnect, which is a mount option, right?
You mean recover_session=clean?
https://docs.ceph.com/en/pacific/man/8/mount.ceph/#basic
Not just in that scenario, no. The kernel has gotten better about not flushing dirty data when blocklisted. I'm not sure how safe it is at this time, Jeff would need to comment.
So anybody who wants to plan to use range blocklisting can just not use bad remounts.
You can't (yet) skip blocklisted superblocks. They stick around and prevent new mounts from even connecting. Jeff is working on a fix for that presently which will be upstream soon.
Or, they can arrange to reboot, which I get the impression is going to happen anyway since other layers need to do updates and fallbacks.
That was the rationale for the new generation number we discussed in various places including:
https://lists.ceph.io/hyperkitty/list/dev@ceph.io/thread/3FTOBHPZQGTEY3RHMO2EXKOLSP3SJGNW/
Yeah. That proposal isn’t gonna happen any time soon, if ever. I can knock this ticket out this or next week.
Also, I think compound blocklist commands should be supported as there will probably be several CIDR ranges involved.
Meh; we can handle three osdmap commits if needed but multiple ranges was specifically not part of the request — just a cluster with a single fixed range.
ACK.
- Status changed from New to Resolved
- Pull request ID set to 44151
- Status changed from Resolved to Pending Backport
- Related to Bug #55516: qa: fs suite tests failing with "json.decoder.JSONDecodeError: Extra data: line 2 column 82 (char 82)" added
- Backport set to quincy,pacific
The Backport field was empty, therefore no backport tickets were created.
- Copied to Backport #55747: pacific: Support blocklisting a CIDR range added
- Tags set to backport_processed
- Status changed from Pending Backport to Resolved
Also available in: Atom
PDF