Project

General

Profile

Actions

Feature #47277

closed

implement new mount "device" syntax for kcephfs

Added by Jeff Layton over 3 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Component(FS):
kceph
Labels (FS):
Pull request ID:

Description

Currently, a mount has to pass in a device string like this:

mon_addr1,mon_addr2:/path

It's problematic for a couple of reasons:

  1. mounts to the same cluster but with different fsnames or creds have identical device strings. This is confusing for things like xfstests
  2. the mount helper can fill in the mon addrs automatically, which is good, but that means that the device shown in /proc/mounts is different from what was mounted
  3. if mon addrs change, then that won't be reflected in the device string

Change this to a new device string format. Possibly something like this:

<name>@<fs_name>[.<ceph_fsid>]:/path

Note that there is no place there for mon addrs. Those would have to move to a new (mandatory) mount option, which would also make it cleaner for remounts after mon addrs have changed.

This will require work in both the kernel and userland mount helper. The difficult part here will be dealing with old kernel + new userland or vice versa. The kernel will need to be able to accept both syntax variants, and the userland mount helper will most likely need to be able to try the new syntax and fall back to the old if it's not recognized.


Related issues 1 (0 open1 closed)

Related to CephFS - Bug #47276: MDSMonitor: add command to rename file systemsResolvedRamana Raja

Actions
Actions #1

Updated by Jeff Layton over 3 years ago

Proposed syntax looks wrong in the description. I meant this:

[name@][fs_name[.ceph_fsid]]:/path

Note that if the name and fs_name and ceph_fsid are not specified, then the syntax looks more or less identical to the old one.

Actions #2

Updated by Patrick Donnelly over 3 years ago

  • Related to Bug #47276: MDSMonitor: add command to rename file systems added
Actions #3

Updated by Patrick Donnelly over 3 years ago

Jeff Layton wrote:

Proposed syntax looks wrong in the description. I meant this:

[...]

Note that if the name and fs_name and ceph_fsid are not specified, then the syntax looks more or less identical to the old one.

A word of caution, we probably don't want to make too many parts of the device id optional. I'm thinking we should require "name@fsname" at the very least. "guest" is the default name which I don't think is very common in the wild. That means most people are specifying the name in the mount options but we'd really rather they do that in the device name. So let's require it!

Further, Jeff noted that ":/" is searched for the in the device string in existing kernels. Since we're changing the syntax, we should avoid that pattern so that old kernels don't try to parse a new format device string. This allows a newer userspace mount.ceph to fallback to the old format if the kernel is too old. To achieve that, I suggest this format:

name@fsname[.ceph_fsid]=/path

Although, I don't feel super strongly about "="; I'm open to alternatives.

Actions #4

Updated by Venky Shankar over 3 years ago

  • Assignee changed from Jeff Layton to Venky Shankar

I will start taking a look next week

Actions #5

Updated by Venky Shankar over 3 years ago

Patrick Donnelly wrote:

Jeff Layton wrote:

Proposed syntax looks wrong in the description. I meant this:

[...]

Note that if the name and fs_name and ceph_fsid are not specified, then the syntax looks more or less identical to the old one.

A word of caution, we probably don't want to make too many parts of the device id optional. I'm thinking we should require "name@fsname" at the very least. "guest" is the default name which I don't think is very common in the wild. That means most people are specifying the name in the mount options but we'd really rather they do that in the device name. So let's require it!

Further, Jeff noted that ":/" is searched for the in the device string in existing kernels. Since we're changing the syntax, we should avoid that pattern so that old kernels don't try to parse a new format device string. This allows a newer userspace mount.ceph to fallback to the old format if the kernel is too old. To achieve that, I suggest this format:

[...]

Although, I don't feel super strongly about "="; I'm open to alternatives.

The "=" is a bit offputting. FWIW, mount helper tries to resolve (parse host/IP:port + getnameinfo()) the mon addr before passing it on the kernel. On (resolve) failure, mount helper could then try to interpret it as "<name>@<fsname>" format. Would that suffice?

Actions #6

Updated by Jeff Layton over 3 years ago

Venky Shankar wrote:

The "=" is a bit offputting. FWIW, mount helper tries to resolve (parse host/IP:port + getnameinfo()) the mon addr before passing it on the kernel. On (resolve) failure, mount helper could then try to interpret it as "<name>@<fsname>" format. Would that suffice?

Agreed. I think we should avoid '=' as that has connotations in scripts, etc. and may need to be escaped in some cases.

I would probably not try to resolve names first. That can be slow if DNS is unavailable and it could be a security issue too if someone can add a bogus DNS entry. If you're going to require name@fsname format, then you could just use the presence of '@' in the string as a hint that this is the new format?

Actions #7

Updated by Jeff Layton over 3 years ago

One idea might be to just get rid of the ':' ?

name@fsname[.fscid]/path

...but that fsname/path looks like a path itself. Another idea is to just replace it with '+':

name@fsname[.fscid]+/path

...none of them look as nice as ":/" though.

Actions #8

Updated by Patrick Donnelly over 3 years ago

Venky Shankar wrote:

Patrick Donnelly wrote:

Jeff Layton wrote:

Proposed syntax looks wrong in the description. I meant this:

[...]

Note that if the name and fs_name and ceph_fsid are not specified, then the syntax looks more or less identical to the old one.

A word of caution, we probably don't want to make too many parts of the device id optional. I'm thinking we should require "name@fsname" at the very least. "guest" is the default name which I don't think is very common in the wild. That means most people are specifying the name in the mount options but we'd really rather they do that in the device name. So let's require it!

Further, Jeff noted that ":/" is searched for the in the device string in existing kernels. Since we're changing the syntax, we should avoid that pattern so that old kernels don't try to parse a new format device string. This allows a newer userspace mount.ceph to fallback to the old format if the kernel is too old. To achieve that, I suggest this format:

[...]

Although, I don't feel super strongly about "="; I'm open to alternatives.

The "=" is a bit offputting.

The "=" is strange but it's not parsed by the shell already so I think it's advantageous in that regard.

FWIW, mount helper tries to resolve (parse host/IP:port + getnameinfo()) the mon addr before passing it on the kernel. On (resolve) failure, mount helper could then try to interpret it as "<name>@<fsname>" format. Would that suffice?

"@" is not a valid char in a ipaddr/hostname so I think just checking for that first makes sense to me.

Actions #9

Updated by Venky Shankar over 3 years ago

Patrick Donnelly wrote:

Venky Shankar wrote:

Patrick Donnelly wrote:

Jeff Layton wrote:

Proposed syntax looks wrong in the description. I meant this:

[...]

Note that if the name and fs_name and ceph_fsid are not specified, then the syntax looks more or less identical to the old one.

A word of caution, we probably don't want to make too many parts of the device id optional. I'm thinking we should require "name@fsname" at the very least. "guest" is the default name which I don't think is very common in the wild. That means most people are specifying the name in the mount options but we'd really rather they do that in the device name. So let's require it!

Further, Jeff noted that ":/" is searched for the in the device string in existing kernels. Since we're changing the syntax, we should avoid that pattern so that old kernels don't try to parse a new format device string. This allows a newer userspace mount.ceph to fallback to the old format if the kernel is too old. To achieve that, I suggest this format:

[...]

Although, I don't feel super strongly about "="; I'm open to alternatives.

The "=" is a bit offputting.

The "=" is strange but it's not parsed by the shell already so I think it's advantageous in that regard.

FWIW, mount helper tries to resolve (parse host/IP:port + getnameinfo()) the mon addr before passing it on the kernel. On (resolve) failure, mount helper could then try to interpret it as "<name>@<fsname>" format. Would that suffice?

"@" is not a valid char in a ipaddr/hostname so I think just checking for that first makes sense to me.

ACK. Let's stick with ":/" and differentiate based on "@" in the device string.

Actions #10

Updated by Patrick Donnelly over 3 years ago

Venky Shankar wrote:

Patrick Donnelly wrote:

Venky Shankar wrote:

Patrick Donnelly wrote:

Jeff Layton wrote:

Proposed syntax looks wrong in the description. I meant this:

[...]

Note that if the name and fs_name and ceph_fsid are not specified, then the syntax looks more or less identical to the old one.

A word of caution, we probably don't want to make too many parts of the device id optional. I'm thinking we should require "name@fsname" at the very least. "guest" is the default name which I don't think is very common in the wild. That means most people are specifying the name in the mount options but we'd really rather they do that in the device name. So let's require it!

Further, Jeff noted that ":/" is searched for the in the device string in existing kernels. Since we're changing the syntax, we should avoid that pattern so that old kernels don't try to parse a new format device string. This allows a newer userspace mount.ceph to fallback to the old format if the kernel is too old. To achieve that, I suggest this format:

[...]

Although, I don't feel super strongly about "="; I'm open to alternatives.

The "=" is a bit offputting.

The "=" is strange but it's not parsed by the shell already so I think it's advantageous in that regard.

FWIW, mount helper tries to resolve (parse host/IP:port + getnameinfo()) the mon addr before passing it on the kernel. On (resolve) failure, mount helper could then try to interpret it as "<name>@<fsname>" format. Would that suffice?

"@" is not a valid char in a ipaddr/hostname so I think just checking for that first makes sense to me.

ACK. Let's stick with ":/" and differentiate based on "@" in the device string.

That helps the userspace mount tool detect the new format but it is not what we want for the device string passed to the kernel through the mount syscall. We want to rely on the behavior in older kernels that reject device strings not containing ":/". So, the userspace mount tool can by default always start by trying "=/path" with the mount syscall and then fallback to "monip1,monip2:/path".

Actions #11

Updated by Jeff Layton over 3 years ago

There are other alternates too, fwiw (e.g.):

name@fs#/path

...or maybe just omit the ':' or anything to replace it:

name@fs/path

The main thing is to make sure you test all of the different combinations of the new format you choose with old kernel + old userspace, so that we know how current OS's in the field will react to it.

Actions #12

Updated by Venky Shankar over 3 years ago

Patrick Donnelly wrote:

Venky Shankar wrote:

Patrick Donnelly wrote:

Venky Shankar wrote:

Patrick Donnelly wrote:

Jeff Layton wrote:

Proposed syntax looks wrong in the description. I meant this:

[...]

Note that if the name and fs_name and ceph_fsid are not specified, then the syntax looks more or less identical to the old one.

A word of caution, we probably don't want to make too many parts of the device id optional. I'm thinking we should require "name@fsname" at the very least. "guest" is the default name which I don't think is very common in the wild. That means most people are specifying the name in the mount options but we'd really rather they do that in the device name. So let's require it!

Further, Jeff noted that ":/" is searched for the in the device string in existing kernels. Since we're changing the syntax, we should avoid that pattern so that old kernels don't try to parse a new format device string. This allows a newer userspace mount.ceph to fallback to the old format if the kernel is too old. To achieve that, I suggest this format:

[...]

Although, I don't feel super strongly about "="; I'm open to alternatives.

The "=" is a bit offputting.

The "=" is strange but it's not parsed by the shell already so I think it's advantageous in that regard.

FWIW, mount helper tries to resolve (parse host/IP:port + getnameinfo()) the mon addr before passing it on the kernel. On (resolve) failure, mount helper could then try to interpret it as "<name>@<fsname>" format. Would that suffice?

"@" is not a valid char in a ipaddr/hostname so I think just checking for that first makes sense to me.

ACK. Let's stick with ":/" and differentiate based on "@" in the device string.

That helps the userspace mount tool detect the new format but it is not what we want for the device string passed to the kernel through the mount syscall. We want to rely on the behavior in older kernels that reject device strings not containing ":/". So, the userspace mount tool can by default always start by trying "=/path" with the mount syscall and then fallback to "monip1,monip2:/path".

Right. I wasn't clear mentioning that, sorry!

Actions #13

Updated by Venky Shankar almost 3 years ago

  • Status changed from New to In Progress
Actions #14

Updated by Venky Shankar almost 3 years ago

I did some work on this a while back but never go to posting a patch (or changing the tracker status). I'm reviving the changes this week and will push out an RFC patch.

Actions #15

Updated by Venky Shankar almost 3 years ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 41334
Actions #16

Updated by Venky Shankar over 2 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF