Project

General

Profile

Feature #58112

Add OpenMPI and ROMIO modules to support "striping_factor" from MPI standard

Added by Eric Chamberland over 1 year ago. Updated over 1 year ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

Hi,

I am creating this feature request but maybe there is already an ongoing work about adding CephFS support for MPI IO usages...?

The first thing that comes to my mind is about controlling the file creation with MPI hints, for example "striping_factor" which is a major performance-changer.

More parameters exists, see section "14.2.8 File Info" in https://www.mpi-forum.org/docs/mpi-4.0/mpi40-report.pdf, page 656 (or 696 of pdf).

"striping_unit" is another important parameter.

While discussing on OpenMPI list, Gilles Gouaillardet replied:


Hi Eric,

Currently, Open MPI does not provide specific support for CephFS.
MPI-IO is either implemented by ROMIO (imported from MPICH, it does not support CephFS today)
or the "native" ompio component (that also does not support CephFS today).

A proof of concept for CephFS in ompio might not be a huge work for someone motivated:
That could be as simple as (so to speak, since things are generally not easy) creating a new fs/ceph component
(e.g. in ompi/mca/fs/ceph) and implement the "file_open" callback that uses the ceph API.
I think the fs/lustre component can be used as an inspiration.

I cannot commit to do this, but if you are willing to take a crack at it, I can create such a component
so you can go directly to implementing the callback without spending too much time on some Open MPI internals
(e.g. component creation).

Cheers,

Gilles

So, I come here to ask because I feel I am not the best one to add support for CephFS, neither test it, into a so widely used library as OpenMPI or MPICH.

Thank you!

Eric

History

#1 Updated by Eric Chamberland over 1 year ago

People at OpenMPI created skeleton for CephFS support:

https://github.com/open-mpi/ompi/pull/11122

If anyone wants to jump in! :)

Eric

Also available in: Atom PDF