Ceph » rgw

same crash on pacific:

Oct 27 11:31:33 dael conmon[1881242]: *** Caught signal (Segmentation fault) **
Oct 27 11:31:33 dael conmon[1881242]:  in thread 7f84b85f6700 thread_name:ganesha.nfsd
Oct 27 11:31:33 dael conmon[1881242]:  ceph version 16.2.6-242-g9c8bdbc8 (9c8bdbc8afe90dacd18f2ee15044f48109538abe) pacific (stable)
Oct 27 11:31:33 dael conmon[1881242]:  1: /lib64/libpthread.so.0(+0x12b20) [0x7f852ad6eb20]
Oct 27 11:31:33 dael conmon[1881242]:  2: (rgw::ARN::ARN(rgw_bucket const&)+0x42) [0x7f84ec9a04d2]
Oct 27 11:31:33 dael conmon[1881242]:  3: (verify_bucket_permission(DoutPrefixProvider const*, perm_state_base*, rgw_bucket const&, RGWAccessControlPolicy*, RGWAccessControlPolicy*, boost::optional<rgw::IAM::Policy> const&, std::vector<rgw::IAM::Policy, std::allocator<rgw::IAM::Policy> > const&, std::vector<rgw::IAM::Policy, std::allocator<rgw::IAM::Policy> > const&, unsigned long)+0xa2) [0x7f84ec5e4a32]
Oct 27 11:31:33 dael conmon[1881242]:  4: (verify_bucket_permission(DoutPrefixProvider const*, req_state*, unsigned long)+0x83) [0x7f84ec5e5863]
Oct 27 11:31:33 dael conmon[1881242]:  5: (RGWListBucket::verify_permission(optional_yield)+0x12e) [0x7f84ec6a568e]
Oct 27 11:31:33 dael conmon[1881242]:  6: (rgw::RGWLibProcess::process_request(rgw::RGWLibRequest*, rgw::RGWLibIO*)+0xd66) [0x7f84ec504b86]
Oct 27 11:31:33 dael conmon[1881242]:  7: (rgw::RGWLibProcess::process_request(rgw::RGWLibRequest*)+0x49) [0x7f84ec505d09]
Oct 27 11:31:33 dael conmon[1881242]:  8: (rgw::RGWFileHandle::readdir(bool (*)(char const*, void*, unsigned long, stat*, unsigned int, unsigned int), void*, boost::variant<unsigned long*, char const*>, bool*, unsigned int)+0x184) [0x7f84ec51b2a4]
Oct 27 11:31:33 dael conmon[1881242]:  9: rgw_readdir2()
Oct 27 11:31:33 dael conmon[1881242]:  10: /usr/lib64/ganesha/libfsalrgw.so(+0x629c) [0x7f850c3eb29c]
Oct 27 11:31:33 dael conmon[1881242]:  11: /lib64/libganesha_nfsd.so.3.5(+0x11fd87) [0x7f852c8fcd87]
Oct 27 11:31:33 dael conmon[1881242]:  12: /lib64/libganesha_nfsd.so.3.5(+0x1215b3) [0x7f852c8fe5b3]
Oct 27 11:31:33 dael conmon[1881242]:  13: fsal_readdir()
Oct 27 11:31:33 dael conmon[1881242]:  14: /lib64/libganesha_nfsd.so.3.5(+0xe2b22) [0x7f852c8bfb22]
Oct 27 11:31:33 dael conmon[1881242]:  15: /lib64/libganesha_nfsd.so.3.5(+0xcacaf) [0x7f852c8a7caf]
Oct 27 11:31:33 dael conmon[1881242]:  16: /lib64/libganesha_nfsd.so.3.5(+0xcbdf7) [0x7f852c8a8df7]
Oct 27 11:31:33 dael conmon[1881242]:  17: /lib64/libganesha_nfsd.so.3.5(+0x4d116) [0x7f852c82a116]
Oct 27 11:31:33 dael conmon[1881242]:  18: /lib64/libntirpc.so.3.4(+0x22efc) [0x7f852c5bcefc]
Oct 27 11:31:33 dael conmon[1881242]:  19: /lib64/libntirpc.so.3.4(+0x20301) [0x7f852c5ba301]
Oct 27 11:31:33 dael conmon[1881242]:  20: /lib64/libntirpc.so.3.4(+0x20db8) [0x7f852c5badb8]
Oct 27 11:31:33 dael conmon[1881242]:  21: /lib64/libntirpc.so.3.4(+0x2b584) [0x7f852c5c5584]
Oct 27 11:31:33 dael conmon[1881242]:  22: /lib64/libpthread.so.0(+0x814a) [0x7f852ad6414a]
Oct 27 11:31:33 dael conmon[1881242]:  23: clone()

If I try the octopus container I get EACCESS from getdents64

Hi Folks,

I was able to build and run a ceph master librgw with nfs-ganesha next, as of thursday afternoon, without apparent problems, using a ganesha.conf based on the one provided.

A couple of notes:

1. I did create a "sage" user using radosgw-adamin and used it as the mounting user, as implied
2. the ganesha.conf is (exactly) as below--so the only differences from a cephadm-generated one would be in the rados-url stubs and substitution, so possibly the RGW {} config section could differ slightly, but that seems unlikely to make any kind of difference

have a look:

###################################################
#
# EXPORT
#
# To function, all that is required is an EXPORT
#
# Define the absolute minimal export
#
###################################################

EXPORT {
    FSAL {
        name = "RGW";
        user_id = "sage";
        access_key_id = "5MRIMLJVB1ODIEREBD08";
        secret_access_key = "TlfACgAqHZclGlAZbF3aoRof8ibkDqsiLYh0ADV3";
    }
    export_id = 2;
    path = "";
    #path = "/"; # I thought path might need to be /, but "" worked fine
    pseudo = "/sage";>
    access_type = "RW";
    squash = "none";
    attr_expiration_time = 0;
    security_label = true;
    protocols = 4;
    transports = "TCP";
}

RGW {
    ceph_conf = "/home/mbenjamin/ceph-cp/build/ceph.conf";
    #init_args = "--debug-rgw=16 --log-file=/tmp/nfs.log";
    init_args = "--debug-rgw=20";
}

NFS_Core_Param {
        #Nb_Worker = 3;
    #RPC_Debug_Flags = 2147483648;
        mount_path_pseudo = true;
}

Cache_Inode {
    #Dir_Max = 10000;

    #Dir_Max = 200000;
    #Cache_Size = 1000000;
    #Entries_Hwmark = 1000000;
    #Dir_Chunk = 0;

#Cache_Size = 191;
    #Entries_Hwmark = 191;
}

NFSV4 {
    Graceless = true;
    Allow_Numeric_Owners = true;
    Only_Numeric_Owners = true;
}

LOG {
    # Default_Log_Level is unknown token??
    #Default_Log_Level = NIV_FULL_DEBUG;
    Components {
        #ALL = FULL_DEBUG;
        MEMLEAKS = FATAL;
        FSAL = FATAL;
        NFSPROTO = FULL_DEBUG;
        NFS_V4 = FULL_DEBUG;
        EXPORT = FATAL;
        FILEHANDLE = FATAL;
        DISPATCH = FATAL;
        CACHE_INODE = FATAL;
        CACHE_INODE_LRU = FATAL;
        HASHTABLE = FATAL;
        HASHTABLE_CACHE = FATAL;
        DUPREQ = FATAL;
        INIT = FATAL;
        MAIN = FATAL;
        IDMAPPER = FATAL;
        NFS_READDIR = FULL_DEBUG;
        NFS_V4_LOCK = FATAL;
        CONFIG = FATAL;
        CLIENTID = FATAL;
        SESSIONS = FATAL;
        PNFS = FATAL;
        RW_LOCK = FATAL;
        NLM = FATAL;
        RPC = FATAL;
        NFS_CB = FATAL;
        THREAD = FATAL;
        NFS_V4_ACL = FATAL;
        STATE = FATAL;
#        9P = FATAL;
#        9P_DISPATCH = FATAL;
        FSAL_UP = FATAL;
        DBUS = FATAL;
    }
    Facility {
        name = FILE;
        destination = "/tmp/ganesha-rgw.log";
        enable = active;
    }
}