Feature #48513
closeduses librgw2 to directly access the rados cluster for hadoop
0%
Description
Hadoop S3 accesses Ceph through the object gateway. The radosgw daemon process needs to be deployed. Messages are sent from Hadoop to radosgw and then to the rados cluster. In the new solution, Hadoop uses librgw2 to directly access the rados cluster, avoiding radosgw forwarding and improving performance.
Updated by chunsong feng over 3 years ago
The adapter invokes the librgw2 interface through JNI. The following interfaces need to be implemented:
static const JNINativeMethod gMethods[] = {
{"native_initialize", "()V", (void*)native_initialize},
{"native_ceph_create", "(Ljava/lang/String;)I", (void*)native_ceph_create},
{"native_ceph_mount",
"(Lcom/ceph/rgw/CephRgwAdapter;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)J",
(void*)native_ceph_mount},
{"native_ceph_unmount", "(J)I", (void*)native_ceph_unmount},
{"native_ceph_release", "(J)I", (void*)native_ceph_release},
{"native_ceph_statfs", "(JJLjava/lang/String;Lcom/ceph/rgw/CephStatVFS;)I", (void*)native_ceph_statfs},
{"native_ceph_listdir",
"(JJLjava/lang/String;Lcom/ceph/rgw/CephRgwAdapter$ListDirHandler;)I",
(void*)native_ceph_listdir},
{"native_ceph_unlink", "(JJLjava/lang/String;)I", (void*)native_ceph_unlink},
{"native_ceph_rename",
"(JJLjava/lang/String;Ljava/lang/String;JLjava/lang/String;Ljava/lang/String;)I",
(void*)native_ceph_rename},
{"native_ceph_mkdirs", "(JJLjava/lang/String;Ljava/lang/String;I)Z", (void*)native_ceph_mkdirs},
{"native_ceph_lstat", "(JJLjava/lang/String;Lcom/ceph/rgw/CephStat;)I", (void*)native_ceph_lstat},
{"native_ceph_setattr", "(JJLjava/lang/String;Lcom/ceph/rgw/CephStat;I)I", (void*)native_ceph_setattr},
{"native_ceph_open", "(JJLjava/lang/String;II)J", (void*)native_ceph_open},
{"native_ceph_close", "(JJ)I", (void*)native_ceph_close},
{"native_ceph_read", "(JJJ[BJ)J", (void*)native_ceph_read},
{"native_ceph_write", "(JJJ[BJ)J", (void*)native_ceph_write},
{"native_ceph_fsync", "(JJZ)I", (void*)native_ceph_fsync},
};