You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Mukul Kumar Singh (Jira)" <ji...@apache.org> on 2021/04/21 09:47:00 UTC
[jira] [Comment Edited] (HDDS-4977) Ozone go client and FUSE driver
[ https://issues.apache.org/jira/browse/HDDS-4977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17325956#comment-17325956 ]
Mukul Kumar Singh edited comment on HDDS-4977 at 4/21/21, 9:46 AM:
-------------------------------------------------------------------
Can we please add more details about the use cases which we are planning to solve in here.
The kind of API's which need to be developed in order to support a Go Client or a non JVM client for Ozone.
was (Author: msingh):
Can we please add more details about the use cases which we are planning to solve in here.
The kind of API's which need to be developed in order to support a Go Client or a non JVM client for Ozone.
The current proposal is incomplete as it does not talk about the details of the API usages. Whats kind of use cases are we planning to solve with this proposal here ?
> Ozone go client and FUSE driver
> -------------------------------
>
> Key: HDDS-4977
> URL: https://issues.apache.org/jira/browse/HDDS-4977
> Project: Apache Ozone
> Issue Type: New Feature
> Reporter: Marton Elek
> Priority: Major
>
> Based on the [discussion|https://lists.apache.org/thread.html/rf7dc7f417188103cd76cdc0536b11af8a089bb23e3faa388e82c7d23%40%3Cdev.ozone.apache.org%3E] on the dev list a Golang client can help the adoption of Apache Ozone in non-jvm environment.
> Ozone go-client
> Apache Ozone is an object store for Hadoop ecosystem and cloud-native environments. As it was originally developed as a Hadoop subproject it was implemented in Java language.
> One of the power of Ozone compared to Hdfs that it can be used with multiple interfaces: it supports Hadoop Compatible File Systems, S3 compatible interfaces, and it also supports CSI specification (but today only with the help of 3rd party fuse driver.
> These interfaces help to use Ozone from multiple type of environments but Ozone provides only Java based SDK which makes harder the adoption for non JVM languages.
> This Jira proposes to create a Go based Apache Ozone SDK and additional tools on top of it:
> • With the help of a Go SDK we can create easily native Fuse drivers which can be an additional option for the existing Ozone CSI driver (in addition to the existing, 3rd party S3 fuse drivers)
> • With the help of the Go SDK we can create a native C library which can be used from many other languages (like Python or Rust)
> Implementation details:
> The Go SDK would follow the design of the Java based Ozone Client:
> • We need low-level Om and Datanode (Grpc and Ratis) clients
> • And we can provide a higher level interface which uses those clients and provides better programming experience
> • As a first step, fuse driver can be based (github.com/hanwen/go-fuse) which already implements the file-system like interface/abstract. This implementation would use the low-level clients of the Go SDK.
> • C library (and headers) also can be provided using the low-level API which can be tested with Python
> • A simple command line tool can be created (similar to ozone sh) which uses the higher level interface. It can be used to test the clients from similar approach as the Ozone main repository is tested (robotframework, docker-compose)
> • Goals for the first phase: Implement fully functional Fuse driver + CLI for non-secure, non-HA clusters. It would be enough to test the functionality in multiple environments (including testing it as an underlying implementation for the CSI driver)
> • In next phase we ca incrementally implement HA support and security
> In the current form, it’s not supposed to be released, and it should be clearly communicated that all of these elements are just experiments and poc. Once we have production ready SDK / Fuse driver / CLI we can start separate discussion about release lifecycle.
> Initial features (poc) are demostrated here: https://youtu.be/ouNxn9i5DcM
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org