[incubator-pinot] branch master updated: Adding pluggable storage docs (#3751)

[je...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

jenniferdai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git


The following commit(s) were added to refs/heads/master by this push:
     new 42d00ce  Adding pluggable storage docs (#3751)
42d00ce is described below

commit 42d00ce151086216c9da5c8d0245c80c8b6cac8b
Author: Jennifer Dai <je...@users.noreply.github.com>
AuthorDate: Tue Jan 29 10:12:14 2019 -0800

    Adding pluggable storage docs (#3751)
---
 docs/index.rst             |  1 +
 docs/pluggable_storage.rst | 21 +++++++++++++++++++++
 2 files changed, 22 insertions(+)

diff --git a/docs/index.rst b/docs/index.rst
index f1defaa..f929f57 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -36,6 +36,7 @@ Customizing Pinot
 
    pluggable_streams
    segment_fetcher
+   pluggable_storage
 
 ################
 Design Documents
diff --git a/docs/pluggable_storage.rst b/docs/pluggable_storage.rst
new file mode 100644
index 0000000..f276072
--- /dev/null
+++ b/docs/pluggable_storage.rst
@@ -0,0 +1,21 @@
+Pluggable Storage
+=================
+
+Pinot enables its users to write a PinotFS abstraction layer to store data in a source of truth data layer of their
+choice for offline segments. We do not yet have support for realtime consumption in deep storage.
+
+Some examples of storage backends(other than local storage) currently supported are:
+
+* `HadoopFS <https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/FileSystemShell.html>`_
+* `Azure Data Lake <https://azure.microsoft.com/en-us/solutions/data-lake/>`_
+
+If the above two filesystems do not meet your needs, please feel free to get in touch with us,
+and we can help you out.
+
+New Storage Type implementation
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+In order to add a new type of storage backend (say, Amazon s3) implement the following class:
+
+#. S3FS extends `PinotFS <https://github.com/apache/incubator-pinot/blob/master/pinot-filesystem/src/main/java/org/apache/pinot/filesystem/PinotFS.java>`_
+
+The properties for the stream implementation are to be set in your controller and server configurations, `like so <https://github.com/apache/incubator-pinot/wiki/Pluggable-Storage>`_.
\ No newline at end of file


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org