You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@predictionio.apache.org by ch...@apache.org on 2018/03/01 21:36:44 UTC
[7/8] predictionio git commit: [PIO-151] Add S3 storage docs
[PIO-151] Add S3 storage docs
commit 8fbf38c2e4f8bb20a8e2c4ef43e97f8a18931336
Author: James Ward <ja...@jamesward.org>
Date: Thu Dec 14 15:06:08 2017 -0700
add S3 storage provider docs
Project: http://git-wip-us.apache.org/repos/asf/predictionio/repo
Commit: http://git-wip-us.apache.org/repos/asf/predictionio/commit/6eda07fd
Tree: http://git-wip-us.apache.org/repos/asf/predictionio/tree/6eda07fd
Diff: http://git-wip-us.apache.org/repos/asf/predictionio/diff/6eda07fd
Branch: refs/heads/develop
Commit: 6eda07fdc3732b83840e6b4a8d55d0a7af18e890
Parents: e156cb4
Author: James Ward <ja...@jamesward.org>
Authored: Thu Mar 1 13:26:35 2018 -0800
Committer: Chan Lee <ch...@gmail.com>
Committed: Thu Mar 1 13:26:35 2018 -0800
----------------------------------------------------------------------
.../source/system/anotherdatastore.html.md | 43 ++++++++++++++++++++
1 file changed, 43 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/predictionio/blob/6eda07fd/docs/manual/source/system/anotherdatastore.html.md
----------------------------------------------------------------------
diff --git a/docs/manual/source/system/anotherdatastore.html.md b/docs/manual/source/system/anotherdatastore.html.md
index b3a3709..071a197 100644
--- a/docs/manual/source/system/anotherdatastore.html.md
+++ b/docs/manual/source/system/anotherdatastore.html.md
@@ -114,6 +114,10 @@ PredictionIO comes with the following sources:
* Type name is **hdfs**.
* Can be used for *Model Data* repository
+- **S3**:
+ * Type name is **s3**.
+ * Can be used for *Model Data* repository
+
Each repository can be configured to use different sources as shown above.
Each source has its own set of configuration parameters. Configuration variables will be explained in more details in later sections below (see Data Store Configuration).
@@ -293,6 +297,45 @@ supported.
`PIO_STORAGE_SOURCES_HDFS_PATH=/mymodels`
+#### S3 Configuration
+
+Variable Format: `PIO_STORAGE_SOURCES_[NAME]_TYPE=s3`
+
+Supported Repositories: **model**
+
+To provide authentication information, you can set the `AWS_ACCESS_KEY_ID`
+and `AWS_SECRET_ACCESS_KEY` environment variables or use one of the other
+methods in the [AWS Setup Docs](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#config-settings-and-precedence)
+
+When `TYPE` is set to `s3`, the following configuration keys are
+supported.
+
+- REGION (mandatory)
+
+ AWS Region to use, e.g.
+ `PIO_STORAGE_SOURCES_S3_REGION=us-east-1`
+
+- BUCKET_NAME (mandatory)
+
+ S3 Bucket where models are stored, e.g.
+ `PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket`
+
+- BASE_PATH (optional)
+
+ S3 base path where models are stored, e.g.
+ `PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model`
+
+- DISABLE_CHUNKED_ENCODING (optional)
+
+ Disable the use of Chunked Encoding when transferring files to/from S3, e.g.
+ `PIO_STORAGE_SOURCES_S3_DISABLE_CHUNKED_ENCODING=true`
+
+- ENDPOINT (optional)
+
+ S3 Endpoint to use, e.g.
+ `PIO_STORAGE_SOURCES_S3_ENDPOINT=http://localstack:4572`
+
+
## Adding Support of Other Backends
It is quite straightforward to implement support of other backends. A good