You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@predictionio.apache.org by do...@apache.org on 2018/08/26 23:25:32 UTC
[27/40] predictionio git commit: [PIO-151] Add S3 storage docs
[PIO-151] Add S3 storage docs
This closes #449
Project: http://git-wip-us.apache.org/repos/asf/predictionio/repo
Commit: http://git-wip-us.apache.org/repos/asf/predictionio/commit/c1270e23
Tree: http://git-wip-us.apache.org/repos/asf/predictionio/tree/c1270e23
Diff: http://git-wip-us.apache.org/repos/asf/predictionio/diff/c1270e23
Branch: refs/heads/master
Commit: c1270e23b00ff459e76a4e9dde589bb2e1d0d0db
Parents: e156cb4
Author: James Ward <ja...@jamesward.org>
Authored: Thu Mar 1 13:26:35 2018 -0800
Committer: Chan Lee <ch...@gmail.com>
Committed: Thu Mar 1 13:48:59 2018 -0800
----------------------------------------------------------------------
.../source/system/anotherdatastore.html.md | 43 ++++++++++++++++++++
1 file changed, 43 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/predictionio/blob/c1270e23/docs/manual/source/system/anotherdatastore.html.md
----------------------------------------------------------------------
diff --git a/docs/manual/source/system/anotherdatastore.html.md b/docs/manual/source/system/anotherdatastore.html.md
index b3a3709..071a197 100644
--- a/docs/manual/source/system/anotherdatastore.html.md
+++ b/docs/manual/source/system/anotherdatastore.html.md
@@ -114,6 +114,10 @@ PredictionIO comes with the following sources:
* Type name is **hdfs**.
* Can be used for *Model Data* repository
+- **S3**:
+ * Type name is **s3**.
+ * Can be used for *Model Data* repository
+
Each repository can be configured to use different sources as shown above.
Each source has its own set of configuration parameters. Configuration variables will be explained in more details in later sections below (see Data Store Configuration).
@@ -293,6 +297,45 @@ supported.
`PIO_STORAGE_SOURCES_HDFS_PATH=/mymodels`
+#### S3 Configuration
+
+Variable Format: `PIO_STORAGE_SOURCES_[NAME]_TYPE=s3`
+
+Supported Repositories: **model**
+
+To provide authentication information, you can set the `AWS_ACCESS_KEY_ID`
+and `AWS_SECRET_ACCESS_KEY` environment variables or use one of the other
+methods in the [AWS Setup Docs](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html#config-settings-and-precedence)
+
+When `TYPE` is set to `s3`, the following configuration keys are
+supported.
+
+- REGION (mandatory)
+
+ AWS Region to use, e.g.
+ `PIO_STORAGE_SOURCES_S3_REGION=us-east-1`
+
+- BUCKET_NAME (mandatory)
+
+ S3 Bucket where models are stored, e.g.
+ `PIO_STORAGE_SOURCES_S3_BUCKET_NAME=pio_bucket`
+
+- BASE_PATH (optional)
+
+ S3 base path where models are stored, e.g.
+ `PIO_STORAGE_SOURCES_S3_BASE_PATH=pio_model`
+
+- DISABLE_CHUNKED_ENCODING (optional)
+
+ Disable the use of Chunked Encoding when transferring files to/from S3, e.g.
+ `PIO_STORAGE_SOURCES_S3_DISABLE_CHUNKED_ENCODING=true`
+
+- ENDPOINT (optional)
+
+ S3 Endpoint to use, e.g.
+ `PIO_STORAGE_SOURCES_S3_ENDPOINT=http://localstack:4572`
+
+
## Adding Support of Other Backends
It is quite straightforward to implement support of other backends. A good