You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by vi...@apache.org on 2021/09/07 09:58:00 UTC
[drill-site] branch master updated: DRILL-7996: Documentation for
OCI OS Storage Plugin (#8)
This is an automated email from the ASF dual-hosted git repository.
vitalii pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/drill-site.git
The following commit(s) were added to refs/heads/master by this push:
new a24547a DRILL-7996: Documentation for OCI OS Storage Plugin (#8)
a24547a is described below
commit a24547a79f6f20c8a8e8442512aa425ce3560b77
Author: Vitalii Diravka <vi...@apache.org>
AuthorDate: Tue Sep 7 12:56:52 2021 +0300
DRILL-7996: Documentation for OCI OS Storage Plugin (#8)
* DRILL-7996: Documentation for OCI OS Storage Plugin
- added page for Drill OCI OS Storage Plugin
- added "**Introduced in release:** 1.20"
- fix outdated info in S3 plugin
---
.../plugins/110-s3-storage-plugin.md | 2 +-
.../plugins/111-OCI-OS-storage-plugin.md | 71 ++++++++++++++++++++++
2 files changed, 72 insertions(+), 1 deletion(-)
diff --git a/_docs/en/connect-a-data-source/plugins/110-s3-storage-plugin.md b/_docs/en/connect-a-data-source/plugins/110-s3-storage-plugin.md
index e0fcfa0..9eead44 100644
--- a/_docs/en/connect-a-data-source/plugins/110-s3-storage-plugin.md
+++ b/_docs/en/connect-a-data-source/plugins/110-s3-storage-plugin.md
@@ -14,7 +14,7 @@ To connect Drill to S3:
For additional information, refer to the [HDFS S3 documentation](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html).
-**Note:** Drill does not use HDFS 3.x, therefore Drill does not support AWS temporary credentials, as described in the s3a documentation.
+**Note:** Drill started using HDFS 3.0, but support of AWS temporary credentials (as described in the s3a documentation) wasn't verified yet.
## Providing AWS Credentials
diff --git a/_docs/en/connect-a-data-source/plugins/111-OCI-OS-storage-plugin.md b/_docs/en/connect-a-data-source/plugins/111-OCI-OS-storage-plugin.md
new file mode 100644
index 0000000..97a8076
--- /dev/null
+++ b/_docs/en/connect-a-data-source/plugins/111-OCI-OS-storage-plugin.md
@@ -0,0 +1,71 @@
+---
+title: "OCI OS Storage Plugin"
+slug: "OCI OS Storage Plugin"
+parent: "Connect a Data Source"
+---
+
+**Introduced in release:** 1.20
+
+Similar to S3 Storage Plugin Drill can be configured to query Oracle Cloud Infrastructure (OCI) Object Storage (OS).
+The ability to query this cloud storage is implemented by using Oracle HDFS library.
+
+To connect Drill to OCI OS:
+
+- Provide your OCI credentials.
+- Configure the OCI OS storage plugin with an OS bucket name.
+
+For additional information, refer to the [HDFS Connector for Object Storage](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/hdfsconnector.htm) documentation.
+
+## Configuring the OCI OS Storage Plugin
+
+The **Storage** page in the Drill Web UI provides an OCI storage plugin that you configure to connect Drill to the OCI OS distributed file system registered in `core-site.xml`. If you did not define your OCI credentials in the `core-site.xml` file, you can define them in the storage plugin configuration. You can define the credentials directly in the OCI storage plugin configuration, or you can configure the OCI storage plugin to use an external provider.
+
+To configure the OCI OS storage plugin, log in to the Drill Web UI at `http://<drill-hostname>:8047`. The `drill-hostname` is a node on which Drill is running. Go to the **Storage** page and click **Update** next to the OCI OS storage plugin option or **Create** new if it doesn't exist yet.
+
+ {
+ "type": "file",
+ "connection": "oci://{bucket_name}@{namespace}/",
+ "config": {
+ "fs.oci.client.hostname": "https://objectstorage.us-ashburn-1.oraclecloud.com",
+ "fs.oci.client.auth.tenantId": "ocid1.tenancy.oc1..exampleuniqueID",
+ "fs.oci.client.auth.userId": "ocid1.user.oc1..exampleuniqueID",
+ "fs.oci.client.auth.fingerprint": "20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34",
+ "fs.oci.client.auth.pemfilepath": "/opt/drill/conf/oci_api_key.pem"
+ },
+ "workspaces": {
+ ...
+ }
+
+**Note:** The `"config"` block in the OCI storage plugin configuration contains properties to define your OCI credentials. Do not include the `"config"` block in your OCI OS storage plugin configuration if you defined your OCI credentials in the `core-site.xml` file.
+
+To configure the plugin in core-site.xml file, navigate to the $DRILL_HOME/conf or $DRILL_SITE directory, and rename the core-site-example.xml file to core-site.xml
+
+Configure the OCI OS storage plugin configuration to use an external provider for credentials or directly add the credentials in the configuration itself, as described below. Click **Update** to save the configuration when done.
+
+## Providing OCI OS Credentials
+
+You can use different [Plugin credentials provider](https://github.com/apache/drill/blob/master/docs/dev/PluginCredentialsProvider.md) for OCI OS Storage Plugin
+
+### Providing OCI OS Credentials
+
+[VaultCredentialsProvider](`VaultCredentialsProvider`) can be configured to keep OCI OS properties as a secrets. The example of Drill OCI Storage plugin with `VaultCredentialsProvider`:
+```
+{
+ "type": "file",
+ "connection": "oci://objectstorage-test-my-app-site-548c762a46e549ba@id1kowo7idm1/",
+ "workspaces": {...},
+ "formats": {...},
+ "credentialsProvider": {
+ "credentialsProviderType": "VaultCredentialsProvider",
+ "secretPath": "secret/drill/oci",
+ "propertyNames": {
+ "fs.oci.client.hostname": "fs.oci.client.hostname",
+ "fs.oci.client.auth.tenantId": "fs.oci.client.auth.tenantId",
+ "fs.oci.client.auth.userId": "fs.oci.client.auth.userId",
+ "fs.oci.client.auth.fingerprint": "fs.oci.client.auth.fingerprint",
+ "fs.oci.client.auth.pemfilepath": "fs.oci.client.auth.pemfilepath"
+ }
+ },
+ "enabled": true
+}
+```
\ No newline at end of file