You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by GitBox <gi...@apache.org> on 2021/09/03 19:29:22 UTC
[GitHub] [drill] vdiravka opened a new pull request #2309: DRILL-7996: Documentation for OCI OS Storage Plugin
vdiravka opened a new pull request #2309:
URL: https://github.com/apache/drill/pull/2309
# [DRILL-7996](https://issues.apache.org/jira/browse/DRILL-7996): Documentation for OCI OS Storage Plugin
## Description
This the DOC page for Drill web-site about Oracle Cloud Infrastructure Object Storage. See [DRILL-7995](https://issues.apache.org/jira/browse/DRILL-7995) for more info.
## Documentation
This changes fully about doc
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] cgivre commented on a change in pull request #2309: DRILL-7996: Documentation for OCI OS Storage Plugin
Posted by GitBox <gi...@apache.org>.
cgivre commented on a change in pull request #2309:
URL: https://github.com/apache/drill/pull/2309#discussion_r702121669
##########
File path: _docs/en/connect-a-data-source/plugins/111-OCI-OS-storage-plugin.md
##########
@@ -0,0 +1,44 @@
+---
+title: "OCI OS Storage Plugin"
+slug: "OCI OS Storage Plugin"
+parent: "Connect a Data Source"
+---
+Similar to S3 Storage Plugin Drill can be configured to query Oracle Cloud Infrastructure (OCI) Object Storage (OS).
+The ability to query this cloud storage is implemented by using Oracle HDFS library.
+
+To connect Drill to OCI OS:
+
+- Provide your OCI credentials.
+- Configure the OCI OS storage plugin with an OS bucket name.
+
+For additional information, refer to the [HDFS Connector for Object Storage](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/hdfsconnector.htm) documentation.
+
+## Configuring the OCI OS Storage Plugin
+
+The **Storage** page in the Drill Web UI provides an OS storage plugin that you configure to connect Drill to the OS distributed file system registered in `core-site.xml`. If you did not define your OCI credentials in the `core-site.xml` file, you can define them in the storage plugin configuration. You can define the credentials directly in the OS storage plugin configuration, or you can configure the OS storage plugin to use an external provider.
+
+To configure the OCI OS storage plugin, log in to the Drill Web UI at `http://<drill-hostname>:8047`. The `drill-hostname` is a node on which Drill is running. Go to the **Storage** page and click **Update** next to the OS storage plugin option or **Create** new if it doesn't exist yet.
+
+ {
+ "type": "file",
+ "connection": "oci://{bucket_name}@{namespace}/",
+ "config": {
+ "fs.oci.client.hostname": "https://objectstorage.us-ashburn-1.oraclecloud.com",
+ "fs.oci.client.auth.tenantId": "ocid1.tenancy.oc1..exampleuniqueID",
+ "fs.oci.client.auth.userId": "ocid1.user.oc1..exampleuniqueID",
+ "fs.oci.client.auth.fingerprint": "20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34",
Review comment:
@vdiravka
Is this fingerprint ok to publish here?
##########
File path: _docs/en/connect-a-data-source/plugins/111-OCI-OS-storage-plugin.md
##########
@@ -0,0 +1,44 @@
+---
+title: "OCI OS Storage Plugin"
+slug: "OCI OS Storage Plugin"
+parent: "Connect a Data Source"
+---
+Similar to S3 Storage Plugin Drill can be configured to query Oracle Cloud Infrastructure (OCI) Object Storage (OS).
+The ability to query this cloud storage is implemented by using Oracle HDFS library.
+
+To connect Drill to OCI OS:
+
+- Provide your OCI credentials.
+- Configure the OCI OS storage plugin with an OS bucket name.
+
+For additional information, refer to the [HDFS Connector for Object Storage](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/hdfsconnector.htm) documentation.
+
+## Configuring the OCI OS Storage Plugin
+
+The **Storage** page in the Drill Web UI provides an OS storage plugin that you configure to connect Drill to the OS distributed file system registered in `core-site.xml`. If you did not define your OCI credentials in the `core-site.xml` file, you can define them in the storage plugin configuration. You can define the credentials directly in the OS storage plugin configuration, or you can configure the OS storage plugin to use an external provider.
+
+To configure the OCI OS storage plugin, log in to the Drill Web UI at `http://<drill-hostname>:8047`. The `drill-hostname` is a node on which Drill is running. Go to the **Storage** page and click **Update** next to the OS storage plugin option or **Create** new if it doesn't exist yet.
+
+ {
+ "type": "file",
+ "connection": "oci://{bucket_name}@{namespace}/",
+ "config": {
+ "fs.oci.client.hostname": "https://objectstorage.us-ashburn-1.oraclecloud.com",
+ "fs.oci.client.auth.tenantId": "ocid1.tenancy.oc1..exampleuniqueID",
+ "fs.oci.client.auth.userId": "ocid1.user.oc1..exampleuniqueID",
+ "fs.oci.client.auth.fingerprint": "20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34",
+ "fs.oci.client.auth.pemfilepath": "/opt/drill/conf/oci_api_key.pem"
+ },
+ "workspaces": {
+ ...
+ }
+
+**Note:** The `"config"` block in the OS storage plugin configuration contains properties to define your OCI credentials. Do not include the `"config"` block in your OS storage plugin configuration if you defined your OCI credentials in the `core-site.xml` file.
+
+To configure the plugin in core-site.xml file, navigate to the $DRILL_HOME/conf or $DRILL_SITE directory, and rename the core-site-example.xml file to core-site.xml
+
+Configure the OS storage plugin configuration to use an external provider for credentials or directly add the credentials in the configuration itself, as described below. Click **Update** to save the configuration when done.
+
+## Providing OCI OS Credentials
Review comment:
I know it is documented elsewhere, but could we provide information about how to use the password vault and/or an example here as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] vdiravka commented on pull request #2309: DRILL-7996: Documentation for OCI OS Storage Plugin
Posted by GitBox <gi...@apache.org>.
vdiravka commented on pull request #2309:
URL: https://github.com/apache/drill/pull/2309#issuecomment-912775972
> @vdiravka
> Thanks for writing this up. A few minor changes, but @dzamo recently changed the way Drill is doing the web site and docs and I believe that this needs to go on this repo (https://github.com/apache/drill-site) rather than the `gh-pages` branch of the Drill repo.
Ah, exactly. Forgot about that. I'll add info about Vault Credential Provider and reopen it for https://github.com/apache/drill-site
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] dzamo commented on pull request #2309: DRILL-7996: Documentation for OCI OS Storage Plugin
Posted by GitBox <gi...@apache.org>.
dzamo commented on pull request #2309:
URL: https://github.com/apache/drill/pull/2309#issuecomment-912767158
Yes, this PR (and all future website-related PRs) should go to drill-site/master please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] vdiravka commented on a change in pull request #2309: DRILL-7996: Documentation for OCI OS Storage Plugin
Posted by GitBox <gi...@apache.org>.
vdiravka commented on a change in pull request #2309:
URL: https://github.com/apache/drill/pull/2309#discussion_r702130962
##########
File path: _docs/en/connect-a-data-source/plugins/111-OCI-OS-storage-plugin.md
##########
@@ -0,0 +1,44 @@
+---
+title: "OCI OS Storage Plugin"
+slug: "OCI OS Storage Plugin"
+parent: "Connect a Data Source"
+---
+Similar to S3 Storage Plugin Drill can be configured to query Oracle Cloud Infrastructure (OCI) Object Storage (OS).
+The ability to query this cloud storage is implemented by using Oracle HDFS library.
+
+To connect Drill to OCI OS:
+
+- Provide your OCI credentials.
+- Configure the OCI OS storage plugin with an OS bucket name.
+
+For additional information, refer to the [HDFS Connector for Object Storage](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/hdfsconnector.htm) documentation.
+
+## Configuring the OCI OS Storage Plugin
+
+The **Storage** page in the Drill Web UI provides an OS storage plugin that you configure to connect Drill to the OS distributed file system registered in `core-site.xml`. If you did not define your OCI credentials in the `core-site.xml` file, you can define them in the storage plugin configuration. You can define the credentials directly in the OS storage plugin configuration, or you can configure the OS storage plugin to use an external provider.
+
+To configure the OCI OS storage plugin, log in to the Drill Web UI at `http://<drill-hostname>:8047`. The `drill-hostname` is a node on which Drill is running. Go to the **Storage** page and click **Update** next to the OS storage plugin option or **Create** new if it doesn't exist yet.
+
+ {
+ "type": "file",
+ "connection": "oci://{bucket_name}@{namespace}/",
+ "config": {
+ "fs.oci.client.hostname": "https://objectstorage.us-ashburn-1.oraclecloud.com",
+ "fs.oci.client.auth.tenantId": "ocid1.tenancy.oc1..exampleuniqueID",
+ "fs.oci.client.auth.userId": "ocid1.user.oc1..exampleuniqueID",
+ "fs.oci.client.auth.fingerprint": "20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34",
+ "fs.oci.client.auth.pemfilepath": "/opt/drill/conf/oci_api_key.pem"
+ },
+ "workspaces": {
+ ...
+ }
+
+**Note:** The `"config"` block in the OS storage plugin configuration contains properties to define your OCI credentials. Do not include the `"config"` block in your OS storage plugin configuration if you defined your OCI credentials in the `core-site.xml` file.
+
+To configure the plugin in core-site.xml file, navigate to the $DRILL_HOME/conf or $DRILL_SITE directory, and rename the core-site-example.xml file to core-site.xml
+
+Configure the OS storage plugin configuration to use an external provider for credentials or directly add the credentials in the configuration itself, as described below. Click **Update** to save the configuration when done.
+
+## Providing OCI OS Credentials
Review comment:
I'll set up Vault and add the plugin configs example here then
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] dzamo closed pull request #2309: DRILL-7996: Documentation for OCI OS Storage Plugin
Posted by GitBox <gi...@apache.org>.
dzamo closed pull request #2309:
URL: https://github.com/apache/drill/pull/2309
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [drill] vdiravka commented on a change in pull request #2309: DRILL-7996: Documentation for OCI OS Storage Plugin
Posted by GitBox <gi...@apache.org>.
vdiravka commented on a change in pull request #2309:
URL: https://github.com/apache/drill/pull/2309#discussion_r702129924
##########
File path: _docs/en/connect-a-data-source/plugins/111-OCI-OS-storage-plugin.md
##########
@@ -0,0 +1,44 @@
+---
+title: "OCI OS Storage Plugin"
+slug: "OCI OS Storage Plugin"
+parent: "Connect a Data Source"
+---
+Similar to S3 Storage Plugin Drill can be configured to query Oracle Cloud Infrastructure (OCI) Object Storage (OS).
+The ability to query this cloud storage is implemented by using Oracle HDFS library.
+
+To connect Drill to OCI OS:
+
+- Provide your OCI credentials.
+- Configure the OCI OS storage plugin with an OS bucket name.
+
+For additional information, refer to the [HDFS Connector for Object Storage](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/hdfsconnector.htm) documentation.
+
+## Configuring the OCI OS Storage Plugin
+
+The **Storage** page in the Drill Web UI provides an OS storage plugin that you configure to connect Drill to the OS distributed file system registered in `core-site.xml`. If you did not define your OCI credentials in the `core-site.xml` file, you can define them in the storage plugin configuration. You can define the credentials directly in the OS storage plugin configuration, or you can configure the OS storage plugin to use an external provider.
+
+To configure the OCI OS storage plugin, log in to the Drill Web UI at `http://<drill-hostname>:8047`. The `drill-hostname` is a node on which Drill is running. Go to the **Storage** page and click **Update** next to the OS storage plugin option or **Create** new if it doesn't exist yet.
+
+ {
+ "type": "file",
+ "connection": "oci://{bucket_name}@{namespace}/",
+ "config": {
+ "fs.oci.client.hostname": "https://objectstorage.us-ashburn-1.oraclecloud.com",
+ "fs.oci.client.auth.tenantId": "ocid1.tenancy.oc1..exampleuniqueID",
+ "fs.oci.client.auth.userId": "ocid1.user.oc1..exampleuniqueID",
+ "fs.oci.client.auth.fingerprint": "20:3b:97:13:55:1c:5b:0d:d3:37:d8:50:4e:c5:3a:34",
Review comment:
Yes, it is :)
It is an example from https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/hdfsconnector.htm#ariaid-title4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org