You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2016/11/29 22:42:49 UTC
[11/19] drill git commit: add new sql commands for Dynamic UDFs
add new sql commands for Dynamic UDFs
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/b9b1d234
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/b9b1d234
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/b9b1d234
Branch: refs/heads/gh-pages
Commit: b9b1d23411a85644e96c8719591840372a86de61
Parents: 02fd4b1
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Mon Nov 21 16:22:07 2016 -0800
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Mon Nov 21 16:22:07 2016 -0800
----------------------------------------------------------------------
...ng-custom-functions-to-drill-introduction.md | 6 ++--
.../020-dynamic-udfs.md | 16 +++++-----
.../034-create-function-using-jar.md | 29 ++++++++++++++++++
.../sql-commands/054-drop-function-using-jar.md | 32 ++++++++++++++++++++
4 files changed, 72 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/b9b1d234/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md b/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
index 06a391d..2a5ead2 100644
--- a/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
+++ b/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
@@ -1,10 +1,10 @@
---
title: "Adding Custom Functions to Drill Introduction"
-date: 2016-11-21 21:25:56 UTC
+date: 2016-11-22 00:22:09 UTC
parent: "Adding Custom Functions to Drill"
---
As of Drill 1.9, there are two methods for adding custom functions to Drill. An administrator can manually add functions to Drill, or provide users access to a staging directory where they can upload JAR files and register their UDFs using the CREATE FUNCTION USING JAR command. The CREATE FUNCTION USING JAR command is part of the Dynamic UDF feature.
-- For manual instructions, see Manually Adding Custom Functions to Drill.
-- For Dynamic UDF instructions, see Dynamic UDFs.
+- For manual instructions, see [Manually Adding Custom Functions to Drill]({{site.baseurl}}/docs/manually-adding-custom-functions-to-drill/).
+- For Dynamic UDF information and instructions, see [Dynamic UDFs]({{site.baseurl}}/docs/dynamic-udfs/).
http://git-wip-us.apache.org/repos/asf/drill/blob/b9b1d234/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md b/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md
index 29d4114..85868aa 100644
--- a/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md
+++ b/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md
@@ -1,6 +1,6 @@
---
title: "Dynamic UDFs"
-date: 2016-11-21 21:25:57 UTC
+date: 2016-11-22 00:22:09 UTC
parent: "Adding Custom Functions to Drill"
---
@@ -10,7 +10,7 @@ The Dynamic UDF feature eliminates the need to restart drillbits, which can disr
The Dynamic UDF feature is enabled by default. An administrator can enable or disable the feature using the ALTER SYSTEM SET command with the `exec.udf.enable_dynamic_support option`. When the feature is enabled, users must upload their UDF (source and binary) JAR files to a staging directory in the distributed file system before issuing the CREATE FUNCTION USING JAR command to register a UDF.
-If users do not have write access to the staging directory, the registration attempt fails. When a user issues the CREATE FUNCTION USING JAR command to register a UDF, Drill uses specific directories while validating and registering the UDFs. ZooKeeper stores the list of UDFs and associated JAR files. Drillbits refer to this list when registering and unregistering UDFs.
+If users do not have write access to the staging directory, the registration attempt fails. When a user issues the CREATE FUNCTION USING JAR command to register a UDF, Drill uses specific directories while validating and registering the UDFs. The [persistent configuration store]({{site.baseurl}}/docs/persistent-configuration-storage/) stores the list of UDFs and associated JAR files. Drillbits refer to this list when registering and unregistering UDFs.
##UDF Directories
@@ -51,7 +51,7 @@ The following table lists optional directories that you can add:
Currently, any user can register UDFs if they
have access to the staging directory. Since Drill does not provide full
authorization and authentication support, an administrator may want to disable
-the Dynamic UDF feature. See Enabling and Disabling the Dynamic UDF Feature.
+the Dynamic UDF feature. See [Enabling and Disabling the Dynamic UDF Feature]({{site.baseurl}}/docs/dynamic-udfs/#enabling-and-disabling-the-dynamic-udf-feature).
Drill moves all JAR files from the staging directory to the other UDF directories as the user that started the drillbit, not as the user that submitted the JAR files. Drill behaves this way even if impersonation is enabled.
@@ -59,7 +59,7 @@ Drill moves all JAR files from the staging directory to the other UDF directorie
Before users can successfully register and unregister their UDFs using the Dynamic UDF feature, administrators and users must complete certain prerequisites. The prerequisites for each role are listed below.
###Administrators
-Before users can issue the CREATE FUNCTION USING JAR or DROP FUNCTION USING JAR commands to register or unregister UDFs, verify that the `exec.udf.enable_dynamic_support` option is enabled and that the staging directory is accessible to users. See Enabling and Disabling the Dynamic UDF Feature.
+Before users can issue the CREATE FUNCTION USING JAR or DROP FUNCTION USING JAR commands to register or unregister UDFs, verify that the `exec.udf.enable_dynamic_support` option is enabled and the staging directory is accessible to users. See [Enabling and Disabling the Dynamic UDF Feature]({{site.baseurl}}/docs/dynamic-udfs/#enabling-and-disabling-the-dynamic-udf-feature).
###Users
Create a UDF using Drill\u2019s [simple]({{site.baseurl}}/docs/developing-a-simple-function/) or [aggregate]({{site.baseurl}}/docs/developing-an-aggregate-function/) function interface. Add a `drill-module.conf` file to the root of the class JAR file. The `drill-module.conf` file should contain the packages to scan for the functions `drill.classpath.scanning.packages+= "com.mydomain.drill.fn"`, as shown in the following example:
@@ -68,7 +68,7 @@ Create a UDF using Drill\u2019s [simple]({{site.baseurl}}/docs/developing-a-simple-
Replace `com.mydomain.drill.fn` with the package name(s) of your UDF(s). If there are multiple packages, separate package names with a comma.
-Once the UDF is created, copy the source and binary JAR files to the staging directory. If you do not know the location of the staging directory, contact your administrator. Now, you can register your UDF using the CREATE FUNCTION USING JAR command. See Registering a UDF.
+Once the UDF is created, copy the source and binary JAR files to the staging directory. If you do not know the location of the staging directory, contact your administrator. Now, you can register your UDF using the CREATE FUNCTION USING JAR command. See [Registering a UDF]({{site.baseurl}}/docs/dynamic-udfs/#registering-a-udf).
##Enabling and Disabling the Dynamic UDF Feature
An administrator can enable or disable the Dynamic UDF feature. The feature is enabled by default. The `exec.udf.enable_dynamic_support` option turns the Dynamic UDF feature on and off. If security is a concern, the administrator can disable the feature to prevent users from registering and unregistering UDFs.
@@ -83,7 +83,7 @@ Copy the UDF source and binary JAR files to the DFS staging directory and then i
If you do not know the location of the staging directory or you need access to the directory, contact your administrator.
-When you issue the command, Drill uses the JAR file name to register the JAR name in the Dynamic UDF registry (UDF list stored in ZooKeeper) and then copies the source and binary JAR files to the local UDF directory on each drillbit.
+When you issue the command, Drill uses the JAR file name to register the JAR name in the Dynamic UDF registry ([persistent store]({{site.baseurl}}/docs/persistent-configuration-storage/) and then copies the source and binary JAR files to the local UDF directory on each drillbit.
Upon successful registration, Drill returns a message with a list of registered UDFs:
@@ -98,7 +98,7 @@ Issue the DROP FUNCTION USING JAR command to unregister a UDF, as follows:
DROP FUNCTION USING JAR \u2018<jar_name>.jar\u2019
-When you issue the command, Drill unregisters UDFs based on the JAR file name and removes the JAR files from the UDF directory. Drill deletes all UDFs associated with the JAR file from the UDF registry (UDF list stored in ZooKeeper), signaling drillbits to start the local unregistering process.
+When you issue the command, Drill unregisters UDFs based on the JAR file name and removes the JAR files from the UDF directory. Drill deletes all UDFs associated with the JAR file from the UDF registry ([persistent store]({{site.baseurl}}/docs/persistent-configuration-storage/), signaling drillbits to start the local unregistering process.
Drill returns a message with the list of unregistered UDFs:
@@ -110,7 +110,7 @@ Drill returns a message with the list of unregistered UDFs:
##Migrating UDFs from Dynamic to Built-In
-You can migrate UDFs registered using the Dynamic UDF feature to built-in UDFs to free up space in the UDF directories and the Dynamic UDF registry (UDF list stored in ZooKeeper). You can migrate all of the UDFs or you can migrate a portion of the UDFs. If you migrate all of the UDFs, you cannot issue the DROP FUNCTION USING JAR command to unregister the UDFs that have been migrated from dynamic to built-in.
+You can migrate UDFs registered using the Dynamic UDF feature to built-in UDFs to free up space in the UDF directories and the Dynamic UDF registry ([persistent store]({{site.baseurl}}/docs/persistent-configuration-storage/). You can migrate all of the UDFs or you can migrate a portion of the UDFs. If you migrate all of the UDFs, you cannot issue the DROP FUNCTION USING JAR command to unregister the UDFs that have been migrated from dynamic to built-in.
###Migrating All Registered UDFs to Built-In UDFs
To migrate all registered UDFs to built-in UDFs, complete the following steps:
http://git-wip-us.apache.org/repos/asf/drill/blob/b9b1d234/_docs/sql-reference/sql-commands/034-create-function-using-jar.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/sql-commands/034-create-function-using-jar.md b/_docs/sql-reference/sql-commands/034-create-function-using-jar.md
new file mode 100644
index 0000000..7c467e9
--- /dev/null
+++ b/_docs/sql-reference/sql-commands/034-create-function-using-jar.md
@@ -0,0 +1,29 @@
+---
+title: "CREATE FUNCTION USING JAR"
+date: 2016-05-05 21:44:33 UTC
+parent: "SQL Commands"
+---
+The CREATE FUNCTION USING JAR command registers UDFs in Drill. See [Dynamic UDFs]({{site.baseurl}}/docs/dynamic-udfs/) for more information.
+
+## Syntax
+
+The CREATE FUNCTION USING JAR command supports the following syntax:
+
+ CREATE FUNCTION USING JAR '<jar_name>.jar';
+
+## Parameters
+*jar_name*
+The name of the JAR file that contains the UDFs.
+
+## Usage Notes
+Before you issue the CREATE FUNCTION USING JAR command, you must Copy the UDF source and binary JAR files to the DFS staging directory. When you issue this command, Drill uses the JAR file name to register the JAR name in the Dynamic UDF registry ([persistent store]({{site.baseurl}}/docs/persistent-configuration-storage/)) and then copies the source and binary JAR files to the local UDF directory on each drillbit upon request.
+
+## Example
+ 0: jdbc:drill:zk=local> CREATE FUNCTION USING JAR 'simple_functions.jar';
+ +---------------+--------------------------------------------------------------------------------------------------------------+
+ | ok
+ |summary
+ +---------------+--------------------------------------------------------------------------------------------------------------+
+ | true
+ | The following UDFs in jar simple_function.jar have been registered: ... |
+ +---------------+--------------------------------------------------------------------------------------------------------------+
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/drill/blob/b9b1d234/_docs/sql-reference/sql-commands/054-drop-function-using-jar.md
----------------------------------------------------------------------
diff --git a/_docs/sql-reference/sql-commands/054-drop-function-using-jar.md b/_docs/sql-reference/sql-commands/054-drop-function-using-jar.md
new file mode 100644
index 0000000..4aa39cd
--- /dev/null
+++ b/_docs/sql-reference/sql-commands/054-drop-function-using-jar.md
@@ -0,0 +1,32 @@
+---
+title: "DROP FUNCTION USING JAR"
+date: 2016-08-04 00:23:10 UTC
+parent: "SQL Commands"
+---
+
+The DROP FUNCTION USING JAR command unregisters a UDF. As of Drill 1.9, you can use this command to unregister a UDF based on the JAR file name. See Dynamic UDFs for more information.
+
+## Syntax
+
+The DROP FUNCTION USING JAR command supports the following syntax:
+
+ DROP FUNCTION USING JAR '<jar_name>.jar';
+
+## Parameters
+
+*jar_name*
+The name of the JAR file that contains the UDFs.
+
+## Usage Notes
+When you issue the DROP FUNCTION USING JAR command, Drill unregisters the UDFs based on the JAR file name and removes the JAR file from the UDF directory. Drill deletes all UDFs associated with the JAR file from the UDF registry (UDF list stored in ZooKeeper). Drill returns a message with the list of unregistered UDFs.
+
+## Example
+
+ 0: jdbc:drill:zk=local> DROP FUNCTION USING JAR 'simple_functions.jar';
+ +---------------+--------------------------------------------------------------------------------------------------------------+
+ | ok
+ |summary
+ +---------------+--------------------------------------------------------------------------------------------------------------+
+ | true
+ | The following UDFs in jar simple_function.jar have been unregistered: ... |
+ +---------------+--------------------------------------------------------------------------------------------------------------+
\ No newline at end of file