You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@drill.apache.org by br...@apache.org on 2016/11/29 22:42:40 UTC
[02/19] drill git commit: number adding custom functions to drill
folder
number adding custom functions to drill folder
Project: http://git-wip-us.apache.org/repos/asf/drill/repo
Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/6e0c6240
Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/6e0c6240
Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/6e0c6240
Branch: refs/heads/gh-pages
Commit: 6e0c62409c0f2263db19a5fe15d529c570bbe2cf
Parents: bb638b1
Author: Bridget Bevens <bb...@maprtech.com>
Authored: Fri Nov 18 14:08:08 2016 -0800
Committer: Bridget Bevens <bb...@maprtech.com>
Committed: Fri Nov 18 14:08:08 2016 -0800
----------------------------------------------------------------------
...ng-custom-functions-to-drill-introduction.md | 10 ++
...manually-adding-custom-functions-to-drill.md | 23 ++++
.../020-dynamic-udfs.md | 136 +++++++++++++++++++
...ng-custom-functions-to-drill-introduction.md | 10 --
...manually-adding-custom-functions-to-drill.md | 23 ----
.../020-dynamic-udfs.md | 136 -------------------
6 files changed, 169 insertions(+), 169 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/drill/blob/6e0c6240/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md b/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
new file mode 100644
index 0000000..2f5b547
--- /dev/null
+++ b/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
@@ -0,0 +1,10 @@
+---
+title: "Adding Custom Functions to Drill Introduction"
+date: 2016-10-04 23:35:34 UTC
+parent: "Adding Custom Functions to Drill"
+---
+
+As of Drill 1.9, there are two methods for adding custom functions to Drill. An administrator can manually add functions to Drill, or provide users access to a staging directory where they can upload JAR files and register their UDFs using the CREATE FUNCTION USING JAR command. The CREATE FUNCTION USING JAR command is part of the Dynamic UDF feature.
+
+- For manual instructions see Manually Adding Custom Functions to Drill.
+- For Dynamic UDF instructions, see Dynamic UDFs.
http://git-wip-us.apache.org/repos/asf/drill/blob/6e0c6240/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md b/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md
new file mode 100644
index 0000000..851df63
--- /dev/null
+++ b/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md
@@ -0,0 +1,23 @@
+---
+title: "Manually Adding Custom Functions to Drill"
+date: 2016-10-04 23:35:34 UTC
+parent: "Adding Custom Functions to Drill"
+---
+
+Administrators can manually add custom functions to Drill. After the custom function is developed, generate the sources and classes JAR files. Add both JAR files to the Drill classpath on each node, and include the name of the package that contains the classes to the main Drill configuration file. Restart the drillbit on each node to refresh the configuration.
+
+To add a custom function to Drill, complete the following steps:
+
+1. Add the sources and classes JAR file for the custom function to the Drill classpath on all drillbits by copying the files to `<drill installation directory>/jars/3rdparty`.
+2. Include a `drill-module.conf` file in the class JAR file, at its root.
+3. Add the following code to `drill-module.conf` (src/main/resources/drill-module.conf), and replace `com.yourgroupidentifier.udf` with the package name(s) of your UDF(s), as shown below:
+
+ drill.classpath.scanning.packages += "com.yourgroupidentifier.udf"
+**Note:** Separate package names with a comma.
+4. Verify that `DRILL_HOME/conf/drill-override.conf` does not contain any information regarding UDF packages.
+5. Issue the following command to restart Drill:
+
+ <drill_installation_directory>/bin/drillbit.sh restart
+
+ Now, you can use the custom function(s) in Drill queries.
+
http://git-wip-us.apache.org/repos/asf/drill/blob/6e0c6240/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/020-dynamic-udfs.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/020-dynamic-udfs.md b/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/020-dynamic-udfs.md
new file mode 100644
index 0000000..e0b3178
--- /dev/null
+++ b/_docs/develop-custom-functions/009-adding-custom-functions-to-drill/020-dynamic-udfs.md
@@ -0,0 +1,136 @@
+---
+title: "Dynamic UDFs"
+date: 2016-11-11 23:35:34 UTC
+parent: "Adding Custom Functions to Drill"
+---
+
+Drill 1.9 introduces support for Dynamic UDFs. The Dynamic UDF feature enables users to register and unregister UDFs on their own using the CREATE FUNCTION USING JAR and DROP FUNCTION USING JAR commands.
+
+The Dynamic UDF feature eliminates the need to restart drillbits, which can disrupt users, when administrators manually load and unload UDFs in a multi-tenant environment. Users can issue the CREATE FUNCTION USING JAR command to register manually loaded (built-in) UDFs. Also, users can migrate registered UDFs to built-in UDFs.
+
+The Dynamic UDF feature is enabled by default. An administrator can enable or disable the feature using the ALTER SYSTEM SET command with the `exec.udf.enable_dynamic_support option`. When the feature is enabled, users must upload their UDF (source and binary) JAR files to a staging directory in the distributed file system before issuing the CREATE FUNCTION USING JAR command to register a UDF.
+
+If users do not have write access to the staging directory, the registration attempt fails. When a user issues the CREATE FUNCTION USING JAR command to register a UDF, Drill uses specific directories while validating and registering the UDFs. ZooKeeper stores the list of UDFs and associated JAR files. Drillbits refer to this list when registering and unregistering UDFs.
+
+##UDF Directories
+
+The directories that Drill uses when registering UDFs are configured in the `drill.exec.udf` stanza of the `drill-override.conf` file. Upon startup, Drill verifies that these directories exist in the file system. If the directories do not exist, Drill creates them. If Drill is unable to create the directories, the start-up attempt fails. An administrator can modify the directory locations in `drill-override.conf`.
+
+The configuration file contains the following default properties and directories required to use the Dynamic UDF feature:
+
+ drill.exec.udf: {
+ retry-attempts: 5,
+ directory: {
+ base: ${drill.exec.zk.root}"/udf",
+ local: ${drill.exec.udf.directory.base}"/local",
+ staging: ${drill.exec.udf.directory.base}"/staging",
+ registry: ${drill.exec.udf.directory.base}"/registry",
+ tmp: ${drill.exec.udf.directory.base}"/tmp"
+ }
+ }
+
+The following table describes the configuration properties and UDF directories, where `drill.exec.udf.directory.base` is the relative directory used to generate all of the UDF directories (local and remote):
+
+| Property | Description |
+|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| retry-attempts | The number of times that the UDF registry update can fail before Drill returns an error. Drill checks the registry version before updating the remote function registry to avoid overriding changes made by another user. If the registry version has changed, Drill validates the functions among the updated registry again. The default is 5. |
+| base: ${drill.exec.zk.root}"/udf" | The property used to separate the UDF directories between clusters that use the same file system. |
+| local: ${drill.exec.udf.directory.base}"/local" | The relative path concatenated to the Drill temporary directory to indicate the local UDF directory. The local UDF directory is used as a temporary directory for the Dynamic UDF JAR files. Drill cleans this directory out upon exiting. |
+| staging: ${drill.exec.udf.directory.base}"/staging" | The location to which users copy their binary and source JAR files. This directory must be accessible to users in order to register their UDFs. When a UDF is registered, Drill deletes both of the JAR files (source and binary) from this directory. If Drill fails to register the UDFs from the JAR files, the JAR files remain here. You can change the location of this directory. |
+| registry: ${drill.exec.udf.directory.base}"/registry" | The location to which Drill copies the source and binary JAR files after validating the UDFs. Drill copies the JAR files from the registry directory to a local UDF directory on each drillbit. When you unregister UDFs, Drill deletes the appropriate JAR files from the local UDF directory on each drillbit. DO NOT delete the JAR files from the registry directory. Deleting JAR files from the registry directory results in inconsistencies in the Dynamic UDF registry that point to the directory where JAR files are stored. You can change the location of this directory. |
+| tmp: ${drill.exec.udf.directory.base}"/tmp" | The location to which Drill backs up the binary and source JAR files before starting the registration process. Drill places each binary and source file in a unique folder in this directory. At the end of registration, Drill deletes both JAR files from this directory. You can change the location of this directory. |
+
+The following table lists optional directories that you can add:
+
+| Property | Description |
+|-------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| drill.exec.udf.directory.fs | Changes the file system from the default. If there are multiple drillbits in the cluster, and the default file system is not distributed, you must include this property, and set it to a distributed file system. For example, file:///, hdfs:///, or maprfs:///, as shown below: drill.exec.udf.directory.fs: "hdfs:///" |
+| drill.exec.udf.directory.root | Changes the root directory for remote UDF directories. By default, this property is set to the home directory of the user that started the drillbit. For example, on Linux the location is /home/some_user. On DFS, the location is /user/<user_name>. And, on Windows, the location is /C:/User/<user_name>. |
+
+##Security and Authentication Impact
+Currently, any user can register UDFs if they
+have access to the staging directory. Since Drill does not provide full
+authorization and authentication support, an administrator may want to disable
+the Dynamic UDF feature. See Enabling and Disabling the Dynamic UDF Feature.
+
+Drill moves all JAR files from the staging directory to the other UDF directories as the user that started the drillbit, not as the user that submitted the JAR files. Drill behaves this way even if impersonation is enabled.
+
+##Before Using the Dynamic UDF Feature
+Before users can issue the CREATE FUNCTION USING JAR or DROP FUNCTION USING JAR commands to register or unregister UDFs, an administrator should verify that the option is enabled and that the staging directory is accessible to users.
+
+Users create a UDF using Drill\u2019s simple or aggregate function interface. Add a `drill-module.conf` file to the root of the class JAR file. The `drill-module.conf` file should contain the packages to scan for the functions `drill.classpath.scanning.packages+= "com.mydomain.drill.fn"`. Separate package names with a comma, as shown in the following example:
+
+ drill.classpath.scanning.package+= "com.mydomain.drill.fn"
+
+Once the UDF is created, copy the source and binary JAR files to the staging directory. Now, you can register your UDF using the CREATE FUNCTION USING JAR command. See Registering a UDF.
+
+##Enabling and Disabling the Dynamic UDF Feature
+An administrator can enable or disable the Dynamic UDF feature. The feature is enabled by default. The `exec.udf.enable_dynamic_support` option turns the Dynamic UDF feature on and off. If security is a concern, the administrator can disable the feature to prevent users from registering and unregistering UDFs.
+
+
+Use the [ALTER SYSTEM SET]({{site.baseurl}}/docs/alter-system/) command with the `exec.udf.enable_dynamic_support` system option to turn the feature on or off.
+
+##Registering a UDF
+Copy the UDF source and binary JAR files to the DFS staging directory and then issue the CREATE FUNCTION USING JAR command to register the UDF, as follows:
+
+ CREATE FUNCTION USING JAR \u2018<jar_name>.jar\u2019
+
+If you do not know the location of the staging directory or you need access to the directory, contact your administrator.
+
+When you issue the command, Drill uses the JAR file name to register the JAR name in the Dynamic UDF registry (UDF list stored in ZooKeeper) and then copies the source and binary JAR files to the local UDF directory on each drillbit.
+
+Upon successful registration, Drill returns a message with a list of registered UDFs:
+
+ +---------------+-----------------------------------------------------------------------------------------------------------+
+ | ok | summary |
+ +---------------+-----------------------------------------------------------------------------------------------------------+
+ | true | The following UDFs in jar %s have been registered: %s |
+ +---------------+-----------------------------------------------------------------------------------------------------------+
+
+##Unregistering a UDF
+Issue the DROP FUNCTION USING JAR command to unregister a UDF, as follows:
+
+ DROP FUNCTION USING JAR \u2018<jar_name>.jar\u2019
+
+When you issue the command, Drill unregisters UDFs based on the JAR file name and removes the JAR files from the UDF directory. Drill deletes all UDFs associated with the JAR file from the UDF registry (UDF list stored in ZooKeeper), signaling drillbits to start the local unregistering process.
+
+Drill returns a message with the list of unregistered UDFs:
+
+ +---------------+-----------------------------------------------------------------------------------------------------------+
+ | ok | summary |
+ +---------------+-----------------------------------------------------------------------------------------------------------+
+ | true | The following UDFs in jar %s have been unregistered: %s |
+ +---------------+-----------------------------------------------------------------------------------------------------------+
+
+##Migrating UDFs from Dynamic to Built-In
+
+You can migrate UDFs registered using the Dynamic UDF feature to built-in UDFs to free up space in the UDF directories and the Dynamic UDF registry (UDF list stored in ZooKeeper). You can migrate all of the UDFs or you can migrate a portion of the UDFs. If you migrate all of the UDFs, you cannot issue the DROP FUNCTION USING JAR command to unregister the UDFs that have been migrated from dynamic to built-in.
+
+###Migrating All Registered UDFs to Built-In UDFs
+To migrate all registered UDFs to built-in UDFs, complete the following steps:
+
+1. Stop all drillbits in the cluster.
+2. Move the UDF source and binary JAR files to the $DRILL_SITE/jars directory on each drillbit. (Must be included in the classpath.)
+3. Remove the remote function registry from ZooKeeper.
+4. Start all drillbits in the cluster.
+
+###Migrating Some of the Registered UDF JAR Files to Built-In UDFs
+To migrate a portion of the UDF JAR files to built-in UDFs, complete the following steps:
+
+1. Copy (not move) the JAR files from the UDF registry directory to the $DRILL_SITE/jars directory on each drillbit. (Must be included in the classpath.)
+2. Issue the DROP FUNCTION USING JAR command for each JAR file.
+3. Stop all drillbits in the cluster.
+4. Start all drillbits in the cluster.
+
+##Limitations
+The Dynamic UDF feature has the following known limitations:
+
+* If a user drops a UDF while a query that references the UDF is running, the query may fail. Users should verify that no queries reference a UDF prior to issuing the DROP command.
+* The DROP command only operates at the JAR level name. A user cannot unregister only one UDF from a JAR where several UDFs are present. To avoid this situation, a user can create one UDF per jar.
+* All UDF directories (remote or local) are created upon drillbit startup, even if Dynamic UDF support is disabled. Drillbit startup fails if the user who started the drillbit does not have write access to these directories.
+
+
+
+
+
+
http://git-wip-us.apache.org/repos/asf/drill/blob/6e0c6240/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md b/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
deleted file mode 100644
index 2f5b547..0000000
--- a/_docs/develop-custom-functions/adding-custom-functions-to-drill/009-adding-custom-functions-to-drill-introduction.md
+++ /dev/null
@@ -1,10 +0,0 @@
----
-title: "Adding Custom Functions to Drill Introduction"
-date: 2016-10-04 23:35:34 UTC
-parent: "Adding Custom Functions to Drill"
----
-
-As of Drill 1.9, there are two methods for adding custom functions to Drill. An administrator can manually add functions to Drill, or provide users access to a staging directory where they can upload JAR files and register their UDFs using the CREATE FUNCTION USING JAR command. The CREATE FUNCTION USING JAR command is part of the Dynamic UDF feature.
-
-- For manual instructions see Manually Adding Custom Functions to Drill.
-- For Dynamic UDF instructions, see Dynamic UDFs.
http://git-wip-us.apache.org/repos/asf/drill/blob/6e0c6240/_docs/develop-custom-functions/adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md b/_docs/develop-custom-functions/adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md
deleted file mode 100644
index 851df63..0000000
--- a/_docs/develop-custom-functions/adding-custom-functions-to-drill/010-manually-adding-custom-functions-to-drill.md
+++ /dev/null
@@ -1,23 +0,0 @@
----
-title: "Manually Adding Custom Functions to Drill"
-date: 2016-10-04 23:35:34 UTC
-parent: "Adding Custom Functions to Drill"
----
-
-Administrators can manually add custom functions to Drill. After the custom function is developed, generate the sources and classes JAR files. Add both JAR files to the Drill classpath on each node, and include the name of the package that contains the classes to the main Drill configuration file. Restart the drillbit on each node to refresh the configuration.
-
-To add a custom function to Drill, complete the following steps:
-
-1. Add the sources and classes JAR file for the custom function to the Drill classpath on all drillbits by copying the files to `<drill installation directory>/jars/3rdparty`.
-2. Include a `drill-module.conf` file in the class JAR file, at its root.
-3. Add the following code to `drill-module.conf` (src/main/resources/drill-module.conf), and replace `com.yourgroupidentifier.udf` with the package name(s) of your UDF(s), as shown below:
-
- drill.classpath.scanning.packages += "com.yourgroupidentifier.udf"
-**Note:** Separate package names with a comma.
-4. Verify that `DRILL_HOME/conf/drill-override.conf` does not contain any information regarding UDF packages.
-5. Issue the following command to restart Drill:
-
- <drill_installation_directory>/bin/drillbit.sh restart
-
- Now, you can use the custom function(s) in Drill queries.
-
http://git-wip-us.apache.org/repos/asf/drill/blob/6e0c6240/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md
----------------------------------------------------------------------
diff --git a/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md b/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md
deleted file mode 100644
index e0b3178..0000000
--- a/_docs/develop-custom-functions/adding-custom-functions-to-drill/020-dynamic-udfs.md
+++ /dev/null
@@ -1,136 +0,0 @@
----
-title: "Dynamic UDFs"
-date: 2016-11-11 23:35:34 UTC
-parent: "Adding Custom Functions to Drill"
----
-
-Drill 1.9 introduces support for Dynamic UDFs. The Dynamic UDF feature enables users to register and unregister UDFs on their own using the CREATE FUNCTION USING JAR and DROP FUNCTION USING JAR commands.
-
-The Dynamic UDF feature eliminates the need to restart drillbits, which can disrupt users, when administrators manually load and unload UDFs in a multi-tenant environment. Users can issue the CREATE FUNCTION USING JAR command to register manually loaded (built-in) UDFs. Also, users can migrate registered UDFs to built-in UDFs.
-
-The Dynamic UDF feature is enabled by default. An administrator can enable or disable the feature using the ALTER SYSTEM SET command with the `exec.udf.enable_dynamic_support option`. When the feature is enabled, users must upload their UDF (source and binary) JAR files to a staging directory in the distributed file system before issuing the CREATE FUNCTION USING JAR command to register a UDF.
-
-If users do not have write access to the staging directory, the registration attempt fails. When a user issues the CREATE FUNCTION USING JAR command to register a UDF, Drill uses specific directories while validating and registering the UDFs. ZooKeeper stores the list of UDFs and associated JAR files. Drillbits refer to this list when registering and unregistering UDFs.
-
-##UDF Directories
-
-The directories that Drill uses when registering UDFs are configured in the `drill.exec.udf` stanza of the `drill-override.conf` file. Upon startup, Drill verifies that these directories exist in the file system. If the directories do not exist, Drill creates them. If Drill is unable to create the directories, the start-up attempt fails. An administrator can modify the directory locations in `drill-override.conf`.
-
-The configuration file contains the following default properties and directories required to use the Dynamic UDF feature:
-
- drill.exec.udf: {
- retry-attempts: 5,
- directory: {
- base: ${drill.exec.zk.root}"/udf",
- local: ${drill.exec.udf.directory.base}"/local",
- staging: ${drill.exec.udf.directory.base}"/staging",
- registry: ${drill.exec.udf.directory.base}"/registry",
- tmp: ${drill.exec.udf.directory.base}"/tmp"
- }
- }
-
-The following table describes the configuration properties and UDF directories, where `drill.exec.udf.directory.base` is the relative directory used to generate all of the UDF directories (local and remote):
-
-| Property | Description |
-|---------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| retry-attempts | The number of times that the UDF registry update can fail before Drill returns an error. Drill checks the registry version before updating the remote function registry to avoid overriding changes made by another user. If the registry version has changed, Drill validates the functions among the updated registry again. The default is 5. |
-| base: ${drill.exec.zk.root}"/udf" | The property used to separate the UDF directories between clusters that use the same file system. |
-| local: ${drill.exec.udf.directory.base}"/local" | The relative path concatenated to the Drill temporary directory to indicate the local UDF directory. The local UDF directory is used as a temporary directory for the Dynamic UDF JAR files. Drill cleans this directory out upon exiting. |
-| staging: ${drill.exec.udf.directory.base}"/staging" | The location to which users copy their binary and source JAR files. This directory must be accessible to users in order to register their UDFs. When a UDF is registered, Drill deletes both of the JAR files (source and binary) from this directory. If Drill fails to register the UDFs from the JAR files, the JAR files remain here. You can change the location of this directory. |
-| registry: ${drill.exec.udf.directory.base}"/registry" | The location to which Drill copies the source and binary JAR files after validating the UDFs. Drill copies the JAR files from the registry directory to a local UDF directory on each drillbit. When you unregister UDFs, Drill deletes the appropriate JAR files from the local UDF directory on each drillbit. DO NOT delete the JAR files from the registry directory. Deleting JAR files from the registry directory results in inconsistencies in the Dynamic UDF registry that point to the directory where JAR files are stored. You can change the location of this directory. |
-| tmp: ${drill.exec.udf.directory.base}"/tmp" | The location to which Drill backs up the binary and source JAR files before starting the registration process. Drill places each binary and source file in a unique folder in this directory. At the end of registration, Drill deletes both JAR files from this directory. You can change the location of this directory. |
-
-The following table lists optional directories that you can add:
-
-| Property | Description |
-|-------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| drill.exec.udf.directory.fs | Changes the file system from the default. If there are multiple drillbits in the cluster, and the default file system is not distributed, you must include this property, and set it to a distributed file system. For example, file:///, hdfs:///, or maprfs:///, as shown below: drill.exec.udf.directory.fs: "hdfs:///" |
-| drill.exec.udf.directory.root | Changes the root directory for remote UDF directories. By default, this property is set to the home directory of the user that started the drillbit. For example, on Linux the location is /home/some_user. On DFS, the location is /user/<user_name>. And, on Windows, the location is /C:/User/<user_name>. |
-
-##Security and Authentication Impact
-Currently, any user can register UDFs if they
-have access to the staging directory. Since Drill does not provide full
-authorization and authentication support, an administrator may want to disable
-the Dynamic UDF feature. See Enabling and Disabling the Dynamic UDF Feature.
-
-Drill moves all JAR files from the staging directory to the other UDF directories as the user that started the drillbit, not as the user that submitted the JAR files. Drill behaves this way even if impersonation is enabled.
-
-##Before Using the Dynamic UDF Feature
-Before users can issue the CREATE FUNCTION USING JAR or DROP FUNCTION USING JAR commands to register or unregister UDFs, an administrator should verify that the option is enabled and that the staging directory is accessible to users.
-
-Users create a UDF using Drill\u2019s simple or aggregate function interface. Add a `drill-module.conf` file to the root of the class JAR file. The `drill-module.conf` file should contain the packages to scan for the functions `drill.classpath.scanning.packages+= "com.mydomain.drill.fn"`. Separate package names with a comma, as shown in the following example:
-
- drill.classpath.scanning.package+= "com.mydomain.drill.fn"
-
-Once the UDF is created, copy the source and binary JAR files to the staging directory. Now, you can register your UDF using the CREATE FUNCTION USING JAR command. See Registering a UDF.
-
-##Enabling and Disabling the Dynamic UDF Feature
-An administrator can enable or disable the Dynamic UDF feature. The feature is enabled by default. The `exec.udf.enable_dynamic_support` option turns the Dynamic UDF feature on and off. If security is a concern, the administrator can disable the feature to prevent users from registering and unregistering UDFs.
-
-
-Use the [ALTER SYSTEM SET]({{site.baseurl}}/docs/alter-system/) command with the `exec.udf.enable_dynamic_support` system option to turn the feature on or off.
-
-##Registering a UDF
-Copy the UDF source and binary JAR files to the DFS staging directory and then issue the CREATE FUNCTION USING JAR command to register the UDF, as follows:
-
- CREATE FUNCTION USING JAR \u2018<jar_name>.jar\u2019
-
-If you do not know the location of the staging directory or you need access to the directory, contact your administrator.
-
-When you issue the command, Drill uses the JAR file name to register the JAR name in the Dynamic UDF registry (UDF list stored in ZooKeeper) and then copies the source and binary JAR files to the local UDF directory on each drillbit.
-
-Upon successful registration, Drill returns a message with a list of registered UDFs:
-
- +---------------+-----------------------------------------------------------------------------------------------------------+
- | ok | summary |
- +---------------+-----------------------------------------------------------------------------------------------------------+
- | true | The following UDFs in jar %s have been registered: %s |
- +---------------+-----------------------------------------------------------------------------------------------------------+
-
-##Unregistering a UDF
-Issue the DROP FUNCTION USING JAR command to unregister a UDF, as follows:
-
- DROP FUNCTION USING JAR \u2018<jar_name>.jar\u2019
-
-When you issue the command, Drill unregisters UDFs based on the JAR file name and removes the JAR files from the UDF directory. Drill deletes all UDFs associated with the JAR file from the UDF registry (UDF list stored in ZooKeeper), signaling drillbits to start the local unregistering process.
-
-Drill returns a message with the list of unregistered UDFs:
-
- +---------------+-----------------------------------------------------------------------------------------------------------+
- | ok | summary |
- +---------------+-----------------------------------------------------------------------------------------------------------+
- | true | The following UDFs in jar %s have been unregistered: %s |
- +---------------+-----------------------------------------------------------------------------------------------------------+
-
-##Migrating UDFs from Dynamic to Built-In
-
-You can migrate UDFs registered using the Dynamic UDF feature to built-in UDFs to free up space in the UDF directories and the Dynamic UDF registry (UDF list stored in ZooKeeper). You can migrate all of the UDFs or you can migrate a portion of the UDFs. If you migrate all of the UDFs, you cannot issue the DROP FUNCTION USING JAR command to unregister the UDFs that have been migrated from dynamic to built-in.
-
-###Migrating All Registered UDFs to Built-In UDFs
-To migrate all registered UDFs to built-in UDFs, complete the following steps:
-
-1. Stop all drillbits in the cluster.
-2. Move the UDF source and binary JAR files to the $DRILL_SITE/jars directory on each drillbit. (Must be included in the classpath.)
-3. Remove the remote function registry from ZooKeeper.
-4. Start all drillbits in the cluster.
-
-###Migrating Some of the Registered UDF JAR Files to Built-In UDFs
-To migrate a portion of the UDF JAR files to built-in UDFs, complete the following steps:
-
-1. Copy (not move) the JAR files from the UDF registry directory to the $DRILL_SITE/jars directory on each drillbit. (Must be included in the classpath.)
-2. Issue the DROP FUNCTION USING JAR command for each JAR file.
-3. Stop all drillbits in the cluster.
-4. Start all drillbits in the cluster.
-
-##Limitations
-The Dynamic UDF feature has the following known limitations:
-
-* If a user drops a UDF while a query that references the UDF is running, the query may fail. Users should verify that no queries reference a UDF prior to issuing the DROP command.
-* The DROP command only operates at the JAR level name. A user cannot unregister only one UDF from a JAR where several UDFs are present. To avoid this situation, a user can create one UDF per jar.
-* All UDF directories (remote or local) are created upon drillbit startup, even if Dynamic UDF support is disabled. Drillbit startup fails if the user who started the drillbit does not have write access to these directories.
-
-
-
-
-
-