You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by ji...@apache.org on 2022/10/14 02:40:09 UTC
[doris-website] branch master updated: fix-doc27 (#144)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 106b4fd2b22 fix-doc27 (#144)
106b4fd2b22 is described below
commit 106b4fd2b22032a71ca282746e98cfa3e9fa1858
Author: Liqf <10...@users.noreply.github.com>
AuthorDate: Fri Oct 14 10:40:05 2022 +0800
fix-doc27 (#144)
---
docs/ecosystem/udf/java-user-defined-function.md | 183 ---------------------
.../ecosystem/udf/java-user-defined-function.md | 182 --------------------
2 files changed, 365 deletions(-)
diff --git a/docs/ecosystem/udf/java-user-defined-function.md b/docs/ecosystem/udf/java-user-defined-function.md
deleted file mode 100644
index aff3c7874fc..00000000000
--- a/docs/ecosystem/udf/java-user-defined-function.md
+++ /dev/null
@@ -1,183 +0,0 @@
----
-{
-"title": "Java UDF",
-"language": "en"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Java UDF
-
-Java UDF provides users with a Java interface written in UDF to facilitate the execution of user-defined functions in Java language. Compared with native UDF implementation, Java UDF has the following advantages and limitations:
-1. The advantages
-* Compatibility: Using Java UDF can be compatible with different Doris versions, so when upgrading Doris version, Java UDF does not need additional migration. At the same time, Java UDF also follows the same programming specifications as hive / spark and other engines, so that users can directly move Hive / Spark UDF jar to Doris.
-* Security: The failure or crash of Java UDF execution will only cause the JVM to report an error, not the Doris process to crash.
-* Flexibility: In Java UDF, users can package the third-party dependencies together in the user jar.
-
-2. Restrictions on use
-* Performance: Compared with native UDF, Java UDF will bring additional JNI overhead, but through batch execution, we have minimized the JNI overhead as much as possible.
-* Vectorized engine: Java UDF is only supported on vectorized engine now.
-
-### Type correspondence
-
-|Type|UDF Argument Type|
-|----|---------|
-|Bool|Boolean|
-|TinyInt|Byte|
-|SmallInt|Short|
-|Int|Integer|
-|BigInt|Long|
-|LargeInt|BigInteger|
-|Float|Float|
-|Double|Double|
-|Date|LocalDate|
-|Datetime|LocalDateTime|
-|Char|String|
-|Varchar|String|
-|Decimal|BigDecimal|
-
-## Write UDF functions
-
-This section mainly introduces how to develop a Java UDF. Samples for the Java version are provided under `samples/doris-demo/java-udf-demo/` for your reference, Check it out [here](https://github.com/apache/incubator-doris/tree/master/samples/doris-demo/java-udf-demo)
-
-To use Java UDF, the main entry of UDF must be the `evaluate` function. This is consistent with other engines such as Hive. In the example of `AddOne`, we have completed the operation of adding an integer as the UDF.
-
-It is worth mentioning that this example is not only the Java UDF supported by Doris, but also the UDF supported by Hive, that's to say, for users, Hive UDF can be directly migrated to Doris.
-
-## Create UDF
-
-```sql
-CREATE FUNCTION
-name ([,...])
-[RETURNS] rettype
-PROPERTIES (["key"="value"][,...])
-```
-Instructions:
-
-1. `symbol` in properties represents the class name containing UDF classes. This parameter must be set.
-2. The jar package containing UDF represented by `file` in properties must be set.
-3. The UDF call type represented by `type` in properties is native by default. When using java UDF, it is transferred to `Java_UDF`.
-4. `name`: A function belongs to a DB and name is of the form`dbName`.`funcName`. When `dbName` is not explicitly specified, the db of the current session is used`dbName`.
-
-Sample：
-```sql
-CREATE FUNCTION java_udf_add_one(int) RETURNS int PROPERTIES (
- "file"="file:///path/to/java-udf-demo-jar-with-dependencies.jar",
- "symbol"="org.apache.doris.udf.AddOne",
- "type"="JAVA_UDF"
-);
-```
-
-## Create UDAF
-<br/>
-When using Java code to write UDAF, there are some functions that must be implemented (mark required) and an inner class State, which will be explained with a specific example below.
-The following SimpleDemo will implement a simple function similar to sum, the input parameter is INT, and the output parameter is INT
-
-```JAVA
-package org.apache.doris.udf;
-
-public class SimpleDemo {
- //Need an inner class to store data
- /*required*/
- public static class State {
- /*some variables if you need */
- public int sum = 0;
- }
-
- /*required*/
- public State create() {
- /* here could do some init work if needed */
- return new State();
- }
-
- /*required*/
- public void destroy(State state) {
- /* here could do some destroy work if needed */
- }
-
- /*required*/
- //first argument is State, then other types your input
- public void add(State state, Integer val) {
- /* here doing update work when input data*/
- if (val != null) {
- state.sum += val;
- }
- }
-
- /*required*/
- public void serialize(State state, DataOutputStream out) {
- /* serialize some data into buffer */
- out.writeInt(state.sum);
- }
-
- /*required*/
- public void deserialize(State state, DataInputStream in) {
- /* deserialize get data from buffer before you put */
- int val = in.readInt();
- state.sum = val;
- }
-
- /*required*/
- public void merge(State state, State rhs) {
- /* merge data from state */
- state.sum += rhs.sum;
- }
-
- /*required*/
- //return Type you defined
- public Integer getValue(State state) {
- /* return finally result */
- return state.sum;
- }
-}
-
-```
-
-```sql
-CREATE AGGREGATE FUNCTION simple_sum(INT) RETURNS INT PROPERTIES (
- "file"="file:///pathTo/java-udaf.jar",
- "symbol"="org.apache.doris.udf.SimpleDemo",
- "type"="JAVA_UDF"
-);
-```
-
-Currently, UDTF are not supported.
-
-<br/>
-
-## Use UDF
-
-Users must have the `SELECT` permission of the corresponding database to use UDF/UDAF.
-
-The use of UDF is consistent with ordinary function methods. The only difference is that the scope of built-in functions is global, and the scope of UDF is internal to DB. When the link session is inside the data, directly using the UDF name will find the corresponding UDF inside the current DB. Otherwise, the user needs to display the specified UDF database name, such as `dbName`.`funcName`.
-
-## Delete UDF
-
-When you no longer need UDF functions, you can delete a UDF function by the following command, you can refer to `DROP FUNCTION`.
-
-## Example
-Examples of Java UDF are provided in the `samples/doris-demo/java-udf-demo/` directory. See the `README.md` in each directory for details on how to use it, Check it out [here](https://github.com/apache/incubator-doris/tree/master/samples/doris-demo/java-udf-demo)
-
-## Unsupported Use Case
-At present, Java UDF is still in the process of continuous development, so some features are **not completed**.
-1. Complex data types (HLL, bitmap) are not supported.
-2. Memory management and statistics of JVM and Doris have not been unified.
-
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/udf/java-user-defined-function.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/udf/java-user-defined-function.md
deleted file mode 100644
index 8cb870cc0b0..00000000000
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/udf/java-user-defined-function.md
+++ /dev/null
@@ -1,182 +0,0 @@
----
-{
-"title": "Java UDF",
-"language": "zh-CN"
-}
----
-
-<!--
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements. See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership. The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied. See the License for the
-specific language governing permissions and limitations
-under the License.
--->
-
-# Java UDF
-
-Java UDF 为用户提供UDF编写的Java接口，以方便用户使用Java语言进行自定义函数的执行。相比于 Native 的 UDF 实现，Java UDF 有如下优势和限制：
-1. 优势
-* 兼容性：使用Java UDF可以兼容不同的Doris版本，所以在进行Doris版本升级时，Java UDF不需要进行额外的迁移操作。与此同时，Java UDF同样遵循了和Hive/Spark等引擎同样的编程规范，使得用户可以直接将Hive/Spark的UDF jar包迁移至Doris使用。
-* 安全：Java UDF 执行失败或崩溃仅会导致JVM报错，而不会导致 Doris 进程崩溃。
-* 灵活：Java UDF 中用户通过把第三方依赖打进用户jar包，而不需要额外处理引入的三方库。
-
-2. 使用限制
-* 性能：相比于 Native UDF，Java UDF会带来额外的JNI开销，不过通过批式执行的方式，我们已经尽可能的将JNI开销降到最低。
-* 向量化引擎：Java UDF当前只支持向量化引擎。
-
-### 类型对应关系
-
-|Type|UDF Argument Type|
-|----|---------|
-|Bool|Boolean|
-|TinyInt|Byte|
-|SmallInt|Short|
-|Int|Integer|
-|BigInt|Long|
-|LargeInt|BigInteger|
-|Float|Float|
-|Double|Double|
-|Date|LocalDate|
-|Datetime|LocalDateTime|
-|Char|String|
-|Varchar|String|
-|Decimal|BigDecimal|
-
-## 编写 UDF 函数
-
-本小节主要介绍如何开发一个 Java UDF。在 `samples/doris-demo/java-udf-demo/` 下提供了示例，可供参考，查看点击[这里](https://github.com/apache/incubator-doris/tree/master/samples/doris-demo/java-udf-demo)
-
-使用Java代码编写UDF，UDF的主入口必须为 `evaluate` 函数。这一点与Hive等其他引擎保持一致。在本示例中，我们编写了 `AddOne` UDF来完成对整型输入进行加一的操作。
-值得一提的是，本例不只是Doris支持的Java UDF，同时还是Hive支持的UDF，也就是说，对于用户来讲，Hive UDF是可以直接迁移至Doris的。
-
-## 创建 UDF
-
-```sql
-CREATE FUNCTION
-name ([,...])
-[RETURNS] rettype
-PROPERTIES (["key"="value"][,...])
-```
-说明：
-
-1. PROPERTIES中`symbol`表示的是包含UDF类的类名，这个参数是必须设定的。
-2. PROPERTIES中`file`表示的包含用户UDF的jar包，这个参数是必须设定的。
-3. PROPERTIES中`type`表示的 UDF 调用类型，默认为 Native，使用 Java UDF时传 JAVA_UDF。
-4. name: 一个function是要归属于某个DB的，name的形式为`dbName`.`funcName`。当`dbName`没有明确指定的时候，就是使用当前session所在的db作为`dbName`。
-
-示例：
-```sql
-CREATE FUNCTION java_udf_add_one(int) RETURNS int PROPERTIES (
- "file"="file:///path/to/java-udf-demo-jar-with-dependencies.jar",
- "symbol"="org.apache.doris.udf.AddOne",
- "type"="JAVA_UDF"
-);
-```
-
-## 编写 UDAF 函数
-<br/>
-
-在使用Java代码编写UDAF时，有一些必须实现的函数(标记required)和一个内部类State，下面将以一个具体的实例来说明
-下面的SimpleDemo将实现一个类似的sum的简单函数,输入参数INT，输出参数是INT
-```JAVA
-package org.apache.doris.udf;
-
-public class SimpleDemo {
- //Need an inner class to store data
- /*required*/
- public static class State {
- /*some variables if you need */
- public int sum = 0;
- }
-
- /*required*/
- public State create() {
- /* here could do some init work if needed */
- return new State();
- }
-
- /*required*/
- public void destroy(State state) {
- /* here could do some destroy work if needed */
- }
-
- /*required*/
- //first argument is State, then other types your input
- public void add(State state, Integer val) {
- /* here doing update work when input data*/
- if (val != null) {
- state.sum += val;
- }
- }
-
- /*required*/
- public void serialize(State state, DataOutputStream out) {
- /* serialize some data into buffer */
- out.writeInt(state.sum);
- }
-
- /*required*/
- public void deserialize(State state, DataInputStream in) {
- /* deserialize get data from buffer before you put */
- int val = in.readInt();
- state.sum = val;
- }
-
- /*required*/
- public void merge(State state, State rhs) {
- /* merge data from state */
- state.sum += rhs.sum;
- }
-
- /*required*/
- //return Type you defined
- public Integer getValue(State state) {
- /* return finally result */
- return state.sum;
- }
-}
-
-```
-
-```sql
-CREATE AGGREGATE FUNCTION simple_sum(int) RETURNS int PROPERTIES (
- "file"="file:///pathTo/java-udaf.jar",
- "symbol"="org.apache.doris.udf.SimpleDemo",
- "type"="JAVA_UDF"
-);
-```
-
-目前还暂不支持UDTF
-
-<br/>
-
-## 使用 UDF
-
-用户使用 UDF 必须拥有对应数据库的 `SELECT` 权限。
-
-UDF 的使用与普通的函数方式一致，唯一的区别在于，内置函数的作用域是全局的，而 UDF 的作用域是 DB内部。当链接 session 位于数据内部时，直接使用 UDF 名字会在当前DB内部查找对应的 UDF。否则用户需要显示的指定 UDF 的数据库名字，例如 `dbName`.`funcName`。
-
-## 删除 UDF
-
-当你不再需要 UDF 函数时，你可以通过下述命令来删除一个 UDF 函数, 可以参考 `DROP FUNCTION`。
-
-## 示例
-在`samples/doris-demo/java-udf-demo/` 目录中提供了具体示例。具体使用方法见每个目录下的`README.md`，查看点击[这里](https://github.com/apache/incubator-doris/tree/master/samples/doris-demo/java-udf-demo)
-
-## 暂不支持的场景
-当前Java UDF仍然处在持续的开发过程中，所以部分功能**尚不完善**。包括：
-1. 不支持复杂数据类型（HLL，Bitmap）
-2. 尚未统一JVM和Doris的内存管理以及统计信息
-
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org