You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampark.apache.org by GitBox <gi...@apache.org> on 2022/10/09 06:30:07 UTC

[GitHub] [incubator-streampark] green241 opened a new issue, #1782: [Feature] About an idea to add a UDF management module for StreamPark

green241 opened a new issue, #1782:
URL: https://github.com/apache/incubator-streampark/issues/1782

   ### Search before asking
   
   - [X] I had searched in the [feature](https://github.com/apache/streampark/issues?q=is%3Aissue+label%3A%22Feature%22) and found no similar feature requirement.
   
   
   ### Description
   
   I have been in contact with StreamPark for some time now and have had a pretty good experience in terms of ease of use and stability. Currently, StreamPark itself supports UDF functionality, but there does not seem to be a unified UDF management menu, so it is recommended that a new UDF management menu be added for UDF management.
   
   ### Usage Scenario
   
   Note: 
   1. This feature is currently only implemented based on SQL jobs in Yarn Application mode; 
   2. the JAR is saved on top of HDFS.
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-streampark] green241 commented on issue #1782: [Feature] About an idea to add a UDF management module for StreamPark

Posted by GitBox <gi...@apache.org>.
green241 commented on issue #1782:
URL: https://github.com/apache/incubator-streampark/issues/1782#issuecomment-1272820406

   **The current plan is mainly based on the yarn application model, so the following is mainly idea of implement.**
   
   1. When creating a job, select the required UDF (e.g., a drop-down box showing the UDF available to the current user, associated with udfId);
   
   2. When starting a job, it will query the paths of these udf stores according to the selected udfId (there can be more than one), and at the same time stitch these storage paths into strings, and finally pass them into yarn.provided.lib.dirs when submitting the job to achieve dynamic loading.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-streampark] green241 commented on issue #1782: [Feature] About an idea to add a UDF management module for StreamPark

Posted by GitBox <gi...@apache.org>.
green241 commented on issue #1782:
URL: https://github.com/apache/incubator-streampark/issues/1782#issuecomment-1293003105

   hi datayangl,
   - So nice,warmly welcome.
   - After the next version , we may make a discussion in weekly-meeting including the proposal、solutions etc.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [incubator-streampark] datayangl commented on issue #1782: [Feature] About an idea to add a UDF management module for StreamPark

Posted by GitBox <gi...@apache.org>.
datayangl commented on issue #1782:
URL: https://github.com/apache/incubator-streampark/issues/1782#issuecomment-1290207831

   Actually,  your plan is pretty much like zepplin's way of managing udf. I would like to contribute. Firstly, overall design for udf management.Secondly, stages to implement. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org