[GitHub] [dolphinscheduler] reele edited a comment on issue #5925: [Improvement][Common template API | Standalone import tool] About migrating to DolphinScheduler

[GitBox <gi...@apache.org>]

reele edited a comment on issue #5925:
URL: https://github.com/apache/dolphinscheduler/issues/5925#issuecomment-891811825


   > In my opinion, this function is generally implemented by users themselves. There are too many scheduling platforms (including self-development), complex structures and more uncertainties, which makes it difficult to parse and initialize into DS general format (especially on front-end, how to initialize location to define the coordinates of each task?), we can collect the user's own scripts, summarize them to the tool set on the DS official website, and download and modify them by ourselves.
   
   In my understanding, the main reasons for migration difficulties are as follows:
   1. Job data structure is complex and difficult to be solved by simple mapping
   2. Difficult for non-developer user
   
   Typically, the migration of large numbers of jobs (thousands) is done with little regard for the DAG graphics, and the dependencies are likely to be too complex for the data warehouse user to develop with the DAG graphical interface, like mine(One of the simpler):
   
   ![auto_generated_dag](https://user-images.githubusercontent.com/38578667/128014335-ff35b5c0-424b-4d72-912b-efa688ee68ba.png)
   
   In general, the scheduler requires the following basic data:
   1.Task definition
   2.Task Dependencies
   3.Scheduling of tasks
   4.DAG drawing definition
   5.Task group definition
   
   I think, 1-3 are required and 4,5 are optional.
   
   If the common interface of 1-3 can be implemented, the data migration will be very friendly to other platforms.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org