You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/05/19 15:10:40 UTC
[GitHub] [dolphinscheduler] jieguangzhou opened a new pull request, #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
jieguangzhou opened a new pull request, #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
Update MLflow task plugin to enable Dolphin users to schedule [MLflow](https://mlflow.org/).
This PR relates to https://github.com/apache/dolphinscheduler/issues/10149.
Because I want to continue enriching the capabilities of this plugin, leave the issue open for now.
- [x] MLFLOW: Use `MLflow models serve` to deploy a model service
- [x] Docker: Run the container after packaging the docker image
<img width="484" alt="image" src="https://user-images.githubusercontent.com/31528124/169329245-b0079f20-05a0-4eda-817d-3a01340e5332.png">
## Brief change log
- Add MLflow Models component
- Update Doc
- The MLFlow plugin code has been reworked to expand more functionality
<!--*(for example:)*
- *Add maven-checkstyle-plugin to root pom.xml*
-->
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#issuecomment-1132651184
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10150)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL) [8 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=new_coverage&view=list) [85.2% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] songjianet merged pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
songjianet merged PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#issuecomment-1132350384
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10150)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] jieguangzhou commented on pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
jieguangzhou commented on PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#issuecomment-1132585449
> LGTM
> LGTM
Thank you for your code review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#issuecomment-1131892362
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10150)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10150&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10150&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] jieguangzhou commented on pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
jieguangzhou commented on PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#issuecomment-1132565000
PTAL @zhongjiajie @songjianet @labbomb , thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#discussion_r877861232
##########
docs/docs/en/guide/task/mlflow.md:
##########
@@ -5,77 +5,140 @@
[MLflow](https://mlflow.org) is an excellent open source platform to manage the ML lifecycle, including experimentation,
reproducibility, deployment, and a central model registry.
-Mlflow task is used to perform mlflow project tasks, which include basic algorithmic and autoML capabilities (
-User-defined MLFlow project task execution will be supported in the near future)
+MLflow task plugin used to execute MLflow tasks，Currently contains Mlflow Projects and MLflow Models.（Model Registry will soon be rewarded for support）
+
+- Mlflow Projects: Package data science code in a format to reproduce runs on any platform.
+- MLflow Models: Deploy machine learning models in diverse serving environments.
+- Model Registry: Store, annotate, discover, and manage models in a central repository.
+
+The Mlflow plugin currently supports and will support the following:
+
+- [ ] MLflow Projects
+ - [x] BasicAlgorithm: contains lr, svm, lightgbm, xgboost
+ - [x] AutoML: AutoML tool，contains autosklean, flaml
+ - [ ] Custom projects: Support for running your own MLflow projects
+- [ ] MLflow Models
+ - [x] MLFLOW: Use `MLflow models serve` to deploy a model service
+ - [x] Docker: Run the container after packaging the docker image
+ - [ ] Docker Compose: Use docker compose to run the container, Will replace the docker run above
+ - [ ] Seldon core: Use Selcon core to deploy model to k8s cluster
+ - [ ] k8s: Deploy containers directly to K8S
+ - [ ] mlflow deployments: Built-in deployment modules, such as built-in deployment to SageMaker, etc
+- [ ] Model Registry
+ - [ ] Register Model: Allows artifacts (Including model and related parameters, indicators) to be registered directly into the model center
Review Comment:
It is a document for users, I think we should not add something we do not support yet, it is for developers or contributors, maybe we should only add what we support here, but I do not have strong opinions here.
##########
docs/docs/en/guide/task/mlflow.md:
##########
@@ -5,77 +5,140 @@
[MLflow](https://mlflow.org) is an excellent open source platform to manage the ML lifecycle, including experimentation,
reproducibility, deployment, and a central model registry.
-Mlflow task is used to perform mlflow project tasks, which include basic algorithmic and autoML capabilities (
-User-defined MLFlow project task execution will be supported in the near future)
+MLflow task plugin used to execute MLflow tasks，Currently contains Mlflow Projects and MLflow Models.（Model Registry will soon be rewarded for support）
+
+- Mlflow Projects: Package data science code in a format to reproduce runs on any platform.
+- MLflow Models: Deploy machine learning models in diverse serving environments.
+- Model Registry: Store, annotate, discover, and manage models in a central repository.
+
+The Mlflow plugin currently supports and will support the following:
+
+- [ ] MLflow Projects
+ - [x] BasicAlgorithm: contains lr, svm, lightgbm, xgboost
+ - [x] AutoML: AutoML tool，contains autosklean, flaml
+ - [ ] Custom projects: Support for running your own MLflow projects
+- [ ] MLflow Models
+ - [x] MLFLOW: Use `MLflow models serve` to deploy a model service
+ - [x] Docker: Run the container after packaging the docker image
+ - [ ] Docker Compose: Use docker compose to run the container, Will replace the docker run above
+ - [ ] Seldon core: Use Selcon core to deploy model to k8s cluster
+ - [ ] k8s: Deploy containers directly to K8S
+ - [ ] mlflow deployments: Built-in deployment modules, such as built-in deployment to SageMaker, etc
+- [ ] Model Registry
+ - [ ] Register Model: Allows artifacts (Including model and related parameters, indicators) to be registered directly into the model center
+
+
## Create Task
- Click `Project -> Management-Project -> Name-Workflow Definition`, and click the "Create Workflow" button to enter the
DAG editing page.
- Drag from the toolbar <img src="/img/tasks/icons/mlflow.png" width="15"/> task node to canvas.
-## Task Parameter
-
-- DolphinScheduler common parameters
- - **Node name**: The node name in a workflow definition is unique.
- - **Run flag**: Identifies whether this node schedules normally, if it does not need to execute, select
- the `prohibition execution`.
- - **Descriptive information**: Describe the function of the node.
- - **Task priority**: When the number of worker threads is insufficient, execute in the order of priority from high
- to low, and tasks with the same priority will execute in a first-in first-out order.
- - **Worker grouping**: Assign tasks to the machines of the worker group to execute. If `Default` is selected,
- randomly select a worker machine for execution.
- - **Environment Name**: Configure the environment name in which run the script.
- - **Times of failed retry attempts**: The number of times the task failed to resubmit.
- - **Failed retry interval**: The time interval (unit minute) for resubmitting the task after a failed task.
- - **Delayed execution time**: The time (unit minute) that a task delays in execution.
- - **Timeout alarm**: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm
- email will send and the task execution will fail.
- - **Custom parameter**: It is a local user-defined parameter for mlflow, and will replace the content
- with `${variable}` in the script.
- - **Predecessor task**: Selecting a predecessor task for the current task, will set the selected predecessor task as
- upstream of the current task.
-
-- MLflow task specific parameters
- - **mlflow server tracking uri** ：MLflow server uri, default http://localhost:5000.
- - **experiment name** ：The experiment in which the task is running, if none, is created.
- - **register model** ：Register the model or not. If register is selected, the following parameters are expanded.
- - **model name** : The registered model name is added to the original model version and registered as
- Production.
- - **job type** : The type of task to run, currently including the underlying algorithm and AutoML. (User-defined
- MLFlow project task execution will be supported in the near future)
- - BasicAlgorithm specific parameters
- - **algorithm** ：The selected algorithm currently supports `LR`, `SVM`, `LightGBM` and `XGboost` based
- on [scikit-learn](https://scikit-learn.org/) form.
- - **Parameter search space** : Parameter search space when running the corresponding algorithm, which can be
- empty. For example, the parameter `max_depth=[5, 10];n_estimators=[100, 200]` for lightgbm 。The convention
- will be passed with '; 'shards each parameter, using the name before the equal sign as the parameter name,
- and using the name after the equal sign to get the corresponding parameter value through `python eval()`.
- - AutoML specific parameters
- - **AutoML tool** : The AutoML tool used, currently
- supports [autosklearn](https://github.com/automl/auto-sklearn)
- and [flaml](https://github.com/microsoft/FLAML)
- - Parameters common to BasicAlgorithm and AutoML
- - **data path** : The absolute path of the file or folder. Ends with .csv for file or contain train.csv and
- test.csv for folder（In the suggested way, users should build their own test sets for model evaluation）。
- - **parameters** : Parameter when initializing the algorithm/AutoML model, which can be empty. For example
- parameters `"time_budget=30;estimator_list=['lgbm']"` for flaml 。The convention will be passed with '; 'shards
- each parameter, using the name before the equal sign as the parameter name, and using the name after the equal
- sign to get the corresponding parameter value through `python eval()`.
- - BasicAlgorithm
- - [lr](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)
- - [SVM](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html?highlight=svc#sklearn.svm.SVC)
- - [lightgbm](https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.LGBMClassifier.html#lightgbm.LGBMClassifier)
- - [xgboost](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBClassifier)
- - AutoML
- - [flaml](https://microsoft.github.io/FLAML/docs/reference/automl#automl-objects)
- - [autosklearn](https://automl.github.io/auto-sklearn/master/api.html)
-
## Task Example
-### Preparation
+First, introduce some general parameters of DolphinScheduler
+
+- **Node name**: The node name in a workflow definition is unique.
+- **Run flag**: Identifies whether this node schedules normally, if it does not need to execute, select
+ the `prohibition execution`.
+- **Descriptive information**: Describe the function of the node.
+- **Task priority**: When the number of worker threads is insufficient, execute in the order of priority from high
+ to low, and tasks with the same priority will execute in a first-in first-out order.
+- **Worker grouping**: Assign tasks to the machines of the worker group to execute. If `Default` is selected,
+ randomly select a worker machine for execution.
+- **Environment Name**: Configure the environment name in which run the script.
+- **Times of failed retry attempts**: The number of times the task failed to resubmit.
+- **Failed retry interval**: The time interval (unit minute) for resubmitting the task after a failed task.
+- **Delayed execution time**: The time (unit minute) that a task delays in execution.
+- **Timeout alarm**: Check the timeout alarm and timeout failure. When the task runs exceed the "timeout", an alarm
+ email will send and the task execution will fail.
+- **Predecessor task**: Selecting a predecessor task for the current task, will set the selected predecessor task as
+ upstream of the current task.
+
+### MLflow Projects
+
+#### BasicAlgorithm
+
+
+
+**Task Parameter**
+
+- **mlflow server tracking uri** ：MLflow server uri, default http://localhost:5000.
+- **job type** : The type of task to run, currently including the underlying algorithm and AutoML. (User-defined
+ MLFlow project task execution will be supported in the near future)
+- **experiment name** ：The experiment in which the task is running, if none, is created.
+- **register model** ：Register the model or not. If register is selected, the following parameters are expanded.
+ - **model name** : The registered model name is added to the original model version and registered as
+ Production.
+- **data path** : The absolute path of the file or folder. Ends with .csv for file or contain train.csv and
+ test.csv for folder（In the suggested way, users should build their own test sets for model evaluation）。
+- **parameters** : Parameter when initializing the algorithm/AutoML model, which can be empty. For example
+ parameters `"time_budget=30;estimator_list=['lgbm']"` for flaml 。The convention will be passed with '; 'shards
+ each parameter, using the name before the equal sign as the parameter name, and using the name after the equal
+ sign to get the corresponding parameter value through `python eval()`.
+ - [lr](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)
Review Comment:
I do not like abbreviation in the document, I do not think all the reader will know its mean.
```suggestion
- [Logistic Regression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#issuecomment-1132347518
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10150?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#10150](https://codecov.io/gh/apache/dolphinscheduler/pull/10150?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5258f2e) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/bf4b8ff0d98990a46dbc7adac01164c515b13ad3?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (bf4b8ff) will **increase** coverage by `0.09%`.
> The diff coverage is `83.58%`.
```diff
@@ Coverage Diff @@
## dev #10150 +/- ##
============================================
+ Coverage 40.83% 40.93% +0.09%
- Complexity 4714 4729 +15
============================================
Files 854 854
Lines 34503 34552 +49
Branches 3814 3818 +4
============================================
+ Hits 14090 14143 +53
+ Misses 19061 19056 -5
- Partials 1352 1353 +1
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/10150?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...nscheduler/plugin/task/mlflow/MlflowConstants.java](https://codecov.io/gh/apache/dolphinscheduler/pull/10150/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stbWxmbG93L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi90YXNrL21sZmxvdy9NbGZsb3dDb25zdGFudHMuamF2YQ==) | `0.00% <ø> (ø)` | |
| [...scheduler/plugin/task/mlflow/MlflowParameters.java](https://codecov.io/gh/apache/dolphinscheduler/pull/10150/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stbWxmbG93L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi90YXNrL21sZmxvdy9NbGZsb3dQYXJhbWV0ZXJzLmphdmE=) | `63.09% <81.81%> (+9.47%)` | :arrow_up: |
| [...olphinscheduler/plugin/task/mlflow/MlflowTask.java](https://codecov.io/gh/apache/dolphinscheduler/pull/10150/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci10YXNrLXBsdWdpbi9kb2xwaGluc2NoZWR1bGVyLXRhc2stbWxmbG93L3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9kb2xwaGluc2NoZWR1bGVyL3BsdWdpbi90YXNrL21sZmxvdy9NbGZsb3dUYXNrLmphdmE=) | `66.17% <84.44%> (+42.64%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10150?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10150?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [bf4b8ff...5258f2e](https://codecov.io/gh/apache/dolphinscheduler/pull/10150?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] jieguangzhou commented on a diff in pull request #10150: [Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker)
Posted by GitBox <gi...@apache.org>.
jieguangzhou commented on code in PR #10150:
URL: https://github.com/apache/dolphinscheduler/pull/10150#discussion_r877884703
##########
docs/docs/en/guide/task/mlflow.md:
##########
@@ -5,77 +5,140 @@
[MLflow](https://mlflow.org) is an excellent open source platform to manage the ML lifecycle, including experimentation,
reproducibility, deployment, and a central model registry.
-Mlflow task is used to perform mlflow project tasks, which include basic algorithmic and autoML capabilities (
-User-defined MLFlow project task execution will be supported in the near future)
+MLflow task plugin used to execute MLflow tasks，Currently contains Mlflow Projects and MLflow Models.（Model Registry will soon be rewarded for support）
+
+- Mlflow Projects: Package data science code in a format to reproduce runs on any platform.
+- MLflow Models: Deploy machine learning models in diverse serving environments.
+- Model Registry: Store, annotate, discover, and manage models in a central repository.
+
+The Mlflow plugin currently supports and will support the following:
+
+- [ ] MLflow Projects
+ - [x] BasicAlgorithm: contains lr, svm, lightgbm, xgboost
+ - [x] AutoML: AutoML tool，contains autosklean, flaml
+ - [ ] Custom projects: Support for running your own MLflow projects
+- [ ] MLflow Models
+ - [x] MLFLOW: Use `MLflow models serve` to deploy a model service
+ - [x] Docker: Run the container after packaging the docker image
+ - [ ] Docker Compose: Use docker compose to run the container, Will replace the docker run above
+ - [ ] Seldon core: Use Selcon core to deploy model to k8s cluster
+ - [ ] k8s: Deploy containers directly to K8S
+ - [ ] mlflow deployments: Built-in deployment modules, such as built-in deployment to SageMaker, etc
+- [ ] Model Registry
+ - [ ] Register Model: Allows artifacts (Including model and related parameters, indicators) to be registered directly into the model center
Review Comment:
I think a list of future supported components is not only shown for developers but users. This can attract users by showing them future features.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org