You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@liminal.apache.org by GitBox <gi...@apache.org> on 2021/08/04 15:06:43 UTC
[GitHub] [incubator-liminal] assapin opened a new pull request #67: Spark example improvements
assapin opened a new pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67
turned the spark code into more structured
used spark ml tools to normalize the data and transform the labels
updated the training code to also compute accuracy and do cross validation
updated the validation code to ensure accuracy is about 90% before promoting to production
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-liminal] liorsav merged pull request #67: Spark example improvements
Posted by GitBox <gi...@apache.org>.
liorsav merged pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-liminal] zionrubin commented on a change in pull request #67: Spark example improvements
Posted by GitBox <gi...@apache.org>.
zionrubin commented on a change in pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67#discussion_r684648586
##########
File path: examples/spark-app-demo/k8s/liminal.yml
##########
@@ -52,20 +52,22 @@ pipelines:
type: spark
description: prepare the data for training
application_arguments:
+ - '--input_uri'
Review comment:
application arguments can be a dict. you can do the following instead:
```YAML
application_arguments:
--input_uri: '{{input_root_dir}}data/iris.csv'
--output_uri: '{{training_data_path}}'
```
the result will be the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-liminal] zionrubin commented on a change in pull request #67: Spark example improvements
Posted by GitBox <gi...@apache.org>.
zionrubin commented on a change in pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67#discussion_r684648586
##########
File path: examples/spark-app-demo/k8s/liminal.yml
##########
@@ -52,20 +52,22 @@ pipelines:
type: spark
description: prepare the data for training
application_arguments:
+ - '--input_uri'
Review comment:
application arguments can be a dict. you can do the following instead:
```YAML
application_arguments:
--input_uri: '{{input_root_dir}}data/iris.csv'
--output_uri: '{{training_data_path}}'
```
the result will be the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-liminal] assapin commented on pull request #67: Spark example improvements
Posted by GitBox <gi...@apache.org>.
assapin commented on pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67#issuecomment-892737047
@zionrubin @aviemzur
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-liminal] lior-schachter commented on pull request #67: Spark example improvements
Posted by GitBox <gi...@apache.org>.
lior-schachter commented on pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67#issuecomment-900108229
LGTM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-liminal] assapin commented on pull request #67: Spark example improvements
Posted by GitBox <gi...@apache.org>.
assapin commented on pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67#issuecomment-892737047
@zionrubin @aviemzur
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-liminal] zionrubin commented on a change in pull request #67: Spark example improvements
Posted by GitBox <gi...@apache.org>.
zionrubin commented on a change in pull request #67:
URL: https://github.com/apache/incubator-liminal/pull/67#discussion_r684648954
##########
File path: examples/spark-app-demo/k8s/liminal.yml
##########
@@ -20,7 +20,7 @@ name: MyFirstLiminalSparkApp
super: InfraSpark
Review comment:
Can you update the code snippet in the doc? (docs/getting-started/spark-app-demo/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@liminal.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org