You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by tfournier314 <gi...@git.apache.org> on 2016/10/23 14:05:15 UTC
[GitHub] flink pull request #2684: Add EvaluateDataSet Operation for LabeledVector - ...
GitHub user tfournier314 opened a pull request:
https://github.com/apache/flink/pull/2684
Add EvaluateDataSet Operation for LabeledVector - This closes #4865
Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful description of your changes.
- [ ] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [ ] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tfournier314/flink AddEvaluateDataSetOperation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/2684.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2684
----
commit 5dfa434c083c4cdc7ba0e9a2ae2aaff121ef18a7
Author: Thomas FOURNIER <tf...@nextperf.com>
Date: 2016-10-23T14:04:14Z
Add EvaluateDataSet Operation for LabeledVector - This closes #4865
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2684: Add EvaluateDataSet Operation for LabeledVector - This cl...
Posted by thvasilo <gi...@git.apache.org>.
Github user thvasilo commented on the issue:
https://github.com/apache/flink/pull/2684
Hello @tillrohrmann I think you make some valid points. My original idea was to move completely to using `LabeledVector` to the de facto format for supervised learning problems and do away with `(Vector, Double)` tuples.
What you suggest would add some extra flexibility though, esp. in terms of `(FeatureVector, LabelVector)` tuples which we have discussed in the context of ANNs.
@tfournier314 Could you investigate the applicability of the scheme that Till recommended?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2684: Add EvaluateDataSet Operation for LabeledVector - This cl...
Posted by thvasilo <gi...@git.apache.org>.
Github user thvasilo commented on the issue:
https://github.com/apache/flink/pull/2684
> We don't need to care about Jenkins/Travis fails, do we ?
Normally we should, but there many unstable tests and issues with Travis that it's hard to follow now.
You should generally make sure the failures don't originate from code you have contributed, and then it's up to the commiter to decide.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2684: Add EvaluateDataSet Operation for LabeledVector - This cl...
Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:
https://github.com/apache/flink/pull/2684
Thanks for your contribution @tfournier314.
I'm wondering whether we should introduce a new `EvaluateDataSetOperation` for the special case of a `LabeledVector`. Can't you easily achieve the same by `val evaluateDS = labeledVectorDS.map(x => (x.vector, x.label))` and then using `evaluateDS` for the `evaluate` operation?
In general I think it would be better to write a generic `EvaluateDataSetOperation` which can take a `ValueExtractor` which can extract the value from the `Testing` type. Then we could offer a `ValueExtractor` for the `LabeledVector`. That way we could extend it easily for different types as well.
A general remark: It's always helpful to write a PR description and format the title according to the contribution guidelines [1]. Furthermore, your code does not adhere to the existing code style. Even though Flink does not have a strict code style, it is always good to stick to the code style of the existing code.
[1] http://flink.apache.org/how-to-contribute.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2684: Add EvaluateDataSet Operation for LabeledVector - This cl...
Posted by thvasilo <gi...@git.apache.org>.
Github user thvasilo commented on the issue:
https://github.com/apache/flink/pull/2684
Thank you for your contribution Thomas!
The changes look mostly fine I think, my only concern is the need to cast the types in lines [215-216](https://github.com/apache/flink/pull/2684/files#diff-c74d49c94a86f962939a42362f3ea190R215), I'm not sure if there is a more elegant way to do that.
I'm going to ping @chiwanpark, @mbalassi and @tillrohrmann, if anyone has some time to review and merge this let us know in the comments.
As a side note, I've noticed that we don't include the evaluate operation in the documentation of all Predictors, so I've created [FLINK-4908](https://issues.apache.org/jira/browse/FLINK-4908) to track that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2684: Add EvaluateDataSet Operation for LabeledVector - This cl...
Posted by thvasilo <gi...@git.apache.org>.
Github user thvasilo commented on the issue:
https://github.com/apache/flink/pull/2684
One last thing, could you change this PR's title to `[FLINK-4865] [ml] Add EvaluateDataSet operation for LabeledVector`
This is the standard naming for PRs, issue number, followed by component, followed by the description. That way the PR gets linked back to the JIRA issue automatically.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2684: [FLINK-4865] [ml] Add EvaluateDataSet operation for Label...
Posted by tfournier314 <gi...@git.apache.org>.
Github user tfournier314 commented on the issue:
https://github.com/apache/flink/pull/2684
@thvasilo @tillrohrmann Ok I will investigate this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #2684: Add EvaluateDataSet Operation for LabeledVector - This cl...
Posted by tfournier314 <gi...@git.apache.org>.
Github user tfournier314 commented on the issue:
https://github.com/apache/flink/pull/2684
Ok thanks !
We don't need to care about Jenkins/Travis fails, do we ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---