You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by gallenvara <gi...@git.apache.org> on 2016/05/09 11:58:11 UTC
[GitHub] flink pull request: [FLINK-2184] Cannot get last element with maxB...
GitHub user gallenvara opened a pull request:
https://github.com/apache/flink/pull/1975
[FLINK-2184] Cannot get last element with maxBy/minBy.
Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful description of your changes.
- [X] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the JIRA id)
- [X] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
Extended the minBy/maxBy methods to support returning last element.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gallenvara/flink flink-2184
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1975.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1975
----
commit 8ae9bb689320a0b5b3fe199ecc61c62504ff0e7d
Author: gallenvara <ga...@126.com>
Date: 2016-05-09T11:51:24Z
Extend minBy/maxBy methods to support returning last element.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83318228
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/KeyedStream.scala ---
@@ -224,45 +224,95 @@ class KeyedStream[T, K](javaStream: KeyedJavaStream[T, K]) extends DataStream[T]
/**
* Applies an aggregation that that gives the current minimum element of the data stream by
* the given position by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * When equality, the first element is returned with the minimal value by default.
*
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType
- .MINBY, position)
-
- /**
+ .MINBY, position, true)
+
+ /**
* Applies an aggregation that that gives the current minimum element of the data stream by
- * the given field by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * the given position by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType
+ .MINBY, position, first)
+
+ /**
+ * Applies an aggregation that that gives the current minimum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, the first element is returned with the minimal value by default.
*
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType
- .MINBY, field )
+ .MINBY, field, true)
+
+ /**
+ * Applies an aggregation that that gives the current minimum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType
+ .MINBY, field, first)
/**
* Applies an aggregation that that gives the current maximum element of the data stream by
* the given position by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the maximal value.
+ * When equality, the first element is returned with the maximal value by default.
*
*/
def maxBy(position: Int): DataStream[T] =
- aggregate(AggregationType.MAXBY, position)
-
- /**
+ aggregate(AggregationType.MAXBY, position, true)
+
+ /**
+ * Applies an aggregation that that gives the current maximum element of the data stream by
+ * the given position by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] =
+ aggregate(AggregationType.MAXBY, position, first)
+
+ /**
* Applies an aggregation that that gives the current maximum element of the data stream by
* the given field by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the maximal value.
+ * When equality, the first element is returned with the maximal value by default.
*
*/
def maxBy(field: String): DataStream[T] =
- aggregate(AggregationType.MAXBY, field)
-
+ aggregate(AggregationType.MAXBY, field, true)
+
+ /**
+ * Applies an aggregation that that gives the current maximum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] =
+ aggregate(AggregationType.MAXBY, field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ aggregate(aggregationType, position, false)
+ }
+
private def aggregate(aggregationType: AggregationType, field: String): DataStream[T] = {
+ aggregate(aggregationType, field, false)
+ }
+
+ private def aggregate(
+ aggregationType: AggregationType,
+ field: String,
+ first: Boolean): DataStream[T] = {
val position = fieldNames2Indices(javaStream.getType(), Array(field))(0)
- aggregate(aggregationType, position)
+ aggregate(aggregationType, position, first)
}
- private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ private def aggregate(
+ aggregationType: AggregationType,
+ position: Int,
+ first: Boolean): DataStream[T] = {
--- End diff --
`first` is not used.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83310395
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala ---
@@ -361,38 +361,84 @@ class AllWindowedStream[T, W <: Window](javaStream: JavaAllWStream[T, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
- * Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
--- End diff --
`maximum` -> `minimum`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83310338
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala ---
@@ -361,38 +361,84 @@ class AllWindowedStream[T, W <: Window](javaStream: JavaAllWStream[T, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
- * Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * Applies an aggregation that that gives the maximum element of the window by
--- End diff --
`maximum` -> `minimum`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83319240
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/WindowedStream.scala ---
@@ -362,38 +362,84 @@ class WindowedStream[T, K, W <: Window](javaStream: JavaWStream[T, K, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType.MINBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
--- End diff --
This method may not be changed to `private` because it is a stable `@Public` interface.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83310436
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala ---
@@ -361,38 +361,84 @@ class AllWindowedStream[T, W <: Window](javaStream: JavaAllWStream[T, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
- * Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType.MINBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
--- End diff --
`maximum` -> `minimum`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83309671
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala ---
@@ -361,38 +361,84 @@ class AllWindowedStream[T, W <: Window](javaStream: JavaAllWStream[T, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
--- End diff --
Please annotate all new public methods with `@PublicEvolving`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83319084
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/WindowedStream.scala ---
@@ -362,38 +362,84 @@ class WindowedStream[T, K, W <: Window](javaStream: JavaWStream[T, K, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType.MINBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ aggregate(aggregationType, position, false)
+ }
private def aggregate(aggregationType: AggregationType, field: String): DataStream[T] = {
+ aggregate(aggregationType, field, false)
+ }
+
+ private def aggregate(
+ aggregationType: AggregationType,
+ field: String,
+ first: Boolean): DataStream[T] = {
val position = fieldNames2Indices(getInputType(), Array(field))(0)
aggregate(aggregationType, position)
}
- def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ private def aggregate(
+ aggregationType: AggregationType,
+ position: Int,
+ first: Boolean): DataStream[T] = {
--- End diff --
`first` is not used.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83319026
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/WindowedStream.scala ---
@@ -362,38 +362,84 @@ class WindowedStream[T, K, W <: Window](javaStream: JavaWStream[T, K, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType.MINBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ aggregate(aggregationType, position, false)
--- End diff --
default should be `true` instead of `false`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83308865
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala ---
@@ -361,38 +361,84 @@ class AllWindowedStream[T, W <: Window](javaStream: JavaAllWStream[T, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
--- End diff --
typo `that that` copied from the other docs in this file. Could you please fix all of them? Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83308572
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/AllWindowedStream.scala ---
@@ -361,38 +361,84 @@ class AllWindowedStream[T, W <: Window](javaStream: JavaAllWStream[T, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
- * Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType.MINBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
--- End diff --
This method may not be `private`. All public method of this class are declared stable `@Public` and may not be changed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2184] Cannot get last element with maxB...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1975#issuecomment-219966941
Hi @gallenvara, thanks for reminding.
I put it on my list but might take some time until I get to review it. Thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83318661
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/WindowedStream.scala ---
@@ -362,38 +362,84 @@ class WindowedStream[T, K, W <: Window](javaStream: JavaWStream[T, K, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
--- End diff --
Please add `@PublicEvolving` to new public methods.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83318311
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/KeyedStream.scala ---
@@ -224,45 +224,95 @@ class KeyedStream[T, K](javaStream: KeyedJavaStream[T, K]) extends DataStream[T]
/**
* Applies an aggregation that that gives the current minimum element of the data stream by
* the given position by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * When equality, the first element is returned with the minimal value by default.
*
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType
- .MINBY, position)
-
- /**
+ .MINBY, position, true)
+
+ /**
* Applies an aggregation that that gives the current minimum element of the data stream by
- * the given field by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * the given position by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType
+ .MINBY, position, first)
+
+ /**
+ * Applies an aggregation that that gives the current minimum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, the first element is returned with the minimal value by default.
*
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType
- .MINBY, field )
+ .MINBY, field, true)
+
+ /**
+ * Applies an aggregation that that gives the current minimum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType
+ .MINBY, field, first)
/**
* Applies an aggregation that that gives the current maximum element of the data stream by
* the given position by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the maximal value.
+ * When equality, the first element is returned with the maximal value by default.
*
*/
def maxBy(position: Int): DataStream[T] =
- aggregate(AggregationType.MAXBY, position)
-
- /**
+ aggregate(AggregationType.MAXBY, position, true)
+
+ /**
+ * Applies an aggregation that that gives the current maximum element of the data stream by
+ * the given position by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] =
+ aggregate(AggregationType.MAXBY, position, first)
+
+ /**
* Applies an aggregation that that gives the current maximum element of the data stream by
* the given field by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the maximal value.
+ * When equality, the first element is returned with the maximal value by default.
*
*/
def maxBy(field: String): DataStream[T] =
- aggregate(AggregationType.MAXBY, field)
-
+ aggregate(AggregationType.MAXBY, field, true)
+
+ /**
+ * Applies an aggregation that that gives the current maximum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] =
+ aggregate(AggregationType.MAXBY, field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ aggregate(aggregationType, position, false)
+ }
+
private def aggregate(aggregationType: AggregationType, field: String): DataStream[T] = {
+ aggregate(aggregationType, field, false)
--- End diff --
default should be `true` instead of `false`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83318300
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/KeyedStream.scala ---
@@ -224,45 +224,95 @@ class KeyedStream[T, K](javaStream: KeyedJavaStream[T, K]) extends DataStream[T]
/**
* Applies an aggregation that that gives the current minimum element of the data stream by
* the given position by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * When equality, the first element is returned with the minimal value by default.
*
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType
- .MINBY, position)
-
- /**
+ .MINBY, position, true)
+
+ /**
* Applies an aggregation that that gives the current minimum element of the data stream by
- * the given field by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * the given position by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType
+ .MINBY, position, first)
+
+ /**
+ * Applies an aggregation that that gives the current minimum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, the first element is returned with the minimal value by default.
*
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType
- .MINBY, field )
+ .MINBY, field, true)
+
+ /**
+ * Applies an aggregation that that gives the current minimum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType
+ .MINBY, field, first)
/**
* Applies an aggregation that that gives the current maximum element of the data stream by
* the given position by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the maximal value.
+ * When equality, the first element is returned with the maximal value by default.
*
*/
def maxBy(position: Int): DataStream[T] =
- aggregate(AggregationType.MAXBY, position)
-
- /**
+ aggregate(AggregationType.MAXBY, position, true)
+
+ /**
+ * Applies an aggregation that that gives the current maximum element of the data stream by
+ * the given position by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] =
+ aggregate(AggregationType.MAXBY, position, first)
+
+ /**
* Applies an aggregation that that gives the current maximum element of the data stream by
* the given field by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the maximal value.
+ * When equality, the first element is returned with the maximal value by default.
*
*/
def maxBy(field: String): DataStream[T] =
- aggregate(AggregationType.MAXBY, field)
-
+ aggregate(AggregationType.MAXBY, field, true)
+
+ /**
+ * Applies an aggregation that that gives the current maximum element of the data stream by
+ * the given field by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] =
+ aggregate(AggregationType.MAXBY, field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ aggregate(aggregationType, position, false)
--- End diff --
default should be `true` instead of `false`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83319036
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/WindowedStream.scala ---
@@ -362,38 +362,84 @@ class WindowedStream[T, K, W <: Window](javaStream: JavaWStream[T, K, W]) {
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def maxBy(position: Int): DataStream[T] = aggregate(AggregationType.MAXBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the maximum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def maxBy(field: String): DataStream[T] = aggregate(AggregationType.MAXBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the maximum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def maxBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MAXBY,
+ field, first)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given position. When equality, returns the first.
+ * the given position. When equality, returns the first by default.
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType.MINBY,
- position)
+ position, true)
/**
* Applies an aggregation that that gives the minimum element of the window by
- * the given field. When equality, returns the first.
+ * the given position. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ position, first)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns the first by default.
*/
def minBy(field: String): DataStream[T] = aggregate(AggregationType.MINBY,
- field)
+ field, true)
+
+ /**
+ * Applies an aggregation that that gives the minimum element of the window by
+ * the given field. When equality, returns either the first or last one depending
+ * on the parameter setting.
+ */
+ def minBy(field: String, first: Boolean): DataStream[T] = aggregate(AggregationType.MINBY,
+ field, first)
+
+ private def aggregate(aggregationType: AggregationType, position: Int): DataStream[T] = {
+ aggregate(aggregationType, position, false)
+ }
private def aggregate(aggregationType: AggregationType, field: String): DataStream[T] = {
+ aggregate(aggregationType, field, false)
--- End diff --
default should be `true` instead of `false`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1975#discussion_r83316946
--- Diff: flink-streaming-scala/src/main/scala/org/apache/flink/streaming/api/scala/KeyedStream.scala ---
@@ -224,45 +224,95 @@ class KeyedStream[T, K](javaStream: KeyedJavaStream[T, K]) extends DataStream[T]
/**
* Applies an aggregation that that gives the current minimum element of the data stream by
* the given position by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * When equality, the first element is returned with the minimal value by default.
*
*/
def minBy(position: Int): DataStream[T] = aggregate(AggregationType
- .MINBY, position)
-
- /**
+ .MINBY, position, true)
+
+ /**
* Applies an aggregation that that gives the current minimum element of the data stream by
- * the given field by the given key. An independent aggregate is kept per key.
- * When equality, the first element is returned with the minimal value.
+ * the given position by the given key. An independent aggregate is kept per key.
+ * When equality, returns either the first or last one depending on the parameter setting.
+ *
+ */
+ def minBy(position: Int, first: Boolean): DataStream[T] = aggregate(AggregationType
--- End diff --
Please add `@PublicEvolving` to new public methods.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #1975: [FLINK-2184] Cannot get last element with maxBy/mi...
Posted by gallenvara <gi...@git.apache.org>.
Github user gallenvara closed the pull request at:
https://github.com/apache/flink/pull/1975
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2184] Cannot get last element with maxB...
Posted by gallenvara <gi...@git.apache.org>.
Github user gallenvara commented on the pull request:
https://github.com/apache/flink/pull/1975#issuecomment-219362150
Hi @fhueske , can you help review this PR? Thanks! :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---