You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by ChengXiangLi <gi...@git.apache.org> on 2015/11/03 08:08:24 UTC
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
GitHub user ChengXiangLi opened a pull request:
https://github.com/apache/flink/pull/1318
[FLINK-2955] Add operators description in Table API page
On the Table API page, there is no formal introduction of current supported operations, it should be nice to have it.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ChengXiangLi/flink tableApiOperatorIntroduction
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1318.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1318
----
commit f36a6cfae48da767c34ac925bbfa4c59700768df
Author: chengxiang li <ch...@intel.com>
Date: 2015-11-03T07:11:09Z
[FLINK-2955] [Documentation] Add operators description in Table API page.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44402293
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
--- End diff --
If you use, "{{ site.baseurl }}/api/java/org/apache/flink/api/table/Table.html", the correct version will be injected when the markdown file is compiled to html.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44281719
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+
+<br />
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Operators</th>
+ <th class="text-center">Description</th>
+ </tr>
+ </thead>
+
+ <tbody>
+ <tr>
+ <td><strong>Select</strong></td>
+ <td>
+ <p>Similar to a SQL SELECT statement. Perform a select operation.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.select("a, b, c as d");
--- End diff --
Leave out `a` or `b` to demonstrate the projection?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by ChengXiangLi <gi...@git.apache.org>.
Github user ChengXiangLi commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44366143
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
--- End diff --
Where should i link to? The javdoc link of 1.0 version does not exist yet, should i link to 1.0 here anyway, or link to 0.9 here, and let release manager update to 1.0 while release 1.0?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on the pull request:
https://github.com/apache/flink/pull/1318#issuecomment-155086121
Thanks for extending the documentation of the Table API. I added a few comments to the Java section. Most apply to the Scala section as well.
Thanks, Fabian
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44282588
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+
+<br />
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Operators</th>
+ <th class="text-center">Description</th>
+ </tr>
+ </thead>
+
+ <tbody>
+ <tr>
+ <td><strong>Select</strong></td>
+ <td>
+ <p>Similar to a SQL SELECT statement. Perform a select operation.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.select("a, b, c as d");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>As</strong></td>
+ <td>
+ <p>Rename the fields name.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.as("d, e, f");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>Filter</strong></td>
+ <td>
+ <p>Similar to a SQL WHERE clause. Filter out elements that do not pass the filter predicate.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.filter("a % 2 = 0");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>Where</strong></td>
+ <td>
+ <p>Similar to a SQL WHERE clause. Filter out elements that do not pass the filter predicate.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.where("b = 'red'");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>GroupBy</strong></td>
+ <td>
+ <p>Similar to a SQL GROUPBY clause. Group the elements on the grouping keys, with a following aggregation</p>
+ <p>operator to aggregate on per-group basis.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.groupby("a").select("a, b.sum");
+{% endhighlight %}
+ </td>
+ </tr>
-A `Table` supports to following operations: `select`, `where`, `groupBy`, `join` (Plus `filter` as
-an alias for `where`.). These are also documented in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html)
-of Table.
+ <tr>
+ <td><strong>Join</strong></td>
+ <td>
+ <p>Similar to a SQL JOIN clause. Join two tables, the fields of two joined tables must not overlap.</p>
--- End diff --
I would rephrase "the fields of two joined tables must not overlap" to "both tables must have distinct field names". It would also be good to mention that the join condition is mandatory and specified with `where()`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44280518
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
--- End diff --
query -> queries
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44282858
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+
+<br />
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Operators</th>
+ <th class="text-center">Description</th>
+ </tr>
+ </thead>
+
+ <tbody>
+ <tr>
+ <td><strong>Select</strong></td>
+ <td>
+ <p>Similar to a SQL SELECT statement. Perform a select operation.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.select("a, b, c as d");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>As</strong></td>
+ <td>
+ <p>Rename the fields name.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.as("d, e, f");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>Filter</strong></td>
+ <td>
+ <p>Similar to a SQL WHERE clause. Filter out elements that do not pass the filter predicate.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.filter("a % 2 = 0");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>Where</strong></td>
+ <td>
+ <p>Similar to a SQL WHERE clause. Filter out elements that do not pass the filter predicate.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.where("b = 'red'");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>GroupBy</strong></td>
+ <td>
+ <p>Similar to a SQL GROUPBY clause. Group the elements on the grouping keys, with a following aggregation</p>
+ <p>operator to aggregate on per-group basis.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.groupby("a").select("a, b.sum");
+{% endhighlight %}
+ </td>
+ </tr>
-A `Table` supports to following operations: `select`, `where`, `groupBy`, `join` (Plus `filter` as
-an alias for `where`.). These are also documented in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html)
-of Table.
+ <tr>
+ <td><strong>Join</strong></td>
+ <td>
+ <p>Similar to a SQL JOIN clause. Join two tables, the fields of two joined tables must not overlap.</p>
+{% highlight java %}
+Table left = tableEnv.fromDataSet(ds1, "a, b, c");
+Table right = tableEnv.fromDataSet(ds2, "d, e, f");
+Table result = left.join(right).where("a = d").select("a, b, e");
+{% endhighlight %}
+ </td>
+ </tr>
-Some of these expect an expression. These can either be specified using an embedded Scala DSL or
+ <tr>
+ <td><strong>Union</strong></td>
+ <td>
+ <p>Similar to a SQL UNION ALL clause. Union two tables, the fields of two unioned tables must overlap.</p>
--- End diff --
Rephrase "the fields of two unioned tables must overlap" to "Both tables must have identical schema (field names and types)."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44281162
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
--- End diff --
We shouldn't put a hard link to the latest JavaDocs here but link to the correct version.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44281637
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+
+<br />
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Operators</th>
+ <th class="text-center">Description</th>
+ </tr>
+ </thead>
+
+ <tbody>
+ <tr>
+ <td><strong>Select</strong></td>
+ <td>
+ <p>Similar to a SQL SELECT statement. Perform a select operation.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.select("a, b, c as d");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>As</strong></td>
+ <td>
+ <p>Rename the fields name.</p>
--- End diff --
"Rename the fields name" is redundant. How about "Rename fields"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/1318#issuecomment-157998634
Thanks for adding this. :+1:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by fhueske <gi...@git.apache.org>.
Github user fhueske commented on a diff in the pull request:
https://github.com/apache/flink/pull/1318#discussion_r44282172
--- Diff: docs/libs/table.md ---
@@ -123,13 +123,205 @@ DataSet<WC> result = tableEnv.toDataSet(filtered, WC.class);
When using Java, the embedded DSL for specifying expressions cannot be used. Only String expressions
are supported. They support exactly the same feature set as the expression DSL.
-## Expression Syntax
+## Table API Operators
+Table API provide a domain-spcific language to execute language-integrated query on structured data in Scala and Java.
+This section gives a brief overview of all available operators. You can find more details of operators in the [Javadoc](http://flink.apache.org/docs/latest/api/java/org/apache/flink/api/table/Table.html).
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+
+<br />
+
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 20%">Operators</th>
+ <th class="text-center">Description</th>
+ </tr>
+ </thead>
+
+ <tbody>
+ <tr>
+ <td><strong>Select</strong></td>
+ <td>
+ <p>Similar to a SQL SELECT statement. Perform a select operation.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.select("a, b, c as d");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>As</strong></td>
+ <td>
+ <p>Rename the fields name.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.as("d, e, f");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>Filter</strong></td>
+ <td>
+ <p>Similar to a SQL WHERE clause. Filter out elements that do not pass the filter predicate.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.filter("a % 2 = 0");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>Where</strong></td>
+ <td>
+ <p>Similar to a SQL WHERE clause. Filter out elements that do not pass the filter predicate.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.where("b = 'red'");
+{% endhighlight %}
+ </td>
+ </tr>
+
+ <tr>
+ <td><strong>GroupBy</strong></td>
+ <td>
+ <p>Similar to a SQL GROUPBY clause. Group the elements on the grouping keys, with a following aggregation</p>
+ <p>operator to aggregate on per-group basis.</p>
+{% highlight java %}
+Table in = tableEnv.fromDataSet(ds, "a, b, c");
+Table result = in.groupby("a").select("a, b.sum");
--- End diff --
can you assign an attribute name to `b.sum`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/1318#issuecomment-157998150
I'm merging
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by ChengXiangLi <gi...@git.apache.org>.
Github user ChengXiangLi closed the pull request at:
https://github.com/apache/flink/pull/1318
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/1318#issuecomment-157728654
@fhueske is this good to merge now?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request: [FLINK-2955] Add operators description in Tabl...
Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the pull request:
https://github.com/apache/flink/pull/1318#issuecomment-157999295
Github seems to not want to close this. @ChengXiangLi Could you please do it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---