You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Robert Metzger <rm...@apache.org> on 2017/05/18 09:01:03 UTC
[DISCUSS] Reorganize Table API / SQL documentation
Hi,
I'm wondering whether we should make the Table API a bit more prominent in
our documentation by upgrading it from below "Libraries" to the same level
as "DataSet" and "DataStream".
This would also allow us to split it from one large page into smaller
sub-pages.
I think it would be nice to do this change as part of the documentation
writing of the new 1.3 features.
Let me know what you think.
Regards,
Robert
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Fabian Hueske <fh...@gmail.com>.
Hi,
I merged the feature branch back to the master and the release-1.3 branch
and deleted it.
There are still a few gaps in the docs that need to be filled (there are
JIRAs for those), but the overall structure looks good IMO and most aspects
of the APIs are covered.
Cheers, Fabian
2017-05-28 15:21 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
> Hi everybody,
>
> I added a couple of subissues to https://issues.apache.org/
> jira/browse/FLINK-5354 and started to work on the first two.
> I'll open PRs for those soon.
>
> Please pick an issue and assign it to yourself if you want to help with
> the Table API / SQL documentation.
>
> Thanks,
> Fabian
>
>
> 2017-05-23 17:49 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
>
>> Hi everybody,
>>
>> I pushed the branch to the ASF Flink repository as a feature branch to
>> keep all PRs in one place:
>>
>> --> https://github.com/apache/flink/tree/tableDocs
>>
>> Thanks,
>> Fabian
>>
>> 2017-05-23 16:25 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
>>
>>> Hi everybody,
>>>
>>> I prepared a branch that creates the proposed structure and copied the
>>> existing documentation into the corresponding pages / sections.
>>> There are plenty of gaps that need to be filled or reworked.
>>>
>>> --> https://github.com/fhueske/flink/tree/tableDocs
>>>
>>> How do we go on from here?
>>> I think the easiest would be if everybody who's interested in working on
>>> the documentation picks a page and prepares a PR against my branch (we
>>> could also push this into a feature branch in the Flink repository if
>>> somebody prefers that). The PRs are cross-checked and we merge everything
>>> into to the master when the docs are ready.
>>>
>>> Any opinions or other proposals?
>>>
>>> Cheers, Fabian
>>>
>>> 2017-05-23 10:31 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
>>>
>>>> Hi everybody,
>>>>
>>>> Thanks for the feedback. I'll go ahead and create the proposed
>>>> structure and move the content of the existing docs with comments of what
>>>> needs to be adapted.
>>>> I'll put this into branch of my Github repo and let you know when I'm
>>>> done.
>>>> From there, we can distribute working on the missing parts / parts that
>>>> need adaption.
>>>>
>>>> Cheers, Fabian
>>>>
>>>> 2017-05-19 9:44 GMT+01:00 jincheng sun <su...@gmail.com>:
>>>>
>>>>> Hi, Fabian,
>>>>>
>>>>> Thanks for the sketch. The structure is pretty well to me, And glad
>>>>> to
>>>>> join in the discussion in google doc.
>>>>>
>>>>> Cheers,
>>>>> SunJincheng
>>>>>
>>>>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <ws...@gmail.com>:
>>>>>
>>>>> > Hello Fabian,
>>>>> > Thanks for drafting the proposal. I like the entire organization in
>>>>> general
>>>>> > and left a few comments. I think this will be a very good kick off to
>>>>> > reorganize the tableAPI&SQL doc.
>>>>> >
>>>>> > -shaoxuan
>>>>> >
>>>>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <fh...@gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > > Hi everybody,
>>>>> > >
>>>>> > > I came up with a proposal for the structure of the Table API / SQL
>>>>> > > documentation:
>>>>> > >
>>>>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>>>>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>>>>> > >
>>>>> > > Feedback and comments are very welcome.
>>>>> > > Once we agree on a structure, we can create skeletons and
>>>>> distribute the
>>>>> > > work.
>>>>> > >
>>>>> > > Cheers,
>>>>> > > Fabian
>>>>> > >
>>>>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
>>>>> > >
>>>>> > > > +1
>>>>> > > >
>>>>> > > > The Table / SQL component has made significant progress in the
>>>>> last few
>>>>> > > > months (kudos to all contributors).
>>>>> > > >
>>>>> > > > It is a good time to have a documentation to reflect all the
>>>>> changes in
>>>>> > > the
>>>>> > > > Table / SQL side.
>>>>> > > >
>>>>> > > >
>>>>> > > >
>>>>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <
>>>>> rmetzger@apache.org>
>>>>> > > > wrote:
>>>>> > > >
>>>>> > > > > Thank you Fabian for working on the proposal.
>>>>> > > > >
>>>>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <
>>>>> fhueske@gmail.com>
>>>>> > > > wrote:
>>>>> > > > >
>>>>> > > > > > Thanks for starting this discussion Robert.
>>>>> > > > > >
>>>>> > > > > > I think with the next release the Table API / SQL should be
>>>>> moved
>>>>> > up
>>>>> > > in
>>>>> > > > > the
>>>>> > > > > > Application Development menu.
>>>>> > > > > > I also though about restructuring the docs, but it won't be
>>>>> trivial
>>>>> > > to
>>>>> > > > do
>>>>> > > > > > this, IMO because there are many orthogonal aspects:
>>>>> > > > > > - Stream/Batch
>>>>> > > > > > - Table/SQL
>>>>> > > > > > - Scala/Java
>>>>> > > > > >
>>>>> > > > > > and sometimes also common concepts.
>>>>> > > > > > At the moment there are also many new features missing like
>>>>> OVER
>>>>> > > > windows,
>>>>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator
>>>>> attributes,
>>>>> > > filter
>>>>> > > > > > pushdown, ...
>>>>> > > > > >
>>>>> > > > > > I will try to sketch a new structure in a Google Doc in the
>>>>> next
>>>>> > days
>>>>> > > > and
>>>>> > > > > > share it in this thread.
>>>>> > > > > >
>>>>> > > > > > Cheers, Fabian
>>>>> > > > > >
>>>>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>>>>> > > k.kloudas@data-artisans.com
>>>>> > > > >:
>>>>> > > > > >
>>>>> > > > > > > A big +1 as well.
>>>>> > > > > > >
>>>>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uce@apache.org
>>>>> >
>>>>> > wrote:
>>>>> > > > > > > >
>>>>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>>>>> > > > trohrmann@apache.org
>>>>> > > > > >
>>>>> > > > > > > wrote:
>>>>> > > > > > > >> I think we have a history of creating too long
>>>>> monolithic
>>>>> > > > > > documentation
>>>>> > > > > > > >> pages which are hard to digest. So a big +1 for
>>>>> splitting the
>>>>> > > > Table
>>>>> > > > > > > API/SQL
>>>>> > > > > > > >> documentation up into more easily digestible pieces.
>>>>> > > > > > > >
>>>>> > > > > > > > +1
>>>>> > > > > > > >
>>>>> > > > > > > > Thanks for bringing it up
>>>>> > > > > > >
>>>>> > > > > > >
>>>>> > > > > >
>>>>> > > > >
>>>>> > > >
>>>>> > >
>>>>> >
>>>>>
>>>>
>>>>
>>>
>>
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Fabian Hueske <fh...@gmail.com>.
Hi everybody,
I added a couple of subissues to
https://issues.apache.org/jira/browse/FLINK-5354 and started to work on the
first two.
I'll open PRs for those soon.
Please pick an issue and assign it to yourself if you want to help with the
Table API / SQL documentation.
Thanks,
Fabian
2017-05-23 17:49 GMT+02:00 Fabian Hueske <fh...@gmail.com>:
> Hi everybody,
>
> I pushed the branch to the ASF Flink repository as a feature branch to
> keep all PRs in one place:
>
> --> https://github.com/apache/flink/tree/tableDocs
>
> Thanks,
> Fabian
>
> 2017-05-23 16:25 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
>
>> Hi everybody,
>>
>> I prepared a branch that creates the proposed structure and copied the
>> existing documentation into the corresponding pages / sections.
>> There are plenty of gaps that need to be filled or reworked.
>>
>> --> https://github.com/fhueske/flink/tree/tableDocs
>>
>> How do we go on from here?
>> I think the easiest would be if everybody who's interested in working on
>> the documentation picks a page and prepares a PR against my branch (we
>> could also push this into a feature branch in the Flink repository if
>> somebody prefers that). The PRs are cross-checked and we merge everything
>> into to the master when the docs are ready.
>>
>> Any opinions or other proposals?
>>
>> Cheers, Fabian
>>
>> 2017-05-23 10:31 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
>>
>>> Hi everybody,
>>>
>>> Thanks for the feedback. I'll go ahead and create the proposed structure
>>> and move the content of the existing docs with comments of what needs to be
>>> adapted.
>>> I'll put this into branch of my Github repo and let you know when I'm
>>> done.
>>> From there, we can distribute working on the missing parts / parts that
>>> need adaption.
>>>
>>> Cheers, Fabian
>>>
>>> 2017-05-19 9:44 GMT+01:00 jincheng sun <su...@gmail.com>:
>>>
>>>> Hi, Fabian,
>>>>
>>>> Thanks for the sketch. The structure is pretty well to me, And glad to
>>>> join in the discussion in google doc.
>>>>
>>>> Cheers,
>>>> SunJincheng
>>>>
>>>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <ws...@gmail.com>:
>>>>
>>>> > Hello Fabian,
>>>> > Thanks for drafting the proposal. I like the entire organization in
>>>> general
>>>> > and left a few comments. I think this will be a very good kick off to
>>>> > reorganize the tableAPI&SQL doc.
>>>> >
>>>> > -shaoxuan
>>>> >
>>>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <fh...@gmail.com>
>>>> wrote:
>>>> >
>>>> > > Hi everybody,
>>>> > >
>>>> > > I came up with a proposal for the structure of the Table API / SQL
>>>> > > documentation:
>>>> > >
>>>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>>>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>>>> > >
>>>> > > Feedback and comments are very welcome.
>>>> > > Once we agree on a structure, we can create skeletons and
>>>> distribute the
>>>> > > work.
>>>> > >
>>>> > > Cheers,
>>>> > > Fabian
>>>> > >
>>>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
>>>> > >
>>>> > > > +1
>>>> > > >
>>>> > > > The Table / SQL component has made significant progress in the
>>>> last few
>>>> > > > months (kudos to all contributors).
>>>> > > >
>>>> > > > It is a good time to have a documentation to reflect all the
>>>> changes in
>>>> > > the
>>>> > > > Table / SQL side.
>>>> > > >
>>>> > > >
>>>> > > >
>>>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <
>>>> rmetzger@apache.org>
>>>> > > > wrote:
>>>> > > >
>>>> > > > > Thank you Fabian for working on the proposal.
>>>> > > > >
>>>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <
>>>> fhueske@gmail.com>
>>>> > > > wrote:
>>>> > > > >
>>>> > > > > > Thanks for starting this discussion Robert.
>>>> > > > > >
>>>> > > > > > I think with the next release the Table API / SQL should be
>>>> moved
>>>> > up
>>>> > > in
>>>> > > > > the
>>>> > > > > > Application Development menu.
>>>> > > > > > I also though about restructuring the docs, but it won't be
>>>> trivial
>>>> > > to
>>>> > > > do
>>>> > > > > > this, IMO because there are many orthogonal aspects:
>>>> > > > > > - Stream/Batch
>>>> > > > > > - Table/SQL
>>>> > > > > > - Scala/Java
>>>> > > > > >
>>>> > > > > > and sometimes also common concepts.
>>>> > > > > > At the moment there are also many new features missing like
>>>> OVER
>>>> > > > windows,
>>>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator
>>>> attributes,
>>>> > > filter
>>>> > > > > > pushdown, ...
>>>> > > > > >
>>>> > > > > > I will try to sketch a new structure in a Google Doc in the
>>>> next
>>>> > days
>>>> > > > and
>>>> > > > > > share it in this thread.
>>>> > > > > >
>>>> > > > > > Cheers, Fabian
>>>> > > > > >
>>>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>>>> > > k.kloudas@data-artisans.com
>>>> > > > >:
>>>> > > > > >
>>>> > > > > > > A big +1 as well.
>>>> > > > > > >
>>>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org>
>>>> > wrote:
>>>> > > > > > > >
>>>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>>>> > > > trohrmann@apache.org
>>>> > > > > >
>>>> > > > > > > wrote:
>>>> > > > > > > >> I think we have a history of creating too long monolithic
>>>> > > > > > documentation
>>>> > > > > > > >> pages which are hard to digest. So a big +1 for
>>>> splitting the
>>>> > > > Table
>>>> > > > > > > API/SQL
>>>> > > > > > > >> documentation up into more easily digestible pieces.
>>>> > > > > > > >
>>>> > > > > > > > +1
>>>> > > > > > > >
>>>> > > > > > > > Thanks for bringing it up
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>
>>>
>>
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Fabian Hueske <fh...@gmail.com>.
Hi everybody,
I pushed the branch to the ASF Flink repository as a feature branch to keep
all PRs in one place:
--> https://github.com/apache/flink/tree/tableDocs
Thanks,
Fabian
2017-05-23 16:25 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
> Hi everybody,
>
> I prepared a branch that creates the proposed structure and copied the
> existing documentation into the corresponding pages / sections.
> There are plenty of gaps that need to be filled or reworked.
>
> --> https://github.com/fhueske/flink/tree/tableDocs
>
> How do we go on from here?
> I think the easiest would be if everybody who's interested in working on
> the documentation picks a page and prepares a PR against my branch (we
> could also push this into a feature branch in the Flink repository if
> somebody prefers that). The PRs are cross-checked and we merge everything
> into to the master when the docs are ready.
>
> Any opinions or other proposals?
>
> Cheers, Fabian
>
> 2017-05-23 10:31 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
>
>> Hi everybody,
>>
>> Thanks for the feedback. I'll go ahead and create the proposed structure
>> and move the content of the existing docs with comments of what needs to be
>> adapted.
>> I'll put this into branch of my Github repo and let you know when I'm
>> done.
>> From there, we can distribute working on the missing parts / parts that
>> need adaption.
>>
>> Cheers, Fabian
>>
>> 2017-05-19 9:44 GMT+01:00 jincheng sun <su...@gmail.com>:
>>
>>> Hi, Fabian,
>>>
>>> Thanks for the sketch. The structure is pretty well to me, And glad to
>>> join in the discussion in google doc.
>>>
>>> Cheers,
>>> SunJincheng
>>>
>>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <ws...@gmail.com>:
>>>
>>> > Hello Fabian,
>>> > Thanks for drafting the proposal. I like the entire organization in
>>> general
>>> > and left a few comments. I think this will be a very good kick off to
>>> > reorganize the tableAPI&SQL doc.
>>> >
>>> > -shaoxuan
>>> >
>>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <fh...@gmail.com>
>>> wrote:
>>> >
>>> > > Hi everybody,
>>> > >
>>> > > I came up with a proposal for the structure of the Table API / SQL
>>> > > documentation:
>>> > >
>>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>>> > >
>>> > > Feedback and comments are very welcome.
>>> > > Once we agree on a structure, we can create skeletons and distribute
>>> the
>>> > > work.
>>> > >
>>> > > Cheers,
>>> > > Fabian
>>> > >
>>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
>>> > >
>>> > > > +1
>>> > > >
>>> > > > The Table / SQL component has made significant progress in the
>>> last few
>>> > > > months (kudos to all contributors).
>>> > > >
>>> > > > It is a good time to have a documentation to reflect all the
>>> changes in
>>> > > the
>>> > > > Table / SQL side.
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <
>>> rmetzger@apache.org>
>>> > > > wrote:
>>> > > >
>>> > > > > Thank you Fabian for working on the proposal.
>>> > > > >
>>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <
>>> fhueske@gmail.com>
>>> > > > wrote:
>>> > > > >
>>> > > > > > Thanks for starting this discussion Robert.
>>> > > > > >
>>> > > > > > I think with the next release the Table API / SQL should be
>>> moved
>>> > up
>>> > > in
>>> > > > > the
>>> > > > > > Application Development menu.
>>> > > > > > I also though about restructuring the docs, but it won't be
>>> trivial
>>> > > to
>>> > > > do
>>> > > > > > this, IMO because there are many orthogonal aspects:
>>> > > > > > - Stream/Batch
>>> > > > > > - Table/SQL
>>> > > > > > - Scala/Java
>>> > > > > >
>>> > > > > > and sometimes also common concepts.
>>> > > > > > At the moment there are also many new features missing like
>>> OVER
>>> > > > windows,
>>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator
>>> attributes,
>>> > > filter
>>> > > > > > pushdown, ...
>>> > > > > >
>>> > > > > > I will try to sketch a new structure in a Google Doc in the
>>> next
>>> > days
>>> > > > and
>>> > > > > > share it in this thread.
>>> > > > > >
>>> > > > > > Cheers, Fabian
>>> > > > > >
>>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>>> > > k.kloudas@data-artisans.com
>>> > > > >:
>>> > > > > >
>>> > > > > > > A big +1 as well.
>>> > > > > > >
>>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org>
>>> > wrote:
>>> > > > > > > >
>>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>>> > > > trohrmann@apache.org
>>> > > > > >
>>> > > > > > > wrote:
>>> > > > > > > >> I think we have a history of creating too long monolithic
>>> > > > > > documentation
>>> > > > > > > >> pages which are hard to digest. So a big +1 for splitting
>>> the
>>> > > > Table
>>> > > > > > > API/SQL
>>> > > > > > > >> documentation up into more easily digestible pieces.
>>> > > > > > > >
>>> > > > > > > > +1
>>> > > > > > > >
>>> > > > > > > > Thanks for bringing it up
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Fabian Hueske <fh...@gmail.com>.
Hi everybody,
I prepared a branch that creates the proposed structure and copied the
existing documentation into the corresponding pages / sections.
There are plenty of gaps that need to be filled or reworked.
--> https://github.com/fhueske/flink/tree/tableDocs
How do we go on from here?
I think the easiest would be if everybody who's interested in working on
the documentation picks a page and prepares a PR against my branch (we
could also push this into a feature branch in the Flink repository if
somebody prefers that). The PRs are cross-checked and we merge everything
into to the master when the docs are ready.
Any opinions or other proposals?
Cheers, Fabian
2017-05-23 10:31 GMT+01:00 Fabian Hueske <fh...@gmail.com>:
> Hi everybody,
>
> Thanks for the feedback. I'll go ahead and create the proposed structure
> and move the content of the existing docs with comments of what needs to be
> adapted.
> I'll put this into branch of my Github repo and let you know when I'm done.
> From there, we can distribute working on the missing parts / parts that
> need adaption.
>
> Cheers, Fabian
>
> 2017-05-19 9:44 GMT+01:00 jincheng sun <su...@gmail.com>:
>
>> Hi, Fabian,
>>
>> Thanks for the sketch. The structure is pretty well to me, And glad to
>> join in the discussion in google doc.
>>
>> Cheers,
>> SunJincheng
>>
>> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <ws...@gmail.com>:
>>
>> > Hello Fabian,
>> > Thanks for drafting the proposal. I like the entire organization in
>> general
>> > and left a few comments. I think this will be a very good kick off to
>> > reorganize the tableAPI&SQL doc.
>> >
>> > -shaoxuan
>> >
>> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <fh...@gmail.com>
>> wrote:
>> >
>> > > Hi everybody,
>> > >
>> > > I came up with a proposal for the structure of the Table API / SQL
>> > > documentation:
>> > >
>> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
>> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>> > >
>> > > Feedback and comments are very welcome.
>> > > Once we agree on a structure, we can create skeletons and distribute
>> the
>> > > work.
>> > >
>> > > Cheers,
>> > > Fabian
>> > >
>> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
>> > >
>> > > > +1
>> > > >
>> > > > The Table / SQL component has made significant progress in the last
>> few
>> > > > months (kudos to all contributors).
>> > > >
>> > > > It is a good time to have a documentation to reflect all the
>> changes in
>> > > the
>> > > > Table / SQL side.
>> > > >
>> > > >
>> > > >
>> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <rmetzger@apache.org
>> >
>> > > > wrote:
>> > > >
>> > > > > Thank you Fabian for working on the proposal.
>> > > > >
>> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <fhueske@gmail.com
>> >
>> > > > wrote:
>> > > > >
>> > > > > > Thanks for starting this discussion Robert.
>> > > > > >
>> > > > > > I think with the next release the Table API / SQL should be
>> moved
>> > up
>> > > in
>> > > > > the
>> > > > > > Application Development menu.
>> > > > > > I also though about restructuring the docs, but it won't be
>> trivial
>> > > to
>> > > > do
>> > > > > > this, IMO because there are many orthogonal aspects:
>> > > > > > - Stream/Batch
>> > > > > > - Table/SQL
>> > > > > > - Scala/Java
>> > > > > >
>> > > > > > and sometimes also common concepts.
>> > > > > > At the moment there are also many new features missing like OVER
>> > > > windows,
>> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
>> > > filter
>> > > > > > pushdown, ...
>> > > > > >
>> > > > > > I will try to sketch a new structure in a Google Doc in the next
>> > days
>> > > > and
>> > > > > > share it in this thread.
>> > > > > >
>> > > > > > Cheers, Fabian
>> > > > > >
>> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
>> > > k.kloudas@data-artisans.com
>> > > > >:
>> > > > > >
>> > > > > > > A big +1 as well.
>> > > > > > >
>> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org>
>> > wrote:
>> > > > > > > >
>> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
>> > > > trohrmann@apache.org
>> > > > > >
>> > > > > > > wrote:
>> > > > > > > >> I think we have a history of creating too long monolithic
>> > > > > > documentation
>> > > > > > > >> pages which are hard to digest. So a big +1 for splitting
>> the
>> > > > Table
>> > > > > > > API/SQL
>> > > > > > > >> documentation up into more easily digestible pieces.
>> > > > > > > >
>> > > > > > > > +1
>> > > > > > > >
>> > > > > > > > Thanks for bringing it up
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Fabian Hueske <fh...@gmail.com>.
Hi everybody,
Thanks for the feedback. I'll go ahead and create the proposed structure
and move the content of the existing docs with comments of what needs to be
adapted.
I'll put this into branch of my Github repo and let you know when I'm done.
From there, we can distribute working on the missing parts / parts that
need adaption.
Cheers, Fabian
2017-05-19 9:44 GMT+01:00 jincheng sun <su...@gmail.com>:
> Hi, Fabian,
>
> Thanks for the sketch. The structure is pretty well to me, And glad to
> join in the discussion in google doc.
>
> Cheers,
> SunJincheng
>
> 2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <ws...@gmail.com>:
>
> > Hello Fabian,
> > Thanks for drafting the proposal. I like the entire organization in
> general
> > and left a few comments. I think this will be a very good kick off to
> > reorganize the tableAPI&SQL doc.
> >
> > -shaoxuan
> >
> > On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <fh...@gmail.com>
> wrote:
> >
> > > Hi everybody,
> > >
> > > I came up with a proposal for the structure of the Table API / SQL
> > > documentation:
> > >
> > > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
> > > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
> > >
> > > Feedback and comments are very welcome.
> > > Once we agree on a structure, we can create skeletons and distribute
> the
> > > work.
> > >
> > > Cheers,
> > > Fabian
> > >
> > > 2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
> > >
> > > > +1
> > > >
> > > > The Table / SQL component has made significant progress in the last
> few
> > > > months (kudos to all contributors).
> > > >
> > > > It is a good time to have a documentation to reflect all the changes
> in
> > > the
> > > > Table / SQL side.
> > > >
> > > >
> > > >
> > > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <rm...@apache.org>
> > > > wrote:
> > > >
> > > > > Thank you Fabian for working on the proposal.
> > > > >
> > > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <fh...@gmail.com>
> > > > wrote:
> > > > >
> > > > > > Thanks for starting this discussion Robert.
> > > > > >
> > > > > > I think with the next release the Table API / SQL should be moved
> > up
> > > in
> > > > > the
> > > > > > Application Development menu.
> > > > > > I also though about restructuring the docs, but it won't be
> trivial
> > > to
> > > > do
> > > > > > this, IMO because there are many orthogonal aspects:
> > > > > > - Stream/Batch
> > > > > > - Table/SQL
> > > > > > - Scala/Java
> > > > > >
> > > > > > and sometimes also common concepts.
> > > > > > At the moment there are also many new features missing like OVER
> > > > windows,
> > > > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
> > > filter
> > > > > > pushdown, ...
> > > > > >
> > > > > > I will try to sketch a new structure in a Google Doc in the next
> > days
> > > > and
> > > > > > share it in this thread.
> > > > > >
> > > > > > Cheers, Fabian
> > > > > >
> > > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
> > > k.kloudas@data-artisans.com
> > > > >:
> > > > > >
> > > > > > > A big +1 as well.
> > > > > > >
> > > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org>
> > wrote:
> > > > > > > >
> > > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> > > > trohrmann@apache.org
> > > > > >
> > > > > > > wrote:
> > > > > > > >> I think we have a history of creating too long monolithic
> > > > > > documentation
> > > > > > > >> pages which are hard to digest. So a big +1 for splitting
> the
> > > > Table
> > > > > > > API/SQL
> > > > > > > >> documentation up into more easily digestible pieces.
> > > > > > > >
> > > > > > > > +1
> > > > > > > >
> > > > > > > > Thanks for bringing it up
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by jincheng sun <su...@gmail.com>.
Hi, Fabian,
Thanks for the sketch. The structure is pretty well to me, And glad to
join in the discussion in google doc.
Cheers,
SunJincheng
2017-05-19 14:55 GMT+08:00 Shaoxuan Wang <ws...@gmail.com>:
> Hello Fabian,
> Thanks for drafting the proposal. I like the entire organization in general
> and left a few comments. I think this will be a very good kick off to
> reorganize the tableAPI&SQL doc.
>
> -shaoxuan
>
> On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <fh...@gmail.com> wrote:
>
> > Hi everybody,
> >
> > I came up with a proposal for the structure of the Table API / SQL
> > documentation:
> >
> > https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
> > lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
> >
> > Feedback and comments are very welcome.
> > Once we agree on a structure, we can create skeletons and distribute the
> > work.
> >
> > Cheers,
> > Fabian
> >
> > 2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
> >
> > > +1
> > >
> > > The Table / SQL component has made significant progress in the last few
> > > months (kudos to all contributors).
> > >
> > > It is a good time to have a documentation to reflect all the changes in
> > the
> > > Table / SQL side.
> > >
> > >
> > >
> > > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <rm...@apache.org>
> > > wrote:
> > >
> > > > Thank you Fabian for working on the proposal.
> > > >
> > > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <fh...@gmail.com>
> > > wrote:
> > > >
> > > > > Thanks for starting this discussion Robert.
> > > > >
> > > > > I think with the next release the Table API / SQL should be moved
> up
> > in
> > > > the
> > > > > Application Development menu.
> > > > > I also though about restructuring the docs, but it won't be trivial
> > to
> > > do
> > > > > this, IMO because there are many orthogonal aspects:
> > > > > - Stream/Batch
> > > > > - Table/SQL
> > > > > - Scala/Java
> > > > >
> > > > > and sometimes also common concepts.
> > > > > At the moment there are also many new features missing like OVER
> > > windows,
> > > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
> > filter
> > > > > pushdown, ...
> > > > >
> > > > > I will try to sketch a new structure in a Google Doc in the next
> days
> > > and
> > > > > share it in this thread.
> > > > >
> > > > > Cheers, Fabian
> > > > >
> > > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
> > k.kloudas@data-artisans.com
> > > >:
> > > > >
> > > > > > A big +1 as well.
> > > > > >
> > > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org>
> wrote:
> > > > > > >
> > > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> > > trohrmann@apache.org
> > > > >
> > > > > > wrote:
> > > > > > >> I think we have a history of creating too long monolithic
> > > > > documentation
> > > > > > >> pages which are hard to digest. So a big +1 for splitting the
> > > Table
> > > > > > API/SQL
> > > > > > >> documentation up into more easily digestible pieces.
> > > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > Thanks for bringing it up
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Shaoxuan Wang <ws...@gmail.com>.
Hello Fabian,
Thanks for drafting the proposal. I like the entire organization in general
and left a few comments. I think this will be a very good kick off to
reorganize the tableAPI&SQL doc.
-shaoxuan
On Fri, May 19, 2017 at 7:06 AM, Fabian Hueske <fh...@gmail.com> wrote:
> Hi everybody,
>
> I came up with a proposal for the structure of the Table API / SQL
> documentation:
>
> https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_
> lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
>
> Feedback and comments are very welcome.
> Once we agree on a structure, we can create skeletons and distribute the
> work.
>
> Cheers,
> Fabian
>
> 2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
>
> > +1
> >
> > The Table / SQL component has made significant progress in the last few
> > months (kudos to all contributors).
> >
> > It is a good time to have a documentation to reflect all the changes in
> the
> > Table / SQL side.
> >
> >
> >
> > On Thu, May 18, 2017 at 8:12 AM Robert Metzger <rm...@apache.org>
> > wrote:
> >
> > > Thank you Fabian for working on the proposal.
> > >
> > > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <fh...@gmail.com>
> > wrote:
> > >
> > > > Thanks for starting this discussion Robert.
> > > >
> > > > I think with the next release the Table API / SQL should be moved up
> in
> > > the
> > > > Application Development menu.
> > > > I also though about restructuring the docs, but it won't be trivial
> to
> > do
> > > > this, IMO because there are many orthogonal aspects:
> > > > - Stream/Batch
> > > > - Table/SQL
> > > > - Scala/Java
> > > >
> > > > and sometimes also common concepts.
> > > > At the moment there are also many new features missing like OVER
> > windows,
> > > > UDAGGs, retraction, StreamTableSinks, time indicator attributes,
> filter
> > > > pushdown, ...
> > > >
> > > > I will try to sketch a new structure in a Google Doc in the next days
> > and
> > > > share it in this thread.
> > > >
> > > > Cheers, Fabian
> > > >
> > > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <
> k.kloudas@data-artisans.com
> > >:
> > > >
> > > > > A big +1 as well.
> > > > >
> > > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org> wrote:
> > > > > >
> > > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> > trohrmann@apache.org
> > > >
> > > > > wrote:
> > > > > >> I think we have a history of creating too long monolithic
> > > > documentation
> > > > > >> pages which are hard to digest. So a big +1 for splitting the
> > Table
> > > > > API/SQL
> > > > > >> documentation up into more easily digestible pieces.
> > > > > >
> > > > > > +1
> > > > > >
> > > > > > Thanks for bringing it up
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Fabian Hueske <fh...@gmail.com>.
Hi everybody,
I came up with a proposal for the structure of the Table API / SQL
documentation:
https://docs.google.com/document/d/1ENY8tcPadZjoZ4AQ_lRRwWiVpScDkm_4rgxIGWGT5E0/edit?usp=sharing
Feedback and comments are very welcome.
Once we agree on a structure, we can create skeletons and distribute the
work.
Cheers,
Fabian
2017-05-18 21:01 GMT+02:00 Haohui Mai <ri...@gmail.com>:
> +1
>
> The Table / SQL component has made significant progress in the last few
> months (kudos to all contributors).
>
> It is a good time to have a documentation to reflect all the changes in the
> Table / SQL side.
>
>
>
> On Thu, May 18, 2017 at 8:12 AM Robert Metzger <rm...@apache.org>
> wrote:
>
> > Thank you Fabian for working on the proposal.
> >
> > On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <fh...@gmail.com>
> wrote:
> >
> > > Thanks for starting this discussion Robert.
> > >
> > > I think with the next release the Table API / SQL should be moved up in
> > the
> > > Application Development menu.
> > > I also though about restructuring the docs, but it won't be trivial to
> do
> > > this, IMO because there are many orthogonal aspects:
> > > - Stream/Batch
> > > - Table/SQL
> > > - Scala/Java
> > >
> > > and sometimes also common concepts.
> > > At the moment there are also many new features missing like OVER
> windows,
> > > UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
> > > pushdown, ...
> > >
> > > I will try to sketch a new structure in a Google Doc in the next days
> and
> > > share it in this thread.
> > >
> > > Cheers, Fabian
> > >
> > > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <k.kloudas@data-artisans.com
> >:
> > >
> > > > A big +1 as well.
> > > >
> > > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org> wrote:
> > > > >
> > > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <
> trohrmann@apache.org
> > >
> > > > wrote:
> > > > >> I think we have a history of creating too long monolithic
> > > documentation
> > > > >> pages which are hard to digest. So a big +1 for splitting the
> Table
> > > > API/SQL
> > > > >> documentation up into more easily digestible pieces.
> > > > >
> > > > > +1
> > > > >
> > > > > Thanks for bringing it up
> > > >
> > > >
> > >
> >
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Haohui Mai <ri...@gmail.com>.
+1
The Table / SQL component has made significant progress in the last few
months (kudos to all contributors).
It is a good time to have a documentation to reflect all the changes in the
Table / SQL side.
On Thu, May 18, 2017 at 8:12 AM Robert Metzger <rm...@apache.org> wrote:
> Thank you Fabian for working on the proposal.
>
> On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <fh...@gmail.com> wrote:
>
> > Thanks for starting this discussion Robert.
> >
> > I think with the next release the Table API / SQL should be moved up in
> the
> > Application Development menu.
> > I also though about restructuring the docs, but it won't be trivial to do
> > this, IMO because there are many orthogonal aspects:
> > - Stream/Batch
> > - Table/SQL
> > - Scala/Java
> >
> > and sometimes also common concepts.
> > At the moment there are also many new features missing like OVER windows,
> > UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
> > pushdown, ...
> >
> > I will try to sketch a new structure in a Google Doc in the next days and
> > share it in this thread.
> >
> > Cheers, Fabian
> >
> > 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <k....@data-artisans.com>:
> >
> > > A big +1 as well.
> > >
> > > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org> wrote:
> > > >
> > > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <trohrmann@apache.org
> >
> > > wrote:
> > > >> I think we have a history of creating too long monolithic
> > documentation
> > > >> pages which are hard to digest. So a big +1 for splitting the Table
> > > API/SQL
> > > >> documentation up into more easily digestible pieces.
> > > >
> > > > +1
> > > >
> > > > Thanks for bringing it up
> > >
> > >
> >
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Robert Metzger <rm...@apache.org>.
Thank you Fabian for working on the proposal.
On Thu, May 18, 2017 at 3:51 PM, Fabian Hueske <fh...@gmail.com> wrote:
> Thanks for starting this discussion Robert.
>
> I think with the next release the Table API / SQL should be moved up in the
> Application Development menu.
> I also though about restructuring the docs, but it won't be trivial to do
> this, IMO because there are many orthogonal aspects:
> - Stream/Batch
> - Table/SQL
> - Scala/Java
>
> and sometimes also common concepts.
> At the moment there are also many new features missing like OVER windows,
> UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
> pushdown, ...
>
> I will try to sketch a new structure in a Google Doc in the next days and
> share it in this thread.
>
> Cheers, Fabian
>
> 2017-05-18 14:03 GMT+02:00 Kostas Kloudas <k....@data-artisans.com>:
>
> > A big +1 as well.
> >
> > > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org> wrote:
> > >
> > > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <tr...@apache.org>
> > wrote:
> > >> I think we have a history of creating too long monolithic
> documentation
> > >> pages which are hard to digest. So a big +1 for splitting the Table
> > API/SQL
> > >> documentation up into more easily digestible pieces.
> > >
> > > +1
> > >
> > > Thanks for bringing it up
> >
> >
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Fabian Hueske <fh...@gmail.com>.
Thanks for starting this discussion Robert.
I think with the next release the Table API / SQL should be moved up in the
Application Development menu.
I also though about restructuring the docs, but it won't be trivial to do
this, IMO because there are many orthogonal aspects:
- Stream/Batch
- Table/SQL
- Scala/Java
and sometimes also common concepts.
At the moment there are also many new features missing like OVER windows,
UDAGGs, retraction, StreamTableSinks, time indicator attributes, filter
pushdown, ...
I will try to sketch a new structure in a Google Doc in the next days and
share it in this thread.
Cheers, Fabian
2017-05-18 14:03 GMT+02:00 Kostas Kloudas <k....@data-artisans.com>:
> A big +1 as well.
>
> > On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org> wrote:
> >
> > On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <tr...@apache.org>
> wrote:
> >> I think we have a history of creating too long monolithic documentation
> >> pages which are hard to digest. So a big +1 for splitting the Table
> API/SQL
> >> documentation up into more easily digestible pieces.
> >
> > +1
> >
> > Thanks for bringing it up
>
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Kostas Kloudas <k....@data-artisans.com>.
A big +1 as well.
> On May 18, 2017, at 1:55 PM, Ufuk Celebi <uc...@apache.org> wrote:
>
> On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <tr...@apache.org> wrote:
>> I think we have a history of creating too long monolithic documentation
>> pages which are hard to digest. So a big +1 for splitting the Table API/SQL
>> documentation up into more easily digestible pieces.
>
> +1
>
> Thanks for bringing it up
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Ufuk Celebi <uc...@apache.org>.
On Thu, May 18, 2017 at 1:52 PM, Till Rohrmann <tr...@apache.org> wrote:
> I think we have a history of creating too long monolithic documentation
> pages which are hard to digest. So a big +1 for splitting the Table API/SQL
> documentation up into more easily digestible pieces.
+1
Thanks for bringing it up
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Till Rohrmann <tr...@apache.org>.
I think we have a history of creating too long monolithic documentation
pages which are hard to digest. So a big +1 for splitting the Table API/SQL
documentation up into more easily digestible pieces.
Cheers,
Till
On Thu, May 18, 2017 at 12:01 PM, Shaoxuan Wang <ws...@gmail.com> wrote:
> Hi Robert,
> This sounds great to me. While I am in the middle of writing-up UDAGG doc
> (FLINK-5905), I also feel it's not good to have entire tableAPI&SQL
> introduction in one page.
> We can move tableAPI&SQL under "application development", and split it into
> small sub-topics, such as basic/UDF/UDTF/UDAGG/window aggregate, etc.
> If this sounds good to you, I can help to refactor the pages.
>
> Regards,
> Shaoxuan
>
>
> On Thu, May 18, 2017 at 5:01 PM, Robert Metzger <rm...@apache.org>
> wrote:
>
> > Hi,
> >
> > I'm wondering whether we should make the Table API a bit more prominent
> in
> > our documentation by upgrading it from below "Libraries" to the same
> level
> > as "DataSet" and "DataStream".
> >
> > This would also allow us to split it from one large page into smaller
> > sub-pages.
> >
> > I think it would be nice to do this change as part of the documentation
> > writing of the new 1.3 features.
> >
> > Let me know what you think.
> >
> >
> > Regards,
> > Robert
> >
>
Re: [DISCUSS] Reorganize Table API / SQL documentation
Posted by Shaoxuan Wang <ws...@gmail.com>.
Hi Robert,
This sounds great to me. While I am in the middle of writing-up UDAGG doc
(FLINK-5905), I also feel it's not good to have entire tableAPI&SQL
introduction in one page.
We can move tableAPI&SQL under "application development", and split it into
small sub-topics, such as basic/UDF/UDTF/UDAGG/window aggregate, etc.
If this sounds good to you, I can help to refactor the pages.
Regards,
Shaoxuan
On Thu, May 18, 2017 at 5:01 PM, Robert Metzger <rm...@apache.org> wrote:
> Hi,
>
> I'm wondering whether we should make the Table API a bit more prominent in
> our documentation by upgrading it from below "Libraries" to the same level
> as "DataSet" and "DataStream".
>
> This would also allow us to split it from one large page into smaller
> sub-pages.
>
> I think it would be nice to do this change as part of the documentation
> writing of the new 1.3 features.
>
> Let me know what you think.
>
>
> Regards,
> Robert
>