You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by amit pal <am...@gmail.com> on 2015/02/25 11:09:02 UTC
Understanding and Contributing to Flink
Hi Guys,
I have been reading the Flink documentation and the mail list discussions
since last two days, but haven't completely got hold of what all is there
in Flink.
My next step is to start with the examples present on github.
Can someone give me a gist of what Flink is all about and some details of
the internals?
--
Thanks and Regards,
Amit Pal
Re: Understanding and Contributing to Flink
Posted by Max Michels <mx...@apache.org>.
Hi Amit,
Extensions of Flink should go in the flink-contrib project. Try
implementing a simple algorithm for your first task. If you extend it
and it becomes useful for other people besides you, make a pull
request via GitHub, so other people can check it out.
Your general understanding if the Flink architecture is correct. Of
course there are many more details. The compiler generates the plan
and optimizes it using estimates of the data input/data generation
processes within a Flink job. The schedule within the job manager then
distributes the individual tasks that are extracted from the plan.
Best regards,
Max
On Wed, Feb 25, 2015 at 12:37 PM, amit pal <am...@gmail.com> wrote:
> Hey all,
>
> I will start with some starter JIRA tasks, along with building some
> application of Flink, particularly in the domain of NLP/text processing.
> Any good NLT idea to implement over Flink? I am thinking on picking this
> JIRA <https://issues.apache.org/jira/browse/FLINK-1450> one to start on the
> development.
>
> Any pointers where to start for understanding the code-base. I am imported
> the maven project in eclipse and I see a lot of modules.
>
> Moreover, another point i would like a clarification on is that we write an
> Execution pipeline using Flink-Core, Java/Scala APIs, then pass that Jar to
> a Flink-Runtime (similar to what we would do in a hadoop cluster). The
> Runtime now generates an execution plan for the data pipeline and overloads
> the task to the worker nodes. Is my understanding of Flink usage correct?
> Where does the Flink-compiler come into the picture.
>
> Appreciate all the help in advance.
>
>
> On Wed, Feb 25, 2015 at 4:05 PM, Max Michels <mx...@apache.org> wrote:
>
>> Hi Amit,
>>
>> Nice to hear you're interested in Flink. The first thing you could do
>> is run the provided examples. Next, you could try to implement a
>> simple Flink job yourself. Besides the "How to contribute" guide, take
>> a look at the material page [1], the Flink blog [2], and the Data
>> Artisans blog [3].
>>
>> Kind regards,
>> Max
>>
>> [1] http://flink.apache.org/material.html
>> [2] http://flink.apache.org/blog/index.html
>> [3] http://data-artisans.com/blog.html
>>
>> On Wed, Feb 25, 2015 at 11:17 AM, Robert Metzger <rm...@apache.org>
>> wrote:
>> > Hey,
>> >
>> > since you've already read the documentation, I can recommend checking out
>> > some slides about Flink on Slideshare as well.
>> >
>> > Here is our "How to Contribute" guide:
>> > http://flink.apache.org/how-to-contribute.html
>> >
>> > Best,
>> > Robert
>> >
>> > On Wed, Feb 25, 2015 at 11:09 AM, amit pal <am...@gmail.com> wrote:
>> >
>> >> Hi Guys,
>> >>
>> >> I have been reading the Flink documentation and the mail list
>> discussions
>> >> since last two days, but haven't completely got hold of what all is
>> there
>> >> in Flink.
>> >>
>> >> My next step is to start with the examples present on github.
>> >>
>> >> Can someone give me a gist of what Flink is all about and some details
>> of
>> >> the internals?
>> >>
>> >> --
>> >> Thanks and Regards,
>> >> Amit Pal
>> >>
>>
>
>
>
> --
> Thanks and Regards,
> Amit Pal
Re: Understanding and Contributing to Flink
Posted by amit pal <am...@gmail.com>.
Hey all,
I will start with some starter JIRA tasks, along with building some
application of Flink, particularly in the domain of NLP/text processing.
Any good NLT idea to implement over Flink? I am thinking on picking this
JIRA <https://issues.apache.org/jira/browse/FLINK-1450> one to start on the
development.
Any pointers where to start for understanding the code-base. I am imported
the maven project in eclipse and I see a lot of modules.
Moreover, another point i would like a clarification on is that we write an
Execution pipeline using Flink-Core, Java/Scala APIs, then pass that Jar to
a Flink-Runtime (similar to what we would do in a hadoop cluster). The
Runtime now generates an execution plan for the data pipeline and overloads
the task to the worker nodes. Is my understanding of Flink usage correct?
Where does the Flink-compiler come into the picture.
Appreciate all the help in advance.
On Wed, Feb 25, 2015 at 4:05 PM, Max Michels <mx...@apache.org> wrote:
> Hi Amit,
>
> Nice to hear you're interested in Flink. The first thing you could do
> is run the provided examples. Next, you could try to implement a
> simple Flink job yourself. Besides the "How to contribute" guide, take
> a look at the material page [1], the Flink blog [2], and the Data
> Artisans blog [3].
>
> Kind regards,
> Max
>
> [1] http://flink.apache.org/material.html
> [2] http://flink.apache.org/blog/index.html
> [3] http://data-artisans.com/blog.html
>
> On Wed, Feb 25, 2015 at 11:17 AM, Robert Metzger <rm...@apache.org>
> wrote:
> > Hey,
> >
> > since you've already read the documentation, I can recommend checking out
> > some slides about Flink on Slideshare as well.
> >
> > Here is our "How to Contribute" guide:
> > http://flink.apache.org/how-to-contribute.html
> >
> > Best,
> > Robert
> >
> > On Wed, Feb 25, 2015 at 11:09 AM, amit pal <am...@gmail.com> wrote:
> >
> >> Hi Guys,
> >>
> >> I have been reading the Flink documentation and the mail list
> discussions
> >> since last two days, but haven't completely got hold of what all is
> there
> >> in Flink.
> >>
> >> My next step is to start with the examples present on github.
> >>
> >> Can someone give me a gist of what Flink is all about and some details
> of
> >> the internals?
> >>
> >> --
> >> Thanks and Regards,
> >> Amit Pal
> >>
>
--
Thanks and Regards,
Amit Pal
Re: Understanding and Contributing to Flink
Posted by Max Michels <mx...@apache.org>.
Hi Amit,
Nice to hear you're interested in Flink. The first thing you could do
is run the provided examples. Next, you could try to implement a
simple Flink job yourself. Besides the "How to contribute" guide, take
a look at the material page [1], the Flink blog [2], and the Data
Artisans blog [3].
Kind regards,
Max
[1] http://flink.apache.org/material.html
[2] http://flink.apache.org/blog/index.html
[3] http://data-artisans.com/blog.html
On Wed, Feb 25, 2015 at 11:17 AM, Robert Metzger <rm...@apache.org> wrote:
> Hey,
>
> since you've already read the documentation, I can recommend checking out
> some slides about Flink on Slideshare as well.
>
> Here is our "How to Contribute" guide:
> http://flink.apache.org/how-to-contribute.html
>
> Best,
> Robert
>
> On Wed, Feb 25, 2015 at 11:09 AM, amit pal <am...@gmail.com> wrote:
>
>> Hi Guys,
>>
>> I have been reading the Flink documentation and the mail list discussions
>> since last two days, but haven't completely got hold of what all is there
>> in Flink.
>>
>> My next step is to start with the examples present on github.
>>
>> Can someone give me a gist of what Flink is all about and some details of
>> the internals?
>>
>> --
>> Thanks and Regards,
>> Amit Pal
>>
Re: Understanding and Contributing to Flink
Posted by Robert Metzger <rm...@apache.org>.
Hey,
since you've already read the documentation, I can recommend checking out
some slides about Flink on Slideshare as well.
Here is our "How to Contribute" guide:
http://flink.apache.org/how-to-contribute.html
Best,
Robert
On Wed, Feb 25, 2015 at 11:09 AM, amit pal <am...@gmail.com> wrote:
> Hi Guys,
>
> I have been reading the Flink documentation and the mail list discussions
> since last two days, but haven't completely got hold of what all is there
> in Flink.
>
> My next step is to start with the examples present on github.
>
> Can someone give me a gist of what Flink is all about and some details of
> the internals?
>
> --
> Thanks and Regards,
> Amit Pal
>