You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Greg Hogan <co...@greghogan.com> on 2017/04/30 11:16:17 UTC
Supported Hadoop versions
Hi Flink,
I filed a ticket [0] that our download page [1] still states
“You don’t have to install Hadoop to use Flink, but if you plan to use Flink with data stored in Hadoop, pick the version matching your installed Hadoop version. If you don’t want to do this, pick the Hadoop 1 version.”
1) We no longer offer the Hadoop 1 version, which I expect was chosen due to its smaller size. What is the new recommendation?
2) Should we reference, for example, “Hadoop 2.7” (as with the binary filenames) rather than “Hadoop 2.7.0” since we are actually testing and releasing against Hadoop 2.7.2?
3) Should Flink 1.3.0 support the recently released Hadoop 2.8.0? Is this the time to drop older versions (which users can easily build)? This would also be the time to bump the patch versions in .travis.yml and create-release-files.sh.
I ask because I think it is important to present this choice well since it is likely to be a new users first decision point.
[0] https://issues.apache.org/jira/browse/FLINK-6399 <https://issues.apache.org/jira/browse/FLINK-6399>
[1] https://flink.apache.org/downloads.html <https://flink.apache.org/downloads.html>
Greg
Re: Supported Hadoop versions
Posted by Greg Hogan <co...@greghogan.com>.
1) Flink 1.2 dropped Hadoop 1 but we don’t have a replacement “default”.
2) I’ll create a ticket to bump to the latest patch versions.
3) Uncertain if we can drop Hadoop 2.3 and/or 2.4.
> On Apr 30, 2017, at 10:46 AM, Ted Yu <yu...@gmail.com> wrote:
>
> For #1, +1 on dropping hadoop 1.
>
> For #2, we can reference Hadoop 2.7
> BTW I think we can bump to 2.7.3 as dependency.
>
> For #3, Hadoop 2.8.0 was marked not production ready. Junping is in the
> process of releasing 2.8.1
> It would be good idea to start testing against 2.8.0 now
>
> Cheers
>
> On Sun, Apr 30, 2017 at 4:16 AM, Greg Hogan <code@greghogan.com <ma...@greghogan.com>> wrote:
>
>> Hi Flink,
>>
>> I filed a ticket [0] that our download page [1] still states
>>
>> “You don’t have to install Hadoop to use Flink, but if you plan to use
>> Flink with data stored in Hadoop, pick the version matching your installed
>> Hadoop version. If you don’t want to do this, pick the Hadoop 1 version.”
>>
>> 1) We no longer offer the Hadoop 1 version, which I expect was chosen due
>> to its smaller size. What is the new recommendation?
>>
>> 2) Should we reference, for example, “Hadoop 2.7” (as with the binary
>> filenames) rather than “Hadoop 2.7.0” since we are actually testing and
>> releasing against Hadoop 2.7.2?
>>
>> 3) Should Flink 1.3.0 support the recently released Hadoop 2.8.0? Is this
>> the time to drop older versions (which users can easily build)? This would
>> also be the time to bump the patch versions in .travis.yml and
>> create-release-files.sh.
>>
>> I ask because I think it is important to present this choice well since it
>> is likely to be a new users first decision point.
>>
>> [0] https://issues.apache.org/jira/browse/FLINK-6399 <
>> https://issues.apache.org/jira/browse/FLINK-6399 <https://issues.apache.org/jira/browse/FLINK-6399>>
>> [1] https://flink.apache.org/downloads.html <https://flink.apache.org/downloads.html> <https://flink.apache.org/ <https://flink.apache.org/>
>> downloads.html>
>>
>> Greg
Re: Supported Hadoop versions
Posted by Ted Yu <yu...@gmail.com>.
For #1, +1 on dropping hadoop 1.
For #2, we can reference Hadoop 2.7
BTW I think we can bump to 2.7.3 as dependency.
For #3, Hadoop 2.8.0 was marked not production ready. Junping is in the
process of releasing 2.8.1
It would be good idea to start testing against 2.8.0 now
Cheers
On Sun, Apr 30, 2017 at 4:16 AM, Greg Hogan <co...@greghogan.com> wrote:
> Hi Flink,
>
> I filed a ticket [0] that our download page [1] still states
>
> “You don’t have to install Hadoop to use Flink, but if you plan to use
> Flink with data stored in Hadoop, pick the version matching your installed
> Hadoop version. If you don’t want to do this, pick the Hadoop 1 version.”
>
> 1) We no longer offer the Hadoop 1 version, which I expect was chosen due
> to its smaller size. What is the new recommendation?
>
> 2) Should we reference, for example, “Hadoop 2.7” (as with the binary
> filenames) rather than “Hadoop 2.7.0” since we are actually testing and
> releasing against Hadoop 2.7.2?
>
> 3) Should Flink 1.3.0 support the recently released Hadoop 2.8.0? Is this
> the time to drop older versions (which users can easily build)? This would
> also be the time to bump the patch versions in .travis.yml and
> create-release-files.sh.
>
> I ask because I think it is important to present this choice well since it
> is likely to be a new users first decision point.
>
> [0] https://issues.apache.org/jira/browse/FLINK-6399 <
> https://issues.apache.org/jira/browse/FLINK-6399>
> [1] https://flink.apache.org/downloads.html <https://flink.apache.org/
> downloads.html>
>
> Greg