You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Danny Cranmer <da...@apache.org> on 2021/08/20 20:59:09 UTC
Scala 2.12 End to End Tests
Hello,
I am working on reviewing a PR [1] to add JSON support for the AWS Glue
Schema Registry integration. The module only builds for Scala 2.12 due to a
dependency incompatibility. The end-to-end tests have been implemented
using the newer Java framework rather than bash scripts. However, it
appears as though end-to-end tests are only run for Scala 2.11, therefore
the new tests are not actually being exercised. Questions:
- Please correct me if I am wrong here, E2E java tests are only run for
Scala 2.11 [2]?
- Is there a reason we are not running tests for Scala 2.11 AND Scala 2.12
- How do we proceed? I would be inclined with number 1, unless there is a
good reason not to, besides increase in time (they already take a long time)
1. Enable ALL Scala 2.12 tests?
2. Just run the new tests with Scala 2.12?
3. Do not run the new tests
[1] https://github.com/apache/flink/pull/16513
[2]
https://github.com/apache/flink/blob/master/flink-end-to-end-tests/run-nightly-tests.sh#L264
Thanks,
Danny Cranmer.
Re: Scala 2.12 End to End Tests
Posted by Danny Cranmer <da...@apache.org>.
I have raised the Jira [1]. If no one picks this up I will try to find some
time next week to make a start.
Thanks.
[1] https://issues.apache.org/jira/browse/FLINK-23996
On Tue, Aug 24, 2021 at 11:37 AM Till Rohrmann <tr...@apache.org> wrote:
> Great, thanks Danny.
>
> On Mon, Aug 23, 2021 at 9:41 PM Danny Cranmer <da...@gmail.com>
> wrote:
>
> > Thanks Till!
> >
> > +1 making Scala 2.12 the default. I will raise a Jira tomorrow, not at
> the
> > laptop today.
> >
> > On Mon, 23 Aug 2021, 09:43 Martijn Visser, <ma...@ververica.com>
> wrote:
> >
> > > +1 to switch to 2.12 by default
> > >
> > > On Sat, 21 Aug 2021 at 13:43, Chesnay Schepler <ch...@apache.org>
> > wrote:
> > >
> > > > I'm with Till that we should switch to 2.12 by default .
> > > >
> > > > On 21/08/2021 11:12, Till Rohrmann wrote:
> > > > > Hi Danny,
> > > > >
> > > > > I think in the nightly builds we do run the e2e with Scala 2.12
> [1].
> > > The
> > > > > way it is configured is via the PROFILE env variable if I am not
> > > > mistaken.
> > > > >
> > > > > Independent of this we might wanna start a discussion whether we
> > don't
> > > > want
> > > > > to switch to Scala 2.12. per default.
> > > > >
> > > > > [1]
> > > > >
> > > >
> > >
> >
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22579&view=logs&j=08866332-78f7-59e4-4f7e-49a56faa3179
> > > > >
> > > > > Cheers,
> > > > > Till
> > > > >
> > > > > On Fri, Aug 20, 2021 at 10:59 PM Danny Cranmer <
> > > dannycranmer@apache.org>
> > > > > wrote:
> > > > >
> > > > >> Hello,
> > > > >>
> > > > >> I am working on reviewing a PR [1] to add JSON support for the AWS
> > > Glue
> > > > >> Schema Registry integration. The module only builds for Scala 2.12
> > due
> > > > to a
> > > > >> dependency incompatibility. The end-to-end tests have been
> > implemented
> > > > >> using the newer Java framework rather than bash scripts. However,
> it
> > > > >> appears as though end-to-end tests are only run for Scala 2.11,
> > > > therefore
> > > > >> the new tests are not actually being exercised. Questions:
> > > > >> - Please correct me if I am wrong here, E2E java tests are only
> run
> > > for
> > > > >> Scala 2.11 [2]?
> > > > >> - Is there a reason we are not running tests for Scala 2.11 AND
> > Scala
> > > > 2.12
> > > > >> - How do we proceed? I would be inclined with number 1, unless
> there
> > > is
> > > > a
> > > > >> good reason not to, besides increase in time (they already take a
> > long
> > > > >> time)
> > > > >> 1. Enable ALL Scala 2.12 tests?
> > > > >> 2. Just run the new tests with Scala 2.12?
> > > > >> 3. Do not run the new tests
> > > > >>
> > > > >> [1] https://github.com/apache/flink/pull/16513
> > > > >> [2]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/run-nightly-tests.sh#L264
> > > > >>
> > > > >> Thanks,
> > > > >> Danny Cranmer.
> > > > >>
> > > >
> > > >
> > >
> >
>
Re: Scala 2.12 End to End Tests
Posted by Till Rohrmann <tr...@apache.org>.
Great, thanks Danny.
On Mon, Aug 23, 2021 at 9:41 PM Danny Cranmer <da...@gmail.com>
wrote:
> Thanks Till!
>
> +1 making Scala 2.12 the default. I will raise a Jira tomorrow, not at the
> laptop today.
>
> On Mon, 23 Aug 2021, 09:43 Martijn Visser, <ma...@ververica.com> wrote:
>
> > +1 to switch to 2.12 by default
> >
> > On Sat, 21 Aug 2021 at 13:43, Chesnay Schepler <ch...@apache.org>
> wrote:
> >
> > > I'm with Till that we should switch to 2.12 by default .
> > >
> > > On 21/08/2021 11:12, Till Rohrmann wrote:
> > > > Hi Danny,
> > > >
> > > > I think in the nightly builds we do run the e2e with Scala 2.12 [1].
> > The
> > > > way it is configured is via the PROFILE env variable if I am not
> > > mistaken.
> > > >
> > > > Independent of this we might wanna start a discussion whether we
> don't
> > > want
> > > > to switch to Scala 2.12. per default.
> > > >
> > > > [1]
> > > >
> > >
> >
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22579&view=logs&j=08866332-78f7-59e4-4f7e-49a56faa3179
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Fri, Aug 20, 2021 at 10:59 PM Danny Cranmer <
> > dannycranmer@apache.org>
> > > > wrote:
> > > >
> > > >> Hello,
> > > >>
> > > >> I am working on reviewing a PR [1] to add JSON support for the AWS
> > Glue
> > > >> Schema Registry integration. The module only builds for Scala 2.12
> due
> > > to a
> > > >> dependency incompatibility. The end-to-end tests have been
> implemented
> > > >> using the newer Java framework rather than bash scripts. However, it
> > > >> appears as though end-to-end tests are only run for Scala 2.11,
> > > therefore
> > > >> the new tests are not actually being exercised. Questions:
> > > >> - Please correct me if I am wrong here, E2E java tests are only run
> > for
> > > >> Scala 2.11 [2]?
> > > >> - Is there a reason we are not running tests for Scala 2.11 AND
> Scala
> > > 2.12
> > > >> - How do we proceed? I would be inclined with number 1, unless there
> > is
> > > a
> > > >> good reason not to, besides increase in time (they already take a
> long
> > > >> time)
> > > >> 1. Enable ALL Scala 2.12 tests?
> > > >> 2. Just run the new tests with Scala 2.12?
> > > >> 3. Do not run the new tests
> > > >>
> > > >> [1] https://github.com/apache/flink/pull/16513
> > > >> [2]
> > > >>
> > > >>
> > >
> >
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/run-nightly-tests.sh#L264
> > > >>
> > > >> Thanks,
> > > >> Danny Cranmer.
> > > >>
> > >
> > >
> >
>
Re: Scala 2.12 End to End Tests
Posted by Danny Cranmer <da...@gmail.com>.
Thanks Till!
+1 making Scala 2.12 the default. I will raise a Jira tomorrow, not at the
laptop today.
On Mon, 23 Aug 2021, 09:43 Martijn Visser, <ma...@ververica.com> wrote:
> +1 to switch to 2.12 by default
>
> On Sat, 21 Aug 2021 at 13:43, Chesnay Schepler <ch...@apache.org> wrote:
>
> > I'm with Till that we should switch to 2.12 by default .
> >
> > On 21/08/2021 11:12, Till Rohrmann wrote:
> > > Hi Danny,
> > >
> > > I think in the nightly builds we do run the e2e with Scala 2.12 [1].
> The
> > > way it is configured is via the PROFILE env variable if I am not
> > mistaken.
> > >
> > > Independent of this we might wanna start a discussion whether we don't
> > want
> > > to switch to Scala 2.12. per default.
> > >
> > > [1]
> > >
> >
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22579&view=logs&j=08866332-78f7-59e4-4f7e-49a56faa3179
> > >
> > > Cheers,
> > > Till
> > >
> > > On Fri, Aug 20, 2021 at 10:59 PM Danny Cranmer <
> dannycranmer@apache.org>
> > > wrote:
> > >
> > >> Hello,
> > >>
> > >> I am working on reviewing a PR [1] to add JSON support for the AWS
> Glue
> > >> Schema Registry integration. The module only builds for Scala 2.12 due
> > to a
> > >> dependency incompatibility. The end-to-end tests have been implemented
> > >> using the newer Java framework rather than bash scripts. However, it
> > >> appears as though end-to-end tests are only run for Scala 2.11,
> > therefore
> > >> the new tests are not actually being exercised. Questions:
> > >> - Please correct me if I am wrong here, E2E java tests are only run
> for
> > >> Scala 2.11 [2]?
> > >> - Is there a reason we are not running tests for Scala 2.11 AND Scala
> > 2.12
> > >> - How do we proceed? I would be inclined with number 1, unless there
> is
> > a
> > >> good reason not to, besides increase in time (they already take a long
> > >> time)
> > >> 1. Enable ALL Scala 2.12 tests?
> > >> 2. Just run the new tests with Scala 2.12?
> > >> 3. Do not run the new tests
> > >>
> > >> [1] https://github.com/apache/flink/pull/16513
> > >> [2]
> > >>
> > >>
> >
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/run-nightly-tests.sh#L264
> > >>
> > >> Thanks,
> > >> Danny Cranmer.
> > >>
> >
> >
>
Re: Scala 2.12 End to End Tests
Posted by Martijn Visser <ma...@ververica.com>.
+1 to switch to 2.12 by default
On Sat, 21 Aug 2021 at 13:43, Chesnay Schepler <ch...@apache.org> wrote:
> I'm with Till that we should switch to 2.12 by default .
>
> On 21/08/2021 11:12, Till Rohrmann wrote:
> > Hi Danny,
> >
> > I think in the nightly builds we do run the e2e with Scala 2.12 [1]. The
> > way it is configured is via the PROFILE env variable if I am not
> mistaken.
> >
> > Independent of this we might wanna start a discussion whether we don't
> want
> > to switch to Scala 2.12. per default.
> >
> > [1]
> >
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22579&view=logs&j=08866332-78f7-59e4-4f7e-49a56faa3179
> >
> > Cheers,
> > Till
> >
> > On Fri, Aug 20, 2021 at 10:59 PM Danny Cranmer <da...@apache.org>
> > wrote:
> >
> >> Hello,
> >>
> >> I am working on reviewing a PR [1] to add JSON support for the AWS Glue
> >> Schema Registry integration. The module only builds for Scala 2.12 due
> to a
> >> dependency incompatibility. The end-to-end tests have been implemented
> >> using the newer Java framework rather than bash scripts. However, it
> >> appears as though end-to-end tests are only run for Scala 2.11,
> therefore
> >> the new tests are not actually being exercised. Questions:
> >> - Please correct me if I am wrong here, E2E java tests are only run for
> >> Scala 2.11 [2]?
> >> - Is there a reason we are not running tests for Scala 2.11 AND Scala
> 2.12
> >> - How do we proceed? I would be inclined with number 1, unless there is
> a
> >> good reason not to, besides increase in time (they already take a long
> >> time)
> >> 1. Enable ALL Scala 2.12 tests?
> >> 2. Just run the new tests with Scala 2.12?
> >> 3. Do not run the new tests
> >>
> >> [1] https://github.com/apache/flink/pull/16513
> >> [2]
> >>
> >>
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/run-nightly-tests.sh#L264
> >>
> >> Thanks,
> >> Danny Cranmer.
> >>
>
>
Re: Scala 2.12 End to End Tests
Posted by Chesnay Schepler <ch...@apache.org>.
I'm with Till that we should switch to 2.12 by default .
On 21/08/2021 11:12, Till Rohrmann wrote:
> Hi Danny,
>
> I think in the nightly builds we do run the e2e with Scala 2.12 [1]. The
> way it is configured is via the PROFILE env variable if I am not mistaken.
>
> Independent of this we might wanna start a discussion whether we don't want
> to switch to Scala 2.12. per default.
>
> [1]
> https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22579&view=logs&j=08866332-78f7-59e4-4f7e-49a56faa3179
>
> Cheers,
> Till
>
> On Fri, Aug 20, 2021 at 10:59 PM Danny Cranmer <da...@apache.org>
> wrote:
>
>> Hello,
>>
>> I am working on reviewing a PR [1] to add JSON support for the AWS Glue
>> Schema Registry integration. The module only builds for Scala 2.12 due to a
>> dependency incompatibility. The end-to-end tests have been implemented
>> using the newer Java framework rather than bash scripts. However, it
>> appears as though end-to-end tests are only run for Scala 2.11, therefore
>> the new tests are not actually being exercised. Questions:
>> - Please correct me if I am wrong here, E2E java tests are only run for
>> Scala 2.11 [2]?
>> - Is there a reason we are not running tests for Scala 2.11 AND Scala 2.12
>> - How do we proceed? I would be inclined with number 1, unless there is a
>> good reason not to, besides increase in time (they already take a long
>> time)
>> 1. Enable ALL Scala 2.12 tests?
>> 2. Just run the new tests with Scala 2.12?
>> 3. Do not run the new tests
>>
>> [1] https://github.com/apache/flink/pull/16513
>> [2]
>>
>> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/run-nightly-tests.sh#L264
>>
>> Thanks,
>> Danny Cranmer.
>>
Re: Scala 2.12 End to End Tests
Posted by Till Rohrmann <tr...@apache.org>.
Hi Danny,
I think in the nightly builds we do run the e2e with Scala 2.12 [1]. The
way it is configured is via the PROFILE env variable if I am not mistaken.
Independent of this we might wanna start a discussion whether we don't want
to switch to Scala 2.12. per default.
[1]
https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=22579&view=logs&j=08866332-78f7-59e4-4f7e-49a56faa3179
Cheers,
Till
On Fri, Aug 20, 2021 at 10:59 PM Danny Cranmer <da...@apache.org>
wrote:
> Hello,
>
> I am working on reviewing a PR [1] to add JSON support for the AWS Glue
> Schema Registry integration. The module only builds for Scala 2.12 due to a
> dependency incompatibility. The end-to-end tests have been implemented
> using the newer Java framework rather than bash scripts. However, it
> appears as though end-to-end tests are only run for Scala 2.11, therefore
> the new tests are not actually being exercised. Questions:
> - Please correct me if I am wrong here, E2E java tests are only run for
> Scala 2.11 [2]?
> - Is there a reason we are not running tests for Scala 2.11 AND Scala 2.12
> - How do we proceed? I would be inclined with number 1, unless there is a
> good reason not to, besides increase in time (they already take a long
> time)
> 1. Enable ALL Scala 2.12 tests?
> 2. Just run the new tests with Scala 2.12?
> 3. Do not run the new tests
>
> [1] https://github.com/apache/flink/pull/16513
> [2]
>
> https://github.com/apache/flink/blob/master/flink-end-to-end-tests/run-nightly-tests.sh#L264
>
> Thanks,
> Danny Cranmer.
>