You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by ramkrishna vasudevan <ra...@gmail.com> on 2022/12/06 17:22:45 UTC
ABFS/WASB IT test cases not running
Hi devs,
I was trying to write test cases for the Streaming sink support for ABFS as
part of https://issues.apache.org/jira/browse/FLINK-30128.
I found that the AzureFileSystemBehaviorITCase was not running in the CI
build too. So if I want to add the Recoverable support for ABFS , is there
a way I can add test cases ?
Regards
Ram
Re: ABFS/WASB IT test cases not running
Posted by ramkrishna vasudevan <ra...@gmail.com>.
Makes sense. I will get in touch with the PMC once I raise the PR with the
tests.
Regards
Ram
On Wed, Dec 7, 2022 at 1:16 PM Matthias Pohl <ma...@aiven.io.invalid>
wrote:
> You can implement the tests. Access to the underlying filesystem should be
> configurable through system properties (similar to what is done in
> AzureFileSystemBehaviorITCase). This enables contributors to run the tests
> locally (if they set the corresponding resource up). You should reach out
> to the Flink PMC (CC'd Martijn) to discuss providing infrastructure for the
> corresponding tests in CI.
>
> Matthias
>
> On Wed, Dec 7, 2022 at 5:54 AM ramkrishna vasudevan <
> ramvasu.flink@gmail.com>
> wrote:
>
> > Thank you for your email. So with this PR I should be adding ABFS related
> > config and secrets and also ensure that the existing WASB secrets are
> also
> > updated?
> >
> > Regards
> > Ram
> >
> > On Wed, Dec 7, 2022 at 1:51 AM Matthias Pohl <matthias.pohl@aiven.io
> > .invalid>
> > wrote:
> >
> > > Hi Ramkrishna,
> > > There are Jira issues for the Azure e2e tests [1] and
> > > AzureFileSystemBehaviorITCase [2]. Essentially, it comes down to
> setting
> > > things up in Azure and working together with the PMC to add the secrets
> > to
> > > the Azure CI project.
> > > The secrets are only set for CI on master and the release branches as
> > > described in the Repository section of the CI docs [3].
> > >
> > > I hope that helps,
> > > Matthias
> > >
> > > [1] https://issues.apache.org/jira/browse/FLINK-12654
> > > [2] https://issues.apache.org/jira/browse/FLINK-29810
> > > [3]
> > >
> https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration
> > >
> > > On Tue, Dec 6, 2022 at 6:23 PM ramkrishna vasudevan <
> > > ramvasu.flink@gmail.com>
> > > wrote:
> > >
> > > > Hi devs,
> > > >
> > > > I was trying to write test cases for the Streaming sink support for
> > ABFS
> > > as
> > > > part of https://issues.apache.org/jira/browse/FLINK-30128.
> > > >
> > > > I found that the AzureFileSystemBehaviorITCase was not running in the
> > CI
> > > > build too. So if I want to add the Recoverable support for ABFS , is
> > > there
> > > > a way I can add test cases ?
> > > >
> > > > Regards
> > > > Ram
> > > >
> > >
> >
>
Re: ABFS/WASB IT test cases not running
Posted by Matthias Pohl <ma...@aiven.io.INVALID>.
You can implement the tests. Access to the underlying filesystem should be
configurable through system properties (similar to what is done in
AzureFileSystemBehaviorITCase). This enables contributors to run the tests
locally (if they set the corresponding resource up). You should reach out
to the Flink PMC (CC'd Martijn) to discuss providing infrastructure for the
corresponding tests in CI.
Matthias
On Wed, Dec 7, 2022 at 5:54 AM ramkrishna vasudevan <ra...@gmail.com>
wrote:
> Thank you for your email. So with this PR I should be adding ABFS related
> config and secrets and also ensure that the existing WASB secrets are also
> updated?
>
> Regards
> Ram
>
> On Wed, Dec 7, 2022 at 1:51 AM Matthias Pohl <matthias.pohl@aiven.io
> .invalid>
> wrote:
>
> > Hi Ramkrishna,
> > There are Jira issues for the Azure e2e tests [1] and
> > AzureFileSystemBehaviorITCase [2]. Essentially, it comes down to setting
> > things up in Azure and working together with the PMC to add the secrets
> to
> > the Azure CI project.
> > The secrets are only set for CI on master and the release branches as
> > described in the Repository section of the CI docs [3].
> >
> > I hope that helps,
> > Matthias
> >
> > [1] https://issues.apache.org/jira/browse/FLINK-12654
> > [2] https://issues.apache.org/jira/browse/FLINK-29810
> > [3]
> > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration
> >
> > On Tue, Dec 6, 2022 at 6:23 PM ramkrishna vasudevan <
> > ramvasu.flink@gmail.com>
> > wrote:
> >
> > > Hi devs,
> > >
> > > I was trying to write test cases for the Streaming sink support for
> ABFS
> > as
> > > part of https://issues.apache.org/jira/browse/FLINK-30128.
> > >
> > > I found that the AzureFileSystemBehaviorITCase was not running in the
> CI
> > > build too. So if I want to add the Recoverable support for ABFS , is
> > there
> > > a way I can add test cases ?
> > >
> > > Regards
> > > Ram
> > >
> >
>
Re: ABFS/WASB IT test cases not running
Posted by ramkrishna vasudevan <ra...@gmail.com>.
Thank you for your email. So with this PR I should be adding ABFS related
config and secrets and also ensure that the existing WASB secrets are also
updated?
Regards
Ram
On Wed, Dec 7, 2022 at 1:51 AM Matthias Pohl <ma...@aiven.io.invalid>
wrote:
> Hi Ramkrishna,
> There are Jira issues for the Azure e2e tests [1] and
> AzureFileSystemBehaviorITCase [2]. Essentially, it comes down to setting
> things up in Azure and working together with the PMC to add the secrets to
> the Azure CI project.
> The secrets are only set for CI on master and the release branches as
> described in the Repository section of the CI docs [3].
>
> I hope that helps,
> Matthias
>
> [1] https://issues.apache.org/jira/browse/FLINK-12654
> [2] https://issues.apache.org/jira/browse/FLINK-29810
> [3]
> https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration
>
> On Tue, Dec 6, 2022 at 6:23 PM ramkrishna vasudevan <
> ramvasu.flink@gmail.com>
> wrote:
>
> > Hi devs,
> >
> > I was trying to write test cases for the Streaming sink support for ABFS
> as
> > part of https://issues.apache.org/jira/browse/FLINK-30128.
> >
> > I found that the AzureFileSystemBehaviorITCase was not running in the CI
> > build too. So if I want to add the Recoverable support for ABFS , is
> there
> > a way I can add test cases ?
> >
> > Regards
> > Ram
> >
>
Re: ABFS/WASB IT test cases not running
Posted by Matthias Pohl <ma...@aiven.io.INVALID>.
Hi Ramkrishna,
There are Jira issues for the Azure e2e tests [1] and
AzureFileSystemBehaviorITCase [2]. Essentially, it comes down to setting
things up in Azure and working together with the PMC to add the secrets to
the Azure CI project.
The secrets are only set for CI on master and the release branches as
described in the Repository section of the CI docs [3].
I hope that helps,
Matthias
[1] https://issues.apache.org/jira/browse/FLINK-12654
[2] https://issues.apache.org/jira/browse/FLINK-29810
[3] https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration
On Tue, Dec 6, 2022 at 6:23 PM ramkrishna vasudevan <ra...@gmail.com>
wrote:
> Hi devs,
>
> I was trying to write test cases for the Streaming sink support for ABFS as
> part of https://issues.apache.org/jira/browse/FLINK-30128.
>
> I found that the AzureFileSystemBehaviorITCase was not running in the CI
> build too. So if I want to add the Recoverable support for ABFS , is there
> a way I can add test cases ?
>
> Regards
> Ram
>