You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@arrow.apache.org by Jean-Baptiste Onofré <jb...@nanthrax.net> on 2023/12/11 13:49:13 UTC
[DISCUSS] Fix Arrow Flight Core tests
Hi guys,
I noticed that Arrow Flight Core doesn't build "out of the box" due to
the TestTls failing.
The reason what TestTls is failing is because it tries to read
cert0.pem from the testing/data folder (at project root), but testing
is empty by default.
If I create a cert0.pem by hand (with self signing key), it works.
So, I propose three options:
1. We document the required tests to build Arrow java cleanly
2. We create a default self signed cert0.pem in testing/data that we commit.
3. We add a before step to the test to create the pem file
My preference would be for 3.
Thoughts ?
Regards
JB
Re: [DISCUSS] Fix Arrow Flight Core tests
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Thanks David,
Good point. Let me create a PR to point to this when the pem is not found.
Regards
JB
On Mon, Dec 11, 2023 at 3:00 PM David Li <li...@apache.org> wrote:
>
> You can `git submodule update --init` to get the files. This is documented in the environment setup [1], though the failing assertion could be more helpful about pointing this out.
>
> [1]: https://arrow.apache.org/docs/dev/developers/java/building.html#building
>
> On Mon, Dec 11, 2023, at 08:49, Jean-Baptiste Onofré wrote:
> > Hi guys,
> >
> > I noticed that Arrow Flight Core doesn't build "out of the box" due to
> > the TestTls failing.
> > The reason what TestTls is failing is because it tries to read
> > cert0.pem from the testing/data folder (at project root), but testing
> > is empty by default.
> > If I create a cert0.pem by hand (with self signing key), it works.
> >
> > So, I propose three options:
> > 1. We document the required tests to build Arrow java cleanly
> > 2. We create a default self signed cert0.pem in testing/data that we commit.
> > 3. We add a before step to the test to create the pem file
> >
> > My preference would be for 3.
> >
> > Thoughts ?
> >
> > Regards
> > JB
Re: [DISCUSS] Fix Arrow Flight Core tests
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Kou
OK, let me do a PR to include submodules in source distribution. I
will also add a README.md to explain the building/release part.
Thanks,
Regards
JB
On Thu, Dec 14, 2023 at 1:30 AM Sutou Kouhei <ko...@clear-code.com> wrote:
>
> Hi,
>
> I think (2) is better because
> https://arrow.apache.org/docs/developers/java/building.html
> is for building checked out repository instead of a release
> source archive.
>
> Thanks,
> --
> kou
>
> In <CA...@mail.gmail.com>
> "Re: [DISCUSS] Fix Arrow Flight Core tests" on Wed, 13 Dec 2023 20:42:29 +0100,
> Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>
> > That's my point:
> > 1. we should document that the build has to be launched with
> > -DskipTests by default
> > 2. or include submodule in the source distribution we distributed
> >
> > I think (1) is an acceptable and easy path. If there are no
> > objections, I will create a PR about that (also updating
> > https://arrow.apache.org/docs/developers/java/building.html).
> >
> > Thoughts ?
> >
> > Thanks !
> > Regards
> > JB
> >
> > On Wed, Dec 13, 2023 at 8:19 PM David Li <li...@apache.org> wrote:
> >>
> >> Could we include submodules in the source distribution instead?
> >>
> >> Also, you can still build from the source distribution, you just have to -DskipTests.
> >>
> >> On Wed, Dec 13, 2023, at 09:54, Jean-Baptiste Onofré wrote:
> >> > Hi David and foxes,
> >> >
> >> > I checked the release source distribution and I think we should maybe
> >> > include a profile to include TestTls related tests.
> >> >
> >> > As an Apache project, we are supposed to be able to build from the
> >> > source distribution without any external requirements (in the case the
> >> > project goes to the attic or someone wants to create a new release on
> >> > an old branch).
> >> >
> >> > Downloading
> >> > https://dist.apache.org/repos/dist/release/arrow/arrow-14.0.1/apache-arrow-14.0.1.tar.gz
> >> > and trying to build with:
> >> >
> >> > mvn clean install
> >> >
> >> > fails as git submodule is required.
> >> >
> >> > So, I would propose to:
> >> > 1. Exclude TestTls related tests by default
> >> > 2. Add a new profile (tlsTests) including TestTls tests
> >> > 3. Activate the profile on CI (-PtlsTests)
> >> >
> >> > Thoughts ?
> >> >
> >> > Regards
> >> > JB
> >> >
> >> >
> >> > On Mon, Dec 11, 2023 at 3:00 PM David Li <li...@apache.org> wrote:
> >> >>
> >> >> You can `git submodule update --init` to get the files. This is documented in the environment setup [1], though the failing assertion could be more helpful about pointing this out.
> >> >>
> >> >> [1]: https://arrow.apache.org/docs/dev/developers/java/building.html#building
> >> >>
> >> >> On Mon, Dec 11, 2023, at 08:49, Jean-Baptiste Onofré wrote:
> >> >> > Hi guys,
> >> >> >
> >> >> > I noticed that Arrow Flight Core doesn't build "out of the box" due to
> >> >> > the TestTls failing.
> >> >> > The reason what TestTls is failing is because it tries to read
> >> >> > cert0.pem from the testing/data folder (at project root), but testing
> >> >> > is empty by default.
> >> >> > If I create a cert0.pem by hand (with self signing key), it works.
> >> >> >
> >> >> > So, I propose three options:
> >> >> > 1. We document the required tests to build Arrow java cleanly
> >> >> > 2. We create a default self signed cert0.pem in testing/data that we commit.
> >> >> > 3. We add a before step to the test to create the pem file
> >> >> >
> >> >> > My preference would be for 3.
> >> >> >
> >> >> > Thoughts ?
> >> >> >
> >> >> > Regards
> >> >> > JB
Re: [DISCUSS] Fix Arrow Flight Core tests
Posted by Sutou Kouhei <ko...@clear-code.com>.
Hi,
I think (2) is better because
https://arrow.apache.org/docs/developers/java/building.html
is for building checked out repository instead of a release
source archive.
Thanks,
--
kou
In <CA...@mail.gmail.com>
"Re: [DISCUSS] Fix Arrow Flight Core tests" on Wed, 13 Dec 2023 20:42:29 +0100,
Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> That's my point:
> 1. we should document that the build has to be launched with
> -DskipTests by default
> 2. or include submodule in the source distribution we distributed
>
> I think (1) is an acceptable and easy path. If there are no
> objections, I will create a PR about that (also updating
> https://arrow.apache.org/docs/developers/java/building.html).
>
> Thoughts ?
>
> Thanks !
> Regards
> JB
>
> On Wed, Dec 13, 2023 at 8:19 PM David Li <li...@apache.org> wrote:
>>
>> Could we include submodules in the source distribution instead?
>>
>> Also, you can still build from the source distribution, you just have to -DskipTests.
>>
>> On Wed, Dec 13, 2023, at 09:54, Jean-Baptiste Onofré wrote:
>> > Hi David and foxes,
>> >
>> > I checked the release source distribution and I think we should maybe
>> > include a profile to include TestTls related tests.
>> >
>> > As an Apache project, we are supposed to be able to build from the
>> > source distribution without any external requirements (in the case the
>> > project goes to the attic or someone wants to create a new release on
>> > an old branch).
>> >
>> > Downloading
>> > https://dist.apache.org/repos/dist/release/arrow/arrow-14.0.1/apache-arrow-14.0.1.tar.gz
>> > and trying to build with:
>> >
>> > mvn clean install
>> >
>> > fails as git submodule is required.
>> >
>> > So, I would propose to:
>> > 1. Exclude TestTls related tests by default
>> > 2. Add a new profile (tlsTests) including TestTls tests
>> > 3. Activate the profile on CI (-PtlsTests)
>> >
>> > Thoughts ?
>> >
>> > Regards
>> > JB
>> >
>> >
>> > On Mon, Dec 11, 2023 at 3:00 PM David Li <li...@apache.org> wrote:
>> >>
>> >> You can `git submodule update --init` to get the files. This is documented in the environment setup [1], though the failing assertion could be more helpful about pointing this out.
>> >>
>> >> [1]: https://arrow.apache.org/docs/dev/developers/java/building.html#building
>> >>
>> >> On Mon, Dec 11, 2023, at 08:49, Jean-Baptiste Onofré wrote:
>> >> > Hi guys,
>> >> >
>> >> > I noticed that Arrow Flight Core doesn't build "out of the box" due to
>> >> > the TestTls failing.
>> >> > The reason what TestTls is failing is because it tries to read
>> >> > cert0.pem from the testing/data folder (at project root), but testing
>> >> > is empty by default.
>> >> > If I create a cert0.pem by hand (with self signing key), it works.
>> >> >
>> >> > So, I propose three options:
>> >> > 1. We document the required tests to build Arrow java cleanly
>> >> > 2. We create a default self signed cert0.pem in testing/data that we commit.
>> >> > 3. We add a before step to the test to create the pem file
>> >> >
>> >> > My preference would be for 3.
>> >> >
>> >> > Thoughts ?
>> >> >
>> >> > Regards
>> >> > JB
Re: [DISCUSS] Fix Arrow Flight Core tests
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
That's my point:
1. we should document that the build has to be launched with
-DskipTests by default
2. or include submodule in the source distribution we distributed
I think (1) is an acceptable and easy path. If there are no
objections, I will create a PR about that (also updating
https://arrow.apache.org/docs/developers/java/building.html).
Thoughts ?
Thanks !
Regards
JB
On Wed, Dec 13, 2023 at 8:19 PM David Li <li...@apache.org> wrote:
>
> Could we include submodules in the source distribution instead?
>
> Also, you can still build from the source distribution, you just have to -DskipTests.
>
> On Wed, Dec 13, 2023, at 09:54, Jean-Baptiste Onofré wrote:
> > Hi David and foxes,
> >
> > I checked the release source distribution and I think we should maybe
> > include a profile to include TestTls related tests.
> >
> > As an Apache project, we are supposed to be able to build from the
> > source distribution without any external requirements (in the case the
> > project goes to the attic or someone wants to create a new release on
> > an old branch).
> >
> > Downloading
> > https://dist.apache.org/repos/dist/release/arrow/arrow-14.0.1/apache-arrow-14.0.1.tar.gz
> > and trying to build with:
> >
> > mvn clean install
> >
> > fails as git submodule is required.
> >
> > So, I would propose to:
> > 1. Exclude TestTls related tests by default
> > 2. Add a new profile (tlsTests) including TestTls tests
> > 3. Activate the profile on CI (-PtlsTests)
> >
> > Thoughts ?
> >
> > Regards
> > JB
> >
> >
> > On Mon, Dec 11, 2023 at 3:00 PM David Li <li...@apache.org> wrote:
> >>
> >> You can `git submodule update --init` to get the files. This is documented in the environment setup [1], though the failing assertion could be more helpful about pointing this out.
> >>
> >> [1]: https://arrow.apache.org/docs/dev/developers/java/building.html#building
> >>
> >> On Mon, Dec 11, 2023, at 08:49, Jean-Baptiste Onofré wrote:
> >> > Hi guys,
> >> >
> >> > I noticed that Arrow Flight Core doesn't build "out of the box" due to
> >> > the TestTls failing.
> >> > The reason what TestTls is failing is because it tries to read
> >> > cert0.pem from the testing/data folder (at project root), but testing
> >> > is empty by default.
> >> > If I create a cert0.pem by hand (with self signing key), it works.
> >> >
> >> > So, I propose three options:
> >> > 1. We document the required tests to build Arrow java cleanly
> >> > 2. We create a default self signed cert0.pem in testing/data that we commit.
> >> > 3. We add a before step to the test to create the pem file
> >> >
> >> > My preference would be for 3.
> >> >
> >> > Thoughts ?
> >> >
> >> > Regards
> >> > JB
Re: [DISCUSS] Fix Arrow Flight Core tests
Posted by David Li <li...@apache.org>.
Could we include submodules in the source distribution instead?
Also, you can still build from the source distribution, you just have to -DskipTests.
On Wed, Dec 13, 2023, at 09:54, Jean-Baptiste Onofré wrote:
> Hi David and foxes,
>
> I checked the release source distribution and I think we should maybe
> include a profile to include TestTls related tests.
>
> As an Apache project, we are supposed to be able to build from the
> source distribution without any external requirements (in the case the
> project goes to the attic or someone wants to create a new release on
> an old branch).
>
> Downloading
> https://dist.apache.org/repos/dist/release/arrow/arrow-14.0.1/apache-arrow-14.0.1.tar.gz
> and trying to build with:
>
> mvn clean install
>
> fails as git submodule is required.
>
> So, I would propose to:
> 1. Exclude TestTls related tests by default
> 2. Add a new profile (tlsTests) including TestTls tests
> 3. Activate the profile on CI (-PtlsTests)
>
> Thoughts ?
>
> Regards
> JB
>
>
> On Mon, Dec 11, 2023 at 3:00 PM David Li <li...@apache.org> wrote:
>>
>> You can `git submodule update --init` to get the files. This is documented in the environment setup [1], though the failing assertion could be more helpful about pointing this out.
>>
>> [1]: https://arrow.apache.org/docs/dev/developers/java/building.html#building
>>
>> On Mon, Dec 11, 2023, at 08:49, Jean-Baptiste Onofré wrote:
>> > Hi guys,
>> >
>> > I noticed that Arrow Flight Core doesn't build "out of the box" due to
>> > the TestTls failing.
>> > The reason what TestTls is failing is because it tries to read
>> > cert0.pem from the testing/data folder (at project root), but testing
>> > is empty by default.
>> > If I create a cert0.pem by hand (with self signing key), it works.
>> >
>> > So, I propose three options:
>> > 1. We document the required tests to build Arrow java cleanly
>> > 2. We create a default self signed cert0.pem in testing/data that we commit.
>> > 3. We add a before step to the test to create the pem file
>> >
>> > My preference would be for 3.
>> >
>> > Thoughts ?
>> >
>> > Regards
>> > JB
Re: [DISCUSS] Fix Arrow Flight Core tests
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi David and foxes,
I checked the release source distribution and I think we should maybe
include a profile to include TestTls related tests.
As an Apache project, we are supposed to be able to build from the
source distribution without any external requirements (in the case the
project goes to the attic or someone wants to create a new release on
an old branch).
Downloading https://dist.apache.org/repos/dist/release/arrow/arrow-14.0.1/apache-arrow-14.0.1.tar.gz
and trying to build with:
mvn clean install
fails as git submodule is required.
So, I would propose to:
1. Exclude TestTls related tests by default
2. Add a new profile (tlsTests) including TestTls tests
3. Activate the profile on CI (-PtlsTests)
Thoughts ?
Regards
JB
On Mon, Dec 11, 2023 at 3:00 PM David Li <li...@apache.org> wrote:
>
> You can `git submodule update --init` to get the files. This is documented in the environment setup [1], though the failing assertion could be more helpful about pointing this out.
>
> [1]: https://arrow.apache.org/docs/dev/developers/java/building.html#building
>
> On Mon, Dec 11, 2023, at 08:49, Jean-Baptiste Onofré wrote:
> > Hi guys,
> >
> > I noticed that Arrow Flight Core doesn't build "out of the box" due to
> > the TestTls failing.
> > The reason what TestTls is failing is because it tries to read
> > cert0.pem from the testing/data folder (at project root), but testing
> > is empty by default.
> > If I create a cert0.pem by hand (with self signing key), it works.
> >
> > So, I propose three options:
> > 1. We document the required tests to build Arrow java cleanly
> > 2. We create a default self signed cert0.pem in testing/data that we commit.
> > 3. We add a before step to the test to create the pem file
> >
> > My preference would be for 3.
> >
> > Thoughts ?
> >
> > Regards
> > JB
Re: [DISCUSS] Fix Arrow Flight Core tests
Posted by David Li <li...@apache.org>.
You can `git submodule update --init` to get the files. This is documented in the environment setup [1], though the failing assertion could be more helpful about pointing this out.
[1]: https://arrow.apache.org/docs/dev/developers/java/building.html#building
On Mon, Dec 11, 2023, at 08:49, Jean-Baptiste Onofré wrote:
> Hi guys,
>
> I noticed that Arrow Flight Core doesn't build "out of the box" due to
> the TestTls failing.
> The reason what TestTls is failing is because it tries to read
> cert0.pem from the testing/data folder (at project root), but testing
> is empty by default.
> If I create a cert0.pem by hand (with self signing key), it works.
>
> So, I propose three options:
> 1. We document the required tests to build Arrow java cleanly
> 2. We create a default self signed cert0.pem in testing/data that we commit.
> 3. We add a before step to the test to create the pem file
>
> My preference would be for 3.
>
> Thoughts ?
>
> Regards
> JB