You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Dongjoon Hyun <do...@gmail.com> on 2022/01/08 23:10:36 UTC
Apache Spark Jenkins Infra 2022
Happy New Year!
After we sunset our legacy Jenkins Infra on December 23th, 2021,
there were many missing parts in our test coverage combinations.
From Today, January 8th, 2022, the following test coverage is recovered
and newly added as a starter. Although this is a pilot and a small step
forward,
we will continue to build and improve our test coverage for the community.
*## Maven Test Coverage*
Although Apache Spark supports Maven/SBT build and testing,
Maven is our official standard for building Apache Spark distributions.
Since GitHub Action has been covering Maven building only, new Jenkins
infrastructure recovers Maven building and testing.
*## Java 17 on Apple Silicon Coverage*
Since there is no publicly available CI option for us to test Apple Silicon
machines,
the new Jenkins infrastructure is running Java/Scala/Python/R testing on
Apple Silicon.
Please note that Java 17 is the first Java release supporting Apple Silicon
natively.
(JEP 391: macOS/AArch64 Port supporting Apple M1-based machine)
[image: screenshot.png]
This is *maintained by Apache Spark PMC*.
We have more details to be discussed before exposing this infra to the
public.
I want to give my heartfelt thanks to the generous donor and
ASF Foundation Fundraise Team for making this happen.
Thanks,
Dongjoon
Re: Apache Spark Jenkins Infra 2022
Posted by Dongjoon Hyun <do...@gmail.com>.
Thank you, Yikun, Shane, and DB.
Dongjoon
On Sun, Jan 9, 2022 at 10:20 PM DB Tsai <db...@dbtsai.com> wrote:
> Thank you, Dongjoon for driving the build infra.
>
> DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
>
> On Jan 9, 2022, at 6:38 PM, shane knapp ☠ <sk...@berkeley.edu> wrote:
>
>
> apache spark jenkins lives on!
>
> @dongjoon, let me know if there's anything you need. nice work, as
> always. :)
>
> shane
>
> On Sat, Jan 8, 2022 at 7:40 PM Yikun Jiang <yi...@gmail.com> wrote:
>
>> @Dongjoon Hyun <do...@gmail.com> Thanks for your work on “Apache
>> Spark Jenkins Infra 2022”. I think this work has very important and useful
>> for CI job that Github Actions cannot support yet.
>>
>> Regards,
>> Yikun
>>
>>
>> Dongjoon Hyun <do...@gmail.com> 于2022年1月9日周日 07:11写道：
>>
>>> Happy New Year!
>>>
>>> After we sunset our legacy Jenkins Infra on December 23th, 2021,
>>> there were many missing parts in our test coverage combinations.
>>>
>>> From Today, January 8th, 2022, the following test coverage is recovered
>>> and newly added as a starter. Although this is a pilot and a small step
>>> forward,
>>> we will continue to build and improve our test coverage for the
>>> community.
>>>
>>> *## Maven Test Coverage*
>>> Although Apache Spark supports Maven/SBT build and testing,
>>> Maven is our official standard for building Apache Spark distributions.
>>> Since GitHub Action has been covering Maven building only, new Jenkins
>>> infrastructure recovers Maven building and testing.
>>>
>>> *## Java 17 on Apple Silicon Coverage*
>>> Since there is no publicly available CI option for us to test Apple
>>> Silicon machines,
>>> the new Jenkins infrastructure is running Java/Scala/Python/R testing on
>>> Apple Silicon.
>>> Please note that Java 17 is the first Java release supporting Apple
>>> Silicon natively.
>>> (JEP 391: macOS/AArch64 Port supporting Apple M1-based machine)
>>>
>>> <screenshot.png>
>>>
>>> This is *maintained by Apache Spark PMC*.
>>> We have more details to be discussed before exposing this infra to the
>>> public.
>>>
>>> I want to give my heartfelt thanks to the generous donor and
>>> ASF Foundation Fundraise Team for making this happen.
>>>
>>> Thanks,
>>> Dongjoon
>>>
>>
>
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>
>
>
Re: Apache Spark Jenkins Infra 2022
Posted by DB Tsai <db...@dbtsai.com>.
Thank you, Dongjoon for driving the build infra.
DB Tsai | https://www.dbtsai.com/ | PGP 42E5B25A8F7A82C1
> On Jan 9, 2022, at 6:38 PM, shane knapp ☠ <sk...@berkeley.edu> wrote:
>
>
> apache spark jenkins lives on!
>
> @dongjoon, let me know if there's anything you need. nice work, as always. :)
>
> shane
>
> On Sat, Jan 8, 2022 at 7:40 PM Yikun Jiang <yikunkero@gmail.com <ma...@gmail.com>> wrote:
> @Dongjoon Hyun <ma...@gmail.com> Thanks for your work on “Apache Spark Jenkins Infra 2022”. I think this work has very important and useful for CI job that Github Actions cannot support yet.
>
> Regards,
> Yikun
>
>
> Dongjoon Hyun <dongjoon.hyun@gmail.com <ma...@gmail.com>> 于2022年1月9日周日 07:11写道：
> Happy New Year!
>
> After we sunset our legacy Jenkins Infra on December 23th, 2021,
> there were many missing parts in our test coverage combinations.
>
> From Today, January 8th, 2022, the following test coverage is recovered
> and newly added as a starter. Although this is a pilot and a small step forward,
> we will continue to build and improve our test coverage for the community.
>
> ## Maven Test Coverage
> Although Apache Spark supports Maven/SBT build and testing,
> Maven is our official standard for building Apache Spark distributions.
> Since GitHub Action has been covering Maven building only, new Jenkins
> infrastructure recovers Maven building and testing.
>
> ## Java 17 on Apple Silicon Coverage
> Since there is no publicly available CI option for us to test Apple Silicon machines,
> the new Jenkins infrastructure is running Java/Scala/Python/R testing on Apple Silicon.
> Please note that Java 17 is the first Java release supporting Apple Silicon natively.
> (JEP 391: macOS/AArch64 Port supporting Apple M1-based machine)
>
> <screenshot.png>
>
> This is maintained by Apache Spark PMC.
> We have more details to be discussed before exposing this infra to the public.
>
> I want to give my heartfelt thanks to the generous donor and
> ASF Foundation Fundraise Team for making this happen.
>
> Thanks,
> Dongjoon
>
>
> --
> Shane Knapp
> Computer Guy / Voice of Reason
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu <https://rise.cs.berkeley.edu/>
Re: Apache Spark Jenkins Infra 2022
Posted by shane knapp ☠ <sk...@berkeley.edu>.
apache spark jenkins lives on!
@dongjoon, let me know if there's anything you need. nice work, as
always. :)
shane
On Sat, Jan 8, 2022 at 7:40 PM Yikun Jiang <yi...@gmail.com> wrote:
> @Dongjoon Hyun <do...@gmail.com> Thanks for your work on “Apache
> Spark Jenkins Infra 2022”. I think this work has very important and useful
> for CI job that Github Actions cannot support yet.
>
> Regards,
> Yikun
>
>
> Dongjoon Hyun <do...@gmail.com> 于2022年1月9日周日 07:11写道：
>
>> Happy New Year!
>>
>> After we sunset our legacy Jenkins Infra on December 23th, 2021,
>> there were many missing parts in our test coverage combinations.
>>
>> From Today, January 8th, 2022, the following test coverage is recovered
>> and newly added as a starter. Although this is a pilot and a small step
>> forward,
>> we will continue to build and improve our test coverage for the community.
>>
>> *## Maven Test Coverage*
>> Although Apache Spark supports Maven/SBT build and testing,
>> Maven is our official standard for building Apache Spark distributions.
>> Since GitHub Action has been covering Maven building only, new Jenkins
>> infrastructure recovers Maven building and testing.
>>
>> *## Java 17 on Apple Silicon Coverage*
>> Since there is no publicly available CI option for us to test Apple
>> Silicon machines,
>> the new Jenkins infrastructure is running Java/Scala/Python/R testing on
>> Apple Silicon.
>> Please note that Java 17 is the first Java release supporting Apple
>> Silicon natively.
>> (JEP 391: macOS/AArch64 Port supporting Apple M1-based machine)
>>
>> [image: screenshot.png]
>>
>> This is *maintained by Apache Spark PMC*.
>> We have more details to be discussed before exposing this infra to the
>> public.
>>
>> I want to give my heartfelt thanks to the generous donor and
>> ASF Foundation Fundraise Team for making this happen.
>>
>> Thanks,
>> Dongjoon
>>
>
--
Shane Knapp
Computer Guy / Voice of Reason
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu
Re: Apache Spark Jenkins Infra 2022
Posted by Yikun Jiang <yi...@gmail.com>.
@Dongjoon Hyun <do...@gmail.com> Thanks for your work on “Apache
Spark Jenkins Infra 2022”. I think this work has very important and useful
for CI job that Github Actions cannot support yet.
Regards,
Yikun
Dongjoon Hyun <do...@gmail.com> 于2022年1月9日周日 07:11写道：
> Happy New Year!
>
> After we sunset our legacy Jenkins Infra on December 23th, 2021,
> there were many missing parts in our test coverage combinations.
>
> From Today, January 8th, 2022, the following test coverage is recovered
> and newly added as a starter. Although this is a pilot and a small step
> forward,
> we will continue to build and improve our test coverage for the community.
>
> *## Maven Test Coverage*
> Although Apache Spark supports Maven/SBT build and testing,
> Maven is our official standard for building Apache Spark distributions.
> Since GitHub Action has been covering Maven building only, new Jenkins
> infrastructure recovers Maven building and testing.
>
> *## Java 17 on Apple Silicon Coverage*
> Since there is no publicly available CI option for us to test Apple
> Silicon machines,
> the new Jenkins infrastructure is running Java/Scala/Python/R testing on
> Apple Silicon.
> Please note that Java 17 is the first Java release supporting Apple
> Silicon natively.
> (JEP 391: macOS/AArch64 Port supporting Apple M1-based machine)
>
> [image: screenshot.png]
>
> This is *maintained by Apache Spark PMC*.
> We have more details to be discussed before exposing this infra to the
> public.
>
> I want to give my heartfelt thanks to the generous donor and
> ASF Foundation Fundraise Team for making this happen.
>
> Thanks,
> Dongjoon
>