You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Yikun Jiang <yi...@gmail.com> on 2022/09/19 00:15:28 UTC
[DISCUSS] SPIP: Support Docker Official Image for Spark
Hi, all
I would like to start the discussion for supporting Docker Official Image
for Spark.
This SPIP is proposed to add Docker Official Image(DOI)
<https://github.com/docker-library/official-images> to ensure the Spark
Docker images meet the quality standards for Docker images, to provide
these Docker images for users who want to use Apache Spark via Docker image.
There are also several Apache projects that release the Docker Official
Images <https://hub.docker.com/search?q=apache&image_filter=official>, such
as: flink <https://hub.docker.com/_/flink>, storm
<https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
zookeeper <https://hub.docker.com/_/zookeeper>, httpd
<https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each). From
the huge download statistics, we can see the real demands of users, and
from the support of other apache projects, we should also be able to do it.
After support:
-
The Dockerfile will still be maintained by the Apache Spark community
and reviewed by Docker.
-
The images will be maintained by the Docker community to ensure the
quality standards for Docker images of the Docker community.
It will also reduce the extra docker images maintenance effort (such as
frequently rebuilding, image security update) of the Apache Spark community.
See more in SPIP DOC:
https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
cc: Ruifeng (co-author) and Hyukjin (shepherd)
Regards,
Yikun
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Ankit Gupta <in...@gmail.com>.
Thanks Yikun for the explanation and Hyukjin. So, what I understand is,
this is to establish the structure so that we start building the DOIs for
now and slowly and steadily build and add more and more images further.
Regards,
Ankit Prakash Gupta
On Thu, Sep 22, 2022 at 7:02 AM Hyukjin Kwon <gu...@gmail.com> wrote:
> Given that support, I will start the vote officially.
>
> On Thu, 22 Sept 2022 at 08:40, Yikun Jiang <yi...@gmail.com> wrote:
>
>> @Ankit
>>
>> Thanks for your support! Your questions are very valuable, but this SPIP
>> is just a start point to cover existing apache/spark image features first.
>> And we will also set up a build/test/publish image workflow (to make sure
>> the quality of image) and some helper scripts to help developers extend
>> more custom images easily in future.
>>
>> > How do we support deployments of spark-standalone clusters in case the
>> users want to use the same image for spark-standalone clusters ? Since that
>> is also widely used.
>> Yes, it's possible, it can be done by exposing some ports, but still need
>> to validate and then doc them in standalone mode doc.
>>
>> > 2. I am not sure about the End of Support of Hadoop 2 with spark, but
>> if that is not planned sooner, shouldn't we be making it configurable to be
>> able to use spark prebuilt with hadoop 2?
>> DOI required a static dockerfile, so couldn't be configurable in
>> runtime.Of course, all spark published releases can also be supported as a
>> separate image in principle. About supporting more distribution, we also
>> planned to add some scripts to help generate the dockerfile.
>>
>> > 3. Also, don't we want to make it feasible for the users to be able to
>> customize the base linux flavour?
>> This is also a good point, but out of scope of this SPIP. Currently, we
>> start with Ubuntu OS (debian series, apt software manager). We might also
>> consider supporting more OS after this SPIP. Such as
>> rehl/centos/rocky/openEuler serious, yum/dnf software manager. But as you
>> know, different OS's have various package versions and upgrade policies, so
>> it's perhaps not very easy work for maintenance, but I think it's possible.
>>
>> Regards,
>> Yikun
>>
>>
>> On Thu, Sep 22, 2022 at 3:43 AM Ankit Gupta <in...@gmail.com>
>> wrote:
>>
>>> Hi Yikun
>>>
>>> Thanks for all your efforts! This is very much needed. But I have the
>>> below three questions:
>>> 1. How do we support deployments of spark-standalone clusters in case
>>> the users wants to use the same image for spark-standalone clusters ? Since
>>> that is also widely used.
>>> 2. I am not sure about the End of Support of Hadoop 2 with spark, but if
>>> that is not planned sooner, shouldn't we be making it configurable to be
>>> able to use spark prebuilt with hadoop 2?
>>> 3. Also, don't we want to make it feasible for the users to be able to
>>> customise the base linux flavour?
>>>
>>> Thanks and Regards.
>>>
>>> Ankit Prakash Gupta
>>>
>>>
>>> On Wed, Sep 21, 2022 at 9:19 PM Xiao Li <ga...@gmail.com> wrote:
>>>
>>>> +1
>>>>
>>>> Yikun Jiang <yi...@gmail.com> 于2022年9月21日周三 07:22写道：
>>>>
>>>>> Thanks for all your inputs! BTW, I also create a JIRA to track related
>>>>> work: https://issues.apache.org/jira/browse/SPARK-40513
>>>>>
>>>>> > can I be involved in this work?
>>>>>
>>>>> @qian Of course! Thanks!
>>>>>
>>>>> Regards,
>>>>> Yikun
>>>>>
>>>>> On Wed, Sep 21, 2022 at 7:31 PM Xinrong Meng <xi...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> On Tue, Sep 20, 2022 at 11:08 PM Qian SUN <qi...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +1.
>>>>>>> It's valuable, can I be involved in this work?
>>>>>>>
>>>>>>> Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
>>>>>>>
>>>>>>>> Hi, all
>>>>>>>>
>>>>>>>> I would like to start the discussion for supporting Docker Official
>>>>>>>> Image for Spark.
>>>>>>>>
>>>>>>>> This SPIP is proposed to add Docker Official Image(DOI)
>>>>>>>> <https://github.com/docker-library/official-images> to ensure the
>>>>>>>> Spark Docker images meet the quality standards for Docker images, to
>>>>>>>> provide these Docker images for users who want to use Apache Spark via
>>>>>>>> Docker image.
>>>>>>>>
>>>>>>>> There are also several Apache projects that release the Docker
>>>>>>>> Official Images
>>>>>>>> <https://hub.docker.com/search?q=apache&image_filter=official>,
>>>>>>>> such as: flink <https://hub.docker.com/_/flink>, storm
>>>>>>>> <https://hub.docker.com/_/storm>, solr
>>>>>>>> <https://hub.docker.com/_/solr>, zookeeper
>>>>>>>> <https://hub.docker.com/_/zookeeper>, httpd
>>>>>>>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for
>>>>>>>> each). From the huge download statistics, we can see the real demands of
>>>>>>>> users, and from the support of other apache projects, we should also be
>>>>>>>> able to do it.
>>>>>>>>
>>>>>>>> After support:
>>>>>>>>
>>>>>>>> -
>>>>>>>>
>>>>>>>> The Dockerfile will still be maintained by the Apache Spark
>>>>>>>> community and reviewed by Docker.
>>>>>>>> -
>>>>>>>>
>>>>>>>> The images will be maintained by the Docker community to ensure
>>>>>>>> the quality standards for Docker images of the Docker community.
>>>>>>>>
>>>>>>>>
>>>>>>>> It will also reduce the extra docker images maintenance effort
>>>>>>>> (such as frequently rebuilding, image security update) of the Apache Spark
>>>>>>>> community.
>>>>>>>>
>>>>>>>> See more in SPIP DOC:
>>>>>>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>>>>>
>>>>>>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Yikun
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best!
>>>>>>> Qian SUN
>>>>>>>
>>>>>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Hyukjin Kwon <gu...@gmail.com>.
Given that support, I will start the vote officially.
On Thu, 22 Sept 2022 at 08:40, Yikun Jiang <yi...@gmail.com> wrote:
> @Ankit
>
> Thanks for your support! Your questions are very valuable, but this SPIP
> is just a start point to cover existing apache/spark image features first.
> And we will also set up a build/test/publish image workflow (to make sure
> the quality of image) and some helper scripts to help developers extend
> more custom images easily in future.
>
> > How do we support deployments of spark-standalone clusters in case the
> users want to use the same image for spark-standalone clusters ? Since that
> is also widely used.
> Yes, it's possible, it can be done by exposing some ports, but still need
> to validate and then doc them in standalone mode doc.
>
> > 2. I am not sure about the End of Support of Hadoop 2 with spark, but if
> that is not planned sooner, shouldn't we be making it configurable to be
> able to use spark prebuilt with hadoop 2?
> DOI required a static dockerfile, so couldn't be configurable in
> runtime.Of course, all spark published releases can also be supported as a
> separate image in principle. About supporting more distribution, we also
> planned to add some scripts to help generate the dockerfile.
>
> > 3. Also, don't we want to make it feasible for the users to be able to
> customize the base linux flavour?
> This is also a good point, but out of scope of this SPIP. Currently, we
> start with Ubuntu OS (debian series, apt software manager). We might also
> consider supporting more OS after this SPIP. Such as
> rehl/centos/rocky/openEuler serious, yum/dnf software manager. But as you
> know, different OS's have various package versions and upgrade policies, so
> it's perhaps not very easy work for maintenance, but I think it's possible.
>
> Regards,
> Yikun
>
>
> On Thu, Sep 22, 2022 at 3:43 AM Ankit Gupta <in...@gmail.com> wrote:
>
>> Hi Yikun
>>
>> Thanks for all your efforts! This is very much needed. But I have the
>> below three questions:
>> 1. How do we support deployments of spark-standalone clusters in case the
>> users wants to use the same image for spark-standalone clusters ? Since
>> that is also widely used.
>> 2. I am not sure about the End of Support of Hadoop 2 with spark, but if
>> that is not planned sooner, shouldn't we be making it configurable to be
>> able to use spark prebuilt with hadoop 2?
>> 3. Also, don't we want to make it feasible for the users to be able to
>> customise the base linux flavour?
>>
>> Thanks and Regards.
>>
>> Ankit Prakash Gupta
>>
>>
>> On Wed, Sep 21, 2022 at 9:19 PM Xiao Li <ga...@gmail.com> wrote:
>>
>>> +1
>>>
>>> Yikun Jiang <yi...@gmail.com> 于2022年9月21日周三 07:22写道：
>>>
>>>> Thanks for all your inputs! BTW, I also create a JIRA to track related
>>>> work: https://issues.apache.org/jira/browse/SPARK-40513
>>>>
>>>> > can I be involved in this work?
>>>>
>>>> @qian Of course! Thanks!
>>>>
>>>> Regards,
>>>> Yikun
>>>>
>>>> On Wed, Sep 21, 2022 at 7:31 PM Xinrong Meng <xi...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> On Tue, Sep 20, 2022 at 11:08 PM Qian SUN <qi...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +1.
>>>>>> It's valuable, can I be involved in this work?
>>>>>>
>>>>>> Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
>>>>>>
>>>>>>> Hi, all
>>>>>>>
>>>>>>> I would like to start the discussion for supporting Docker Official
>>>>>>> Image for Spark.
>>>>>>>
>>>>>>> This SPIP is proposed to add Docker Official Image(DOI)
>>>>>>> <https://github.com/docker-library/official-images> to ensure the
>>>>>>> Spark Docker images meet the quality standards for Docker images, to
>>>>>>> provide these Docker images for users who want to use Apache Spark via
>>>>>>> Docker image.
>>>>>>>
>>>>>>> There are also several Apache projects that release the Docker
>>>>>>> Official Images
>>>>>>> <https://hub.docker.com/search?q=apache&image_filter=official>,
>>>>>>> such as: flink <https://hub.docker.com/_/flink>, storm
>>>>>>> <https://hub.docker.com/_/storm>, solr
>>>>>>> <https://hub.docker.com/_/solr>, zookeeper
>>>>>>> <https://hub.docker.com/_/zookeeper>, httpd
>>>>>>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for
>>>>>>> each). From the huge download statistics, we can see the real demands of
>>>>>>> users, and from the support of other apache projects, we should also be
>>>>>>> able to do it.
>>>>>>>
>>>>>>> After support:
>>>>>>>
>>>>>>> -
>>>>>>>
>>>>>>> The Dockerfile will still be maintained by the Apache Spark
>>>>>>> community and reviewed by Docker.
>>>>>>> -
>>>>>>>
>>>>>>> The images will be maintained by the Docker community to ensure
>>>>>>> the quality standards for Docker images of the Docker community.
>>>>>>>
>>>>>>>
>>>>>>> It will also reduce the extra docker images maintenance effort (such
>>>>>>> as frequently rebuilding, image security update) of the Apache Spark
>>>>>>> community.
>>>>>>>
>>>>>>> See more in SPIP DOC:
>>>>>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>>>>
>>>>>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>>>>
>>>>>>> Regards,
>>>>>>> Yikun
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best!
>>>>>> Qian SUN
>>>>>>
>>>>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Yikun Jiang <yi...@gmail.com>.
@Ankit
Thanks for your support! Your questions are very valuable, but this SPIP is
just a start point to cover existing apache/spark image features first.
And we will also set up a build/test/publish image workflow (to make sure
the quality of image) and some helper scripts to help developers extend
more custom images easily in future.
> How do we support deployments of spark-standalone clusters in case the
users want to use the same image for spark-standalone clusters ? Since that
is also widely used.
Yes, it's possible, it can be done by exposing some ports, but still need
to validate and then doc them in standalone mode doc.
> 2. I am not sure about the End of Support of Hadoop 2 with spark, but if
that is not planned sooner, shouldn't we be making it configurable to be
able to use spark prebuilt with hadoop 2?
DOI required a static dockerfile, so couldn't be configurable in runtime.Of
course, all spark published releases can also be supported as a separate
image in principle. About supporting more distribution, we also planned to
add some scripts to help generate the dockerfile.
> 3. Also, don't we want to make it feasible for the users to be able to
customize the base linux flavour?
This is also a good point, but out of scope of this SPIP. Currently, we
start with Ubuntu OS (debian series, apt software manager). We might also
consider supporting more OS after this SPIP. Such as
rehl/centos/rocky/openEuler serious, yum/dnf software manager. But as you
know, different OS's have various package versions and upgrade policies, so
it's perhaps not very easy work for maintenance, but I think it's possible.
Regards,
Yikun
On Thu, Sep 22, 2022 at 3:43 AM Ankit Gupta <in...@gmail.com> wrote:
> Hi Yikun
>
> Thanks for all your efforts! This is very much needed. But I have the
> below three questions:
> 1. How do we support deployments of spark-standalone clusters in case the
> users wants to use the same image for spark-standalone clusters ? Since
> that is also widely used.
> 2. I am not sure about the End of Support of Hadoop 2 with spark, but if
> that is not planned sooner, shouldn't we be making it configurable to be
> able to use spark prebuilt with hadoop 2?
> 3. Also, don't we want to make it feasible for the users to be able to
> customise the base linux flavour?
>
> Thanks and Regards.
>
> Ankit Prakash Gupta
>
>
> On Wed, Sep 21, 2022 at 9:19 PM Xiao Li <ga...@gmail.com> wrote:
>
>> +1
>>
>> Yikun Jiang <yi...@gmail.com> 于2022年9月21日周三 07:22写道：
>>
>>> Thanks for all your inputs! BTW, I also create a JIRA to track related
>>> work: https://issues.apache.org/jira/browse/SPARK-40513
>>>
>>> > can I be involved in this work?
>>>
>>> @qian Of course! Thanks!
>>>
>>> Regards,
>>> Yikun
>>>
>>> On Wed, Sep 21, 2022 at 7:31 PM Xinrong Meng <xi...@gmail.com>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> On Tue, Sep 20, 2022 at 11:08 PM Qian SUN <qi...@gmail.com>
>>>> wrote:
>>>>
>>>>> +1.
>>>>> It's valuable, can I be involved in this work?
>>>>>
>>>>> Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
>>>>>
>>>>>> Hi, all
>>>>>>
>>>>>> I would like to start the discussion for supporting Docker Official
>>>>>> Image for Spark.
>>>>>>
>>>>>> This SPIP is proposed to add Docker Official Image(DOI)
>>>>>> <https://github.com/docker-library/official-images> to ensure the
>>>>>> Spark Docker images meet the quality standards for Docker images, to
>>>>>> provide these Docker images for users who want to use Apache Spark via
>>>>>> Docker image.
>>>>>>
>>>>>> There are also several Apache projects that release the Docker
>>>>>> Official Images
>>>>>> <https://hub.docker.com/search?q=apache&image_filter=official>, such
>>>>>> as: flink <https://hub.docker.com/_/flink>, storm
>>>>>> <https://hub.docker.com/_/storm>, solr
>>>>>> <https://hub.docker.com/_/solr>, zookeeper
>>>>>> <https://hub.docker.com/_/zookeeper>, httpd
>>>>>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for
>>>>>> each). From the huge download statistics, we can see the real demands of
>>>>>> users, and from the support of other apache projects, we should also be
>>>>>> able to do it.
>>>>>>
>>>>>> After support:
>>>>>>
>>>>>> -
>>>>>>
>>>>>> The Dockerfile will still be maintained by the Apache Spark
>>>>>> community and reviewed by Docker.
>>>>>> -
>>>>>>
>>>>>> The images will be maintained by the Docker community to ensure
>>>>>> the quality standards for Docker images of the Docker community.
>>>>>>
>>>>>>
>>>>>> It will also reduce the extra docker images maintenance effort (such
>>>>>> as frequently rebuilding, image security update) of the Apache Spark
>>>>>> community.
>>>>>>
>>>>>> See more in SPIP DOC:
>>>>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>>>
>>>>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>>>
>>>>>> Regards,
>>>>>> Yikun
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best!
>>>>> Qian SUN
>>>>>
>>>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Ankit Gupta <in...@gmail.com>.
Hi Yikun
Thanks for all your efforts! This is very much needed. But I have the below
three questions:
1. How do we support deployments of spark-standalone clusters in case the
users wants to use the same image for spark-standalone clusters ? Since
that is also widely used.
2. I am not sure about the End of Support of Hadoop 2 with spark, but if
that is not planned sooner, shouldn't we be making it configurable to be
able to use spark prebuilt with hadoop 2?
3. Also, don't we want to make it feasible for the users to be able to
customise the base linux flavour?
Thanks and Regards.
Ankit Prakash Gupta
On Wed, Sep 21, 2022 at 9:19 PM Xiao Li <ga...@gmail.com> wrote:
> +1
>
> Yikun Jiang <yi...@gmail.com> 于2022年9月21日周三 07:22写道：
>
>> Thanks for all your inputs! BTW, I also create a JIRA to track related
>> work: https://issues.apache.org/jira/browse/SPARK-40513
>>
>> > can I be involved in this work?
>>
>> @qian Of course! Thanks!
>>
>> Regards,
>> Yikun
>>
>> On Wed, Sep 21, 2022 at 7:31 PM Xinrong Meng <xi...@gmail.com>
>> wrote:
>>
>>> +1
>>>
>>> On Tue, Sep 20, 2022 at 11:08 PM Qian SUN <qi...@gmail.com>
>>> wrote:
>>>
>>>> +1.
>>>> It's valuable, can I be involved in this work?
>>>>
>>>> Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
>>>>
>>>>> Hi, all
>>>>>
>>>>> I would like to start the discussion for supporting Docker Official
>>>>> Image for Spark.
>>>>>
>>>>> This SPIP is proposed to add Docker Official Image(DOI)
>>>>> <https://github.com/docker-library/official-images> to ensure the
>>>>> Spark Docker images meet the quality standards for Docker images, to
>>>>> provide these Docker images for users who want to use Apache Spark via
>>>>> Docker image.
>>>>>
>>>>> There are also several Apache projects that release the Docker
>>>>> Official Images
>>>>> <https://hub.docker.com/search?q=apache&image_filter=official>, such
>>>>> as: flink <https://hub.docker.com/_/flink>, storm
>>>>> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
>>>>> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
>>>>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for
>>>>> each). From the huge download statistics, we can see the real demands of
>>>>> users, and from the support of other apache projects, we should also be
>>>>> able to do it.
>>>>>
>>>>> After support:
>>>>>
>>>>> -
>>>>>
>>>>> The Dockerfile will still be maintained by the Apache Spark
>>>>> community and reviewed by Docker.
>>>>> -
>>>>>
>>>>> The images will be maintained by the Docker community to ensure
>>>>> the quality standards for Docker images of the Docker community.
>>>>>
>>>>>
>>>>> It will also reduce the extra docker images maintenance effort (such
>>>>> as frequently rebuilding, image security update) of the Apache Spark
>>>>> community.
>>>>>
>>>>> See more in SPIP DOC:
>>>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>>
>>>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>>
>>>>> Regards,
>>>>> Yikun
>>>>>
>>>>
>>>>
>>>> --
>>>> Best!
>>>> Qian SUN
>>>>
>>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Xiao Li <ga...@gmail.com>.
+1
Yikun Jiang <yi...@gmail.com> 于2022年9月21日周三 07:22写道：
> Thanks for all your inputs! BTW, I also create a JIRA to track related
> work: https://issues.apache.org/jira/browse/SPARK-40513
>
> > can I be involved in this work?
>
> @qian Of course! Thanks!
>
> Regards,
> Yikun
>
> On Wed, Sep 21, 2022 at 7:31 PM Xinrong Meng <xi...@gmail.com>
> wrote:
>
>> +1
>>
>> On Tue, Sep 20, 2022 at 11:08 PM Qian SUN <qi...@gmail.com> wrote:
>>
>>> +1.
>>> It's valuable, can I be involved in this work?
>>>
>>> Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
>>>
>>>> Hi, all
>>>>
>>>> I would like to start the discussion for supporting Docker Official
>>>> Image for Spark.
>>>>
>>>> This SPIP is proposed to add Docker Official Image(DOI)
>>>> <https://github.com/docker-library/official-images> to ensure the
>>>> Spark Docker images meet the quality standards for Docker images, to
>>>> provide these Docker images for users who want to use Apache Spark via
>>>> Docker image.
>>>>
>>>> There are also several Apache projects that release the Docker
>>>> Official Images
>>>> <https://hub.docker.com/search?q=apache&image_filter=official>, such
>>>> as: flink <https://hub.docker.com/_/flink>, storm
>>>> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
>>>> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
>>>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each).
>>>> From the huge download statistics, we can see the real demands of users,
>>>> and from the support of other apache projects, we should also be able to do
>>>> it.
>>>>
>>>> After support:
>>>>
>>>> -
>>>>
>>>> The Dockerfile will still be maintained by the Apache Spark
>>>> community and reviewed by Docker.
>>>> -
>>>>
>>>> The images will be maintained by the Docker community to ensure the
>>>> quality standards for Docker images of the Docker community.
>>>>
>>>>
>>>> It will also reduce the extra docker images maintenance effort (such as
>>>> frequently rebuilding, image security update) of the Apache Spark community.
>>>>
>>>> See more in SPIP DOC:
>>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>
>>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>
>>>> Regards,
>>>> Yikun
>>>>
>>>
>>>
>>> --
>>> Best!
>>> Qian SUN
>>>
>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Yikun Jiang <yi...@gmail.com>.
Thanks for all your inputs! BTW, I also create a JIRA to track related
work: https://issues.apache.org/jira/browse/SPARK-40513
> can I be involved in this work?
@qian Of course! Thanks!
Regards,
Yikun
On Wed, Sep 21, 2022 at 7:31 PM Xinrong Meng <xi...@gmail.com>
wrote:
> +1
>
> On Tue, Sep 20, 2022 at 11:08 PM Qian SUN <qi...@gmail.com> wrote:
>
>> +1.
>> It's valuable, can I be involved in this work?
>>
>> Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
>>
>>> Hi, all
>>>
>>> I would like to start the discussion for supporting Docker Official
>>> Image for Spark.
>>>
>>> This SPIP is proposed to add Docker Official Image(DOI)
>>> <https://github.com/docker-library/official-images> to ensure the Spark
>>> Docker images meet the quality standards for Docker images, to provide
>>> these Docker images for users who want to use Apache Spark via Docker image.
>>>
>>> There are also several Apache projects that release the Docker Official
>>> Images <https://hub.docker.com/search?q=apache&image_filter=official>,
>>> such as: flink <https://hub.docker.com/_/flink>, storm
>>> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
>>> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
>>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each).
>>> From the huge download statistics, we can see the real demands of users,
>>> and from the support of other apache projects, we should also be able to do
>>> it.
>>>
>>> After support:
>>>
>>> -
>>>
>>> The Dockerfile will still be maintained by the Apache Spark
>>> community and reviewed by Docker.
>>> -
>>>
>>> The images will be maintained by the Docker community to ensure the
>>> quality standards for Docker images of the Docker community.
>>>
>>>
>>> It will also reduce the extra docker images maintenance effort (such as
>>> frequently rebuilding, image security update) of the Apache Spark community.
>>>
>>> See more in SPIP DOC:
>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>
>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>
>>> Regards,
>>> Yikun
>>>
>>
>>
>> --
>> Best!
>> Qian SUN
>>
>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Xinrong Meng <xi...@gmail.com>.
+1
On Tue, Sep 20, 2022 at 11:08 PM Qian SUN <qi...@gmail.com> wrote:
> +1.
> It's valuable, can I be involved in this work?
>
> Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
>
>> Hi, all
>>
>> I would like to start the discussion for supporting Docker Official Image
>> for Spark.
>>
>> This SPIP is proposed to add Docker Official Image(DOI)
>> <https://github.com/docker-library/official-images> to ensure the Spark
>> Docker images meet the quality standards for Docker images, to provide
>> these Docker images for users who want to use Apache Spark via Docker image.
>>
>> There are also several Apache projects that release the Docker Official
>> Images <https://hub.docker.com/search?q=apache&image_filter=official>,
>> such as: flink <https://hub.docker.com/_/flink>, storm
>> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
>> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each).
>> From the huge download statistics, we can see the real demands of users,
>> and from the support of other apache projects, we should also be able to do
>> it.
>>
>> After support:
>>
>> -
>>
>> The Dockerfile will still be maintained by the Apache Spark community
>> and reviewed by Docker.
>> -
>>
>> The images will be maintained by the Docker community to ensure the
>> quality standards for Docker images of the Docker community.
>>
>>
>> It will also reduce the extra docker images maintenance effort (such as
>> frequently rebuilding, image security update) of the Apache Spark community.
>>
>> See more in SPIP DOC:
>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>
>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>
>> Regards,
>> Yikun
>>
>
>
> --
> Best!
> Qian SUN
>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Qian SUN <qi...@gmail.com>.
+1.
It's valuable, can I be involved in this work?
Yikun Jiang <yi...@gmail.com> 于2022年9月19日周一 08:15写道：
> Hi, all
>
> I would like to start the discussion for supporting Docker Official Image
> for Spark.
>
> This SPIP is proposed to add Docker Official Image(DOI)
> <https://github.com/docker-library/official-images> to ensure the Spark
> Docker images meet the quality standards for Docker images, to provide
> these Docker images for users who want to use Apache Spark via Docker image.
>
> There are also several Apache projects that release the Docker Official
> Images <https://hub.docker.com/search?q=apache&image_filter=official>,
> such as: flink <https://hub.docker.com/_/flink>, storm
> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each).
> From the huge download statistics, we can see the real demands of users,
> and from the support of other apache projects, we should also be able to do
> it.
>
> After support:
>
> -
>
> The Dockerfile will still be maintained by the Apache Spark community
> and reviewed by Docker.
> -
>
> The images will be maintained by the Docker community to ensure the
> quality standards for Docker images of the Docker community.
>
>
> It will also reduce the extra docker images maintenance effort (such as
> frequently rebuilding, image security update) of the Apache Spark community.
>
> See more in SPIP DOC:
> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>
> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>
> Regards,
> Yikun
>
--
Best!
Qian SUN
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by "416161436@qq.com" <ru...@foxmail.com>.
+1
Ruifeng Zheng
ruifengz@foxmail.com
------------------ Original ------------------
From: "Chao Sun" <sunchao@apache.org>;
Date: Wed, Sep 21, 2022 00:18 AM
To: "Wenchen Fan"<cloud0fan@gmail.com>;
Cc: "Yang,Jie(INF)"<yangjie01@baidu.com>;"Yikun Jiang"<yikunkero@gmail.com>;"Denny Lee"<denny.g.lee@gmail.com>;"bo zhaobo"<bzhaojyathousandy@gmail.com>;"Yuming Wang"<wgyumg@gmail.com>;"Kent Yao"<yao@apache.org>;"Gengliang Wang"<ltnwgl@gmail.com>;"Hyukjin Kwon"<gurwls223@gmail.com>;"dev"<dev@spark.apache.org>;"416161436@qq.com"<ruifengz@foxmail.com>;
Subject: Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
+1 (non-binding)
On Mon, Sep 19, 2022 at 10:17 PM Wenchen Fan <cloud0fan@gmail.com> wrote:
>
> +1
>
> On Mon, Sep 19, 2022 at 2:59 PM Yang,Jie(INF) <yangjie01@baidu.com> wrote:
>>
>> +1 (non-binding)
>>
>>
>>
>> Yang Jie
>>
>> ________________________________
>> 发件人: Yikun Jiang <yikunkero@gmail.com>
>> 发送时间: 2022年9月19日 14:23:14
>> 收件人: Denny Lee
>> 抄送: bo zhaobo; Yuming Wang; Kent Yao; Gengliang Wang; Hyukjin Kwon; dev; zrf
>> 主题: Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
>>
>> Thanks for your support! @all
>>
>> > Count me in to help as well, eh?! :)
>>
>> @Denny Sure, It would be great to have your help! I'm going to create a JIRA and TASKS if the SPIP vote passes.
>>
>>
>> On Mon, Sep 19, 2022 at 10:34 AM Denny Lee <denny.g.lee@gmail.com> wrote:
>>>
>>> +1 (non-binding).
>>>
>>> This is a great idea and we should definitely do this. Count me in to help as well, eh?! :)
>>>
>>> On Sun, Sep 18, 2022 at 7:24 PM bo zhaobo <bzhaojyathousandy@gmail.com> wrote:
>>>>
>>>> +1 (non-binding)
>>>>
>>>> This will bring the good experience to customers. So excited about this. ;-)
>>>>
>>>> Yuming Wang <wgyumg@gmail.com> 于2022年9月19日周一 10:18写道：
>>>>>
>>>>> +1.
>>>>>
>>>>> On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <yao@apache.org> wrote:
>>>>>>
>>>>>> +1
>>>>>>
>>>>>> Gengliang Wang <ltnwgl@gmail.com> 于2022年9月19日周一 09:23写道：
>>>>>> >
>>>>>> > +1, thanks for the work!
>>>>>> >
>>>>>> > On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gurwls223@gmail.com> wrote:
>>>>>> >>
>>>>>> >> +1
>>>>>> >>
>>>>>> >> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yikunkero@gmail.com> wrote:
>>>>>> >>>
>>>>>> >>> Hi, all
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> I would like to start the discussion for supporting Docker Official Image for Spark.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> This SPIP is proposed to add Docker Official Image(DOI) to ensure the Spark Docker images meet the quality standards for Docker images, to provide these Docker images for users who want to use Apache Spark via Docker image.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> There are also several Apache projects that release the Docker Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+ to 1B+ download for each). From the huge download statistics, we can see the real demands of users, and from the support of other apache projects, we should also be able to do it.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> After support:
>>>>>> >>>
>>>>>> >>> The Dockerfile will still be maintained by the Apache Spark community and reviewed by Docker.
>>>>>> >>>
>>>>>> >>> The images will be maintained by the Docker community to ensure the quality standards for Docker images of the Docker community.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> It will also reduce the extra docker images maintenance effort (such as frequently rebuilding, image security update) of the Apache Spark community.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> See more in SPIP DOC: https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> Regards,
>>>>>> >>> Yikun
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Chao Sun <su...@apache.org>.
+1 (non-binding)
On Mon, Sep 19, 2022 at 10:17 PM Wenchen Fan <cl...@gmail.com> wrote:
>
> +1
>
> On Mon, Sep 19, 2022 at 2:59 PM Yang,Jie(INF) <ya...@baidu.com> wrote:
>>
>> +1 (non-binding)
>>
>>
>>
>> Yang Jie
>>
>> ________________________________
>> 发件人: Yikun Jiang <yi...@gmail.com>
>> 发送时间: 2022年9月19日 14:23:14
>> 收件人: Denny Lee
>> 抄送: bo zhaobo; Yuming Wang; Kent Yao; Gengliang Wang; Hyukjin Kwon; dev; zrf
>> 主题: Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
>>
>> Thanks for your support! @all
>>
>> > Count me in to help as well, eh?! :)
>>
>> @Denny Sure, It would be great to have your help! I'm going to create a JIRA and TASKS if the SPIP vote passes.
>>
>>
>> On Mon, Sep 19, 2022 at 10:34 AM Denny Lee <de...@gmail.com> wrote:
>>>
>>> +1 (non-binding).
>>>
>>> This is a great idea and we should definitely do this. Count me in to help as well, eh?! :)
>>>
>>> On Sun, Sep 18, 2022 at 7:24 PM bo zhaobo <bz...@gmail.com> wrote:
>>>>
>>>> +1 (non-binding)
>>>>
>>>> This will bring the good experience to customers. So excited about this. ;-)
>>>>
>>>> Yuming Wang <wg...@gmail.com> 于2022年9月19日周一 10:18写道：
>>>>>
>>>>> +1.
>>>>>
>>>>> On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <ya...@apache.org> wrote:
>>>>>>
>>>>>> +1
>>>>>>
>>>>>> Gengliang Wang <lt...@gmail.com> 于2022年9月19日周一 09:23写道：
>>>>>> >
>>>>>> > +1, thanks for the work!
>>>>>> >
>>>>>> > On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>>>>>> >>
>>>>>> >> +1
>>>>>> >>
>>>>>> >> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com> wrote:
>>>>>> >>>
>>>>>> >>> Hi, all
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> I would like to start the discussion for supporting Docker Official Image for Spark.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> This SPIP is proposed to add Docker Official Image(DOI) to ensure the Spark Docker images meet the quality standards for Docker images, to provide these Docker images for users who want to use Apache Spark via Docker image.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> There are also several Apache projects that release the Docker Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+ to 1B+ download for each). From the huge download statistics, we can see the real demands of users, and from the support of other apache projects, we should also be able to do it.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> After support:
>>>>>> >>>
>>>>>> >>> The Dockerfile will still be maintained by the Apache Spark community and reviewed by Docker.
>>>>>> >>>
>>>>>> >>> The images will be maintained by the Docker community to ensure the quality standards for Docker images of the Docker community.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> It will also reduce the extra docker images maintenance effort (such as frequently rebuilding, image security update) of the Apache Spark community.
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> See more in SPIP DOC: https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> Regards,
>>>>>> >>> Yikun
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Wenchen Fan <cl...@gmail.com>.
+1
On Mon, Sep 19, 2022 at 2:59 PM Yang,Jie(INF) <ya...@baidu.com> wrote:
> +1 (non-binding)
>
>
>
> Yang Jie
> ------------------------------
> *发件人:* Yikun Jiang <yi...@gmail.com>
> *发送时间:* 2022年9月19日 14:23:14
> *收件人:* Denny Lee
> *抄送:* bo zhaobo; Yuming Wang; Kent Yao; Gengliang Wang; Hyukjin Kwon;
> dev; zrf
> *主题:* Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
>
> Thanks for your support! @all
>
> > Count me in to help as well, eh?! :)
>
> @Denny Sure, It would be great to have your help! I'm going to create a
> JIRA and TASKS if the SPIP vote passes.
>
>
> On Mon, Sep 19, 2022 at 10:34 AM Denny Lee <de...@gmail.com> wrote:
>
>> +1 (non-binding).
>>
>> This is a great idea and we should definitely do this. Count me in to
>> help as well, eh?! :)
>>
>> On Sun, Sep 18, 2022 at 7:24 PM bo zhaobo <bz...@gmail.com>
>> wrote:
>>
>>> +1 (non-binding)
>>>
>>> This will bring the good experience to customers. So excited about this.
>>> ;-)
>>>
>>> Yuming Wang <wg...@gmail.com> 于2022年9月19日周一 10:18写道：
>>>
>>>> +1.
>>>>
>>>> On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <ya...@apache.org> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Gengliang Wang <lt...@gmail.com> 于2022年9月19日周一 09:23写道：
>>>>> >
>>>>> > +1, thanks for the work!
>>>>> >
>>>>> > On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> +1
>>>>> >>
>>>>> >> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com>
>>>>> wrote:
>>>>> >>>
>>>>> >>> Hi, all
>>>>> >>>
>>>>> >>>
>>>>> >>> I would like to start the discussion for supporting Docker
>>>>> Official Image for Spark.
>>>>> >>>
>>>>> >>>
>>>>> >>> This SPIP is proposed to add Docker Official Image(DOI) to ensure
>>>>> the Spark Docker images meet the quality standards for Docker images, to
>>>>> provide these Docker images for users who want to use Apache Spark via
>>>>> Docker image.
>>>>> >>>
>>>>> >>>
>>>>> >>> There are also several Apache projects that release the Docker
>>>>> Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+
>>>>> to 1B+ download for each). From the huge download statistics, we can see
>>>>> the real demands of users, and from the support of other apache projects,
>>>>> we should also be able to do it.
>>>>> >>>
>>>>> >>>
>>>>> >>> After support:
>>>>> >>>
>>>>> >>> The Dockerfile will still be maintained by the Apache Spark
>>>>> community and reviewed by Docker.
>>>>> >>>
>>>>> >>> The images will be maintained by the Docker community to ensure
>>>>> the quality standards for Docker images of the Docker community.
>>>>> >>>
>>>>> >>>
>>>>> >>> It will also reduce the extra docker images maintenance effort
>>>>> (such as frequently rebuilding, image security update) of the Apache Spark
>>>>> community.
>>>>> >>>
>>>>> >>>
>>>>> >>> See more in SPIP DOC:
>>>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>>> <https://mailshield.baidu.com/check?q=PDTMrHj%2bNMoH%2fYdHHkWc2nkgQS%2bGR2N3LDR8pOiG8HK2Ow8pf4Ji1nupoSdfjyJuZ1jbIhT%2fnCVCVGfEiwMXIzGYWeBCLjG0%2fu4LHfsANXR841Mr>
>>>>> >>>
>>>>> >>>
>>>>> >>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>>> >>>
>>>>> >>>
>>>>> >>> Regards,
>>>>> >>> Yikun
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>
>>>>>
答复: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by "Yang,Jie(INF)" <ya...@baidu.com>.
+1 (non-binding)
Yang Jie
________________________________
发件人: Yikun Jiang <yi...@gmail.com>
发送时间: 2022年9月19日 14:23:14
收件人: Denny Lee
抄送: bo zhaobo; Yuming Wang; Kent Yao; Gengliang Wang; Hyukjin Kwon; dev; zrf
主题: Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Thanks for your support! @all
> Count me in to help as well, eh?! :)
@Denny Sure, It would be great to have your help! I'm going to create a JIRA and TASKS if the SPIP vote passes.
On Mon, Sep 19, 2022 at 10:34 AM Denny Lee <de...@gmail.com>> wrote:
+1 (non-binding).
This is a great idea and we should definitely do this. Count me in to help as well, eh?! :)
On Sun, Sep 18, 2022 at 7:24 PM bo zhaobo <bz...@gmail.com>> wrote:
+1 (non-binding)
This will bring the good experience to customers. So excited about this. ;-)
Yuming Wang <wg...@gmail.com>> 于2022年9月19日周一 10:18写道：
+1.
On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <ya...@apache.org>> wrote:
+1
Gengliang Wang <lt...@gmail.com>> 于2022年9月19日周一 09:23写道：
>
> +1, thanks for the work!
>
> On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com>> wrote:
>>
>> +1
>>
>> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com>> wrote:
>>>
>>> Hi, all
>>>
>>>
>>> I would like to start the discussion for supporting Docker Official Image for Spark.
>>>
>>>
>>> This SPIP is proposed to add Docker Official Image(DOI) to ensure the Spark Docker images meet the quality standards for Docker images, to provide these Docker images for users who want to use Apache Spark via Docker image.
>>>
>>>
>>> There are also several Apache projects that release the Docker Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+ to 1B+ download for each). From the huge download statistics, we can see the real demands of users, and from the support of other apache projects, we should also be able to do it.
>>>
>>>
>>> After support:
>>>
>>> The Dockerfile will still be maintained by the Apache Spark community and reviewed by Docker.
>>>
>>> The images will be maintained by the Docker community to ensure the quality standards for Docker images of the Docker community.
>>>
>>>
>>> It will also reduce the extra docker images maintenance effort (such as frequently rebuilding, image security update) of the Apache Spark community.
>>>
>>>
>>> See more in SPIP DOC: https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o<https://mailshield.baidu.com/check?q=PDTMrHj%2bNMoH%2fYdHHkWc2nkgQS%2bGR2N3LDR8pOiG8HK2Ow8pf4Ji1nupoSdfjyJuZ1jbIhT%2fnCVCVGfEiwMXIzGYWeBCLjG0%2fu4LHfsANXR841Mr>
>>>
>>>
>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>
>>>
>>> Regards,
>>> Yikun
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org<ma...@spark.apache.org>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Yikun Jiang <yi...@gmail.com>.
Thanks for your support! @all
> Count me in to help as well, eh?! :)
@Denny Sure, It would be great to have your help! I'm going to create a
JIRA and TASKS if the SPIP vote passes.
On Mon, Sep 19, 2022 at 10:34 AM Denny Lee <de...@gmail.com> wrote:
> +1 (non-binding).
>
> This is a great idea and we should definitely do this. Count me in to
> help as well, eh?! :)
>
> On Sun, Sep 18, 2022 at 7:24 PM bo zhaobo <bz...@gmail.com>
> wrote:
>
>> +1 (non-binding)
>>
>> This will bring the good experience to customers. So excited about this.
>> ;-)
>>
>> Yuming Wang <wg...@gmail.com> 于2022年9月19日周一 10:18写道：
>>
>>> +1.
>>>
>>> On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <ya...@apache.org> wrote:
>>>
>>>> +1
>>>>
>>>> Gengliang Wang <lt...@gmail.com> 于2022年9月19日周一 09:23写道：
>>>> >
>>>> > +1, thanks for the work!
>>>> >
>>>> > On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> +1
>>>> >>
>>>> >> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com>
>>>> wrote:
>>>> >>>
>>>> >>> Hi, all
>>>> >>>
>>>> >>>
>>>> >>> I would like to start the discussion for supporting Docker Official
>>>> Image for Spark.
>>>> >>>
>>>> >>>
>>>> >>> This SPIP is proposed to add Docker Official Image(DOI) to ensure
>>>> the Spark Docker images meet the quality standards for Docker images, to
>>>> provide these Docker images for users who want to use Apache Spark via
>>>> Docker image.
>>>> >>>
>>>> >>>
>>>> >>> There are also several Apache projects that release the Docker
>>>> Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+
>>>> to 1B+ download for each). From the huge download statistics, we can see
>>>> the real demands of users, and from the support of other apache projects,
>>>> we should also be able to do it.
>>>> >>>
>>>> >>>
>>>> >>> After support:
>>>> >>>
>>>> >>> The Dockerfile will still be maintained by the Apache Spark
>>>> community and reviewed by Docker.
>>>> >>>
>>>> >>> The images will be maintained by the Docker community to ensure the
>>>> quality standards for Docker images of the Docker community.
>>>> >>>
>>>> >>>
>>>> >>> It will also reduce the extra docker images maintenance effort
>>>> (such as frequently rebuilding, image security update) of the Apache Spark
>>>> community.
>>>> >>>
>>>> >>>
>>>> >>> See more in SPIP DOC:
>>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>> >>>
>>>> >>>
>>>> >>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>> >>>
>>>> >>>
>>>> >>> Regards,
>>>> >>> Yikun
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>
>>>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Denny Lee <de...@gmail.com>.
+1 (non-binding).
This is a great idea and we should definitely do this. Count me in to help
as well, eh?! :)
On Sun, Sep 18, 2022 at 7:24 PM bo zhaobo <bz...@gmail.com>
wrote:
> +1 (non-binding)
>
> This will bring the good experience to customers. So excited about this.
> ;-)
>
> Yuming Wang <wg...@gmail.com> 于2022年9月19日周一 10:18写道：
>
>> +1.
>>
>> On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <ya...@apache.org> wrote:
>>
>>> +1
>>>
>>> Gengliang Wang <lt...@gmail.com> 于2022年9月19日周一 09:23写道：
>>> >
>>> > +1, thanks for the work!
>>> >
>>> > On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com>
>>> wrote:
>>> >>
>>> >> +1
>>> >>
>>> >> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com>
>>> wrote:
>>> >>>
>>> >>> Hi, all
>>> >>>
>>> >>>
>>> >>> I would like to start the discussion for supporting Docker Official
>>> Image for Spark.
>>> >>>
>>> >>>
>>> >>> This SPIP is proposed to add Docker Official Image(DOI) to ensure
>>> the Spark Docker images meet the quality standards for Docker images, to
>>> provide these Docker images for users who want to use Apache Spark via
>>> Docker image.
>>> >>>
>>> >>>
>>> >>> There are also several Apache projects that release the Docker
>>> Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+
>>> to 1B+ download for each). From the huge download statistics, we can see
>>> the real demands of users, and from the support of other apache projects,
>>> we should also be able to do it.
>>> >>>
>>> >>>
>>> >>> After support:
>>> >>>
>>> >>> The Dockerfile will still be maintained by the Apache Spark
>>> community and reviewed by Docker.
>>> >>>
>>> >>> The images will be maintained by the Docker community to ensure the
>>> quality standards for Docker images of the Docker community.
>>> >>>
>>> >>>
>>> >>> It will also reduce the extra docker images maintenance effort (such
>>> as frequently rebuilding, image security update) of the Apache Spark
>>> community.
>>> >>>
>>> >>>
>>> >>> See more in SPIP DOC:
>>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>> >>>
>>> >>>
>>> >>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>> >>>
>>> >>>
>>> >>> Regards,
>>> >>> Yikun
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by bo zhaobo <bz...@gmail.com>.
+1 (non-binding)
This will bring the good experience to customers. So excited about this. ;-)
Yuming Wang <wg...@gmail.com> 于2022年9月19日周一 10:18写道：
> +1.
>
> On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <ya...@apache.org> wrote:
>
>> +1
>>
>> Gengliang Wang <lt...@gmail.com> 于2022年9月19日周一 09:23写道：
>> >
>> > +1, thanks for the work!
>> >
>> > On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com>
>> wrote:
>> >>
>> >> +1
>> >>
>> >> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com>
>> wrote:
>> >>>
>> >>> Hi, all
>> >>>
>> >>>
>> >>> I would like to start the discussion for supporting Docker Official
>> Image for Spark.
>> >>>
>> >>>
>> >>> This SPIP is proposed to add Docker Official Image(DOI) to ensure the
>> Spark Docker images meet the quality standards for Docker images, to
>> provide these Docker images for users who want to use Apache Spark via
>> Docker image.
>> >>>
>> >>>
>> >>> There are also several Apache projects that release the Docker
>> Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+
>> to 1B+ download for each). From the huge download statistics, we can see
>> the real demands of users, and from the support of other apache projects,
>> we should also be able to do it.
>> >>>
>> >>>
>> >>> After support:
>> >>>
>> >>> The Dockerfile will still be maintained by the Apache Spark community
>> and reviewed by Docker.
>> >>>
>> >>> The images will be maintained by the Docker community to ensure the
>> quality standards for Docker images of the Docker community.
>> >>>
>> >>>
>> >>> It will also reduce the extra docker images maintenance effort (such
>> as frequently rebuilding, image security update) of the Apache Spark
>> community.
>> >>>
>> >>>
>> >>> See more in SPIP DOC:
>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>> >>>
>> >>>
>> >>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>> >>>
>> >>>
>> >>> Regards,
>> >>> Yikun
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Yuming Wang <wg...@gmail.com>.
+1.
On Mon, Sep 19, 2022 at 9:44 AM Kent Yao <ya...@apache.org> wrote:
> +1
>
> Gengliang Wang <lt...@gmail.com> 于2022年9月19日周一 09:23写道：
> >
> > +1, thanks for the work!
> >
> > On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com>
> wrote:
> >>
> >> +1
> >>
> >> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com> wrote:
> >>>
> >>> Hi, all
> >>>
> >>>
> >>> I would like to start the discussion for supporting Docker Official
> Image for Spark.
> >>>
> >>>
> >>> This SPIP is proposed to add Docker Official Image(DOI) to ensure the
> Spark Docker images meet the quality standards for Docker images, to
> provide these Docker images for users who want to use Apache Spark via
> Docker image.
> >>>
> >>>
> >>> There are also several Apache projects that release the Docker
> Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+
> to 1B+ download for each). From the huge download statistics, we can see
> the real demands of users, and from the support of other apache projects,
> we should also be able to do it.
> >>>
> >>>
> >>> After support:
> >>>
> >>> The Dockerfile will still be maintained by the Apache Spark community
> and reviewed by Docker.
> >>>
> >>> The images will be maintained by the Docker community to ensure the
> quality standards for Docker images of the Docker community.
> >>>
> >>>
> >>> It will also reduce the extra docker images maintenance effort (such
> as frequently rebuilding, image security update) of the Apache Spark
> community.
> >>>
> >>>
> >>> See more in SPIP DOC:
> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
> >>>
> >>>
> >>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
> >>>
> >>>
> >>> Regards,
> >>> Yikun
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Kent Yao <ya...@apache.org>.
+1
Gengliang Wang <lt...@gmail.com> 于2022年9月19日周一 09:23写道：
>
> +1, thanks for the work!
>
> On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com> wrote:
>>
>> +1
>>
>> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com> wrote:
>>>
>>> Hi, all
>>>
>>>
>>> I would like to start the discussion for supporting Docker Official Image for Spark.
>>>
>>>
>>> This SPIP is proposed to add Docker Official Image(DOI) to ensure the Spark Docker images meet the quality standards for Docker images, to provide these Docker images for users who want to use Apache Spark via Docker image.
>>>
>>>
>>> There are also several Apache projects that release the Docker Official Images, such as: flink, storm, solr, zookeeper, httpd (with 50M+ to 1B+ download for each). From the huge download statistics, we can see the real demands of users, and from the support of other apache projects, we should also be able to do it.
>>>
>>>
>>> After support:
>>>
>>> The Dockerfile will still be maintained by the Apache Spark community and reviewed by Docker.
>>>
>>> The images will be maintained by the Docker community to ensure the quality standards for Docker images of the Docker community.
>>>
>>>
>>> It will also reduce the extra docker images maintenance effort (such as frequently rebuilding, image security update) of the Apache Spark community.
>>>
>>>
>>> See more in SPIP DOC: https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>>
>>>
>>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>>
>>>
>>> Regards,
>>> Yikun
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Gengliang Wang <lt...@gmail.com>.
+1, thanks for the work!
On Sun, Sep 18, 2022 at 6:20 PM Hyukjin Kwon <gu...@gmail.com> wrote:
> +1
>
> On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com> wrote:
>
>> Hi, all
>>
>> I would like to start the discussion for supporting Docker Official Image
>> for Spark.
>>
>> This SPIP is proposed to add Docker Official Image(DOI)
>> <https://github.com/docker-library/official-images> to ensure the Spark
>> Docker images meet the quality standards for Docker images, to provide
>> these Docker images for users who want to use Apache Spark via Docker image.
>>
>> There are also several Apache projects that release the Docker Official
>> Images <https://hub.docker.com/search?q=apache&image_filter=official>,
>> such as: flink <https://hub.docker.com/_/flink>, storm
>> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
>> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
>> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each).
>> From the huge download statistics, we can see the real demands of users,
>> and from the support of other apache projects, we should also be able to do
>> it.
>>
>> After support:
>>
>> -
>>
>> The Dockerfile will still be maintained by the Apache Spark community
>> and reviewed by Docker.
>> -
>>
>> The images will be maintained by the Docker community to ensure the
>> quality standards for Docker images of the Docker community.
>>
>>
>> It will also reduce the extra docker images maintenance effort (such as
>> frequently rebuilding, image security update) of the Apache Spark community.
>>
>> See more in SPIP DOC:
>> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>>
>> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>>
>> Regards,
>> Yikun
>>
>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Hyukjin Kwon <gu...@gmail.com>.
+1
On Mon, 19 Sept 2022 at 09:15, Yikun Jiang <yi...@gmail.com> wrote:
> Hi, all
>
> I would like to start the discussion for supporting Docker Official Image
> for Spark.
>
> This SPIP is proposed to add Docker Official Image(DOI)
> <https://github.com/docker-library/official-images> to ensure the Spark
> Docker images meet the quality standards for Docker images, to provide
> these Docker images for users who want to use Apache Spark via Docker image.
>
> There are also several Apache projects that release the Docker Official
> Images <https://hub.docker.com/search?q=apache&image_filter=official>,
> such as: flink <https://hub.docker.com/_/flink>, storm
> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each).
> From the huge download statistics, we can see the real demands of users,
> and from the support of other apache projects, we should also be able to do
> it.
>
> After support:
>
> -
>
> The Dockerfile will still be maintained by the Apache Spark community
> and reviewed by Docker.
> -
>
> The images will be maintained by the Docker community to ensure the
> quality standards for Docker images of the Docker community.
>
>
> It will also reduce the extra docker images maintenance effort (such as
> frequently rebuilding, image security update) of the Apache Spark community.
>
> See more in SPIP DOC:
> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>
> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>
> Regards,
> Yikun
>
Re: [DISCUSS] SPIP: Support Docker Official Image for Spark
Posted by Martin Grigorov <mg...@apache.org>.
+1
Good idea!
Martin
On Mon, Sep 19, 2022 at 3:16 AM Yikun Jiang <yi...@gmail.com> wrote:
> Hi, all
>
> I would like to start the discussion for supporting Docker Official Image
> for Spark.
>
> This SPIP is proposed to add Docker Official Image(DOI)
> <https://github.com/docker-library/official-images> to ensure the Spark
> Docker images meet the quality standards for Docker images, to provide
> these Docker images for users who want to use Apache Spark via Docker image.
>
> There are also several Apache projects that release the Docker Official
> Images <https://hub.docker.com/search?q=apache&image_filter=official>,
> such as: flink <https://hub.docker.com/_/flink>, storm
> <https://hub.docker.com/_/storm>, solr <https://hub.docker.com/_/solr>,
> zookeeper <https://hub.docker.com/_/zookeeper>, httpd
> <https://hub.docker.com/_/httpd> (with 50M+ to 1B+ download for each).
> From the huge download statistics, we can see the real demands of users,
> and from the support of other apache projects, we should also be able to do
> it.
>
> After support:
>
> -
>
> The Dockerfile will still be maintained by the Apache Spark community
> and reviewed by Docker.
> -
>
> The images will be maintained by the Docker community to ensure the
> quality standards for Docker images of the Docker community.
>
>
> It will also reduce the extra docker images maintenance effort (such as
> frequently rebuilding, image security update) of the Apache Spark community.
>
> See more in SPIP DOC:
> https://docs.google.com/document/d/1nN-pKuvt-amUcrkTvYAQ-bJBgtsWb9nAkNoVNRM2S2o
>
> cc: Ruifeng (co-author) and Hyukjin (shepherd)
>
> Regards,
> Yikun
>