You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Robin Moffatt via dev <de...@iceberg.apache.org> on 2026/01/22 18:21:53 UTC
[DISCUSS] Add a Flink quickstart image
Hi,
Following discussion on the Flink quickstart PR [1], what do people think
about adding an official quickstart Docker image for Flink to the project?
At the moment the Spark quickstart uses tabulario/spark-iceberg so perhaps
that could be brought into the project too.
thanks, Robin.
1: https://github.com/apache/iceberg/pull/15062
Re: [DISCUSS] Add a Flink quickstart image
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi
I will take a look on the PR, especially from a "legal" standpoint
(LICENSE/NOTICE, etc).
Regards
JB
On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
dev@iceberg.apache.org> wrote:
> Hi,
>
> Thanks for the discussion and input.
> It sounds like there are no major blockers. Could someone please review
> https://github.com/apache/iceberg/pull/15124 ?
>
> thanks,
>
> Robin.
>
> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
>
>> Hey folks,
>>
>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark base
>> image and installs all the necessary jars. This is used for our integration
>> test setup [2] and is inspired by databricks/docker-spark-iceberg [3].
>> We've made many improvements such as upgrading to Spark 4, supporting Spark
>> Connect, and better image build caching.
>>
>> This is already self-contained and can be reused by other subprojects. In
>> fact, iceberg-rust already uses it [4] and I try to keep them in sync.
>> I think it would be beneficial for the project to publish this image and
>> something similar for Flink.
>>
>> Let me know what you think.
>>
>> Best,
>> Kevin Liu
>>
>>
>>
>> [1]
>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>> [2]
>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>> [3]
>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>> [4]
>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>
>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com> wrote:
>>
>>> > Since the integration code for both Spark and Flink lives in our
>>> repository, it might make sense to also store the Docker images and the
>>> corresponding scripts there.
>>>
>>> I agree with Peter here.
>>>
>>> The previous thread has some concerns if the Iceberg project should host
>>> those docker images. Not sure if the opinions have changed.
>>>
>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>>
>>>> Thanks Ajantha, I'd not seen that thread.
>>>> Having looked at it, am I understanding the view to be that ideally
>>>> Flink would publish a Docker image that included the Iceberg dependencies?
>>>>
>>>> However we do this, I feel that the user coming to run the Flink
>>>> quickstart should not have to build their own Docker image; this adds
>>>> unnecessary friction that is easily alleviated.
>>>>
>>>> If I've understood the situation correctly, then I'm happy to discuss
>>>> this idea with the Flink community; please let me know before I do so.
>>>>
>>>> thanks, Robin.
>>>>
>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Robin and Peter,
>>>>>
>>>>> I discussed community-maintained Docker images previously:
>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>
>>>>> The consensus was to publish only the REST fixture Docker image
>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
>>>>> total downloads) and use Docker images published by the main engines in the
>>>>> quickstart, instead of maintaining these images ourselves.
>>>>> See the thread above for more details.
>>>>>
>>>>> With respect to adding a Flink quickstart page, I’m in favor of adding
>>>>> it and relying on the Docker images provided by Flink rather than
>>>>> maintaining our own images.
>>>>> - Ajantha
>>>>>
>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>
>>>>>> Hi Robin,
>>>>>> It would be nice to separate them. I expect that we will have some
>>>>>> extra stuff to do with the docker image. For example make sure that we have
>>>>>> ci in place to build it.
>>>>>> Thanks,
>>>>>> Peter
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>
>>>>>>> Thanks for the positive reception of this idea.
>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>
>>>>>>> Also, should I keep this and the quickstart PR [2] as separate PRs,
>>>>>>> or combine them?
>>>>>>>
>>>>>>> thanks, Robin.
>>>>>>>
>>>>>>>
>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>
>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> This is a great idea.
>>>>>>>>
>>>>>>>> If we are moving forward with an "official" Docker image published
>>>>>>>> by the project, we must ensure it is fully compliant with ASF requirements
>>>>>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>>>>>> it is a detail that is often overlooked.
>>>>>>>>
>>>>>>>> I would be happy to help with this process.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> JB
>>>>>>>>
>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey Robin,
>>>>>>>>>
>>>>>>>>> +1 That's a great idea. It's often a bit painful for new users to
>>>>>>>>> get
>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>
>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>
>>>>>>>>> -Max
>>>>>>>>>
>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi Robin,
>>>>>>>>> >
>>>>>>>>> > I would love to see the Flink quickstart image in the Iceberg
>>>>>>>>> repo.
>>>>>>>>> >
>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>> > The conclusion was:
>>>>>>>>> >>
>>>>>>>>> >> we should both remove the vendor reference and bring this back
>>>>>>>>> up to date. My preference would be to rely on the Spark image <
>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>>>>>> Spark project, similar to what we do for the Hive <
>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We
>>>>>>>>> should be able to load all the Iceberg-specific JARs through the
>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>> >
>>>>>>>>> > Thanks for working on this,
>>>>>>>>> > Peter
>>>>>>>>> >
>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>> >>
>>>>>>>>> >> Hi,
>>>>>>>>> >>
>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>>>>> the project?
>>>>>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg
>>>>>>>>> so perhaps that could be brought into the project too.
>>>>>>>>> >>
>>>>>>>>> >> thanks, Robin.
>>>>>>>>> >>
>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Kevin Liu <ke...@apache.org>.
Let's start a new discussion thread about publishing docker images. I see
several existing threads on the same topic already.
On Thu, Feb 19, 2026 at 9:20 AM Robin Moffatt via dev <
dev@iceberg.apache.org> wrote:
> Yay, great news. Thank you for everyone's help with this.
> We discussed the idea of publishing the new docker image - can someone
> guide me on what the process would look like here for that? I'm happy to
> have a go at the PR.
>
> thanks, Robin.
>
> On Thu, 19 Feb 2026 at 16:22, Kevin Liu <ke...@apache.org> wrote:
>
>> https://github.com/apache/iceberg/pull/15062 is merged
>> The Flink Quickstart is live at
>> https://iceberg.apache.org/docs/nightly/flink/
>>
>> Thanks everyone!
>>
>> On Wed, Feb 18, 2026 at 9:38 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi
>>>
>>> I did a new pass, thanks !
>>>
>>> Regards
>>> JB
>>>
>>> On Wed, Feb 18, 2026 at 5:14 PM Kevin Liu <ke...@apache.org> wrote:
>>>
>>>> Here's the latest version of the Flink Quickstart PR,
>>>> https://github.com/apache/iceberg/pull/15062
>>>> Please take a look!
>>>>
>>>> Let's start a new thread to follow up on publishing the Docker image
>>>> (for Flink and Spark)
>>>>
>>>>
>>>>
>>>> On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <mx...@apache.org>
>>>> wrote:
>>>>
>>>>> Thanks Robin! Really great work.
>>>>>
>>>>> Cheers,
>>>>> Max
>>>>>
>>>>> On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <ke...@apache.org>
>>>>> wrote:
>>>>> >
>>>>> > Thanks again for working on this. I took another look at #15124.
>>>>> Let's try to get this merged, it's very self contained.
>>>>> >
>>>>> > Best,
>>>>> > Kevin Liu
>>>>> >
>>>>> > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>> >>
>>>>> >> (I realised I mistakenly sent this only to Péter instead of the
>>>>> whole list)
>>>>> >>
>>>>> >> Could I get a review on
>>>>> https://github.com/apache/iceberg/pull/15124 please.
>>>>> >> Once that's sorted, I'll update the quickstart docs PR (15062) for
>>>>> the necessary changes.
>>>>> >>
>>>>> >> thanks, Robin.
>>>>> >>
>>>>> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io>
>>>>> wrote:
>>>>> >>>
>>>>> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for
>>>>> review please. Once that's sorted, I'll update the quickstart docs PR
>>>>> (15062) for the necessary changes.
>>>>> >>>
>>>>> >>> thanks, Robin.
>>>>> >>>
>>>>> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>> >>>>
>>>>> >>>> In there are no objections then I would prefer it in the
>>>>> docker/iceberg-flink-quickstart
>>>>> >>>>
>>>>> >>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>> (időpont: 2026. febr. 6., P, 11:13):
>>>>> >>>>>
>>>>> >>>>> Hi Peter,
>>>>> >>>>>
>>>>> >>>>> Thanks for the direction. I'll remove the publish step so that
>>>>> we can get the quickstart published, and then work on the publishing
>>>>> subsequently.
>>>>> >>>>>
>>>>> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or
>>>>> docker/iceberg-flink-quickstart ?
>>>>> >>>>>
>>>>> >>>>> thanks, Robin
>>>>> >>>>>
>>>>> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>> >>>>>>
>>>>> >>>>>> I think we have two options:
>>>>> >>>>>>
>>>>> >>>>>> Remove the image publication from this PR (
>>>>> https://github.com/apache/iceberg/pull/15124) for now, and proceed
>>>>> with adding the Docker image and updating the documentation.
>>>>> >>>>>> Alternatively, we could discuss publishing the Flink quickstart
>>>>> image at the next Iceberg Community Sync and use that as an opportunity to
>>>>> simplify both the documentation and the overall user experience.
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>>
>>>>> >>>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>> (időpont: 2026. febr. 4., Sze, 18:52):
>>>>> >>>>>>>
>>>>> >>>>>>> Hi,
>>>>> >>>>>>>
>>>>> >>>>>>> I have perhaps managed to deadlock this process :) I'd
>>>>> appreciate some help untangling it. The recap is in my previous email
>>>>> (below).
>>>>> >>>>>>>
>>>>> >>>>>>> thanks, Robin.
>>>>> >>>>>>>
>>>>> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <
>>>>> robin@confluent.io> wrote:
>>>>> >>>>>>>>
>>>>> >>>>>>>> Hi Kevin,
>>>>> >>>>>>>>
>>>>> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>>>> >>>>>>>> I have two related PRs:
>>>>> >>>>>>>>
>>>>> >>>>>>>> #15124: Add Flink Quickstart docker image
>>>>> >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile
>>>>> too)
>>>>> >>>>>>>>
>>>>> >>>>>>>> I can see a few routes forward:
>>>>> >>>>>>>>
>>>>> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy
>>>>> with the publish script (I've not seen anything raised about it yet tho?)
>>>>> >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still
>>>>> relying on local image build (not sure what this would achieve vs the
>>>>> option above tho?)
>>>>> >>>>>>>> 3. Merge #15124 including publish script, then #15062 using
>>>>> the published image not the local build
>>>>> >>>>>>>>
>>>>> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile
>>>>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart
>>>>> (#15124).
>>>>> >>>>>>>>
>>>>> >>>>>>>> LMK if I've missed an angle here.
>>>>> >>>>>>>>
>>>>> >>>>>>>> thanks, Robin
>>>>> >>>>>>>>
>>>>> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <
>>>>> kevinjqliu@apache.org> wrote:
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Thanks for working on this, Robin! It looks like the
>>>>> complexity here is publishing the docker image. What do you think about
>>>>> isolating that part? (Just move the publish script out of #15124) We can
>>>>> start with the Dockerfile definition, which allows us to build locally.
>>>>> This should unblock us from merging the getting started docs in #15062
>>>>> >>>>>>>>> Thoughts?
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> Best,
>>>>> >>>>>>>>> Kevin Liu
>>>>> >>>>>>>>>
>>>>> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> Hi,
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> Thanks for the discussion and input.
>>>>> >>>>>>>>>> It sounds like there are no major blockers. Could someone
>>>>> please review https://github.com/apache/iceberg/pull/15124 ?
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> thanks,
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> Robin.
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <
>>>>> kevinjqliu@apache.org> wrote:
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Hey folks,
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses
>>>>> the Spark base image and installs all the necessary jars. This is used for
>>>>> our integration test setup [2] and is inspired by
>>>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>>>>> upgrading to Spark 4, supporting Spark Connect, and better image build
>>>>> caching.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> This is already self-contained and can be reused by other
>>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>>>>> them in sync.
>>>>> >>>>>>>>>>> I think it would be beneficial for the project to publish
>>>>> this image and something similar for Flink.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Let me know what you think.
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> Best,
>>>>> >>>>>>>>>>> Kevin Liu
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> [1]
>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>>> >>>>>>>>>>> [2]
>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>>> >>>>>>>>>>> [3]
>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>>> >>>>>>>>>>> [4]
>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>> >>>>>>>>>>>
>>>>> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <
>>>>> stevenz3wu@gmail.com> wrote:
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> > Since the integration code for both Spark and Flink
>>>>> lives in our repository, it might make sense to also store the Docker
>>>>> images and the corresponding scripts there.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> I agree with Peter here.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg
>>>>> project should host those docker images. Not sure if the opinions have
>>>>> changed.
>>>>> >>>>>>>>>>>>
>>>>> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>> >>>>>>>>>>>>> Having looked at it, am I understanding the view to be
>>>>> that ideally Flink would publish a Docker image that included the Iceberg
>>>>> dependencies?
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> However we do this, I feel that the user coming to run
>>>>> the Flink quickstart should not have to build their own Docker image; this
>>>>> adds unnecessary friction that is easily alleviated.
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm
>>>>> happy to discuss this idea with the Flink community; please let me know
>>>>> before I do so.
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> thanks, Robin.
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <
>>>>> ajanthabhat@gmail.com> wrote:
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> Hi Robin and Peter,
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> I discussed community-maintained Docker images
>>>>> previously:
>>>>> >>>>>>>>>>>>>>
>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture
>>>>> Docker image (now at 100K+ total downloads) and use Docker images published
>>>>> by the main engines in the quickstart, instead of maintaining these images
>>>>> ourselves.
>>>>> >>>>>>>>>>>>>> See the thread above for more details.
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in
>>>>> favor of adding it and relying on the Docker images provided by Flink
>>>>> rather than maintaining our own images.
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> - Ajantha
>>>>> >>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> Hi Robin,
>>>>> >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we
>>>>> will have some extra stuff to do with the docker image. For example make
>>>>> sure that we have ci in place to build it.
>>>>> >>>>>>>>>>>>>>> Thanks,
>>>>> >>>>>>>>>>>>>>> Peter
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
>>>>> >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as
>>>>> separate PRs, or combine them?
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> thanks, Robin.
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>> >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>>> jb@nanthrax.net> wrote:
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> Hi,
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> This is a great idea.
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker
>>>>> image published by the project, we must ensure it is fully compliant with
>>>>> ASF requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>>> straightforward, it is a detail that is often overlooked.
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> I would be happy to help with this process.
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> Regards,
>>>>> >>>>>>>>>>>>>>>>> JB
>>>>> >>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>>> mxm@apache.org> wrote:
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> Hey Robin,
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful
>>>>> for new users to get
>>>>> >>>>>>>>>>>>>>>>>> all the dependencies in the right place.
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker
>>>>> images:
>>>>> >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> -Max
>>>>> >>>>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>> >>>>>>>>>>>>>>>>>> >
>>>>> >>>>>>>>>>>>>>>>>> > Hi Robin,
>>>>> >>>>>>>>>>>>>>>>>> >
>>>>> >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in
>>>>> the Iceberg repo.
>>>>> >>>>>>>>>>>>>>>>>> >
>>>>> >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>> https://github.com/apache/iceberg/issues/13519
>>>>> >>>>>>>>>>>>>>>>>> > The conclusion was:
>>>>> >>>>>>>>>>>>>>>>>> >>
>>>>> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and
>>>>> bring this back up to date. My preference would be to rely on the Spark
>>>>> image <https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>> Spark project, similar to what we do for the Hive <
>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>>>>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>>>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html
>>>>> >.
>>>>> >>>>>>>>>>>>>>>>>> >
>>>>> >>>>>>>>>>>>>>>>>> >
>>>>> >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list
>>>>> thread:
>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>> >>>>>>>>>>>>>>>>>> >
>>>>> >>>>>>>>>>>>>>>>>> > Thanks for working on this,
>>>>> >>>>>>>>>>>>>>>>>> > Peter
>>>>> >>>>>>>>>>>>>>>>>> >
>>>>> >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org>
>>>>> ezt írta (időpont: 2026. jan. 22., Cs, 19:23):
>>>>> >>>>>>>>>>>>>>>>>> >>
>>>>> >>>>>>>>>>>>>>>>>> >> Hi,
>>>>> >>>>>>>>>>>>>>>>>> >>
>>>>> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR
>>>>> [1], what do people think about adding an official quickstart Docker image
>>>>> for Flink to the project?
>>>>> >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>> too.
>>>>> >>>>>>>>>>>>>>>>>> >>
>>>>> >>>>>>>>>>>>>>>>>> >> thanks, Robin.
>>>>> >>>>>>>>>>>>>>>>>> >>
>>>>> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>>>>
>>>>> >>>>>>>>>>>>>
>>>>> >>>>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>>
>>>>> >>>>>>>
>>>>> >>>>>
>>>>> >>>
>>>>> >>>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >> Robin Moffatt
>>>>> >> Sr. Principal Advisor, Streaming Data Technologies
>>>>>
>>>>
>
> --
> *Robin Moffatt*
> *Sr. Principal Advisor, Streaming Data Technologies*
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
Yay, great news. Thank you for everyone's help with this.
We discussed the idea of publishing the new docker image - can someone
guide me on what the process would look like here for that? I'm happy to
have a go at the PR.
thanks, Robin.
On Thu, 19 Feb 2026 at 16:22, Kevin Liu <ke...@apache.org> wrote:
> https://github.com/apache/iceberg/pull/15062 is merged
> The Flink Quickstart is live at
> https://iceberg.apache.org/docs/nightly/flink/
>
> Thanks everyone!
>
> On Wed, Feb 18, 2026 at 9:38 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi
>>
>> I did a new pass, thanks !
>>
>> Regards
>> JB
>>
>> On Wed, Feb 18, 2026 at 5:14 PM Kevin Liu <ke...@apache.org> wrote:
>>
>>> Here's the latest version of the Flink Quickstart PR,
>>> https://github.com/apache/iceberg/pull/15062
>>> Please take a look!
>>>
>>> Let's start a new thread to follow up on publishing the Docker image
>>> (for Flink and Spark)
>>>
>>>
>>>
>>> On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <mx...@apache.org>
>>> wrote:
>>>
>>>> Thanks Robin! Really great work.
>>>>
>>>> Cheers,
>>>> Max
>>>>
>>>> On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <ke...@apache.org>
>>>> wrote:
>>>> >
>>>> > Thanks again for working on this. I took another look at #15124.
>>>> Let's try to get this merged, it's very self contained.
>>>> >
>>>> > Best,
>>>> > Kevin Liu
>>>> >
>>>> > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>> >>
>>>> >> (I realised I mistakenly sent this only to Péter instead of the
>>>> whole list)
>>>> >>
>>>> >> Could I get a review on https://github.com/apache/iceberg/pull/15124
>>>> please.
>>>> >> Once that's sorted, I'll update the quickstart docs PR (15062) for
>>>> the necessary changes.
>>>> >>
>>>> >> thanks, Robin.
>>>> >>
>>>> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io>
>>>> wrote:
>>>> >>>
>>>> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for
>>>> review please. Once that's sorted, I'll update the quickstart docs PR
>>>> (15062) for the necessary changes.
>>>> >>>
>>>> >>> thanks, Robin.
>>>> >>>
>>>> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <
>>>> peter.vary.apache@gmail.com> wrote:
>>>> >>>>
>>>> >>>> In there are no objections then I would prefer it in the
>>>> docker/iceberg-flink-quickstart
>>>> >>>>
>>>> >>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>> 2026. febr. 6., P, 11:13):
>>>> >>>>>
>>>> >>>>> Hi Peter,
>>>> >>>>>
>>>> >>>>> Thanks for the direction. I'll remove the publish step so that we
>>>> can get the quickstart published, and then work on the publishing
>>>> subsequently.
>>>> >>>>>
>>>> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or
>>>> docker/iceberg-flink-quickstart ?
>>>> >>>>>
>>>> >>>>> thanks, Robin
>>>> >>>>>
>>>> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <
>>>> peter.vary.apache@gmail.com> wrote:
>>>> >>>>>>
>>>> >>>>>> I think we have two options:
>>>> >>>>>>
>>>> >>>>>> Remove the image publication from this PR (
>>>> https://github.com/apache/iceberg/pull/15124) for now, and proceed
>>>> with adding the Docker image and updating the documentation.
>>>> >>>>>> Alternatively, we could discuss publishing the Flink quickstart
>>>> image at the next Iceberg Community Sync and use that as an opportunity to
>>>> simplify both the documentation and the overall user experience.
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>>
>>>> >>>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>> (időpont: 2026. febr. 4., Sze, 18:52):
>>>> >>>>>>>
>>>> >>>>>>> Hi,
>>>> >>>>>>>
>>>> >>>>>>> I have perhaps managed to deadlock this process :) I'd
>>>> appreciate some help untangling it. The recap is in my previous email
>>>> (below).
>>>> >>>>>>>
>>>> >>>>>>> thanks, Robin.
>>>> >>>>>>>
>>>> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io>
>>>> wrote:
>>>> >>>>>>>>
>>>> >>>>>>>> Hi Kevin,
>>>> >>>>>>>>
>>>> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>>> >>>>>>>> I have two related PRs:
>>>> >>>>>>>>
>>>> >>>>>>>> #15124: Add Flink Quickstart docker image
>>>> >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile
>>>> too)
>>>> >>>>>>>>
>>>> >>>>>>>> I can see a few routes forward:
>>>> >>>>>>>>
>>>> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with
>>>> the publish script (I've not seen anything raised about it yet tho?)
>>>> >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still
>>>> relying on local image build (not sure what this would achieve vs the
>>>> option above tho?)
>>>> >>>>>>>> 3. Merge #15124 including publish script, then #15062 using
>>>> the published image not the local build
>>>> >>>>>>>>
>>>> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile
>>>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart
>>>> (#15124).
>>>> >>>>>>>>
>>>> >>>>>>>> LMK if I've missed an angle here.
>>>> >>>>>>>>
>>>> >>>>>>>> thanks, Robin
>>>> >>>>>>>>
>>>> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org>
>>>> wrote:
>>>> >>>>>>>>>
>>>> >>>>>>>>> Thanks for working on this, Robin! It looks like the
>>>> complexity here is publishing the docker image. What do you think about
>>>> isolating that part? (Just move the publish script out of #15124) We can
>>>> start with the Dockerfile definition, which allows us to build locally.
>>>> This should unblock us from merging the getting started docs in #15062
>>>> >>>>>>>>> Thoughts?
>>>> >>>>>>>>>
>>>> >>>>>>>>> Best,
>>>> >>>>>>>>> Kevin Liu
>>>> >>>>>>>>>
>>>> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Hi,
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Thanks for the discussion and input.
>>>> >>>>>>>>>> It sounds like there are no major blockers. Could someone
>>>> please review https://github.com/apache/iceberg/pull/15124 ?
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> thanks,
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> Robin.
>>>> >>>>>>>>>>
>>>> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <
>>>> kevinjqliu@apache.org> wrote:
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Hey folks,
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the
>>>> Spark base image and installs all the necessary jars. This is used for our
>>>> integration test setup [2] and is inspired by
>>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>>>> upgrading to Spark 4, supporting Spark Connect, and better image build
>>>> caching.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> This is already self-contained and can be reused by other
>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>>>> them in sync.
>>>> >>>>>>>>>>> I think it would be beneficial for the project to publish
>>>> this image and something similar for Flink.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Let me know what you think.
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> Best,
>>>> >>>>>>>>>>> Kevin Liu
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> [1]
>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>> >>>>>>>>>>> [2]
>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>> >>>>>>>>>>> [3]
>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>> >>>>>>>>>>> [4]
>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>> >>>>>>>>>>>
>>>> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <
>>>> stevenz3wu@gmail.com> wrote:
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> > Since the integration code for both Spark and Flink
>>>> lives in our repository, it might make sense to also store the Docker
>>>> images and the corresponding scripts there.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> I agree with Peter here.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg
>>>> project should host those docker images. Not sure if the opinions have
>>>> changed.
>>>> >>>>>>>>>>>>
>>>> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>> >>>>>>>>>>>>> Having looked at it, am I understanding the view to be
>>>> that ideally Flink would publish a Docker image that included the Iceberg
>>>> dependencies?
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> However we do this, I feel that the user coming to run
>>>> the Flink quickstart should not have to build their own Docker image; this
>>>> adds unnecessary friction that is easily alleviated.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm
>>>> happy to discuss this idea with the Flink community; please let me know
>>>> before I do so.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> thanks, Robin.
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <
>>>> ajanthabhat@gmail.com> wrote:
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> Hi Robin and Peter,
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> I discussed community-maintained Docker images
>>>> previously:
>>>> >>>>>>>>>>>>>>
>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture
>>>> Docker image (now at 100K+ total downloads) and use Docker images published
>>>> by the main engines in the quickstart, instead of maintaining these images
>>>> ourselves.
>>>> >>>>>>>>>>>>>> See the thread above for more details.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in
>>>> favor of adding it and relying on the Docker images provided by Flink
>>>> rather than maintaining our own images.
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> - Ajantha
>>>> >>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>> peter.vary.apache@gmail.com> wrote:
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> Hi Robin,
>>>> >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we
>>>> will have some extra stuff to do with the docker image. For example make
>>>> sure that we have ci in place to build it.
>>>> >>>>>>>>>>>>>>> Thanks,
>>>> >>>>>>>>>>>>>>> Peter
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
>>>> >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as
>>>> separate PRs, or combine them?
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> thanks, Robin.
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>> >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>> jb@nanthrax.net> wrote:
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Hi,
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> This is a great idea.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker
>>>> image published by the project, we must ensure it is fully compliant with
>>>> ASF requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>> straightforward, it is a detail that is often overlooked.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> I would be happy to help with this process.
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> Regards,
>>>> >>>>>>>>>>>>>>>>> JB
>>>> >>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>> mxm@apache.org> wrote:
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> Hey Robin,
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for
>>>> new users to get
>>>> >>>>>>>>>>>>>>>>>> all the dependencies in the right place.
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker
>>>> images:
>>>> >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> -Max
>>>> >>>>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>> peter.vary.apache@gmail.com> wrote:
>>>> >>>>>>>>>>>>>>>>>> >
>>>> >>>>>>>>>>>>>>>>>> > Hi Robin,
>>>> >>>>>>>>>>>>>>>>>> >
>>>> >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in
>>>> the Iceberg repo.
>>>> >>>>>>>>>>>>>>>>>> >
>>>> >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>> https://github.com/apache/iceberg/issues/13519
>>>> >>>>>>>>>>>>>>>>>> > The conclusion was:
>>>> >>>>>>>>>>>>>>>>>> >>
>>>> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and
>>>> bring this back up to date. My preference would be to rely on the Spark
>>>> image <https://hub.docker.com/r/apache/spark> provided by the Apache
>>>> Spark project, similar to what we do for the Hive <
>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>>>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>> >>>>>>>>>>>>>>>>>> >
>>>> >>>>>>>>>>>>>>>>>> >
>>>> >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list
>>>> thread:
>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>> >>>>>>>>>>>>>>>>>> >
>>>> >>>>>>>>>>>>>>>>>> > Thanks for working on this,
>>>> >>>>>>>>>>>>>>>>>> > Peter
>>>> >>>>>>>>>>>>>>>>>> >
>>>> >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org>
>>>> ezt írta (időpont: 2026. jan. 22., Cs, 19:23):
>>>> >>>>>>>>>>>>>>>>>> >>
>>>> >>>>>>>>>>>>>>>>>> >> Hi,
>>>> >>>>>>>>>>>>>>>>>> >>
>>>> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR
>>>> [1], what do people think about adding an official quickstart Docker image
>>>> for Flink to the project?
>>>> >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>> too.
>>>> >>>>>>>>>>>>>>>>>> >>
>>>> >>>>>>>>>>>>>>>>>> >> thanks, Robin.
>>>> >>>>>>>>>>>>>>>>>> >>
>>>> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>>>>
>>>> >>>>>>>>>>>>>
>>>> >>>>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>>
>>>> >>>>>>>
>>>> >>>>>
>>>> >>>
>>>> >>>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Robin Moffatt
>>>> >> Sr. Principal Advisor, Streaming Data Technologies
>>>>
>>>
--
*Robin Moffatt*
*Sr. Principal Advisor, Streaming Data Technologies*
Re: [DISCUSS] Add a Flink quickstart image
Posted by Kevin Liu <ke...@apache.org>.
https://github.com/apache/iceberg/pull/15062 is merged
The Flink Quickstart is live at
https://iceberg.apache.org/docs/nightly/flink/
Thanks everyone!
On Wed, Feb 18, 2026 at 9:38 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi
>
> I did a new pass, thanks !
>
> Regards
> JB
>
> On Wed, Feb 18, 2026 at 5:14 PM Kevin Liu <ke...@apache.org> wrote:
>
>> Here's the latest version of the Flink Quickstart PR,
>> https://github.com/apache/iceberg/pull/15062
>> Please take a look!
>>
>> Let's start a new thread to follow up on publishing the Docker image (for
>> Flink and Spark)
>>
>>
>>
>> On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <mx...@apache.org>
>> wrote:
>>
>>> Thanks Robin! Really great work.
>>>
>>> Cheers,
>>> Max
>>>
>>> On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <ke...@apache.org> wrote:
>>> >
>>> > Thanks again for working on this. I took another look at #15124. Let's
>>> try to get this merged, it's very self contained.
>>> >
>>> > Best,
>>> > Kevin Liu
>>> >
>>> > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>> >>
>>> >> (I realised I mistakenly sent this only to Péter instead of the whole
>>> list)
>>> >>
>>> >> Could I get a review on https://github.com/apache/iceberg/pull/15124
>>> please.
>>> >> Once that's sorted, I'll update the quickstart docs PR (15062) for
>>> the necessary changes.
>>> >>
>>> >> thanks, Robin.
>>> >>
>>> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io>
>>> wrote:
>>> >>>
>>> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for
>>> review please. Once that's sorted, I'll update the quickstart docs PR
>>> (15062) for the necessary changes.
>>> >>>
>>> >>> thanks, Robin.
>>> >>>
>>> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <pe...@gmail.com>
>>> wrote:
>>> >>>>
>>> >>>> In there are no objections then I would prefer it in the
>>> docker/iceberg-flink-quickstart
>>> >>>>
>>> >>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>> 2026. febr. 6., P, 11:13):
>>> >>>>>
>>> >>>>> Hi Peter,
>>> >>>>>
>>> >>>>> Thanks for the direction. I'll remove the publish step so that we
>>> can get the quickstart published, and then work on the publishing
>>> subsequently.
>>> >>>>>
>>> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or
>>> docker/iceberg-flink-quickstart ?
>>> >>>>>
>>> >>>>> thanks, Robin
>>> >>>>>
>>> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <
>>> peter.vary.apache@gmail.com> wrote:
>>> >>>>>>
>>> >>>>>> I think we have two options:
>>> >>>>>>
>>> >>>>>> Remove the image publication from this PR (
>>> https://github.com/apache/iceberg/pull/15124) for now, and proceed with
>>> adding the Docker image and updating the documentation.
>>> >>>>>> Alternatively, we could discuss publishing the Flink quickstart
>>> image at the next Iceberg Community Sync and use that as an opportunity to
>>> simplify both the documentation and the overall user experience.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>> (időpont: 2026. febr. 4., Sze, 18:52):
>>> >>>>>>>
>>> >>>>>>> Hi,
>>> >>>>>>>
>>> >>>>>>> I have perhaps managed to deadlock this process :) I'd
>>> appreciate some help untangling it. The recap is in my previous email
>>> (below).
>>> >>>>>>>
>>> >>>>>>> thanks, Robin.
>>> >>>>>>>
>>> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io>
>>> wrote:
>>> >>>>>>>>
>>> >>>>>>>> Hi Kevin,
>>> >>>>>>>>
>>> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>> >>>>>>>> I have two related PRs:
>>> >>>>>>>>
>>> >>>>>>>> #15124: Add Flink Quickstart docker image
>>> >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>>> >>>>>>>>
>>> >>>>>>>> I can see a few routes forward:
>>> >>>>>>>>
>>> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with
>>> the publish script (I've not seen anything raised about it yet tho?)
>>> >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still
>>> relying on local image build (not sure what this would achieve vs the
>>> option above tho?)
>>> >>>>>>>> 3. Merge #15124 including publish script, then #15062 using the
>>> published image not the local build
>>> >>>>>>>>
>>> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile
>>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart
>>> (#15124).
>>> >>>>>>>>
>>> >>>>>>>> LMK if I've missed an angle here.
>>> >>>>>>>>
>>> >>>>>>>> thanks, Robin
>>> >>>>>>>>
>>> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org>
>>> wrote:
>>> >>>>>>>>>
>>> >>>>>>>>> Thanks for working on this, Robin! It looks like the
>>> complexity here is publishing the docker image. What do you think about
>>> isolating that part? (Just move the publish script out of #15124) We can
>>> start with the Dockerfile definition, which allows us to build locally.
>>> This should unblock us from merging the getting started docs in #15062
>>> >>>>>>>>> Thoughts?
>>> >>>>>>>>>
>>> >>>>>>>>> Best,
>>> >>>>>>>>> Kevin Liu
>>> >>>>>>>>>
>>> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> Hi,
>>> >>>>>>>>>>
>>> >>>>>>>>>> Thanks for the discussion and input.
>>> >>>>>>>>>> It sounds like there are no major blockers. Could someone
>>> please review https://github.com/apache/iceberg/pull/15124 ?
>>> >>>>>>>>>>
>>> >>>>>>>>>> thanks,
>>> >>>>>>>>>>
>>> >>>>>>>>>> Robin.
>>> >>>>>>>>>>
>>> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <
>>> kevinjqliu@apache.org> wrote:
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Hey folks,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the
>>> Spark base image and installs all the necessary jars. This is used for our
>>> integration test setup [2] and is inspired by
>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>>> upgrading to Spark 4, supporting Spark Connect, and better image build
>>> caching.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> This is already self-contained and can be reused by other
>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>>> them in sync.
>>> >>>>>>>>>>> I think it would be beneficial for the project to publish
>>> this image and something similar for Flink.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Let me know what you think.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Best,
>>> >>>>>>>>>>> Kevin Liu
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> [1]
>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>> >>>>>>>>>>> [2]
>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>> >>>>>>>>>>> [3]
>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>> >>>>>>>>>>> [4]
>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <
>>> stevenz3wu@gmail.com> wrote:
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> > Since the integration code for both Spark and Flink lives
>>> in our repository, it might make sense to also store the Docker images and
>>> the corresponding scripts there.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I agree with Peter here.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg
>>> project should host those docker images. Not sure if the opinions have
>>> changed.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>> >>>>>>>>>>>>> Having looked at it, am I understanding the view to be
>>> that ideally Flink would publish a Docker image that included the Iceberg
>>> dependencies?
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> However we do this, I feel that the user coming to run the
>>> Flink quickstart should not have to build their own Docker image; this adds
>>> unnecessary friction that is easily alleviated.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy
>>> to discuss this idea with the Flink community; please let me know before I
>>> do so.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> thanks, Robin.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <
>>> ajanthabhat@gmail.com> wrote:
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Hi Robin and Peter,
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> I discussed community-maintained Docker images previously:
>>> >>>>>>>>>>>>>>
>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker
>>> image (now at 100K+ total downloads) and use Docker images published by the
>>> main engines in the quickstart, instead of maintaining these images
>>> ourselves.
>>> >>>>>>>>>>>>>> See the thread above for more details.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in
>>> favor of adding it and relying on the Docker images provided by Flink
>>> rather than maintaining our own images.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> - Ajantha
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>> peter.vary.apache@gmail.com> wrote:
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Hi Robin,
>>> >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will
>>> have some extra stuff to do with the docker image. For example make sure
>>> that we have ci in place to build it.
>>> >>>>>>>>>>>>>>> Thanks,
>>> >>>>>>>>>>>>>>> Peter
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
>>> >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as
>>> separate PRs, or combine them?
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> thanks, Robin.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>> >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>> jb@nanthrax.net> wrote:
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Hi,
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> This is a great idea.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker
>>> image published by the project, we must ensure it is fully compliant with
>>> ASF requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>> straightforward, it is a detail that is often overlooked.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> I would be happy to help with this process.
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Regards,
>>> >>>>>>>>>>>>>>>>> JB
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>> mxm@apache.org> wrote:
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Hey Robin,
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for
>>> new users to get
>>> >>>>>>>>>>>>>>>>>> all the dependencies in the right place.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>> >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> -Max
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>> peter.vary.apache@gmail.com> wrote:
>>> >>>>>>>>>>>>>>>>>> >
>>> >>>>>>>>>>>>>>>>>> > Hi Robin,
>>> >>>>>>>>>>>>>>>>>> >
>>> >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in
>>> the Iceberg repo.
>>> >>>>>>>>>>>>>>>>>> >
>>> >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>> https://github.com/apache/iceberg/issues/13519
>>> >>>>>>>>>>>>>>>>>> > The conclusion was:
>>> >>>>>>>>>>>>>>>>>> >>
>>> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and
>>> bring this back up to date. My preference would be to rely on the Spark
>>> image <https://hub.docker.com/r/apache/spark> provided by the Apache
>>> Spark project, similar to what we do for the Hive <
>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>>> >>>>>>>>>>>>>>>>>> >
>>> >>>>>>>>>>>>>>>>>> >
>>> >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list
>>> thread: https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>> >>>>>>>>>>>>>>>>>> >
>>> >>>>>>>>>>>>>>>>>> > Thanks for working on this,
>>> >>>>>>>>>>>>>>>>>> > Peter
>>> >>>>>>>>>>>>>>>>>> >
>>> >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt
>>> írta (időpont: 2026. jan. 22., Cs, 19:23):
>>> >>>>>>>>>>>>>>>>>> >>
>>> >>>>>>>>>>>>>>>>>> >> Hi,
>>> >>>>>>>>>>>>>>>>>> >>
>>> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR
>>> [1], what do people think about adding an official quickstart Docker image
>>> for Flink to the project?
>>> >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>> too.
>>> >>>>>>>>>>>>>>>>>> >>
>>> >>>>>>>>>>>>>>>>>> >> thanks, Robin.
>>> >>>>>>>>>>>>>>>>>> >>
>>> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>>
>>> >>>>>>>
>>> >>>>>
>>> >>>
>>> >>>
>>> >>
>>> >>
>>> >> --
>>> >> Robin Moffatt
>>> >> Sr. Principal Advisor, Streaming Data Technologies
>>>
>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi
I did a new pass, thanks !
Regards
JB
On Wed, Feb 18, 2026 at 5:14 PM Kevin Liu <ke...@apache.org> wrote:
> Here's the latest version of the Flink Quickstart PR,
> https://github.com/apache/iceberg/pull/15062
> Please take a look!
>
> Let's start a new thread to follow up on publishing the Docker image (for
> Flink and Spark)
>
>
>
> On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <mx...@apache.org> wrote:
>
>> Thanks Robin! Really great work.
>>
>> Cheers,
>> Max
>>
>> On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <ke...@apache.org> wrote:
>> >
>> > Thanks again for working on this. I took another look at #15124. Let's
>> try to get this merged, it's very self contained.
>> >
>> > Best,
>> > Kevin Liu
>> >
>> > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev <
>> dev@iceberg.apache.org> wrote:
>> >>
>> >> (I realised I mistakenly sent this only to Péter instead of the whole
>> list)
>> >>
>> >> Could I get a review on https://github.com/apache/iceberg/pull/15124
>> please.
>> >> Once that's sorted, I'll update the quickstart docs PR (15062) for the
>> necessary changes.
>> >>
>> >> thanks, Robin.
>> >>
>> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io> wrote:
>> >>>
>> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for
>> review please. Once that's sorted, I'll update the quickstart docs PR
>> (15062) for the necessary changes.
>> >>>
>> >>> thanks, Robin.
>> >>>
>> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <pe...@gmail.com>
>> wrote:
>> >>>>
>> >>>> In there are no objections then I would prefer it in the
>> docker/iceberg-flink-quickstart
>> >>>>
>> >>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>> 2026. febr. 6., P, 11:13):
>> >>>>>
>> >>>>> Hi Peter,
>> >>>>>
>> >>>>> Thanks for the direction. I'll remove the publish step so that we
>> can get the quickstart published, and then work on the publishing
>> subsequently.
>> >>>>>
>> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or
>> docker/iceberg-flink-quickstart ?
>> >>>>>
>> >>>>> thanks, Robin
>> >>>>>
>> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <
>> peter.vary.apache@gmail.com> wrote:
>> >>>>>>
>> >>>>>> I think we have two options:
>> >>>>>>
>> >>>>>> Remove the image publication from this PR (
>> https://github.com/apache/iceberg/pull/15124) for now, and proceed with
>> adding the Docker image and updating the documentation.
>> >>>>>> Alternatively, we could discuss publishing the Flink quickstart
>> image at the next Iceberg Community Sync and use that as an opportunity to
>> simplify both the documentation and the overall user experience.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>> 2026. febr. 4., Sze, 18:52):
>> >>>>>>>
>> >>>>>>> Hi,
>> >>>>>>>
>> >>>>>>> I have perhaps managed to deadlock this process :) I'd appreciate
>> some help untangling it. The recap is in my previous email (below).
>> >>>>>>>
>> >>>>>>> thanks, Robin.
>> >>>>>>>
>> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io>
>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi Kevin,
>> >>>>>>>>
>> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>> >>>>>>>> I have two related PRs:
>> >>>>>>>>
>> >>>>>>>> #15124: Add Flink Quickstart docker image
>> >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>> >>>>>>>>
>> >>>>>>>> I can see a few routes forward:
>> >>>>>>>>
>> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with
>> the publish script (I've not seen anything raised about it yet tho?)
>> >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still
>> relying on local image build (not sure what this would achieve vs the
>> option above tho?)
>> >>>>>>>> 3. Merge #15124 including publish script, then #15062 using the
>> published image not the local build
>> >>>>>>>>
>> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile
>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart
>> (#15124).
>> >>>>>>>>
>> >>>>>>>> LMK if I've missed an angle here.
>> >>>>>>>>
>> >>>>>>>> thanks, Robin
>> >>>>>>>>
>> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org>
>> wrote:
>> >>>>>>>>>
>> >>>>>>>>> Thanks for working on this, Robin! It looks like the complexity
>> here is publishing the docker image. What do you think about isolating that
>> part? (Just move the publish script out of #15124) We can start with the
>> Dockerfile definition, which allows us to build locally. This should
>> unblock us from merging the getting started docs in #15062
>> >>>>>>>>> Thoughts?
>> >>>>>>>>>
>> >>>>>>>>> Best,
>> >>>>>>>>> Kevin Liu
>> >>>>>>>>>
>> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>> dev@iceberg.apache.org> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Hi,
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks for the discussion and input.
>> >>>>>>>>>> It sounds like there are no major blockers. Could someone
>> please review https://github.com/apache/iceberg/pull/15124 ?
>> >>>>>>>>>>
>> >>>>>>>>>> thanks,
>> >>>>>>>>>>
>> >>>>>>>>>> Robin.
>> >>>>>>>>>>
>> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org>
>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Hey folks,
>> >>>>>>>>>>>
>> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the
>> Spark base image and installs all the necessary jars. This is used for our
>> integration test setup [2] and is inspired by
>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>> upgrading to Spark 4, supporting Spark Connect, and better image build
>> caching.
>> >>>>>>>>>>>
>> >>>>>>>>>>> This is already self-contained and can be reused by other
>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>> them in sync.
>> >>>>>>>>>>> I think it would be beneficial for the project to publish
>> this image and something similar for Flink.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Let me know what you think.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Best,
>> >>>>>>>>>>> Kevin Liu
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> [1]
>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>> >>>>>>>>>>> [2]
>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>> >>>>>>>>>>> [3]
>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>> >>>>>>>>>>> [4]
>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <
>> stevenz3wu@gmail.com> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> > Since the integration code for both Spark and Flink lives
>> in our repository, it might make sense to also store the Docker images and
>> the corresponding scripts there.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I agree with Peter here.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg project
>> should host those docker images. Not sure if the opinions have changed.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>> dev@iceberg.apache.org> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>> >>>>>>>>>>>>> Having looked at it, am I understanding the view to be that
>> ideally Flink would publish a Docker image that included the Iceberg
>> dependencies?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> However we do this, I feel that the user coming to run the
>> Flink quickstart should not have to build their own Docker image; this adds
>> unnecessary friction that is easily alleviated.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy
>> to discuss this idea with the Flink community; please let me know before I
>> do so.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> thanks, Robin.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <
>> ajanthabhat@gmail.com> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi Robin and Peter,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I discussed community-maintained Docker images previously:
>> >>>>>>>>>>>>>>
>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker
>> image (now at 100K+ total downloads) and use Docker images published by the
>> main engines in the quickstart, instead of maintaining these images
>> ourselves.
>> >>>>>>>>>>>>>> See the thread above for more details.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in
>> favor of adding it and relying on the Docker images provided by Flink
>> rather than maintaining our own images.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> - Ajantha
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>> peter.vary.apache@gmail.com> wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi Robin,
>> >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will
>> have some extra stuff to do with the docker image. For example make sure
>> that we have ci in place to build it.
>> >>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>> Peter
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>> dev@iceberg.apache.org> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
>> >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as
>> separate PRs, or combine them?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> thanks, Robin.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>> >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>> jb@nanthrax.net> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> This is a great idea.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker
>> image published by the project, we must ensure it is fully compliant with
>> ASF requirements regarding LICENSE/NOTICE files, etc. While this may seem
>> straightforward, it is a detail that is often overlooked.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I would be happy to help with this process.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Regards,
>> >>>>>>>>>>>>>>>>> JB
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>> mxm@apache.org> wrote:
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Hey Robin,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for
>> new users to get
>> >>>>>>>>>>>>>>>>>> all the dependencies in the right place.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>> >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> -Max
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>> peter.vary.apache@gmail.com> wrote:
>> >>>>>>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>>>>>> > Hi Robin,
>> >>>>>>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in
>> the Iceberg repo.
>> >>>>>>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>> https://github.com/apache/iceberg/issues/13519
>> >>>>>>>>>>>>>>>>>> > The conclusion was:
>> >>>>>>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and
>> bring this back up to date. My preference would be to rely on the Spark
>> image <https://hub.docker.com/r/apache/spark> provided by the Apache
>> Spark project, similar to what we do for the Hive <
>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>> >>>>>>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list
>> thread: https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>> >>>>>>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>>>>>> > Thanks for working on this,
>> >>>>>>>>>>>>>>>>>> > Peter
>> >>>>>>>>>>>>>>>>>> >
>> >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt
>> írta (időpont: 2026. jan. 22., Cs, 19:23):
>> >>>>>>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>>>>>> >> Hi,
>> >>>>>>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR
>> [1], what do people think about adding an official quickstart Docker image
>> for Flink to the project?
>> >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>> tabulario/spark-iceberg so perhaps that could be brought into the project
>> too.
>> >>>>>>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>>>>>> >> thanks, Robin.
>> >>>>>>>>>>>>>>>>>> >>
>> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >> Robin Moffatt
>> >> Sr. Principal Advisor, Streaming Data Technologies
>>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Kevin Liu <ke...@apache.org>.
Here's the latest version of the Flink Quickstart PR,
https://github.com/apache/iceberg/pull/15062
Please take a look!
Let's start a new thread to follow up on publishing the Docker image (for
Flink and Spark)
On Wed, Feb 18, 2026 at 4:54 AM Maximilian Michels <mx...@apache.org> wrote:
> Thanks Robin! Really great work.
>
> Cheers,
> Max
>
> On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <ke...@apache.org> wrote:
> >
> > Thanks again for working on this. I took another look at #15124. Let's
> try to get this merged, it's very self contained.
> >
> > Best,
> > Kevin Liu
> >
> > On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev <
> dev@iceberg.apache.org> wrote:
> >>
> >> (I realised I mistakenly sent this only to Péter instead of the whole
> list)
> >>
> >> Could I get a review on https://github.com/apache/iceberg/pull/15124
> please.
> >> Once that's sorted, I'll update the quickstart docs PR (15062) for the
> necessary changes.
> >>
> >> thanks, Robin.
> >>
> >> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io> wrote:
> >>>
> >>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for
> review please. Once that's sorted, I'll update the quickstart docs PR
> (15062) for the necessary changes.
> >>>
> >>> thanks, Robin.
> >>>
> >>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <pe...@gmail.com>
> wrote:
> >>>>
> >>>> In there are no objections then I would prefer it in the
> docker/iceberg-flink-quickstart
> >>>>
> >>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
> 2026. febr. 6., P, 11:13):
> >>>>>
> >>>>> Hi Peter,
> >>>>>
> >>>>> Thanks for the direction. I'll remove the publish step so that we
> can get the quickstart published, and then work on the publishing
> subsequently.
> >>>>>
> >>>>> Do you think the Dockerfile is best kept in flink/quickstart, or
> docker/iceberg-flink-quickstart ?
> >>>>>
> >>>>> thanks, Robin
> >>>>>
> >>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <pe...@gmail.com>
> wrote:
> >>>>>>
> >>>>>> I think we have two options:
> >>>>>>
> >>>>>> Remove the image publication from this PR (
> https://github.com/apache/iceberg/pull/15124) for now, and proceed with
> adding the Docker image and updating the documentation.
> >>>>>> Alternatively, we could discuss publishing the Flink quickstart
> image at the next Iceberg Community Sync and use that as an opportunity to
> simplify both the documentation and the overall user experience.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
> 2026. febr. 4., Sze, 18:52):
> >>>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> I have perhaps managed to deadlock this process :) I'd appreciate
> some help untangling it. The recap is in my previous email (below).
> >>>>>>>
> >>>>>>> thanks, Robin.
> >>>>>>>
> >>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io>
> wrote:
> >>>>>>>>
> >>>>>>>> Hi Kevin,
> >>>>>>>>
> >>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
> >>>>>>>> I have two related PRs:
> >>>>>>>>
> >>>>>>>> #15124: Add Flink Quickstart docker image
> >>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
> >>>>>>>>
> >>>>>>>> I can see a few routes forward:
> >>>>>>>>
> >>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with
> the publish script (I've not seen anything raised about it yet tho?)
> >>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still
> relying on local image build (not sure what this would achieve vs the
> option above tho?)
> >>>>>>>> 3. Merge #15124 including publish script, then #15062 using the
> published image not the local build
> >>>>>>>>
> >>>>>>>> Either way, one thing that needs resolving is the Dockerfile
> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart
> (#15124).
> >>>>>>>>
> >>>>>>>> LMK if I've missed an angle here.
> >>>>>>>>
> >>>>>>>> thanks, Robin
> >>>>>>>>
> >>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org>
> wrote:
> >>>>>>>>>
> >>>>>>>>> Thanks for working on this, Robin! It looks like the complexity
> here is publishing the docker image. What do you think about isolating that
> part? (Just move the publish script out of #15124) We can start with the
> Dockerfile definition, which allows us to build locally. This should
> unblock us from merging the getting started docs in #15062
> >>>>>>>>> Thoughts?
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Kevin Liu
> >>>>>>>>>
> >>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
> dev@iceberg.apache.org> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Hi,
> >>>>>>>>>>
> >>>>>>>>>> Thanks for the discussion and input.
> >>>>>>>>>> It sounds like there are no major blockers. Could someone
> please review https://github.com/apache/iceberg/pull/15124 ?
> >>>>>>>>>>
> >>>>>>>>>> thanks,
> >>>>>>>>>>
> >>>>>>>>>> Robin.
> >>>>>>>>>>
> >>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org>
> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hey folks,
> >>>>>>>>>>>
> >>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the
> Spark base image and installs all the necessary jars. This is used for our
> integration test setup [2] and is inspired by
> databricks/docker-spark-iceberg [3]. We've made many improvements such as
> upgrading to Spark 4, supporting Spark Connect, and better image build
> caching.
> >>>>>>>>>>>
> >>>>>>>>>>> This is already self-contained and can be reused by other
> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
> them in sync.
> >>>>>>>>>>> I think it would be beneficial for the project to publish this
> image and something similar for Flink.
> >>>>>>>>>>>
> >>>>>>>>>>> Let me know what you think.
> >>>>>>>>>>>
> >>>>>>>>>>> Best,
> >>>>>>>>>>> Kevin Liu
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> [1]
> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
> >>>>>>>>>>> [2]
> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
> >>>>>>>>>>> [3]
> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
> >>>>>>>>>>> [4]
> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <
> stevenz3wu@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> > Since the integration code for both Spark and Flink lives
> in our repository, it might make sense to also store the Docker images and
> the corresponding scripts there.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I agree with Peter here.
> >>>>>>>>>>>>
> >>>>>>>>>>>> The previous thread has some concerns if the Iceberg project
> should host those docker images. Not sure if the opinions have changed.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
> dev@iceberg.apache.org> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
> >>>>>>>>>>>>> Having looked at it, am I understanding the view to be that
> ideally Flink would publish a Docker image that included the Iceberg
> dependencies?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> However we do this, I feel that the user coming to run the
> Flink quickstart should not have to build their own Docker image; this adds
> unnecessary friction that is easily alleviated.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy
> to discuss this idea with the Flink community; please let me know before I
> do so.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> thanks, Robin.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <
> ajanthabhat@gmail.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Robin and Peter,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I discussed community-maintained Docker images previously:
> >>>>>>>>>>>>>>
> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker
> image (now at 100K+ total downloads) and use Docker images published by the
> main engines in the quickstart, instead of maintaining these images
> ourselves.
> >>>>>>>>>>>>>> See the thread above for more details.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in
> favor of adding it and relying on the Docker images provided by Flink
> rather than maintaining our own images.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> - Ajantha
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
> peter.vary.apache@gmail.com> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Robin,
> >>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will
> have some extra stuff to do with the docker image. For example make sure
> that we have ci in place to build it.
> >>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>> Peter
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
> dev@iceberg.apache.org> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
> >>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as
> separate PRs, or combine them?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> thanks, Robin.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
> >>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
> jb@nanthrax.net> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> This is a great idea.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker image
> published by the project, we must ensure it is fully compliant with ASF
> requirements regarding LICENSE/NOTICE files, etc. While this may seem
> straightforward, it is a detail that is often overlooked.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I would be happy to help with this process.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Regards,
> >>>>>>>>>>>>>>>>> JB
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
> mxm@apache.org> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hey Robin,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for
> new users to get
> >>>>>>>>>>>>>>>>>> all the dependencies in the right place.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
> >>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> -Max
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
> peter.vary.apache@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>> >
> >>>>>>>>>>>>>>>>>> > Hi Robin,
> >>>>>>>>>>>>>>>>>> >
> >>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the
> Iceberg repo.
> >>>>>>>>>>>>>>>>>> >
> >>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
> https://github.com/apache/iceberg/issues/13519
> >>>>>>>>>>>>>>>>>> > The conclusion was:
> >>>>>>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring
> this back up to date. My preference would be to rely on the Spark image <
> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
> project, similar to what we do for the Hive <
> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
> able to load all the Iceberg-specific JARs through the spark.jars.packages
> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
> >>>>>>>>>>>>>>>>>> >
> >>>>>>>>>>>>>>>>>> >
> >>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list
> thread: https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
> >>>>>>>>>>>>>>>>>> >
> >>>>>>>>>>>>>>>>>> > Thanks for working on this,
> >>>>>>>>>>>>>>>>>> > Peter
> >>>>>>>>>>>>>>>>>> >
> >>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt
> írta (időpont: 2026. jan. 22., Cs, 19:23):
> >>>>>>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>>>>>> >> Hi,
> >>>>>>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1],
> what do people think about adding an official quickstart Docker image for
> Flink to the project?
> >>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
> tabulario/spark-iceberg so perhaps that could be brought into the project
> too.
> >>>>>>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>>>>>> >> thanks, Robin.
> >>>>>>>>>>>>>>>>>> >>
> >>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>
> >>>
> >>
> >>
> >> --
> >> Robin Moffatt
> >> Sr. Principal Advisor, Streaming Data Technologies
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Maximilian Michels <mx...@apache.org>.
Thanks Robin! Really great work.
Cheers,
Max
On Sun, Feb 15, 2026 at 9:07 PM Kevin Liu <ke...@apache.org> wrote:
>
> Thanks again for working on this. I took another look at #15124. Let's try to get this merged, it's very self contained.
>
> Best,
> Kevin Liu
>
> On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev <de...@iceberg.apache.org> wrote:
>>
>> (I realised I mistakenly sent this only to Péter instead of the whole list)
>>
>> Could I get a review on https://github.com/apache/iceberg/pull/15124 please.
>> Once that's sorted, I'll update the quickstart docs PR (15062) for the necessary changes.
>>
>> thanks, Robin.
>>
>> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io> wrote:
>>>
>>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for review please. Once that's sorted, I'll update the quickstart docs PR (15062) for the necessary changes.
>>>
>>> thanks, Robin.
>>>
>>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <pe...@gmail.com> wrote:
>>>>
>>>> In there are no objections then I would prefer it in the docker/iceberg-flink-quickstart
>>>>
>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026. febr. 6., P, 11:13):
>>>>>
>>>>> Hi Peter,
>>>>>
>>>>> Thanks for the direction. I'll remove the publish step so that we can get the quickstart published, and then work on the publishing subsequently.
>>>>>
>>>>> Do you think the Dockerfile is best kept in flink/quickstart, or docker/iceberg-flink-quickstart ?
>>>>>
>>>>> thanks, Robin
>>>>>
>>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <pe...@gmail.com> wrote:
>>>>>>
>>>>>> I think we have two options:
>>>>>>
>>>>>> Remove the image publication from this PR (https://github.com/apache/iceberg/pull/15124) for now, and proceed with adding the Docker image and updating the documentation.
>>>>>> Alternatively, we could discuss publishing the Flink quickstart image at the next Iceberg Community Sync and use that as an opportunity to simplify both the documentation and the overall user experience.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026. febr. 4., Sze, 18:52):
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I have perhaps managed to deadlock this process :) I'd appreciate some help untangling it. The recap is in my previous email (below).
>>>>>>>
>>>>>>> thanks, Robin.
>>>>>>>
>>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io> wrote:
>>>>>>>>
>>>>>>>> Hi Kevin,
>>>>>>>>
>>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>>>>>>> I have two related PRs:
>>>>>>>>
>>>>>>>> #15124: Add Flink Quickstart docker image
>>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>>>>>>>>
>>>>>>>> I can see a few routes forward:
>>>>>>>>
>>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with the publish script (I've not seen anything raised about it yet tho?)
>>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still relying on local image build (not sure what this would achieve vs the option above tho?)
>>>>>>>> 3. Merge #15124 including publish script, then #15062 using the published image not the local build
>>>>>>>>
>>>>>>>> Either way, one thing that needs resolving is the Dockerfile location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart (#15124).
>>>>>>>>
>>>>>>>> LMK if I've missed an angle here.
>>>>>>>>
>>>>>>>> thanks, Robin
>>>>>>>>
>>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Thanks for working on this, Robin! It looks like the complexity here is publishing the docker image. What do you think about isolating that part? (Just move the publish script out of #15124) We can start with the Dockerfile definition, which allows us to build locally. This should unblock us from merging the getting started docs in #15062
>>>>>>>>> Thoughts?
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Kevin Liu
>>>>>>>>>
>>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <de...@iceberg.apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Thanks for the discussion and input.
>>>>>>>>>> It sounds like there are no major blockers. Could someone please review https://github.com/apache/iceberg/pull/15124 ?
>>>>>>>>>>
>>>>>>>>>> thanks,
>>>>>>>>>>
>>>>>>>>>> Robin.
>>>>>>>>>>
>>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hey folks,
>>>>>>>>>>>
>>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark base image and installs all the necessary jars. This is used for our integration test setup [2] and is inspired by databricks/docker-spark-iceberg [3]. We've made many improvements such as upgrading to Spark 4, supporting Spark Connect, and better image build caching.
>>>>>>>>>>>
>>>>>>>>>>> This is already self-contained and can be reused by other subprojects. In fact, iceberg-rust already uses it [4] and I try to keep them in sync.
>>>>>>>>>>> I think it would be beneficial for the project to publish this image and something similar for Flink.
>>>>>>>>>>>
>>>>>>>>>>> Let me know what you think.
>>>>>>>>>>>
>>>>>>>>>>> Best,
>>>>>>>>>>> Kevin Liu
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>>>>>>>>> [2] https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>>>>>>>>> [3] https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>>>>>>>>> [4] https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> > Since the integration code for both Spark and Flink lives in our repository, it might make sense to also store the Docker images and the corresponding scripts there.
>>>>>>>>>>>>
>>>>>>>>>>>> I agree with Peter here.
>>>>>>>>>>>>
>>>>>>>>>>>> The previous thread has some concerns if the Iceberg project should host those docker images. Not sure if the opinions have changed.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <de...@iceberg.apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>>>>>>>>>> Having looked at it, am I understanding the view to be that ideally Flink would publish a Docker image that included the Iceberg dependencies?
>>>>>>>>>>>>>
>>>>>>>>>>>>> However we do this, I feel that the user coming to run the Flink quickstart should not have to build their own Docker image; this adds unnecessary friction that is easily alleviated.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy to discuss this idea with the Flink community; please let me know before I do so.
>>>>>>>>>>>>>
>>>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Robin and Peter,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I discussed community-maintained Docker images previously:
>>>>>>>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker image (now at 100K+ total downloads) and use Docker images published by the main engines in the quickstart, instead of maintaining these images ourselves.
>>>>>>>>>>>>>> See the thread above for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor of adding it and relying on the Docker images provided by Flink rather than maintaining our own images.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> - Ajantha
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <pe...@gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Robin,
>>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will have some extra stuff to do with the docker image. For example make sure that we have ci in place to build it.
>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>> Peter
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <de...@iceberg.apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as separate PRs, or combine them?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This is a great idea.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker image published by the project, we must ensure it is fully compliant with ASF requirements regarding LICENSE/NOTICE files, etc. While this may seem straightforward, it is a detail that is often overlooked.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>>> JB
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hey Robin,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new users to get
>>>>>>>>>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> -Max
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <pe...@gmail.com> wrote:
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Hi Robin,
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the Iceberg repo.
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side: https://github.com/apache/iceberg/issues/13519
>>>>>>>>>>>>>>>>>> > The conclusion was:
>>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring this back up to date. My preference would be to rely on the Spark image <https://hub.docker.com/r/apache/spark> provided by the Apache Spark project, similar to what we do for the Hive <https://iceberg.apache.org/hive-quickstart/> quickstart. We should be able to load all the Iceberg-specific JARs through the spark.jars.packages configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread: https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>>>>>>>>>> > Peter
>>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>>> >> Hi,
>>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do people think about adding an official quickstart Docker image for Flink to the project?
>>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so perhaps that could be brought into the project too.
>>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>
>>>
>>
>>
>> --
>> Robin Moffatt
>> Sr. Principal Advisor, Streaming Data Technologies
Re: [DISCUSS] Add a Flink quickstart image
Posted by Kevin Liu <ke...@apache.org>.
Thanks again for working on this. I took another look at #15124. Let's try
to get this merged, it's very self contained.
Best,
Kevin Liu
On Fri, Feb 13, 2026 at 2:49 AM Robin Moffatt via dev <
dev@iceberg.apache.org> wrote:
> (I realised I mistakenly sent this only to Péter instead of the whole list)
>
> Could I get a review on https://github.com/apache/iceberg/pull/15124
> please.
> Once that's sorted, I'll update the quickstart docs PR (15062) for the
> necessary changes.
>
> thanks, Robin.
>
> On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io> wrote:
>
>> OK, https://github.com/apache/iceberg/pull/15124 is now ready for review
>> please. Once that's sorted, I'll update the quickstart docs PR (15062) for
>> the necessary changes.
>>
>> thanks, Robin.
>>
>> On Fri, 6 Feb 2026 at 11:20, Péter Váry <pe...@gmail.com>
>> wrote:
>>
>>> In there are no objections then I would prefer it in the
>>> docker/iceberg-flink-quickstart
>>>
>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
>>> febr. 6., P, 11:13):
>>>
>>>> Hi Peter,
>>>>
>>>> Thanks for the direction. I'll remove the publish step so that we can
>>>> get the quickstart published, and then work on the publishing subsequently.
>>>>
>>>> Do you think the Dockerfile is best kept in flink/quickstart, or
>>>> docker/iceberg-flink-quickstart ?
>>>>
>>>> thanks, Robin
>>>>
>>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <pe...@gmail.com>
>>>> wrote:
>>>>
>>>>> I think we have two options:
>>>>>
>>>>> 1. Remove the image publication from this PR (
>>>>> https://github.com/apache/iceberg/pull/15124) for now, and proceed
>>>>> with adding the Docker image and updating the documentation.
>>>>> 2. Alternatively, we could discuss publishing the Flink quickstart
>>>>> image at the next Iceberg Community Sync and use that as an opportunity to
>>>>> simplify both the documentation and the overall user experience.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>>> 2026. febr. 4., Sze, 18:52):
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I have perhaps managed to deadlock this process :) I'd appreciate
>>>>>> some help untangling it. The recap is in my previous email (below).
>>>>>>
>>>>>> thanks, Robin.
>>>>>>
>>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Kevin,
>>>>>>>
>>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>>>>>> I have two related PRs:
>>>>>>>
>>>>>>> #15124: Add Flink Quickstart docker image
>>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>>>>>>>
>>>>>>> I can see a few routes forward:
>>>>>>>
>>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with the
>>>>>>> publish script (I've not seen anything raised about it yet tho?)
>>>>>>> 2. Merge #15124 minus publish script, and then #15062 still relying
>>>>>>> on local image build (not sure what this would achieve vs the option above
>>>>>>> tho?)
>>>>>>> 3. Merge #15124 including publish script, then #15062 using the
>>>>>>> published image not the local build
>>>>>>>
>>>>>>> Either way, one thing that needs resolving is the Dockerfile
>>>>>>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart
>>>>>>> (#15124).
>>>>>>>
>>>>>>> LMK if I've missed an angle here.
>>>>>>>
>>>>>>> thanks, Robin
>>>>>>>
>>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks for working on this, Robin! It looks like the complexity
>>>>>>>> here is publishing the docker image. What do you think about isolating that
>>>>>>>> part? (Just move the publish script out of #15124) We can start
>>>>>>>> with the Dockerfile definition, which allows us to build locally. This
>>>>>>>> should unblock us from merging the getting started docs in #15062
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Kevin Liu
>>>>>>>>
>>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Thanks for the discussion and input.
>>>>>>>>> It sounds like there are no major blockers. Could someone please
>>>>>>>>> review https://github.com/apache/iceberg/pull/15124 ?
>>>>>>>>>
>>>>>>>>> thanks,
>>>>>>>>>
>>>>>>>>> Robin.
>>>>>>>>>
>>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey folks,
>>>>>>>>>>
>>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark
>>>>>>>>>> base image and installs all the necessary jars. This is used for our
>>>>>>>>>> integration test setup [2] and is inspired by
>>>>>>>>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>>>>>>>>>> upgrading to Spark 4, supporting Spark Connect, and better image build
>>>>>>>>>> caching.
>>>>>>>>>>
>>>>>>>>>> This is already self-contained and can be reused by other
>>>>>>>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>>>>>>>>>> them in sync.
>>>>>>>>>> I think it would be beneficial for the project to publish this
>>>>>>>>>> image and something similar for Flink.
>>>>>>>>>>
>>>>>>>>>> Let me know what you think.
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Kevin Liu
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>>>>>>>> [2]
>>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>>>>>>>> [3]
>>>>>>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>>>>>>>> [4]
>>>>>>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> > Since the integration code for both Spark and Flink lives in
>>>>>>>>>>> our repository, it might make sense to also store the Docker images and the
>>>>>>>>>>> corresponding scripts there.
>>>>>>>>>>>
>>>>>>>>>>> I agree with Peter here.
>>>>>>>>>>>
>>>>>>>>>>> The previous thread has some concerns if the Iceberg project
>>>>>>>>>>> should host those docker images. Not sure if the opinions have changed.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>>>>>>>>> Having looked at it, am I understanding the view to be that
>>>>>>>>>>>> ideally Flink would publish a Docker image that included the Iceberg
>>>>>>>>>>>> dependencies?
>>>>>>>>>>>>
>>>>>>>>>>>> However we do this, I feel that the user coming to run the
>>>>>>>>>>>> Flink quickstart should not have to build their own Docker image; this adds
>>>>>>>>>>>> unnecessary friction that is easily alleviated.
>>>>>>>>>>>>
>>>>>>>>>>>> If I've understood the situation correctly, then I'm happy to
>>>>>>>>>>>> discuss this idea with the Flink community; please let me know before I do
>>>>>>>>>>>> so.
>>>>>>>>>>>>
>>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <
>>>>>>>>>>>> ajanthabhat@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Robin and Peter,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I discussed community-maintained Docker images previously:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>>>>>>>>
>>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker
>>>>>>>>>>>>> image <https://hub.docker.com/r/apache/iceberg-rest-fixture>
>>>>>>>>>>>>> (now at 100K+ total downloads) and use Docker images published by the main
>>>>>>>>>>>>> engines in the quickstart, instead of maintaining these images ourselves.
>>>>>>>>>>>>> See the thread above for more details.
>>>>>>>>>>>>>
>>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor
>>>>>>>>>>>>> of adding it and relying on the Docker images provided by Flink rather than
>>>>>>>>>>>>> maintaining our own images.
>>>>>>>>>>>>> - Ajantha
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Robin,
>>>>>>>>>>>>>> It would be nice to separate them. I expect that we will have
>>>>>>>>>>>>>> some extra stuff to do with the docker image. For example make sure that we
>>>>>>>>>>>>>> have ci in place to build it.
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Peter
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>>>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as
>>>>>>>>>>>>>>> separate PRs, or combine them?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>>>>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This is a great idea.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker image
>>>>>>>>>>>>>>>> published by the project, we must ensure it is fully compliant with ASF
>>>>>>>>>>>>>>>> requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>>>>>>>>>>>>>> straightforward, it is a detail that is often overlooked.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>>> JB
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>>>>>>>>>>>>>> mxm@apache.org> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hey Robin,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new
>>>>>>>>>>>>>>>>> users to get
>>>>>>>>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> -Max
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Hi Robin,
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the
>>>>>>>>>>>>>>>>> Iceberg repo.
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>>>>>>>>>> > The conclusion was:
>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring
>>>>>>>>>>>>>>>>> this back up to date. My preference would be to rely on the Spark image <
>>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the
>>>>>>>>>>>>>>>>> Apache Spark project, similar to what we do for the Hive <
>>>>>>>>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart.
>>>>>>>>>>>>>>>>> We should be able to load all the Iceberg-specific JARs through the
>>>>>>>>>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>>>>>>>>> > Peter
>>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>> >> Hi,
>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1],
>>>>>>>>>>>>>>>>> what do people think about adding an official quickstart Docker image for
>>>>>>>>>>>>>>>>> Flink to the project?
>>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>>>>>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>>>>>>>>>>>>>> too.
>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>
>>
>
> --
> *Robin Moffatt*
> *Sr. Principal Advisor, Streaming Data Technologies*
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
(I realised I mistakenly sent this only to Péter instead of the whole list)
Could I get a review on https://github.com/apache/iceberg/pull/15124
please.
Once that's sorted, I'll update the quickstart docs PR (15062) for the
necessary changes.
thanks, Robin.
On Fri, 6 Feb 2026 at 11:42, Robin Moffatt <ro...@confluent.io> wrote:
> OK, https://github.com/apache/iceberg/pull/15124 is now ready for review
> please. Once that's sorted, I'll update the quickstart docs PR (15062) for
> the necessary changes.
>
> thanks, Robin.
>
> On Fri, 6 Feb 2026 at 11:20, Péter Váry <pe...@gmail.com>
> wrote:
>
>> In there are no objections then I would prefer it in the
>> docker/iceberg-flink-quickstart
>>
>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
>> febr. 6., P, 11:13):
>>
>>> Hi Peter,
>>>
>>> Thanks for the direction. I'll remove the publish step so that we can
>>> get the quickstart published, and then work on the publishing subsequently.
>>>
>>> Do you think the Dockerfile is best kept in flink/quickstart, or
>>> docker/iceberg-flink-quickstart ?
>>>
>>> thanks, Robin
>>>
>>> On Thu, 5 Feb 2026 at 16:10, Péter Váry <pe...@gmail.com>
>>> wrote:
>>>
>>>> I think we have two options:
>>>>
>>>> 1. Remove the image publication from this PR (
>>>> https://github.com/apache/iceberg/pull/15124) for now, and proceed
>>>> with adding the Docker image and updating the documentation.
>>>> 2. Alternatively, we could discuss publishing the Flink quickstart
>>>> image at the next Iceberg Community Sync and use that as an opportunity to
>>>> simplify both the documentation and the overall user experience.
>>>>
>>>>
>>>>
>>>>
>>>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>> 2026. febr. 4., Sze, 18:52):
>>>>
>>>>> Hi,
>>>>>
>>>>> I have perhaps managed to deadlock this process :) I'd appreciate some
>>>>> help untangling it. The recap is in my previous email (below).
>>>>>
>>>>> thanks, Robin.
>>>>>
>>>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io>
>>>>> wrote:
>>>>>
>>>>>> Hi Kevin,
>>>>>>
>>>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>>>>> I have two related PRs:
>>>>>>
>>>>>> #15124: Add Flink Quickstart docker image
>>>>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>>>>>>
>>>>>> I can see a few routes forward:
>>>>>>
>>>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with the
>>>>>> publish script (I've not seen anything raised about it yet tho?)
>>>>>> 2. Merge #15124 minus publish script, and then #15062 still relying
>>>>>> on local image build (not sure what this would achieve vs the option above
>>>>>> tho?)
>>>>>> 3. Merge #15124 including publish script, then #15062 using the
>>>>>> published image not the local build
>>>>>>
>>>>>> Either way, one thing that needs resolving is the Dockerfile
>>>>>> location: flink/quickstart (#15062) vs docker/iceberg-flink-quickstart
>>>>>> (#15124).
>>>>>>
>>>>>> LMK if I've missed an angle here.
>>>>>>
>>>>>> thanks, Robin
>>>>>>
>>>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Thanks for working on this, Robin! It looks like the complexity here
>>>>>>> is publishing the docker image. What do you think about isolating that
>>>>>>> part? (Just move the publish script out of #15124) We can start
>>>>>>> with the Dockerfile definition, which allows us to build locally. This
>>>>>>> should unblock us from merging the getting started docs in #15062
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> Best,
>>>>>>> Kevin Liu
>>>>>>>
>>>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Thanks for the discussion and input.
>>>>>>>> It sounds like there are no major blockers. Could someone please
>>>>>>>> review https://github.com/apache/iceberg/pull/15124 ?
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>>
>>>>>>>> Robin.
>>>>>>>>
>>>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey folks,
>>>>>>>>>
>>>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark
>>>>>>>>> base image and installs all the necessary jars. This is used for our
>>>>>>>>> integration test setup [2] and is inspired by
>>>>>>>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>>>>>>>>> upgrading to Spark 4, supporting Spark Connect, and better image build
>>>>>>>>> caching.
>>>>>>>>>
>>>>>>>>> This is already self-contained and can be reused by other
>>>>>>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>>>>>>>>> them in sync.
>>>>>>>>> I think it would be beneficial for the project to publish this
>>>>>>>>> image and something similar for Flink.
>>>>>>>>>
>>>>>>>>> Let me know what you think.
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Kevin Liu
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>>>>>>> [2]
>>>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>>>>>>> [3]
>>>>>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>>>>>>> [4]
>>>>>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>>>>>>
>>>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> > Since the integration code for both Spark and Flink lives in
>>>>>>>>>> our repository, it might make sense to also store the Docker images and the
>>>>>>>>>> corresponding scripts there.
>>>>>>>>>>
>>>>>>>>>> I agree with Peter here.
>>>>>>>>>>
>>>>>>>>>> The previous thread has some concerns if the Iceberg project
>>>>>>>>>> should host those docker images. Not sure if the opinions have changed.
>>>>>>>>>>
>>>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>>>>>>>> Having looked at it, am I understanding the view to be that
>>>>>>>>>>> ideally Flink would publish a Docker image that included the Iceberg
>>>>>>>>>>> dependencies?
>>>>>>>>>>>
>>>>>>>>>>> However we do this, I feel that the user coming to run the Flink
>>>>>>>>>>> quickstart should not have to build their own Docker image; this adds
>>>>>>>>>>> unnecessary friction that is easily alleviated.
>>>>>>>>>>>
>>>>>>>>>>> If I've understood the situation correctly, then I'm happy to
>>>>>>>>>>> discuss this idea with the Flink community; please let me know before I do
>>>>>>>>>>> so.
>>>>>>>>>>>
>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <
>>>>>>>>>>> ajanthabhat@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Robin and Peter,
>>>>>>>>>>>>
>>>>>>>>>>>> I discussed community-maintained Docker images previously:
>>>>>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>>>>>>>
>>>>>>>>>>>> The consensus was to publish only the REST fixture Docker image
>>>>>>>>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at
>>>>>>>>>>>> 100K+ total downloads) and use Docker images published by the main engines
>>>>>>>>>>>> in the quickstart, instead of maintaining these images ourselves.
>>>>>>>>>>>> See the thread above for more details.
>>>>>>>>>>>>
>>>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor of
>>>>>>>>>>>> adding it and relying on the Docker images provided by Flink rather than
>>>>>>>>>>>> maintaining our own images.
>>>>>>>>>>>> - Ajantha
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Robin,
>>>>>>>>>>>>> It would be nice to separate them. I expect that we will have
>>>>>>>>>>>>> some extra stuff to do with the docker image. For example make sure that we
>>>>>>>>>>>>> have ci in place to build it.
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Peter
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as
>>>>>>>>>>>>>> separate PRs, or combine them?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>>>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This is a great idea.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If we are moving forward with an "official" Docker image
>>>>>>>>>>>>>>> published by the project, we must ensure it is fully compliant with ASF
>>>>>>>>>>>>>>> requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>>>>>>>>>>>>> straightforward, it is a detail that is often overlooked.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>>> JB
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>>>>>>>>>>>>> mxm@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey Robin,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new
>>>>>>>>>>>>>>>> users to get
>>>>>>>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -Max
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Hi Robin,
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the
>>>>>>>>>>>>>>>> Iceberg repo.
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>>>>>>>>> > The conclusion was:
>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring
>>>>>>>>>>>>>>>> this back up to date. My preference would be to rely on the Spark image <
>>>>>>>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the
>>>>>>>>>>>>>>>> Apache Spark project, similar to what we do for the Hive <
>>>>>>>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart.
>>>>>>>>>>>>>>>> We should be able to load all the Iceberg-specific JARs through the
>>>>>>>>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>>>>>>>> > Peter
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>> >> Hi,
>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1],
>>>>>>>>>>>>>>>> what do people think about adding an official quickstart Docker image for
>>>>>>>>>>>>>>>> Flink to the project?
>>>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>>>>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>>>>>>>>>>>>> too.
>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>
>
--
*Robin Moffatt*
*Sr. Principal Advisor, Streaming Data Technologies*
Re: [DISCUSS] Add a Flink quickstart image
Posted by Péter Váry <pe...@gmail.com>.
In there are no objections then I would prefer it in the
docker/iceberg-flink-quickstart
Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
febr. 6., P, 11:13):
> Hi Peter,
>
> Thanks for the direction. I'll remove the publish step so that we can get
> the quickstart published, and then work on the publishing subsequently.
>
> Do you think the Dockerfile is best kept in flink/quickstart, or
> docker/iceberg-flink-quickstart ?
>
> thanks, Robin
>
> On Thu, 5 Feb 2026 at 16:10, Péter Váry <pe...@gmail.com>
> wrote:
>
>> I think we have two options:
>>
>> 1. Remove the image publication from this PR (
>> https://github.com/apache/iceberg/pull/15124) for now, and proceed
>> with adding the Docker image and updating the documentation.
>> 2. Alternatively, we could discuss publishing the Flink quickstart
>> image at the next Iceberg Community Sync and use that as an opportunity to
>> simplify both the documentation and the overall user experience.
>>
>>
>>
>>
>> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
>> febr. 4., Sze, 18:52):
>>
>>> Hi,
>>>
>>> I have perhaps managed to deadlock this process :) I'd appreciate some
>>> help untangling it. The recap is in my previous email (below).
>>>
>>> thanks, Robin.
>>>
>>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io> wrote:
>>>
>>>> Hi Kevin,
>>>>
>>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>>> I have two related PRs:
>>>>
>>>> #15124: Add Flink Quickstart docker image
>>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>>>>
>>>> I can see a few routes forward:
>>>>
>>>> 1. Merge #15062, fast-follow with #15124 once we're happy with the
>>>> publish script (I've not seen anything raised about it yet tho?)
>>>> 2. Merge #15124 minus publish script, and then #15062 still relying on
>>>> local image build (not sure what this would achieve vs the option above
>>>> tho?)
>>>> 3. Merge #15124 including publish script, then #15062 using the
>>>> published image not the local build
>>>>
>>>> Either way, one thing that needs resolving is the Dockerfile location:
>>>> flink/quickstart (#15062) vs docker/iceberg-flink-quickstart (#15124).
>>>>
>>>> LMK if I've missed an angle here.
>>>>
>>>> thanks, Robin
>>>>
>>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org> wrote:
>>>>
>>>>> Thanks for working on this, Robin! It looks like the complexity here
>>>>> is publishing the docker image. What do you think about isolating that
>>>>> part? (Just move the publish script out of #15124) We can start
>>>>> with the Dockerfile definition, which allows us to build locally. This
>>>>> should unblock us from merging the getting started docs in #15062
>>>>> Thoughts?
>>>>>
>>>>> Best,
>>>>> Kevin Liu
>>>>>
>>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Thanks for the discussion and input.
>>>>>> It sounds like there are no major blockers. Could someone please
>>>>>> review https://github.com/apache/iceberg/pull/15124 ?
>>>>>>
>>>>>> thanks,
>>>>>>
>>>>>> Robin.
>>>>>>
>>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey folks,
>>>>>>>
>>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark
>>>>>>> base image and installs all the necessary jars. This is used for our
>>>>>>> integration test setup [2] and is inspired by
>>>>>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>>>>>>> upgrading to Spark 4, supporting Spark Connect, and better image build
>>>>>>> caching.
>>>>>>>
>>>>>>> This is already self-contained and can be reused by other
>>>>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>>>>>>> them in sync.
>>>>>>> I think it would be beneficial for the project to publish this image
>>>>>>> and something similar for Flink.
>>>>>>>
>>>>>>> Let me know what you think.
>>>>>>>
>>>>>>> Best,
>>>>>>> Kevin Liu
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [1]
>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>>>>> [2]
>>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>>>>> [3]
>>>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>>>>> [4]
>>>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>>>>
>>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> > Since the integration code for both Spark and Flink lives in our
>>>>>>>> repository, it might make sense to also store the Docker images and the
>>>>>>>> corresponding scripts there.
>>>>>>>>
>>>>>>>> I agree with Peter here.
>>>>>>>>
>>>>>>>> The previous thread has some concerns if the Iceberg project should
>>>>>>>> host those docker images. Not sure if the opinions have changed.
>>>>>>>>
>>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>
>>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>>>>>> Having looked at it, am I understanding the view to be that
>>>>>>>>> ideally Flink would publish a Docker image that included the Iceberg
>>>>>>>>> dependencies?
>>>>>>>>>
>>>>>>>>> However we do this, I feel that the user coming to run the Flink
>>>>>>>>> quickstart should not have to build their own Docker image; this adds
>>>>>>>>> unnecessary friction that is easily alleviated.
>>>>>>>>>
>>>>>>>>> If I've understood the situation correctly, then I'm happy to
>>>>>>>>> discuss this idea with the Flink community; please let me know before I do
>>>>>>>>> so.
>>>>>>>>>
>>>>>>>>> thanks, Robin.
>>>>>>>>>
>>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Robin and Peter,
>>>>>>>>>>
>>>>>>>>>> I discussed community-maintained Docker images previously:
>>>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>>>>>
>>>>>>>>>> The consensus was to publish only the REST fixture Docker image
>>>>>>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at
>>>>>>>>>> 100K+ total downloads) and use Docker images published by the main engines
>>>>>>>>>> in the quickstart, instead of maintaining these images ourselves.
>>>>>>>>>> See the thread above for more details.
>>>>>>>>>>
>>>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor of
>>>>>>>>>> adding it and relying on the Docker images provided by Flink rather than
>>>>>>>>>> maintaining our own images.
>>>>>>>>>> - Ajantha
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Robin,
>>>>>>>>>>> It would be nice to separate them. I expect that we will have
>>>>>>>>>>> some extra stuff to do with the docker image. For example make sure that we
>>>>>>>>>>> have ci in place to build it.
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Peter
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>>>>>
>>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as separate
>>>>>>>>>>>> PRs, or combine them?
>>>>>>>>>>>>
>>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> This is a great idea.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If we are moving forward with an "official" Docker image
>>>>>>>>>>>>> published by the project, we must ensure it is fully compliant with ASF
>>>>>>>>>>>>> requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>>>>>>>>>>> straightforward, it is a detail that is often overlooked.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> JB
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>>>>>>>>>>> mxm@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hey Robin,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new
>>>>>>>>>>>>>> users to get
>>>>>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -Max
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Hi Robin,
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the
>>>>>>>>>>>>>> Iceberg repo.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>>>>>>> > The conclusion was:
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> we should both remove the vendor reference and bring this
>>>>>>>>>>>>>> back up to date. My preference would be to rely on the Spark image <
>>>>>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the
>>>>>>>>>>>>>> Apache Spark project, similar to what we do for the Hive <
>>>>>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We
>>>>>>>>>>>>>> should be able to load all the Iceberg-specific JARs through the
>>>>>>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>>>>>> > Peter
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> Hi,
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what
>>>>>>>>>>>>>> do people think about adding an official quickstart Docker image for Flink
>>>>>>>>>>>>>> to the project?
>>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>>>>>>>>>>> too.
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>>>>>> >>
>>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>
>>>>
>>>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
Hi Peter,
Thanks for the direction. I'll remove the publish step so that we can get
the quickstart published, and then work on the publishing subsequently.
Do you think the Dockerfile is best kept in flink/quickstart, or
docker/iceberg-flink-quickstart ?
thanks, Robin
On Thu, 5 Feb 2026 at 16:10, Péter Váry <pe...@gmail.com> wrote:
> I think we have two options:
>
> 1. Remove the image publication from this PR (
> https://github.com/apache/iceberg/pull/15124) for now, and proceed
> with adding the Docker image and updating the documentation.
> 2. Alternatively, we could discuss publishing the Flink quickstart
> image at the next Iceberg Community Sync and use that as an opportunity to
> simplify both the documentation and the overall user experience.
>
>
>
>
> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
> febr. 4., Sze, 18:52):
>
>> Hi,
>>
>> I have perhaps managed to deadlock this process :) I'd appreciate some
>> help untangling it. The recap is in my previous email (below).
>>
>> thanks, Robin.
>>
>> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io> wrote:
>>
>>> Hi Kevin,
>>>
>>> Just recapping so that I'm clear, cos I'm getting confused :)
>>> I have two related PRs:
>>>
>>> #15124: Add Flink Quickstart docker image
>>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>>>
>>> I can see a few routes forward:
>>>
>>> 1. Merge #15062, fast-follow with #15124 once we're happy with the
>>> publish script (I've not seen anything raised about it yet tho?)
>>> 2. Merge #15124 minus publish script, and then #15062 still relying on
>>> local image build (not sure what this would achieve vs the option above
>>> tho?)
>>> 3. Merge #15124 including publish script, then #15062 using the
>>> published image not the local build
>>>
>>> Either way, one thing that needs resolving is the Dockerfile location:
>>> flink/quickstart (#15062) vs docker/iceberg-flink-quickstart (#15124).
>>>
>>> LMK if I've missed an angle here.
>>>
>>> thanks, Robin
>>>
>>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org> wrote:
>>>
>>>> Thanks for working on this, Robin! It looks like the complexity here is
>>>> publishing the docker image. What do you think about isolating that part?
>>>> (Just move the publish script out of #15124) We can start
>>>> with the Dockerfile definition, which allows us to build locally. This
>>>> should unblock us from merging the getting started docs in #15062
>>>> Thoughts?
>>>>
>>>> Best,
>>>> Kevin Liu
>>>>
>>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Thanks for the discussion and input.
>>>>> It sounds like there are no major blockers. Could someone please
>>>>> review https://github.com/apache/iceberg/pull/15124 ?
>>>>>
>>>>> thanks,
>>>>>
>>>>> Robin.
>>>>>
>>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
>>>>>
>>>>>> Hey folks,
>>>>>>
>>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark
>>>>>> base image and installs all the necessary jars. This is used for our
>>>>>> integration test setup [2] and is inspired by
>>>>>> databricks/docker-spark-iceberg [3]. We've made many improvements such as
>>>>>> upgrading to Spark 4, supporting Spark Connect, and better image build
>>>>>> caching.
>>>>>>
>>>>>> This is already self-contained and can be reused by other
>>>>>> subprojects. In fact, iceberg-rust already uses it [4] and I try to keep
>>>>>> them in sync.
>>>>>> I think it would be beneficial for the project to publish this image
>>>>>> and something similar for Flink.
>>>>>>
>>>>>> Let me know what you think.
>>>>>>
>>>>>> Best,
>>>>>> Kevin Liu
>>>>>>
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>>>> [2]
>>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>>>> [3]
>>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>>>> [4]
>>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>>>
>>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> > Since the integration code for both Spark and Flink lives in our
>>>>>>> repository, it might make sense to also store the Docker images and the
>>>>>>> corresponding scripts there.
>>>>>>>
>>>>>>> I agree with Peter here.
>>>>>>>
>>>>>>> The previous thread has some concerns if the Iceberg project should
>>>>>>> host those docker images. Not sure if the opinions have changed.
>>>>>>>
>>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>
>>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>>>>> Having looked at it, am I understanding the view to be that ideally
>>>>>>>> Flink would publish a Docker image that included the Iceberg dependencies?
>>>>>>>>
>>>>>>>> However we do this, I feel that the user coming to run the Flink
>>>>>>>> quickstart should not have to build their own Docker image; this adds
>>>>>>>> unnecessary friction that is easily alleviated.
>>>>>>>>
>>>>>>>> If I've understood the situation correctly, then I'm happy to
>>>>>>>> discuss this idea with the Flink community; please let me know before I do
>>>>>>>> so.
>>>>>>>>
>>>>>>>> thanks, Robin.
>>>>>>>>
>>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Robin and Peter,
>>>>>>>>>
>>>>>>>>> I discussed community-maintained Docker images previously:
>>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>>>>
>>>>>>>>> The consensus was to publish only the REST fixture Docker image
>>>>>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at
>>>>>>>>> 100K+ total downloads) and use Docker images published by the main engines
>>>>>>>>> in the quickstart, instead of maintaining these images ourselves.
>>>>>>>>> See the thread above for more details.
>>>>>>>>>
>>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor of
>>>>>>>>> adding it and relying on the Docker images provided by Flink rather than
>>>>>>>>> maintaining our own images.
>>>>>>>>> - Ajantha
>>>>>>>>>
>>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Robin,
>>>>>>>>>> It would be nice to separate them. I expect that we will have
>>>>>>>>>> some extra stuff to do with the docker image. For example make sure that we
>>>>>>>>>> have ci in place to build it.
>>>>>>>>>> Thanks,
>>>>>>>>>> Peter
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>>>>
>>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as separate
>>>>>>>>>>> PRs, or combine them?
>>>>>>>>>>>
>>>>>>>>>>> thanks, Robin.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>
>>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> This is a great idea.
>>>>>>>>>>>>
>>>>>>>>>>>> If we are moving forward with an "official" Docker image
>>>>>>>>>>>> published by the project, we must ensure it is fully compliant with ASF
>>>>>>>>>>>> requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>>>>>>>>>> straightforward, it is a detail that is often overlooked.
>>>>>>>>>>>>
>>>>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> JB
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>>>>>>>>>> mxm@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Robin,
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new users
>>>>>>>>>>>>> to get
>>>>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>>>>
>>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Max
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Hi Robin,
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > I would love to see the Flink quickstart image in the
>>>>>>>>>>>>> Iceberg repo.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>>>>>> > The conclusion was:
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> we should both remove the vendor reference and bring this
>>>>>>>>>>>>> back up to date. My preference would be to rely on the Spark image <
>>>>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>>>>>>>>>> Spark project, similar to what we do for the Hive <
>>>>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We
>>>>>>>>>>>>> should be able to load all the Iceberg-specific JARs through the
>>>>>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>>>>> >
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>>>>> > Peter
>>>>>>>>>>>>> >
>>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Hi,
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what
>>>>>>>>>>>>> do people think about adding an official quickstart Docker image for Flink
>>>>>>>>>>>>> to the project?
>>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>>>>>>>>>> too.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>>>>> >>
>>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>
>>>
>>>
>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Péter Váry <pe...@gmail.com>.
I think we have two options:
1. Remove the image publication from this PR (
https://github.com/apache/iceberg/pull/15124) for now, and proceed with
adding the Docker image and updating the documentation.
2. Alternatively, we could discuss publishing the Flink quickstart image
at the next Iceberg Community Sync and use that as an opportunity to
simplify both the documentation and the overall user experience.
Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
febr. 4., Sze, 18:52):
> Hi,
>
> I have perhaps managed to deadlock this process :) I'd appreciate some
> help untangling it. The recap is in my previous email (below).
>
> thanks, Robin.
>
> On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io> wrote:
>
>> Hi Kevin,
>>
>> Just recapping so that I'm clear, cos I'm getting confused :)
>> I have two related PRs:
>>
>> #15124: Add Flink Quickstart docker image
>> #15062: Add Flink quickstart (which includes the Dockerfile too)
>>
>> I can see a few routes forward:
>>
>> 1. Merge #15062, fast-follow with #15124 once we're happy with the
>> publish script (I've not seen anything raised about it yet tho?)
>> 2. Merge #15124 minus publish script, and then #15062 still relying on
>> local image build (not sure what this would achieve vs the option above
>> tho?)
>> 3. Merge #15124 including publish script, then #15062 using the published
>> image not the local build
>>
>> Either way, one thing that needs resolving is the Dockerfile location:
>> flink/quickstart (#15062) vs docker/iceberg-flink-quickstart (#15124).
>>
>> LMK if I've missed an angle here.
>>
>> thanks, Robin
>>
>> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org> wrote:
>>
>>> Thanks for working on this, Robin! It looks like the complexity here is
>>> publishing the docker image. What do you think about isolating that part?
>>> (Just move the publish script out of #15124) We can start
>>> with the Dockerfile definition, which allows us to build locally. This
>>> should unblock us from merging the getting started docs in #15062
>>> Thoughts?
>>>
>>> Best,
>>> Kevin Liu
>>>
>>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> Thanks for the discussion and input.
>>>> It sounds like there are no major blockers. Could someone please review
>>>> https://github.com/apache/iceberg/pull/15124 ?
>>>>
>>>> thanks,
>>>>
>>>> Robin.
>>>>
>>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
>>>>
>>>>> Hey folks,
>>>>>
>>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark base
>>>>> image and installs all the necessary jars. This is used for our integration
>>>>> test setup [2] and is inspired by databricks/docker-spark-iceberg [3].
>>>>> We've made many improvements such as upgrading to Spark 4, supporting Spark
>>>>> Connect, and better image build caching.
>>>>>
>>>>> This is already self-contained and can be reused by other subprojects.
>>>>> In fact, iceberg-rust already uses it [4] and I try to keep them in sync.
>>>>> I think it would be beneficial for the project to publish this image
>>>>> and something similar for Flink.
>>>>>
>>>>> Let me know what you think.
>>>>>
>>>>> Best,
>>>>> Kevin Liu
>>>>>
>>>>>
>>>>>
>>>>> [1]
>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>>> [2]
>>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>>> [3]
>>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>>> [4]
>>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>>
>>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> > Since the integration code for both Spark and Flink lives in our
>>>>>> repository, it might make sense to also store the Docker images and the
>>>>>> corresponding scripts there.
>>>>>>
>>>>>> I agree with Peter here.
>>>>>>
>>>>>> The previous thread has some concerns if the Iceberg project should
>>>>>> host those docker images. Not sure if the opinions have changed.
>>>>>>
>>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>
>>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>>>> Having looked at it, am I understanding the view to be that ideally
>>>>>>> Flink would publish a Docker image that included the Iceberg dependencies?
>>>>>>>
>>>>>>> However we do this, I feel that the user coming to run the Flink
>>>>>>> quickstart should not have to build their own Docker image; this adds
>>>>>>> unnecessary friction that is easily alleviated.
>>>>>>>
>>>>>>> If I've understood the situation correctly, then I'm happy to
>>>>>>> discuss this idea with the Flink community; please let me know before I do
>>>>>>> so.
>>>>>>>
>>>>>>> thanks, Robin.
>>>>>>>
>>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Robin and Peter,
>>>>>>>>
>>>>>>>> I discussed community-maintained Docker images previously:
>>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>>>
>>>>>>>> The consensus was to publish only the REST fixture Docker image
>>>>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at
>>>>>>>> 100K+ total downloads) and use Docker images published by the main engines
>>>>>>>> in the quickstart, instead of maintaining these images ourselves.
>>>>>>>> See the thread above for more details.
>>>>>>>>
>>>>>>>> With respect to adding a Flink quickstart page, I’m in favor of
>>>>>>>> adding it and relying on the Docker images provided by Flink rather than
>>>>>>>> maintaining our own images.
>>>>>>>> - Ajantha
>>>>>>>>
>>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Robin,
>>>>>>>>> It would be nice to separate them. I expect that we will have some
>>>>>>>>> extra stuff to do with the docker image. For example make sure that we have
>>>>>>>>> ci in place to build it.
>>>>>>>>> Thanks,
>>>>>>>>> Peter
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>>>
>>>>>>>>>> Also, should I keep this and the quickstart PR [2] as separate
>>>>>>>>>> PRs, or combine them?
>>>>>>>>>>
>>>>>>>>>> thanks, Robin.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>
>>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> This is a great idea.
>>>>>>>>>>>
>>>>>>>>>>> If we are moving forward with an "official" Docker image
>>>>>>>>>>> published by the project, we must ensure it is fully compliant with ASF
>>>>>>>>>>> requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>>>>>>>>> straightforward, it is a detail that is often overlooked.
>>>>>>>>>>>
>>>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> JB
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>>>>>>>>> mxm@apache.org> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Robin,
>>>>>>>>>>>>
>>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new users
>>>>>>>>>>>> to get
>>>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>>>
>>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>>>
>>>>>>>>>>>> -Max
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>>> >
>>>>>>>>>>>> > Hi Robin,
>>>>>>>>>>>> >
>>>>>>>>>>>> > I would love to see the Flink quickstart image in the Iceberg
>>>>>>>>>>>> repo.
>>>>>>>>>>>> >
>>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>>>>> > The conclusion was:
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> we should both remove the vendor reference and bring this
>>>>>>>>>>>> back up to date. My preference would be to rely on the Spark image <
>>>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>>>>>>>>> Spark project, similar to what we do for the Hive <
>>>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We
>>>>>>>>>>>> should be able to load all the Iceberg-specific JARs through the
>>>>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>>>> >
>>>>>>>>>>>> >
>>>>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>>>> >
>>>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>>>> > Peter
>>>>>>>>>>>> >
>>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Hi,
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>>>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>>>>>>>> the project?
>>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>>>>>>>>> too.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>>>> >>
>>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>
>>
>>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
Hi,
I have perhaps managed to deadlock this process :) I'd appreciate some help
untangling it. The recap is in my previous email (below).
thanks, Robin.
On Thu, 29 Jan 2026 at 06:20, Robin Moffatt <ro...@confluent.io> wrote:
> Hi Kevin,
>
> Just recapping so that I'm clear, cos I'm getting confused :)
> I have two related PRs:
>
> #15124: Add Flink Quickstart docker image
> #15062: Add Flink quickstart (which includes the Dockerfile too)
>
> I can see a few routes forward:
>
> 1. Merge #15062, fast-follow with #15124 once we're happy with the publish
> script (I've not seen anything raised about it yet tho?)
> 2. Merge #15124 minus publish script, and then #15062 still relying on
> local image build (not sure what this would achieve vs the option above
> tho?)
> 3. Merge #15124 including publish script, then #15062 using the published
> image not the local build
>
> Either way, one thing that needs resolving is the Dockerfile location:
> flink/quickstart (#15062) vs docker/iceberg-flink-quickstart (#15124).
>
> LMK if I've missed an angle here.
>
> thanks, Robin
>
> On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org> wrote:
>
>> Thanks for working on this, Robin! It looks like the complexity here is
>> publishing the docker image. What do you think about isolating that part?
>> (Just move the publish script out of #15124) We can start
>> with the Dockerfile definition, which allows us to build locally. This
>> should unblock us from merging the getting started docs in #15062
>> Thoughts?
>>
>> Best,
>> Kevin Liu
>>
>> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
>> dev@iceberg.apache.org> wrote:
>>
>>> Hi,
>>>
>>> Thanks for the discussion and input.
>>> It sounds like there are no major blockers. Could someone please review
>>> https://github.com/apache/iceberg/pull/15124 ?
>>>
>>> thanks,
>>>
>>> Robin.
>>>
>>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
>>>
>>>> Hey folks,
>>>>
>>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark base
>>>> image and installs all the necessary jars. This is used for our integration
>>>> test setup [2] and is inspired by databricks/docker-spark-iceberg [3].
>>>> We've made many improvements such as upgrading to Spark 4, supporting Spark
>>>> Connect, and better image build caching.
>>>>
>>>> This is already self-contained and can be reused by other subprojects.
>>>> In fact, iceberg-rust already uses it [4] and I try to keep them in sync.
>>>> I think it would be beneficial for the project to publish this image
>>>> and something similar for Flink.
>>>>
>>>> Let me know what you think.
>>>>
>>>> Best,
>>>> Kevin Liu
>>>>
>>>>
>>>>
>>>> [1]
>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>>> [2]
>>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>>> [3]
>>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>>> [4]
>>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>>
>>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com>
>>>> wrote:
>>>>
>>>>> > Since the integration code for both Spark and Flink lives in our
>>>>> repository, it might make sense to also store the Docker images and the
>>>>> corresponding scripts there.
>>>>>
>>>>> I agree with Peter here.
>>>>>
>>>>> The previous thread has some concerns if the Iceberg project should
>>>>> host those docker images. Not sure if the opinions have changed.
>>>>>
>>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>>
>>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>>> Having looked at it, am I understanding the view to be that ideally
>>>>>> Flink would publish a Docker image that included the Iceberg dependencies?
>>>>>>
>>>>>> However we do this, I feel that the user coming to run the Flink
>>>>>> quickstart should not have to build their own Docker image; this adds
>>>>>> unnecessary friction that is easily alleviated.
>>>>>>
>>>>>> If I've understood the situation correctly, then I'm happy to discuss
>>>>>> this idea with the Flink community; please let me know before I do so.
>>>>>>
>>>>>> thanks, Robin.
>>>>>>
>>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Robin and Peter,
>>>>>>>
>>>>>>> I discussed community-maintained Docker images previously:
>>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>>
>>>>>>> The consensus was to publish only the REST fixture Docker image
>>>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at
>>>>>>> 100K+ total downloads) and use Docker images published by the main engines
>>>>>>> in the quickstart, instead of maintaining these images ourselves.
>>>>>>> See the thread above for more details.
>>>>>>>
>>>>>>> With respect to adding a Flink quickstart page, I’m in favor of
>>>>>>> adding it and relying on the Docker images provided by Flink rather than
>>>>>>> maintaining our own images.
>>>>>>> - Ajantha
>>>>>>>
>>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Robin,
>>>>>>>> It would be nice to separate them. I expect that we will have some
>>>>>>>> extra stuff to do with the docker image. For example make sure that we have
>>>>>>>> ci in place to build it.
>>>>>>>> Thanks,
>>>>>>>> Peter
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>>
>>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>>
>>>>>>>>> Also, should I keep this and the quickstart PR [2] as separate
>>>>>>>>> PRs, or combine them?
>>>>>>>>>
>>>>>>>>> thanks, Robin.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>>
>>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <
>>>>>>>>> jb@nanthrax.net> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> This is a great idea.
>>>>>>>>>>
>>>>>>>>>> If we are moving forward with an "official" Docker image
>>>>>>>>>> published by the project, we must ensure it is fully compliant with ASF
>>>>>>>>>> requirements regarding LICENSE/NOTICE files, etc. While this may seem
>>>>>>>>>> straightforward, it is a detail that is often overlooked.
>>>>>>>>>>
>>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> JB
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <
>>>>>>>>>> mxm@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Robin,
>>>>>>>>>>>
>>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new users
>>>>>>>>>>> to get
>>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>>
>>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>>
>>>>>>>>>>> -Max
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>>> >
>>>>>>>>>>> > Hi Robin,
>>>>>>>>>>> >
>>>>>>>>>>> > I would love to see the Flink quickstart image in the Iceberg
>>>>>>>>>>> repo.
>>>>>>>>>>> >
>>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>>>> > The conclusion was:
>>>>>>>>>>> >>
>>>>>>>>>>> >> we should both remove the vendor reference and bring this
>>>>>>>>>>> back up to date. My preference would be to rely on the Spark image <
>>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>>>>>>>> Spark project, similar to what we do for the Hive <
>>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We
>>>>>>>>>>> should be able to load all the Iceberg-specific JARs through the
>>>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>>> >
>>>>>>>>>>> >
>>>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>>> >
>>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>>> > Peter
>>>>>>>>>>> >
>>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>>> >>
>>>>>>>>>>> >> Hi,
>>>>>>>>>>> >>
>>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>>>>>>> the project?
>>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>>>>>>>> too.
>>>>>>>>>>> >>
>>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>>> >>
>>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>
>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
Hi Kevin,
Just recapping so that I'm clear, cos I'm getting confused :)
I have two related PRs:
#15124: Add Flink Quickstart docker image
#15062: Add Flink quickstart (which includes the Dockerfile too)
I can see a few routes forward:
1. Merge #15062, fast-follow with #15124 once we're happy with the publish
script (I've not seen anything raised about it yet tho?)
2. Merge #15124 minus publish script, and then #15062 still relying on
local image build (not sure what this would achieve vs the option above
tho?)
3. Merge #15124 including publish script, then #15062 using the published
image not the local build
Either way, one thing that needs resolving is the Dockerfile location:
flink/quickstart (#15062) vs docker/iceberg-flink-quickstart (#15124).
LMK if I've missed an angle here.
thanks, Robin
On Wed, 28 Jan 2026 at 15:57, Kevin Liu <ke...@apache.org> wrote:
> Thanks for working on this, Robin! It looks like the complexity here is
> publishing the docker image. What do you think about isolating that part?
> (Just move the publish script out of #15124) We can start
> with the Dockerfile definition, which allows us to build locally. This
> should unblock us from merging the getting started docs in #15062
> Thoughts?
>
> Best,
> Kevin Liu
>
> On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
> dev@iceberg.apache.org> wrote:
>
>> Hi,
>>
>> Thanks for the discussion and input.
>> It sounds like there are no major blockers. Could someone please review
>> https://github.com/apache/iceberg/pull/15124 ?
>>
>> thanks,
>>
>> Robin.
>>
>> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
>>
>>> Hey folks,
>>>
>>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark base
>>> image and installs all the necessary jars. This is used for our integration
>>> test setup [2] and is inspired by databricks/docker-spark-iceberg [3].
>>> We've made many improvements such as upgrading to Spark 4, supporting Spark
>>> Connect, and better image build caching.
>>>
>>> This is already self-contained and can be reused by other subprojects.
>>> In fact, iceberg-rust already uses it [4] and I try to keep them in sync.
>>> I think it would be beneficial for the project to publish this image and
>>> something similar for Flink.
>>>
>>> Let me know what you think.
>>>
>>> Best,
>>> Kevin Liu
>>>
>>>
>>>
>>> [1]
>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>>> [2]
>>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>>> [3]
>>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>>> [4]
>>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>>
>>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com> wrote:
>>>
>>>> > Since the integration code for both Spark and Flink lives in our
>>>> repository, it might make sense to also store the Docker images and the
>>>> corresponding scripts there.
>>>>
>>>> I agree with Peter here.
>>>>
>>>> The previous thread has some concerns if the Iceberg project should
>>>> host those docker images. Not sure if the opinions have changed.
>>>>
>>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>>
>>>>> Thanks Ajantha, I'd not seen that thread.
>>>>> Having looked at it, am I understanding the view to be that ideally
>>>>> Flink would publish a Docker image that included the Iceberg dependencies?
>>>>>
>>>>> However we do this, I feel that the user coming to run the Flink
>>>>> quickstart should not have to build their own Docker image; this adds
>>>>> unnecessary friction that is easily alleviated.
>>>>>
>>>>> If I've understood the situation correctly, then I'm happy to discuss
>>>>> this idea with the Flink community; please let me know before I do so.
>>>>>
>>>>> thanks, Robin.
>>>>>
>>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Robin and Peter,
>>>>>>
>>>>>> I discussed community-maintained Docker images previously:
>>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>>
>>>>>> The consensus was to publish only the REST fixture Docker image
>>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
>>>>>> total downloads) and use Docker images published by the main engines in the
>>>>>> quickstart, instead of maintaining these images ourselves.
>>>>>> See the thread above for more details.
>>>>>>
>>>>>> With respect to adding a Flink quickstart page, I’m in favor of
>>>>>> adding it and relying on the Docker images provided by Flink rather than
>>>>>> maintaining our own images.
>>>>>> - Ajantha
>>>>>>
>>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Robin,
>>>>>>> It would be nice to separate them. I expect that we will have some
>>>>>>> extra stuff to do with the docker image. For example make sure that we have
>>>>>>> ci in place to build it.
>>>>>>> Thanks,
>>>>>>> Peter
>>>>>>>
>>>>>>>
>>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>>
>>>>>>>> Thanks for the positive reception of this idea.
>>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>>
>>>>>>>> Also, should I keep this and the quickstart PR [2] as separate PRs,
>>>>>>>> or combine them?
>>>>>>>>
>>>>>>>> thanks, Robin.
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>>
>>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> This is a great idea.
>>>>>>>>>
>>>>>>>>> If we are moving forward with an "official" Docker image published
>>>>>>>>> by the project, we must ensure it is fully compliant with ASF requirements
>>>>>>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>>>>>>> it is a detail that is often overlooked.
>>>>>>>>>
>>>>>>>>> I would be happy to help with this process.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> JB
>>>>>>>>>
>>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Robin,
>>>>>>>>>>
>>>>>>>>>> +1 That's a great idea. It's often a bit painful for new users to
>>>>>>>>>> get
>>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>>
>>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>>
>>>>>>>>>> -Max
>>>>>>>>>>
>>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Hi Robin,
>>>>>>>>>> >
>>>>>>>>>> > I would love to see the Flink quickstart image in the Iceberg
>>>>>>>>>> repo.
>>>>>>>>>> >
>>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>>> > The conclusion was:
>>>>>>>>>> >>
>>>>>>>>>> >> we should both remove the vendor reference and bring this back
>>>>>>>>>> up to date. My preference would be to rely on the Spark image <
>>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>>>>>>> Spark project, similar to what we do for the Hive <
>>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We
>>>>>>>>>> should be able to load all the Iceberg-specific JARs through the
>>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>>> >
>>>>>>>>>> > Thanks for working on this,
>>>>>>>>>> > Peter
>>>>>>>>>> >
>>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>>> >>
>>>>>>>>>> >> Hi,
>>>>>>>>>> >>
>>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>>>>>> the project?
>>>>>>>>>> >> At the moment the Spark quickstart uses
>>>>>>>>>> tabulario/spark-iceberg so perhaps that could be brought into the project
>>>>>>>>>> too.
>>>>>>>>>> >>
>>>>>>>>>> >> thanks, Robin.
>>>>>>>>>> >>
>>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>
>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Kevin Liu <ke...@apache.org>.
Thanks for working on this, Robin! It looks like the complexity here is
publishing the docker image. What do you think about isolating that part?
(Just move the publish script out of #15124) We can start
with the Dockerfile definition, which allows us to build locally. This
should unblock us from merging the getting started docs in #15062
Thoughts?
Best,
Kevin Liu
On Wed, Jan 28, 2026 at 5:57 AM Robin Moffatt via dev <
dev@iceberg.apache.org> wrote:
> Hi,
>
> Thanks for the discussion and input.
> It sounds like there are no major blockers. Could someone please review
> https://github.com/apache/iceberg/pull/15124 ?
>
> thanks,
>
> Robin.
>
> On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
>
>> Hey folks,
>>
>> We have a Dockerfile defined in pyiceberg [1] that uses the Spark base
>> image and installs all the necessary jars. This is used for our integration
>> test setup [2] and is inspired by databricks/docker-spark-iceberg [3].
>> We've made many improvements such as upgrading to Spark 4, supporting Spark
>> Connect, and better image build caching.
>>
>> This is already self-contained and can be reused by other subprojects. In
>> fact, iceberg-rust already uses it [4] and I try to keep them in sync.
>> I think it would be beneficial for the project to publish this image and
>> something similar for Flink.
>>
>> Let me know what you think.
>>
>> Best,
>> Kevin Liu
>>
>>
>>
>> [1]
>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
>> [2]
>> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
>> [3]
>> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
>> [4]
>> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>>
>> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com> wrote:
>>
>>> > Since the integration code for both Spark and Flink lives in our
>>> repository, it might make sense to also store the Docker images and the
>>> corresponding scripts there.
>>>
>>> I agree with Peter here.
>>>
>>> The previous thread has some concerns if the Iceberg project should host
>>> those docker images. Not sure if the opinions have changed.
>>>
>>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>>
>>>> Thanks Ajantha, I'd not seen that thread.
>>>> Having looked at it, am I understanding the view to be that ideally
>>>> Flink would publish a Docker image that included the Iceberg dependencies?
>>>>
>>>> However we do this, I feel that the user coming to run the Flink
>>>> quickstart should not have to build their own Docker image; this adds
>>>> unnecessary friction that is easily alleviated.
>>>>
>>>> If I've understood the situation correctly, then I'm happy to discuss
>>>> this idea with the Flink community; please let me know before I do so.
>>>>
>>>> thanks, Robin.
>>>>
>>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Robin and Peter,
>>>>>
>>>>> I discussed community-maintained Docker images previously:
>>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>>
>>>>> The consensus was to publish only the REST fixture Docker image
>>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
>>>>> total downloads) and use Docker images published by the main engines in the
>>>>> quickstart, instead of maintaining these images ourselves.
>>>>> See the thread above for more details.
>>>>>
>>>>> With respect to adding a Flink quickstart page, I’m in favor of adding
>>>>> it and relying on the Docker images provided by Flink rather than
>>>>> maintaining our own images.
>>>>> - Ajantha
>>>>>
>>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>
>>>>>> Hi Robin,
>>>>>> It would be nice to separate them. I expect that we will have some
>>>>>> extra stuff to do with the docker image. For example make sure that we have
>>>>>> ci in place to build it.
>>>>>> Thanks,
>>>>>> Peter
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>>> dev@iceberg.apache.org> wrote:
>>>>>>
>>>>>>> Thanks for the positive reception of this idea.
>>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>>
>>>>>>> Also, should I keep this and the quickstart PR [2] as separate PRs,
>>>>>>> or combine them?
>>>>>>>
>>>>>>> thanks, Robin.
>>>>>>>
>>>>>>>
>>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>>
>>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> This is a great idea.
>>>>>>>>
>>>>>>>> If we are moving forward with an "official" Docker image published
>>>>>>>> by the project, we must ensure it is fully compliant with ASF requirements
>>>>>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>>>>>> it is a detail that is often overlooked.
>>>>>>>>
>>>>>>>> I would be happy to help with this process.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> JB
>>>>>>>>
>>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey Robin,
>>>>>>>>>
>>>>>>>>> +1 That's a great idea. It's often a bit painful for new users to
>>>>>>>>> get
>>>>>>>>> all the dependencies in the right place.
>>>>>>>>>
>>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>>
>>>>>>>>> -Max
>>>>>>>>>
>>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>>> >
>>>>>>>>> > Hi Robin,
>>>>>>>>> >
>>>>>>>>> > I would love to see the Flink quickstart image in the Iceberg
>>>>>>>>> repo.
>>>>>>>>> >
>>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>>> > The conclusion was:
>>>>>>>>> >>
>>>>>>>>> >> we should both remove the vendor reference and bring this back
>>>>>>>>> up to date. My preference would be to rely on the Spark image <
>>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>>>>>> Spark project, similar to what we do for the Hive <
>>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We
>>>>>>>>> should be able to load all the Iceberg-specific JARs through the
>>>>>>>>> spark.jars.packages configuration <
>>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>>> >
>>>>>>>>> >
>>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>>> >
>>>>>>>>> > Thanks for working on this,
>>>>>>>>> > Peter
>>>>>>>>> >
>>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>>> >>
>>>>>>>>> >> Hi,
>>>>>>>>> >>
>>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>>>>> the project?
>>>>>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg
>>>>>>>>> so perhaps that could be brought into the project too.
>>>>>>>>> >>
>>>>>>>>> >> thanks, Robin.
>>>>>>>>> >>
>>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
Hi,
Thanks for the discussion and input.
It sounds like there are no major blockers. Could someone please review
https://github.com/apache/iceberg/pull/15124 ?
thanks,
Robin.
On Mon, 26 Jan 2026 at 16:36, Kevin Liu <ke...@apache.org> wrote:
> Hey folks,
>
> We have a Dockerfile defined in pyiceberg [1] that uses the Spark base
> image and installs all the necessary jars. This is used for our integration
> test setup [2] and is inspired by databricks/docker-spark-iceberg [3].
> We've made many improvements such as upgrading to Spark 4, supporting Spark
> Connect, and better image build caching.
>
> This is already self-contained and can be reused by other subprojects. In
> fact, iceberg-rust already uses it [4] and I try to keep them in sync.
> I think it would be beneficial for the project to publish this image and
> something similar for Flink.
>
> Let me know what you think.
>
> Best,
> Kevin Liu
>
>
>
> [1]
> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
> [2]
> https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
> [3]
> https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
> [4]
> https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
>
> On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com> wrote:
>
>> > Since the integration code for both Spark and Flink lives in our
>> repository, it might make sense to also store the Docker images and the
>> corresponding scripts there.
>>
>> I agree with Peter here.
>>
>> The previous thread has some concerns if the Iceberg project should host
>> those docker images. Not sure if the opinions have changed.
>>
>> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
>> dev@iceberg.apache.org> wrote:
>>
>>> Thanks Ajantha, I'd not seen that thread.
>>> Having looked at it, am I understanding the view to be that ideally
>>> Flink would publish a Docker image that included the Iceberg dependencies?
>>>
>>> However we do this, I feel that the user coming to run the Flink
>>> quickstart should not have to build their own Docker image; this adds
>>> unnecessary friction that is easily alleviated.
>>>
>>> If I've understood the situation correctly, then I'm happy to discuss
>>> this idea with the Flink community; please let me know before I do so.
>>>
>>> thanks, Robin.
>>>
>>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com>
>>> wrote:
>>>
>>>> Hi Robin and Peter,
>>>>
>>>> I discussed community-maintained Docker images previously:
>>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>>
>>>> The consensus was to publish only the REST fixture Docker image
>>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
>>>> total downloads) and use Docker images published by the main engines in the
>>>> quickstart, instead of maintaining these images ourselves.
>>>> See the thread above for more details.
>>>>
>>>> With respect to adding a Flink quickstart page, I’m in favor of adding
>>>> it and relying on the Docker images provided by Flink rather than
>>>> maintaining our own images.
>>>> - Ajantha
>>>>
>>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <pe...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Robin,
>>>>> It would be nice to separate them. I expect that we will have some
>>>>> extra stuff to do with the docker image. For example make sure that we have
>>>>> ci in place to build it.
>>>>> Thanks,
>>>>> Peter
>>>>>
>>>>>
>>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>>
>>>>>> Thanks for the positive reception of this idea.
>>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>>
>>>>>> Also, should I keep this and the quickstart PR [2] as separate PRs,
>>>>>> or combine them?
>>>>>>
>>>>>> thanks, Robin.
>>>>>>
>>>>>>
>>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>>
>>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> This is a great idea.
>>>>>>>
>>>>>>> If we are moving forward with an "official" Docker image published
>>>>>>> by the project, we must ensure it is fully compliant with ASF requirements
>>>>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>>>>> it is a detail that is often overlooked.
>>>>>>>
>>>>>>> I would be happy to help with this process.
>>>>>>>
>>>>>>> Regards,
>>>>>>> JB
>>>>>>>
>>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hey Robin,
>>>>>>>>
>>>>>>>> +1 That's a great idea. It's often a bit painful for new users to
>>>>>>>> get
>>>>>>>> all the dependencies in the right place.
>>>>>>>>
>>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>>
>>>>>>>> -Max
>>>>>>>>
>>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>>> >
>>>>>>>> > Hi Robin,
>>>>>>>> >
>>>>>>>> > I would love to see the Flink quickstart image in the Iceberg
>>>>>>>> repo.
>>>>>>>> >
>>>>>>>> > Ajantha was working on the Spark side:
>>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>>> > The conclusion was:
>>>>>>>> >>
>>>>>>>> >> we should both remove the vendor reference and bring this back
>>>>>>>> up to date. My preference would be to rely on the Spark image <
>>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache
>>>>>>>> Spark project, similar to what we do for the Hive <
>>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should
>>>>>>>> be able to load all the Iceberg-specific JARs through the
>>>>>>>> spark.jars.packages configuration <
>>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>>> >
>>>>>>>> > Thanks for working on this,
>>>>>>>> > Peter
>>>>>>>> >
>>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta
>>>>>>>> (időpont: 2026. jan. 22., Cs, 19:23):
>>>>>>>> >>
>>>>>>>> >> Hi,
>>>>>>>> >>
>>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>>>> the project?
>>>>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg
>>>>>>>> so perhaps that could be brought into the project too.
>>>>>>>> >>
>>>>>>>> >> thanks, Robin.
>>>>>>>> >>
>>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Kevin Liu <ke...@apache.org>.
Hey folks,
We have a Dockerfile defined in pyiceberg [1] that uses the Spark base
image and installs all the necessary jars. This is used for our integration
test setup [2] and is inspired by databricks/docker-spark-iceberg [3].
We've made many improvements such as upgrading to Spark 4, supporting Spark
Connect, and better image build caching.
This is already self-contained and can be reused by other subprojects. In
fact, iceberg-rust already uses it [4] and I try to keep them in sync.
I think it would be beneficial for the project to publish this image and
something similar for Flink.
Let me know what you think.
Best,
Kevin Liu
[1]
https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/spark/Dockerfile
[2]
https://github.com/apache/iceberg-python/blob/6de6d6acad440885788fb1a24c04ed647b92af0e/dev/docker-compose-integration.yml#L20-L21
[3]
https://github.com/databricks/docker-spark-iceberg/blob/cf617dc29e8672792e76b9bcf6017af52f570020/spark/Dockerfile
[4]
https://github.com/apache/iceberg-rust/blob/330f21da894948fc10b57d541cb2d6f32c8bdbb8/crates/integration_tests/testdata/spark/Dockerfile
On Mon, Jan 26, 2026 at 10:27 AM Steven Wu <st...@gmail.com> wrote:
> > Since the integration code for both Spark and Flink lives in our
> repository, it might make sense to also store the Docker images and the
> corresponding scripts there.
>
> I agree with Peter here.
>
> The previous thread has some concerns if the Iceberg project should host
> those docker images. Not sure if the opinions have changed.
>
> On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
> dev@iceberg.apache.org> wrote:
>
>> Thanks Ajantha, I'd not seen that thread.
>> Having looked at it, am I understanding the view to be that ideally Flink
>> would publish a Docker image that included the Iceberg dependencies?
>>
>> However we do this, I feel that the user coming to run the Flink
>> quickstart should not have to build their own Docker image; this adds
>> unnecessary friction that is easily alleviated.
>>
>> If I've understood the situation correctly, then I'm happy to discuss
>> this idea with the Flink community; please let me know before I do so.
>>
>> thanks, Robin.
>>
>> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com> wrote:
>>
>>> Hi Robin and Peter,
>>>
>>> I discussed community-maintained Docker images previously:
>>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>>
>>> The consensus was to publish only the REST fixture Docker image
>>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
>>> total downloads) and use Docker images published by the main engines in the
>>> quickstart, instead of maintaining these images ourselves.
>>> See the thread above for more details.
>>>
>>> With respect to adding a Flink quickstart page, I’m in favor of adding
>>> it and relying on the Docker images provided by Flink rather than
>>> maintaining our own images.
>>> - Ajantha
>>>
>>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <pe...@gmail.com>
>>> wrote:
>>>
>>>> Hi Robin,
>>>> It would be nice to separate them. I expect that we will have some
>>>> extra stuff to do with the docker image. For example make sure that we have
>>>> ci in place to build it.
>>>> Thanks,
>>>> Peter
>>>>
>>>>
>>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>>
>>>>> Thanks for the positive reception of this idea.
>>>>> I've drafted a PR [1] and would appreciate input :)
>>>>>
>>>>> Also, should I keep this and the quickstart PR [2] as separate PRs, or
>>>>> combine them?
>>>>>
>>>>> thanks, Robin.
>>>>>
>>>>>
>>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>>
>>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> This is a great idea.
>>>>>>
>>>>>> If we are moving forward with an "official" Docker image published by
>>>>>> the project, we must ensure it is fully compliant with ASF requirements
>>>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>>>> it is a detail that is often overlooked.
>>>>>>
>>>>>> I would be happy to help with this process.
>>>>>>
>>>>>> Regards,
>>>>>> JB
>>>>>>
>>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey Robin,
>>>>>>>
>>>>>>> +1 That's a great idea. It's often a bit painful for new users to get
>>>>>>> all the dependencies in the right place.
>>>>>>>
>>>>>>> +1 for building upon the official Flink Docker images:
>>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>>
>>>>>>> -Max
>>>>>>>
>>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>>> >
>>>>>>> > Hi Robin,
>>>>>>> >
>>>>>>> > I would love to see the Flink quickstart image in the Iceberg repo.
>>>>>>> >
>>>>>>> > Ajantha was working on the Spark side:
>>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>>> > The conclusion was:
>>>>>>> >>
>>>>>>> >> we should both remove the vendor reference and bring this back up
>>>>>>> to date. My preference would be to rely on the Spark image <
>>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
>>>>>>> project, similar to what we do for the Hive <
>>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should
>>>>>>> be able to load all the Iceberg-specific JARs through the
>>>>>>> spark.jars.packages configuration <
>>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>>> >
>>>>>>> >
>>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>>> >
>>>>>>> > Thanks for working on this,
>>>>>>> > Peter
>>>>>>> >
>>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>>>>> 2026. jan. 22., Cs, 19:23):
>>>>>>> >>
>>>>>>> >> Hi,
>>>>>>> >>
>>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>>> the project?
>>>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg
>>>>>>> so perhaps that could be brought into the project too.
>>>>>>> >>
>>>>>>> >> thanks, Robin.
>>>>>>> >>
>>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Steven Wu <st...@gmail.com>.
> Since the integration code for both Spark and Flink lives in our
repository, it might make sense to also store the Docker images and the
corresponding scripts there.
I agree with Peter here.
The previous thread has some concerns if the Iceberg project should host
those docker images. Not sure if the opinions have changed.
On Mon, Jan 26, 2026 at 2:43 AM Robin Moffatt via dev <
dev@iceberg.apache.org> wrote:
> Thanks Ajantha, I'd not seen that thread.
> Having looked at it, am I understanding the view to be that ideally Flink
> would publish a Docker image that included the Iceberg dependencies?
>
> However we do this, I feel that the user coming to run the Flink
> quickstart should not have to build their own Docker image; this adds
> unnecessary friction that is easily alleviated.
>
> If I've understood the situation correctly, then I'm happy to discuss this
> idea with the Flink community; please let me know before I do so.
>
> thanks, Robin.
>
> On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com> wrote:
>
>> Hi Robin and Peter,
>>
>> I discussed community-maintained Docker images previously:
>> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>>
>> The consensus was to publish only the REST fixture Docker image
>> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
>> total downloads) and use Docker images published by the main engines in the
>> quickstart, instead of maintaining these images ourselves.
>> See the thread above for more details.
>>
>> With respect to adding a Flink quickstart page, I’m in favor of adding it
>> and relying on the Docker images provided by Flink rather than maintaining
>> our own images.
>> - Ajantha
>>
>> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <pe...@gmail.com>
>> wrote:
>>
>>> Hi Robin,
>>> It would be nice to separate them. I expect that we will have some extra
>>> stuff to do with the docker image. For example make sure that we have ci in
>>> place to build it.
>>> Thanks,
>>> Peter
>>>
>>>
>>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <
>>> dev@iceberg.apache.org> wrote:
>>>
>>>> Thanks for the positive reception of this idea.
>>>> I've drafted a PR [1] and would appreciate input :)
>>>>
>>>> Also, should I keep this and the quickstart PR [2] as separate PRs, or
>>>> combine them?
>>>>
>>>> thanks, Robin.
>>>>
>>>>
>>>> [1] https://github.com/apache/iceberg/pull/15124
>>>> [2] https://github.com/apache/iceberg/pull/15062
>>>>
>>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> This is a great idea.
>>>>>
>>>>> If we are moving forward with an "official" Docker image published by
>>>>> the project, we must ensure it is fully compliant with ASF requirements
>>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>>> it is a detail that is often overlooked.
>>>>>
>>>>> I would be happy to help with this process.
>>>>>
>>>>> Regards,
>>>>> JB
>>>>>
>>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Hey Robin,
>>>>>>
>>>>>> +1 That's a great idea. It's often a bit painful for new users to get
>>>>>> all the dependencies in the right place.
>>>>>>
>>>>>> +1 for building upon the official Flink Docker images:
>>>>>> https://hub.docker.com/r/apache/flink
>>>>>>
>>>>>> -Max
>>>>>>
>>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>>> peter.vary.apache@gmail.com> wrote:
>>>>>> >
>>>>>> > Hi Robin,
>>>>>> >
>>>>>> > I would love to see the Flink quickstart image in the Iceberg repo.
>>>>>> >
>>>>>> > Ajantha was working on the Spark side:
>>>>>> https://github.com/apache/iceberg/issues/13519
>>>>>> > The conclusion was:
>>>>>> >>
>>>>>> >> we should both remove the vendor reference and bring this back up
>>>>>> to date. My preference would be to rely on the Spark image <
>>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
>>>>>> project, similar to what we do for the Hive <
>>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should
>>>>>> be able to load all the Iceberg-specific JARs through the
>>>>>> spark.jars.packages configuration <
>>>>>> https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>>>> >
>>>>>> >
>>>>>> > Ajantha also added the link to the old dev list thread:
>>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>>> >
>>>>>> > Thanks for working on this,
>>>>>> > Peter
>>>>>> >
>>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>>>> 2026. jan. 22., Cs, 19:23):
>>>>>> >>
>>>>>> >> Hi,
>>>>>> >>
>>>>>> >> Following discussion on the Flink quickstart PR [1], what do
>>>>>> people think about adding an official quickstart Docker image for Flink to
>>>>>> the project?
>>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so
>>>>>> perhaps that could be brought into the project too.
>>>>>> >>
>>>>>> >> thanks, Robin.
>>>>>> >>
>>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>>
>>>>>
>>>>
>>>>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
Thanks Ajantha, I'd not seen that thread.
Having looked at it, am I understanding the view to be that ideally Flink
would publish a Docker image that included the Iceberg dependencies?
However we do this, I feel that the user coming to run the Flink quickstart
should not have to build their own Docker image; this adds
unnecessary friction that is easily alleviated.
If I've understood the situation correctly, then I'm happy to discuss this
idea with the Flink community; please let me know before I do so.
thanks, Robin.
On Fri, 23 Jan 2026 at 16:50, Ajantha Bhat <aj...@gmail.com> wrote:
> Hi Robin and Peter,
>
> I discussed community-maintained Docker images previously:
> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>
> The consensus was to publish only the REST fixture Docker image
> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
> total downloads) and use Docker images published by the main engines in the
> quickstart, instead of maintaining these images ourselves.
> See the thread above for more details.
>
> With respect to adding a Flink quickstart page, I’m in favor of adding it
> and relying on the Docker images provided by Flink rather than maintaining
> our own images.
> - Ajantha
>
> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <pe...@gmail.com>
> wrote:
>
>> Hi Robin,
>> It would be nice to separate them. I expect that we will have some extra
>> stuff to do with the docker image. For example make sure that we have ci in
>> place to build it.
>> Thanks,
>> Peter
>>
>>
>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <de...@iceberg.apache.org>
>> wrote:
>>
>>> Thanks for the positive reception of this idea.
>>> I've drafted a PR [1] and would appreciate input :)
>>>
>>> Also, should I keep this and the quickstart PR [2] as separate PRs, or
>>> combine them?
>>>
>>> thanks, Robin.
>>>
>>>
>>> [1] https://github.com/apache/iceberg/pull/15124
>>> [2] https://github.com/apache/iceberg/pull/15062
>>>
>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> This is a great idea.
>>>>
>>>> If we are moving forward with an "official" Docker image published by
>>>> the project, we must ensure it is fully compliant with ASF requirements
>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>> it is a detail that is often overlooked.
>>>>
>>>> I would be happy to help with this process.
>>>>
>>>> Regards,
>>>> JB
>>>>
>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>> wrote:
>>>>
>>>>> Hey Robin,
>>>>>
>>>>> +1 That's a great idea. It's often a bit painful for new users to get
>>>>> all the dependencies in the right place.
>>>>>
>>>>> +1 for building upon the official Flink Docker images:
>>>>> https://hub.docker.com/r/apache/flink
>>>>>
>>>>> -Max
>>>>>
>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>> >
>>>>> > Hi Robin,
>>>>> >
>>>>> > I would love to see the Flink quickstart image in the Iceberg repo.
>>>>> >
>>>>> > Ajantha was working on the Spark side:
>>>>> https://github.com/apache/iceberg/issues/13519
>>>>> > The conclusion was:
>>>>> >>
>>>>> >> we should both remove the vendor reference and bring this back up
>>>>> to date. My preference would be to rely on the Spark image <
>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
>>>>> project, similar to what we do for the Hive <
>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>>>>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>>>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html
>>>>> >.
>>>>> >
>>>>> >
>>>>> > Ajantha also added the link to the old dev list thread:
>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>> >
>>>>> > Thanks for working on this,
>>>>> > Peter
>>>>> >
>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>>> 2026. jan. 22., Cs, 19:23):
>>>>> >>
>>>>> >> Hi,
>>>>> >>
>>>>> >> Following discussion on the Flink quickstart PR [1], what do people
>>>>> think about adding an official quickstart Docker image for Flink to the
>>>>> project?
>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so
>>>>> perhaps that could be brought into the project too.
>>>>> >>
>>>>> >> thanks, Robin.
>>>>> >>
>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>
>>>>
>>>
>>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Péter Váry <pe...@gmail.com>.
Hi Ajantha,
This discussion took place more than a year ago, and I still don’t see
quickstart Docker images available for either Spark or Flink. Since the
integration code for both Spark and Flink lives in our repository, it might
make sense to also store the Docker images and the corresponding scripts
there.
Do we think the Spark community would be open to adding a Dockerfile that
includes Iceberg code to their repository?
Thanks,
Peter
Ajantha Bhat <aj...@gmail.com> ezt írta (időpont: 2026. jan. 23., P,
17:50):
> Hi Robin and Peter,
>
> I discussed community-maintained Docker images previously:
> https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
>
> The consensus was to publish only the REST fixture Docker image
> <https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+
> total downloads) and use Docker images published by the main engines in the
> quickstart, instead of maintaining these images ourselves.
> See the thread above for more details.
>
> With respect to adding a Flink quickstart page, I’m in favor of adding it
> and relying on the Docker images provided by Flink rather than maintaining
> our own images.
> - Ajantha
>
> On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <pe...@gmail.com>
> wrote:
>
>> Hi Robin,
>> It would be nice to separate them. I expect that we will have some extra
>> stuff to do with the docker image. For example make sure that we have ci in
>> place to build it.
>> Thanks,
>> Peter
>>
>>
>> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <de...@iceberg.apache.org>
>> wrote:
>>
>>> Thanks for the positive reception of this idea.
>>> I've drafted a PR [1] and would appreciate input :)
>>>
>>> Also, should I keep this and the quickstart PR [2] as separate PRs, or
>>> combine them?
>>>
>>> thanks, Robin.
>>>
>>>
>>> [1] https://github.com/apache/iceberg/pull/15124
>>> [2] https://github.com/apache/iceberg/pull/15062
>>>
>>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> This is a great idea.
>>>>
>>>> If we are moving forward with an "official" Docker image published by
>>>> the project, we must ensure it is fully compliant with ASF requirements
>>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>>> it is a detail that is often overlooked.
>>>>
>>>> I would be happy to help with this process.
>>>>
>>>> Regards,
>>>> JB
>>>>
>>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>>> wrote:
>>>>
>>>>> Hey Robin,
>>>>>
>>>>> +1 That's a great idea. It's often a bit painful for new users to get
>>>>> all the dependencies in the right place.
>>>>>
>>>>> +1 for building upon the official Flink Docker images:
>>>>> https://hub.docker.com/r/apache/flink
>>>>>
>>>>> -Max
>>>>>
>>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>>> peter.vary.apache@gmail.com> wrote:
>>>>> >
>>>>> > Hi Robin,
>>>>> >
>>>>> > I would love to see the Flink quickstart image in the Iceberg repo.
>>>>> >
>>>>> > Ajantha was working on the Spark side:
>>>>> https://github.com/apache/iceberg/issues/13519
>>>>> > The conclusion was:
>>>>> >>
>>>>> >> we should both remove the vendor reference and bring this back up
>>>>> to date. My preference would be to rely on the Spark image <
>>>>> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
>>>>> project, similar to what we do for the Hive <
>>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>>>>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>>>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html
>>>>> >.
>>>>> >
>>>>> >
>>>>> > Ajantha also added the link to the old dev list thread:
>>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>>> >
>>>>> > Thanks for working on this,
>>>>> > Peter
>>>>> >
>>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>>> 2026. jan. 22., Cs, 19:23):
>>>>> >>
>>>>> >> Hi,
>>>>> >>
>>>>> >> Following discussion on the Flink quickstart PR [1], what do people
>>>>> think about adding an official quickstart Docker image for Flink to the
>>>>> project?
>>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so
>>>>> perhaps that could be brought into the project too.
>>>>> >>
>>>>> >> thanks, Robin.
>>>>> >>
>>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>>
>>>>
>>>
>>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Ajantha Bhat <aj...@gmail.com>.
Hi Robin and Peter,
I discussed community-maintained Docker images previously:
https://lists.apache.org/thread/xl1cwq7vmnh6zgfd2vck2nq7dfd33ncq
The consensus was to publish only the REST fixture Docker image
<https://hub.docker.com/r/apache/iceberg-rest-fixture> (now at 100K+ total
downloads) and use Docker images published by the main engines in the
quickstart, instead of maintaining these images ourselves.
See the thread above for more details.
With respect to adding a Flink quickstart page, I’m in favor of adding it
and relying on the Docker images provided by Flink rather than maintaining
our own images.
- Ajantha
On Fri, Jan 23, 2026 at 9:43 PM Péter Váry <pe...@gmail.com>
wrote:
> Hi Robin,
> It would be nice to separate them. I expect that we will have some extra
> stuff to do with the docker image. For example make sure that we have ci in
> place to build it.
> Thanks,
> Peter
>
>
> On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <de...@iceberg.apache.org>
> wrote:
>
>> Thanks for the positive reception of this idea.
>> I've drafted a PR [1] and would appreciate input :)
>>
>> Also, should I keep this and the quickstart PR [2] as separate PRs, or
>> combine them?
>>
>> thanks, Robin.
>>
>>
>> [1] https://github.com/apache/iceberg/pull/15124
>> [2] https://github.com/apache/iceberg/pull/15062
>>
>> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
>> wrote:
>>
>>> Hi,
>>>
>>> This is a great idea.
>>>
>>> If we are moving forward with an "official" Docker image published by
>>> the project, we must ensure it is fully compliant with ASF requirements
>>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>>> it is a detail that is often overlooked.
>>>
>>> I would be happy to help with this process.
>>>
>>> Regards,
>>> JB
>>>
>>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>>> wrote:
>>>
>>>> Hey Robin,
>>>>
>>>> +1 That's a great idea. It's often a bit painful for new users to get
>>>> all the dependencies in the right place.
>>>>
>>>> +1 for building upon the official Flink Docker images:
>>>> https://hub.docker.com/r/apache/flink
>>>>
>>>> -Max
>>>>
>>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <
>>>> peter.vary.apache@gmail.com> wrote:
>>>> >
>>>> > Hi Robin,
>>>> >
>>>> > I would love to see the Flink quickstart image in the Iceberg repo.
>>>> >
>>>> > Ajantha was working on the Spark side:
>>>> https://github.com/apache/iceberg/issues/13519
>>>> > The conclusion was:
>>>> >>
>>>> >> we should both remove the vendor reference and bring this back up to
>>>> date. My preference would be to rely on the Spark image <
>>>> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
>>>> project, similar to what we do for the Hive <
>>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>>>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>>>> >
>>>> >
>>>> > Ajantha also added the link to the old dev list thread:
>>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>>> >
>>>> > Thanks for working on this,
>>>> > Peter
>>>> >
>>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>>> 2026. jan. 22., Cs, 19:23):
>>>> >>
>>>> >> Hi,
>>>> >>
>>>> >> Following discussion on the Flink quickstart PR [1], what do people
>>>> think about adding an official quickstart Docker image for Flink to the
>>>> project?
>>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so
>>>> perhaps that could be brought into the project too.
>>>> >>
>>>> >> thanks, Robin.
>>>> >>
>>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>>
>>>
>>
>>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Péter Váry <pe...@gmail.com>.
Hi Robin,
It would be nice to separate them. I expect that we will have some extra
stuff to do with the docker image. For example make sure that we have ci in
place to build it.
Thanks,
Peter
On Fri, Jan 23, 2026, 16:55 Robin Moffatt via dev <de...@iceberg.apache.org>
wrote:
> Thanks for the positive reception of this idea.
> I've drafted a PR [1] and would appreciate input :)
>
> Also, should I keep this and the quickstart PR [2] as separate PRs, or
> combine them?
>
> thanks, Robin.
>
>
> [1] https://github.com/apache/iceberg/pull/15124
> [2] https://github.com/apache/iceberg/pull/15062
>
> On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net>
> wrote:
>
>> Hi,
>>
>> This is a great idea.
>>
>> If we are moving forward with an "official" Docker image published by the
>> project, we must ensure it is fully compliant with ASF requirements
>> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
>> it is a detail that is often overlooked.
>>
>> I would be happy to help with this process.
>>
>> Regards,
>> JB
>>
>> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org>
>> wrote:
>>
>>> Hey Robin,
>>>
>>> +1 That's a great idea. It's often a bit painful for new users to get
>>> all the dependencies in the right place.
>>>
>>> +1 for building upon the official Flink Docker images:
>>> https://hub.docker.com/r/apache/flink
>>>
>>> -Max
>>>
>>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <pe...@gmail.com>
>>> wrote:
>>> >
>>> > Hi Robin,
>>> >
>>> > I would love to see the Flink quickstart image in the Iceberg repo.
>>> >
>>> > Ajantha was working on the Spark side:
>>> https://github.com/apache/iceberg/issues/13519
>>> > The conclusion was:
>>> >>
>>> >> we should both remove the vendor reference and bring this back up to
>>> date. My preference would be to rely on the Spark image <
>>> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
>>> project, similar to what we do for the Hive <
>>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>>> >
>>> >
>>> > Ajantha also added the link to the old dev list thread:
>>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>>> >
>>> > Thanks for working on this,
>>> > Peter
>>> >
>>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>>> 2026. jan. 22., Cs, 19:23):
>>> >>
>>> >> Hi,
>>> >>
>>> >> Following discussion on the Flink quickstart PR [1], what do people
>>> think about adding an official quickstart Docker image for Flink to the
>>> project?
>>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so
>>> perhaps that could be brought into the project too.
>>> >>
>>> >> thanks, Robin.
>>> >>
>>> >> 1: https://github.com/apache/iceberg/pull/15062
>>>
>>
>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Robin Moffatt via dev <de...@iceberg.apache.org>.
Thanks for the positive reception of this idea.
I've drafted a PR [1] and would appreciate input :)
Also, should I keep this and the quickstart PR [2] as separate PRs, or
combine them?
thanks, Robin.
[1] https://github.com/apache/iceberg/pull/15124
[2] https://github.com/apache/iceberg/pull/15062
On Fri, 23 Jan 2026 at 13:58, Jean-Baptiste Onofré <jb...@nanthrax.net> wrote:
> Hi,
>
> This is a great idea.
>
> If we are moving forward with an "official" Docker image published by the
> project, we must ensure it is fully compliant with ASF requirements
> regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
> it is a detail that is often overlooked.
>
> I would be happy to help with this process.
>
> Regards,
> JB
>
> On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org> wrote:
>
>> Hey Robin,
>>
>> +1 That's a great idea. It's often a bit painful for new users to get
>> all the dependencies in the right place.
>>
>> +1 for building upon the official Flink Docker images:
>> https://hub.docker.com/r/apache/flink
>>
>> -Max
>>
>> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <pe...@gmail.com>
>> wrote:
>> >
>> > Hi Robin,
>> >
>> > I would love to see the Flink quickstart image in the Iceberg repo.
>> >
>> > Ajantha was working on the Spark side:
>> https://github.com/apache/iceberg/issues/13519
>> > The conclusion was:
>> >>
>> >> we should both remove the vendor reference and bring this back up to
>> date. My preference would be to rely on the Spark image <
>> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
>> project, similar to what we do for the Hive <
>> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
>> able to load all the Iceberg-specific JARs through the spark.jars.packages
>> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>> >
>> >
>> > Ajantha also added the link to the old dev list thread:
>> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>> >
>> > Thanks for working on this,
>> > Peter
>> >
>> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont:
>> 2026. jan. 22., Cs, 19:23):
>> >>
>> >> Hi,
>> >>
>> >> Following discussion on the Flink quickstart PR [1], what do people
>> think about adding an official quickstart Docker image for Flink to the
>> project?
>> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so
>> perhaps that could be brought into the project too.
>> >>
>> >> thanks, Robin.
>> >>
>> >> 1: https://github.com/apache/iceberg/pull/15062
>>
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi,
This is a great idea.
If we are moving forward with an "official" Docker image published by the
project, we must ensure it is fully compliant with ASF requirements
regarding LICENSE/NOTICE files, etc. While this may seem straightforward,
it is a detail that is often overlooked.
I would be happy to help with this process.
Regards,
JB
On Fri, Jan 23, 2026 at 1:52 PM Maximilian Michels <mx...@apache.org> wrote:
> Hey Robin,
>
> +1 That's a great idea. It's often a bit painful for new users to get
> all the dependencies in the right place.
>
> +1 for building upon the official Flink Docker images:
> https://hub.docker.com/r/apache/flink
>
> -Max
>
> On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <pe...@gmail.com>
> wrote:
> >
> > Hi Robin,
> >
> > I would love to see the Flink quickstart image in the Iceberg repo.
> >
> > Ajantha was working on the Spark side:
> https://github.com/apache/iceberg/issues/13519
> > The conclusion was:
> >>
> >> we should both remove the vendor reference and bring this back up to
> date. My preference would be to rely on the Spark image <
> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
> project, similar to what we do for the Hive <
> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
> able to load all the Iceberg-specific JARs through the spark.jars.packages
> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
> >
> >
> > Ajantha also added the link to the old dev list thread:
> https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
> >
> > Thanks for working on this,
> > Peter
> >
> > Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
> jan. 22., Cs, 19:23):
> >>
> >> Hi,
> >>
> >> Following discussion on the Flink quickstart PR [1], what do people
> think about adding an official quickstart Docker image for Flink to the
> project?
> >> At the moment the Spark quickstart uses tabulario/spark-iceberg so
> perhaps that could be brought into the project too.
> >>
> >> thanks, Robin.
> >>
> >> 1: https://github.com/apache/iceberg/pull/15062
>
Re: [DISCUSS] Add a Flink quickstart image
Posted by Maximilian Michels <mx...@apache.org>.
Hey Robin,
+1 That's a great idea. It's often a bit painful for new users to get
all the dependencies in the right place.
+1 for building upon the official Flink Docker images:
https://hub.docker.com/r/apache/flink
-Max
On Fri, Jan 23, 2026 at 12:27 PM Péter Váry <pe...@gmail.com> wrote:
>
> Hi Robin,
>
> I would love to see the Flink quickstart image in the Iceberg repo.
>
> Ajantha was working on the Spark side: https://github.com/apache/iceberg/issues/13519
> The conclusion was:
>>
>> we should both remove the vendor reference and bring this back up to date. My preference would be to rely on the Spark image <https://hub.docker.com/r/apache/spark> provided by the Apache Spark project, similar to what we do for the Hive <https://iceberg.apache.org/hive-quickstart/> quickstart. We should be able to load all the Iceberg-specific JARs through the spark.jars.packages configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
>
>
> Ajantha also added the link to the old dev list thread: https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
>
> Thanks for working on this,
> Peter
>
> Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026. jan. 22., Cs, 19:23):
>>
>> Hi,
>>
>> Following discussion on the Flink quickstart PR [1], what do people think about adding an official quickstart Docker image for Flink to the project?
>> At the moment the Spark quickstart uses tabulario/spark-iceberg so perhaps that could be brought into the project too.
>>
>> thanks, Robin.
>>
>> 1: https://github.com/apache/iceberg/pull/15062
Re: [DISCUSS] Add a Flink quickstart image
Posted by Péter Váry <pe...@gmail.com>.
Hi Robin,
I would love to see the Flink quickstart image in the Iceberg repo.
Ajantha was working on the Spark side:
https://github.com/apache/iceberg/issues/13519
The conclusion was:
> we should both remove the vendor reference and bring this back up to date.
> My preference would be to rely on the Spark image <
> https://hub.docker.com/r/apache/spark> provided by the Apache Spark
> project, similar to what we do for the Hive <
> https://iceberg.apache.org/hive-quickstart/> quickstart. We should be
> able to load all the Iceberg-specific JARs through the spark.jars.packages
> configuration <https://spark.apache.org/docs/3.5.1/configuration.html>.
Ajantha also added the link to the old dev list thread:
https://lists.apache.org/thread/4kknk8mvnffbmhdt63z8t4ps0mt1jbf4
Thanks for working on this,
Peter
Robin Moffatt via dev <de...@iceberg.apache.org> ezt írta (időpont: 2026.
jan. 22., Cs, 19:23):
> Hi,
>
> Following discussion on the Flink quickstart PR [1], what do people think
> about adding an official quickstart Docker image for Flink to the project?
> At the moment the Spark quickstart uses tabulario/spark-iceberg so perhaps
> that could be brought into the project too.
>
> thanks, Robin.
>
> 1: https://github.com/apache/iceberg/pull/15062
>