You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Alex Stephen <al...@google.com.INVALID> on 2025/10/15 17:14:35 UTC

Proposal: Iceberg Terraform Provider

Hello,

I'd like to propose the creation of an Iceberg Terraform Provider for the
Iceberg REST Catalog. There currently exists no vendor-neutral solution for
managing Iceberg resources using Terraform/OpenTofu. The proposal is for
creating a Terraform Provider that can connect to an arbitrary Iceberg REST
Catalog and manage namespaces/tables/views.

The proposal is located at https://s.apache.org/iceberg-catalog-terraform

Please take a look. I'm planning on bringing this up at the next Iceberg
Catalog sync.

Thanks a lot.

-- Alex Stephen

Re: Proposal: Iceberg Terraform Provider

Posted by Alex Stephen <al...@google.com.INVALID>.
Thanks a lot for the conversation in today's Catalog sync. It sounds like
there's a lot of interest in moving forward with the creation of a
Terraform Iceberg Provider.

I've updated the doc <https://s.apache.org/iceberg-catalog-terraform> with
the changes we discussed around licensing and resource identifiers.

On Wed, Oct 15, 2025 at 4:05 PM Alex Stephen <al...@google.com> wrote:

> > IRC doesn't have these concepts. It's only possible with a specific IRC
> implementation.
>
> It would be great to collaborate with the Polaris community, since Polaris
> has many of those concepts built-in already.
>
> The provider could potentially include more implementation-specific
> resources in the future. One other example would be creating catalogs,
> where the provider could interface with Polaris or other vendors to create
> a catalog.
>
> For now, namespaces, tables, and views are the most agnostic resources and
> all supported by the REST Catalog spec.
>
>
> On Wed, Oct 15, 2025 at 3:09 PM Yufei Gu <fl...@gmail.com> wrote:
>
>> > Out of curiosity, why not extend this integration to also cover roles,
>> principals, permissions and etc?
>>
>> IRC doesn't have these concepts. It's only possible with a specific IRC
>> implementation.
>>
>> Yufei
>>
>>
>> On Wed, Oct 15, 2025 at 2:43 PM Jayce Slesar <ja...@gmail.com> wrote:
>>
>>> I think this is a great idea!
>>>
>>> I have done a lot of existing work fir tables + maintenance via IaC via
>>> CDK, but something cloud agnostic would be amazing.
>>>
>>> It would be great to start with a catalog but I am betting there are
>>> other things we could add over time.
>>>
>>> On Wed, Oct 15, 2025 at 5:36 PM Endi Caushi <mr...@gmail.com> wrote:
>>>
>>>> As a user and data platform admin, I think thats a really good idea.
>>>>
>>>> Out of curiosity, why not extend this integration to also cover roles,
>>>> principals, permissions and etc?
>>>>
>>>> Centralizing all of this through terraform would be fantastic IMO.
>>>>
>>>> Best regards,
>>>> Endi
>>>>
>>>>
>>>> On Wed, Oct 15, 2025 at 13:17 Alex Stephen
>>>> <al...@google.com.invalid> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I'd like to propose the creation of an Iceberg Terraform Provider for
>>>>> the Iceberg REST Catalog. There currently exists no vendor-neutral solution
>>>>> for managing Iceberg resources using Terraform/OpenTofu. The proposal is
>>>>> for creating a Terraform Provider that can connect to an arbitrary Iceberg
>>>>> REST Catalog and manage namespaces/tables/views.
>>>>>
>>>>> The proposal is located at
>>>>> https://s.apache.org/iceberg-catalog-terraform
>>>>>
>>>>> Please take a look. I'm planning on bringing this up at the next
>>>>> Iceberg Catalog sync.
>>>>>
>>>>> Thanks a lot.
>>>>>
>>>>> -- Alex Stephen
>>>>>
>>>>

Re: Proposal: Iceberg Terraform Provider

Posted by Alex Stephen <al...@google.com.INVALID>.
> IRC doesn't have these concepts. It's only possible with a specific IRC
implementation.

It would be great to collaborate with the Polaris community, since Polaris
has many of those concepts built-in already.

The provider could potentially include more implementation-specific
resources in the future. One other example would be creating catalogs,
where the provider could interface with Polaris or other vendors to create
a catalog.

For now, namespaces, tables, and views are the most agnostic resources and
all supported by the REST Catalog spec.


On Wed, Oct 15, 2025 at 3:09 PM Yufei Gu <fl...@gmail.com> wrote:

> > Out of curiosity, why not extend this integration to also cover roles,
> principals, permissions and etc?
>
> IRC doesn't have these concepts. It's only possible with a specific IRC
> implementation.
>
> Yufei
>
>
> On Wed, Oct 15, 2025 at 2:43 PM Jayce Slesar <ja...@gmail.com> wrote:
>
>> I think this is a great idea!
>>
>> I have done a lot of existing work fir tables + maintenance via IaC via
>> CDK, but something cloud agnostic would be amazing.
>>
>> It would be great to start with a catalog but I am betting there are
>> other things we could add over time.
>>
>> On Wed, Oct 15, 2025 at 5:36 PM Endi Caushi <mr...@gmail.com> wrote:
>>
>>> As a user and data platform admin, I think thats a really good idea.
>>>
>>> Out of curiosity, why not extend this integration to also cover roles,
>>> principals, permissions and etc?
>>>
>>> Centralizing all of this through terraform would be fantastic IMO.
>>>
>>> Best regards,
>>> Endi
>>>
>>>
>>> On Wed, Oct 15, 2025 at 13:17 Alex Stephen
>>> <al...@google.com.invalid> wrote:
>>>
>>>> Hello,
>>>>
>>>> I'd like to propose the creation of an Iceberg Terraform Provider for
>>>> the Iceberg REST Catalog. There currently exists no vendor-neutral solution
>>>> for managing Iceberg resources using Terraform/OpenTofu. The proposal is
>>>> for creating a Terraform Provider that can connect to an arbitrary Iceberg
>>>> REST Catalog and manage namespaces/tables/views.
>>>>
>>>> The proposal is located at
>>>> https://s.apache.org/iceberg-catalog-terraform
>>>>
>>>> Please take a look. I'm planning on bringing this up at the next
>>>> Iceberg Catalog sync.
>>>>
>>>> Thanks a lot.
>>>>
>>>> -- Alex Stephen
>>>>
>>>

Re: Proposal: Iceberg Terraform Provider

Posted by Yufei Gu <fl...@gmail.com>.
> Out of curiosity, why not extend this integration to also cover roles,
principals, permissions and etc?

IRC doesn't have these concepts. It's only possible with a specific IRC
implementation.

Yufei


On Wed, Oct 15, 2025 at 2:43 PM Jayce Slesar <ja...@gmail.com> wrote:

> I think this is a great idea!
>
> I have done a lot of existing work fir tables + maintenance via IaC via
> CDK, but something cloud agnostic would be amazing.
>
> It would be great to start with a catalog but I am betting there are other
> things we could add over time.
>
> On Wed, Oct 15, 2025 at 5:36 PM Endi Caushi <mr...@gmail.com> wrote:
>
>> As a user and data platform admin, I think thats a really good idea.
>>
>> Out of curiosity, why not extend this integration to also cover roles,
>> principals, permissions and etc?
>>
>> Centralizing all of this through terraform would be fantastic IMO.
>>
>> Best regards,
>> Endi
>>
>>
>> On Wed, Oct 15, 2025 at 13:17 Alex Stephen <al...@google.com.invalid>
>> wrote:
>>
>>> Hello,
>>>
>>> I'd like to propose the creation of an Iceberg Terraform Provider for
>>> the Iceberg REST Catalog. There currently exists no vendor-neutral solution
>>> for managing Iceberg resources using Terraform/OpenTofu. The proposal is
>>> for creating a Terraform Provider that can connect to an arbitrary Iceberg
>>> REST Catalog and manage namespaces/tables/views.
>>>
>>> The proposal is located at
>>> https://s.apache.org/iceberg-catalog-terraform
>>>
>>> Please take a look. I'm planning on bringing this up at the next Iceberg
>>> Catalog sync.
>>>
>>> Thanks a lot.
>>>
>>> -- Alex Stephen
>>>
>>

Re: Proposal: Iceberg Terraform Provider

Posted by Jayce Slesar <ja...@gmail.com>.
I think this is a great idea!

I have done a lot of existing work fir tables + maintenance via IaC via
CDK, but something cloud agnostic would be amazing.

It would be great to start with a catalog but I am betting there are other
things we could add over time.

On Wed, Oct 15, 2025 at 5:36 PM Endi Caushi <mr...@gmail.com> wrote:

> As a user and data platform admin, I think thats a really good idea.
>
> Out of curiosity, why not extend this integration to also cover roles,
> principals, permissions and etc?
>
> Centralizing all of this through terraform would be fantastic IMO.
>
> Best regards,
> Endi
>
>
> On Wed, Oct 15, 2025 at 13:17 Alex Stephen <al...@google.com.invalid>
> wrote:
>
>> Hello,
>>
>> I'd like to propose the creation of an Iceberg Terraform Provider for the
>> Iceberg REST Catalog. There currently exists no vendor-neutral solution for
>> managing Iceberg resources using Terraform/OpenTofu. The proposal is for
>> creating a Terraform Provider that can connect to an arbitrary Iceberg REST
>> Catalog and manage namespaces/tables/views.
>>
>> The proposal is located at https://s.apache.org/iceberg-catalog-terraform
>>
>> Please take a look. I'm planning on bringing this up at the next Iceberg
>> Catalog sync.
>>
>> Thanks a lot.
>>
>> -- Alex Stephen
>>
>

Re: Proposal: Iceberg Terraform Provider

Posted by Endi Caushi <mr...@gmail.com>.
As a user and data platform admin, I think thats a really good idea.

Out of curiosity, why not extend this integration to also cover roles,
principals, permissions and etc?

Centralizing all of this through terraform would be fantastic IMO.

Best regards,
Endi


On Wed, Oct 15, 2025 at 13:17 Alex Stephen <al...@google.com.invalid>
wrote:

> Hello,
>
> I'd like to propose the creation of an Iceberg Terraform Provider for the
> Iceberg REST Catalog. There currently exists no vendor-neutral solution for
> managing Iceberg resources using Terraform/OpenTofu. The proposal is for
> creating a Terraform Provider that can connect to an arbitrary Iceberg REST
> Catalog and manage namespaces/tables/views.
>
> The proposal is located at https://s.apache.org/iceberg-catalog-terraform
>
> Please take a look. I'm planning on bringing this up at the next Iceberg
> Catalog sync.
>
> Thanks a lot.
>
> -- Alex Stephen
>