You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iceberg.apache.org by Alex Stephen via dev <de...@iceberg.apache.org> on 2026/09/04 21:31:29 UTC
Views with Multiple Representations
Hello,
The view spec allows creating a single view with multiple engine-specific
representations. This allows a single view to be queried by multiple
engines, each using its own specific SQL dialect.
In practice, this is difficult to achieve. Users can use the
`replace.drop-dialect.allowed` view property to allow the replacing of an
existing representation. However, there's no way to append a representation
(by creating a new ViewVersion with the existing representations and a new
one). This means that a direct REST API call is the only way to create a
View with multiple representations.
I'd like to propose the creation of a new view property called
`replace.append-dialect.allowed` that allows users to append a new
representation to an existing view.
PR: https://github.com/apache/iceberg/pull/17930
Please take a look. I'd love to hear from the community whether there is
value in this use case.
Thanks!
-- Alex Stephen
Re: Views with Multiple Representations
Posted by Alex Stephen via dev <de...@iceberg.apache.org>.
Hi everyone,
This is a great discussion. I definitely agree that expanding ALTER VIEW
support is the best method to allow for defining views across engines
conveniently.
A common translation layer is the only way to ensure that all
representations are consistent. Any other method requires an engine capable
of parsing SQL from multiple dialects.
Given that constraint, I think it's fine to place the impetus on the user
to ensure that their representations match when they run additional ALTER
VIEW commands to add new representations.
- Alex Stephen
On Tue, Sep 8, 2026 at 12:13 PM Daniel Weeks <dw...@apache.org> wrote:
> Thanks Ajantha for digging up the thread on this discussion.
>
> I think the important takeaway here is understanding what Iceberg is
> trying to solve with views, and what it isn't.
>
> We know that engines have different sql dialects and semantics. Iceberg
> views enable exposing metadata in a way that multiple engines can consume
> in their native dialect (the representation, not the semantics).
>
> However, there is no intention to automate the translation or ensure
> consistency across representations. This is an important detail that the
> user must handle. Translation frameworks or manual verification can be
> used, but achieving consistent behavior is complicated and outside the
> project's scope.
>
> If we think we need more convenient ways to update the syntax, that's
> worth exploring. However, coordinating representations and verifying
> correctness remains the responsibility of the view creator (via API or
> natively through the engine). It's easy to get this wrong, but the utility
> of views outweighs the burden we place on users.
>
> I see 'ALTER VIEW' support as a way to make defining the views more
> convenient (than, say, using PyIceberg which can't validate the syntax of
> other engines). It allows the native engine to parse, resolve all
> references, and ensure the view can be executed, but notably does nothing
> to ensure the results are consistent with any other representation.
>
> -Dan
>
>
> On Mon, Sep 7, 2026 at 11:47 PM Ajantha Bhat <aj...@gmail.com>
> wrote:
>
>> This discussion feels very similar to the earlier thread that I opened on
>> view interoperability a long time ago.
>>
>> One of the takeaways there was that we wanted to avoid each engine
>> manually implementing or maintaining dialect translations. Instead, the
>> preference was to have a centralized translation layer (at the time we
>> discussed exposing that through PyIceberg) rather than pushing that
>> responsibility into every engine. Sharing the previous discussion for
>> context, as it covers many of the same trade-offs:
>> https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81
>> <https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81?utm_source=chatgpt.com>
>>
>> - Ajantha
>>
>> On Tue, Sep 8, 2026 at 12:09 PM Péter Váry <pe...@gmail.com>
>> wrote:
>>
>>> In my opinion, the ideal experience is for users to define both SQL
>>> statements when creating the view. This avoids any period during which the
>>> view is only partially defined and allows the entire workflow to be handled
>>> through a single authoring tool.
>>>
>>> That said, the Iceberg Java API is available, and engines are free to
>>> choose which capabilities they support and how they expose them to users.
>>>
>>> Talat Uyarer <ta...@apache.org> ezt írta (időpont: 2026. szept. 7., H,
>>> 17:18):
>>>
>>>> Hi Péter,
>>>>
>>>> I think a SQL-only workflow is closer than it looks. a Spark user
>>>> doesn't need Spark to parse the Flink SQL, they need to read it, and
>>>> showing another engine's SQL as text is engine-agnostic. What's missing is
>>>> presentation, not translation.
>>>>
>>>> The workflow I'd propose:
>>>>
>>>> 1. Flink: CREATE VIEW v AS SELECT ... — version 1, with a `flink`
>>>> representation.
>>>> 2. Spark user: DESCRIBE EXTENDED VIEW— engines should display *all*
>>>> representations with their dialect labels. The strings are already in the
>>>> metadata, so this is purely a display convention, no spec change. This
>>>> answers "how can the Spark user know the Flink SQL".
>>>> 3. The user reads the Flink SQL and authors the Spark equivalent.
>>>> 4. Spark: ALTER VIEW v AS SELECT ... — upserts the `spark`
>>>> representation and carries the `flink` one forward unchanged. Issuing that
>>>> statement *is* the consistency assertion — the "user authority" you
>>>> described, made concrete. The version summary already records engine
>>>> name/version, so we get per-version provenance for free.
>>>> 5. On read, each engine executes exactly its own dialect.
>>>>
>>>> The rule that makes this flag-free is, I think, just a crisper version
>>>> of what Dan
>>>> said:
>>>>
>>>> ALTER VIEW ... AS = update/add my engine's representation,
>>>> preserve the rest
>>>> CREATE OR REPLACE VIEW = redefine the view; only my representation
>>>> remains
>>>>
>>>> Two intents, two existing syntaxes, no properties, no new grammar.
>>>>
>>>> When someone genuinely changes the view's meaning with CREATE OR
>>>> REPLACE, the other representations are gone, the next read from another
>>>> engine fails fast, and that engine's user re-adds their dialect via steps
>>>> 2–4. The failure is the signal, no cross-engine coordination protocol
>>>> needed.
>>>>
>>>> That only holds if engines actually fail fast, which is Alex's fallback
>>>> point: today sqlFor() silently returns the "closest" representation, which
>>>> is exactly how an engine ends up executing another engine's SQL. That
>>>> leniency was defensible while appending a dialect was impossible; with this
>>>> workflow it loses its justification. #17874 adds an opt-in
>>>> view-dialect-strict to Flink as a first step, and ideally the error message
>>>> itself teaches the workflow: "View v has no 'flink'
>>>> representation (found: spark). Use SHOW CREATE VIEW to see the existing
>>>> SQL and ALTER VIEW to add a flink representation."
>>>>
>>>> Talat
>>>>
>>>>
>>>> On 2026/09/07 14:38:10 Péter Váry wrote:
>>>> > Do I understand correctly that the expected behavior for the user if
>>>> they
>>>> > want to create a view with Flink and Spark representations is:
>>>> >
>>>> > 1. CREATE VIEW my_view AS SELECT ... - in Flink
>>>> > 2. ALTER VIEW my_view AS SELECT ... - in Spark
>>>> >
>>>> > How can a Spark user know the Flink SQL the user used to create the
>>>> view?
>>>> >
>>>> > I'm trying to put together a workflow for creating interoperable
>>>> views and
>>>> > using SQL doesn't seem viable, so as Ryan mentioned the only option is
>>>> > Iceberg API for now.
>>>> >
>>>> >
>>>> > Péter Váry <pe...@gmail.com> ezt írta (időpont: 2026.
>>>> szept.
>>>> > 5., Szo, 9:16):
>>>> >
>>>> > > Thanks Alex for bringing this up.
>>>> > >
>>>> > > We had a very similar discussion with Guo when he wanted to add a
>>>> Flink
>>>> > > representation to a view:
>>>> https://github.com/apache/iceberg/issues/15296.
>>>> > > Talat also has a fresh PR for this issue in Flink:
>>>> > > https://github.com/apache/iceberg/pull/17874
>>>> > >
>>>> > > My main concern is consistency. How can Flink be sure that the
>>>> Flink SQL
>>>> > > returns the same results as the already existing Spark SQL.
>>>> > > In the end we accept the users authority to state that the Flink
>>>> SQL is
>>>> > > consistent with the existing Spark SQL. This means we accept that
>>>> they are
>>>> > > able to parse both, and make decisions.
>>>> > > I do think we should allow a user to update one or more of the SQLs
>>>> > > without needing extra flags.
>>>> > >
>>>> > >
>>>> > >
>>>> > > On Sat, Sep 5, 2026, 04:43 Prashant Singh <prashant010696@gmail.com
>>>> >
>>>> > > wrote:
>>>> > >
>>>> > >> Thanks for starting the thread Alex,
>>>> > >>
>>>> > >> I agree with Dan, an engine should only update / add their own
>>>> dialect
>>>> > >> and not make guarantees about other engine dialects without
>>>> passing through
>>>> > >> the other engine's compiler.
>>>> > >> I would recommend ALTER too, I know engines such as starrocks have
>>>> came
>>>> > >> up with grammar [1] to be more specific that one is adding /
>>>> modifying the
>>>> > >> dialect, given the dialect is of starrocks, that could be an
>>>> option :
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
>>>> > >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
>>>> > >>
>>>> > >>
>>>> > >> [1]
>>>> > >>
>>>> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
>>>> > >>
>>>> > >> Best,
>>>> > >> Prashant Singh
>>>> > >>
>>>> > >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org>
>>>> wrote:
>>>> > >>
>>>> > >>> I feel like if we want to support updating/adding a
>>>> representation for a
>>>> > >>> specific engine, the right way is to allow 'ALTER VIEW' to update
>>>> or add
>>>> > >>> the current engine's representation.
>>>> > >>>
>>>> > >>> I don't think we can expect an engine to add or update other
>>>> engines'
>>>> > >>> representations because that would potentially require parsing a
>>>> different
>>>> > >>> SQL syntax than is native to the engine.
>>>> > >>>
>>>> > >>> For the example you gave where there's an existing
>>>> representation in
>>>> > >>> Trino, then using `ALTER VIEW . . .` would just set the spark
>>>> > >>> representation (as opposed to a `REPLACE` or `CREATE` that would
>>>> create a
>>>> > >>> new representation with only the spake dialect).
>>>> > >>>
>>>> > >>> I don't like the idea of adding properties to toggle behavior
>>>> though as
>>>> > >>> that makes something declarative subject to side-effects of the
>>>> > >>> configuration/session.
>>>> > >>>
>>>> > >>> -Dan
>>>> > >>>
>>>> > >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
>>>> > >>> dev@iceberg.apache.org> wrote:
>>>> > >>>
>>>> > >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as
>>>> much as
>>>> > >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW
>>>> > >>>> VIEWS` (which just calls the underlying List Views API), so they
>>>> know that
>>>> > >>>> they don't need to create a view as much as alter it.
>>>> > >>>>
>>>> > >>>> The Java implementation
>>>> > >>>> <
>>>> https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130>
>>>> surfaces
>>>> > >>>> the first representation when a matching one doesn't exist. This
>>>> assumes
>>>> > >>>> representations are more-or-less compatible between engines,
>>>> which isn't
>>>> > >>>> the case.
>>>> > >>>>
>>>> > >>>> I'm concerned about the user journey where a user attempts to
>>>> query a
>>>> > >>>> view and receives an error message stating they cannot query it
>>>> due to a
>>>> > >>>> wrong engine or invalid SQL syntax (because their engine is
>>>> using the wrong
>>>> > >>>> representation). At this point, their only current recourse is
>>>> to use the
>>>> > >>>> REST endpoints directly to add their new representation.
>>>> > >>>>
>>>> > >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org>
>>>> wrote:
>>>> > >>>>
>>>> > >>>>> I think the intent was for this to happen through API
>>>> integration.
>>>> > >>>>> What is the user interaction that you're trying to achieve? Do
>>>> you want
>>>> > >>>>> each `CREATE VIEW` call to automatically append a new
>>>> representation?
>>>> > >>>>>
>>>> > >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
>>>> > >>>>> dev@iceberg.apache.org> wrote:
>>>> > >>>>>
>>>> > >>>>>> Hello,
>>>> > >>>>>>
>>>> > >>>>>> The view spec allows creating a single view with multiple
>>>> > >>>>>> engine-specific representations. This allows a single view to
>>>> be queried by
>>>> > >>>>>> multiple engines, each using its own specific SQL dialect.
>>>> > >>>>>>
>>>> > >>>>>> In practice, this is difficult to achieve. Users can use the
>>>> > >>>>>> `replace.drop-dialect.allowed` view property to allow the
>>>> replacing of an
>>>> > >>>>>> existing representation. However, there's no way to append a
>>>> representation
>>>> > >>>>>> (by creating a new ViewVersion with the existing
>>>> representations and a new
>>>> > >>>>>> one). This means that a direct REST API call is the only way
>>>> to create a
>>>> > >>>>>> View with multiple representations.
>>>> > >>>>>>
>>>> > >>>>>> I'd like to propose the creation of a new view property called
>>>> > >>>>>> `replace.append-dialect.allowed` that allows users to append a
>>>> new
>>>> > >>>>>> representation to an existing view.
>>>> > >>>>>>
>>>> > >>>>>> PR: https://github.com/apache/iceberg/pull/17930
>>>> > >>>>>>
>>>> > >>>>>> Please take a look. I'd love to hear from the community
>>>> whether there
>>>> > >>>>>> is value in this use case.
>>>> > >>>>>>
>>>> > >>>>>> Thanks!
>>>> > >>>>>>
>>>> > >>>>>> -- Alex Stephen
>>>> > >>>>>>
>>>> > >>>>>
>>>> >
>>>>
>>>
Re: Views with Multiple Representations
Posted by Daniel Weeks <dw...@apache.org>.
Thanks Ajantha for digging up the thread on this discussion.
I think the important takeaway here is understanding what Iceberg is trying
to solve with views, and what it isn't.
We know that engines have different sql dialects and semantics. Iceberg
views enable exposing metadata in a way that multiple engines can consume
in their native dialect (the representation, not the semantics).
However, there is no intention to automate the translation or ensure
consistency across representations. This is an important detail that the
user must handle. Translation frameworks or manual verification can be
used, but achieving consistent behavior is complicated and outside the
project's scope.
If we think we need more convenient ways to update the syntax, that's worth
exploring. However, coordinating representations and verifying correctness
remains the responsibility of the view creator (via API or natively through
the engine). It's easy to get this wrong, but the utility of views
outweighs the burden we place on users.
I see 'ALTER VIEW' support as a way to make defining the views more
convenient (than, say, using PyIceberg which can't validate the syntax of
other engines). It allows the native engine to parse, resolve all
references, and ensure the view can be executed, but notably does nothing
to ensure the results are consistent with any other representation.
-Dan
On Mon, Sep 7, 2026 at 11:47 PM Ajantha Bhat <aj...@gmail.com> wrote:
> This discussion feels very similar to the earlier thread that I opened on
> view interoperability a long time ago.
>
> One of the takeaways there was that we wanted to avoid each engine
> manually implementing or maintaining dialect translations. Instead, the
> preference was to have a centralized translation layer (at the time we
> discussed exposing that through PyIceberg) rather than pushing that
> responsibility into every engine. Sharing the previous discussion for
> context, as it covers many of the same trade-offs:
> https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81
> <https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81?utm_source=chatgpt.com>
>
> - Ajantha
>
> On Tue, Sep 8, 2026 at 12:09 PM Péter Váry <pe...@gmail.com>
> wrote:
>
>> In my opinion, the ideal experience is for users to define both SQL
>> statements when creating the view. This avoids any period during which the
>> view is only partially defined and allows the entire workflow to be handled
>> through a single authoring tool.
>>
>> That said, the Iceberg Java API is available, and engines are free to
>> choose which capabilities they support and how they expose them to users.
>>
>> Talat Uyarer <ta...@apache.org> ezt írta (időpont: 2026. szept. 7., H,
>> 17:18):
>>
>>> Hi Péter,
>>>
>>> I think a SQL-only workflow is closer than it looks. a Spark user
>>> doesn't need Spark to parse the Flink SQL, they need to read it, and
>>> showing another engine's SQL as text is engine-agnostic. What's missing is
>>> presentation, not translation.
>>>
>>> The workflow I'd propose:
>>>
>>> 1. Flink: CREATE VIEW v AS SELECT ... — version 1, with a `flink`
>>> representation.
>>> 2. Spark user: DESCRIBE EXTENDED VIEW— engines should display *all*
>>> representations with their dialect labels. The strings are already in the
>>> metadata, so this is purely a display convention, no spec change. This
>>> answers "how can the Spark user know the Flink SQL".
>>> 3. The user reads the Flink SQL and authors the Spark equivalent.
>>> 4. Spark: ALTER VIEW v AS SELECT ... — upserts the `spark`
>>> representation and carries the `flink` one forward unchanged. Issuing that
>>> statement *is* the consistency assertion — the "user authority" you
>>> described, made concrete. The version summary already records engine
>>> name/version, so we get per-version provenance for free.
>>> 5. On read, each engine executes exactly its own dialect.
>>>
>>> The rule that makes this flag-free is, I think, just a crisper version
>>> of what Dan
>>> said:
>>>
>>> ALTER VIEW ... AS = update/add my engine's representation,
>>> preserve the rest
>>> CREATE OR REPLACE VIEW = redefine the view; only my representation
>>> remains
>>>
>>> Two intents, two existing syntaxes, no properties, no new grammar.
>>>
>>> When someone genuinely changes the view's meaning with CREATE OR
>>> REPLACE, the other representations are gone, the next read from another
>>> engine fails fast, and that engine's user re-adds their dialect via steps
>>> 2–4. The failure is the signal, no cross-engine coordination protocol
>>> needed.
>>>
>>> That only holds if engines actually fail fast, which is Alex's fallback
>>> point: today sqlFor() silently returns the "closest" representation, which
>>> is exactly how an engine ends up executing another engine's SQL. That
>>> leniency was defensible while appending a dialect was impossible; with this
>>> workflow it loses its justification. #17874 adds an opt-in
>>> view-dialect-strict to Flink as a first step, and ideally the error message
>>> itself teaches the workflow: "View v has no 'flink'
>>> representation (found: spark). Use SHOW CREATE VIEW to see the existing
>>> SQL and ALTER VIEW to add a flink representation."
>>>
>>> Talat
>>>
>>>
>>> On 2026/09/07 14:38:10 Péter Váry wrote:
>>> > Do I understand correctly that the expected behavior for the user if
>>> they
>>> > want to create a view with Flink and Spark representations is:
>>> >
>>> > 1. CREATE VIEW my_view AS SELECT ... - in Flink
>>> > 2. ALTER VIEW my_view AS SELECT ... - in Spark
>>> >
>>> > How can a Spark user know the Flink SQL the user used to create the
>>> view?
>>> >
>>> > I'm trying to put together a workflow for creating interoperable views
>>> and
>>> > using SQL doesn't seem viable, so as Ryan mentioned the only option is
>>> > Iceberg API for now.
>>> >
>>> >
>>> > Péter Váry <pe...@gmail.com> ezt írta (időpont: 2026.
>>> szept.
>>> > 5., Szo, 9:16):
>>> >
>>> > > Thanks Alex for bringing this up.
>>> > >
>>> > > We had a very similar discussion with Guo when he wanted to add a
>>> Flink
>>> > > representation to a view:
>>> https://github.com/apache/iceberg/issues/15296.
>>> > > Talat also has a fresh PR for this issue in Flink:
>>> > > https://github.com/apache/iceberg/pull/17874
>>> > >
>>> > > My main concern is consistency. How can Flink be sure that the Flink
>>> SQL
>>> > > returns the same results as the already existing Spark SQL.
>>> > > In the end we accept the users authority to state that the Flink SQL
>>> is
>>> > > consistent with the existing Spark SQL. This means we accept that
>>> they are
>>> > > able to parse both, and make decisions.
>>> > > I do think we should allow a user to update one or more of the SQLs
>>> > > without needing extra flags.
>>> > >
>>> > >
>>> > >
>>> > > On Sat, Sep 5, 2026, 04:43 Prashant Singh <pr...@gmail.com>
>>> > > wrote:
>>> > >
>>> > >> Thanks for starting the thread Alex,
>>> > >>
>>> > >> I agree with Dan, an engine should only update / add their own
>>> dialect
>>> > >> and not make guarantees about other engine dialects without passing
>>> through
>>> > >> the other engine's compiler.
>>> > >> I would recommend ALTER too, I know engines such as starrocks have
>>> came
>>> > >> up with grammar [1] to be more specific that one is adding /
>>> modifying the
>>> > >> dialect, given the dialect is of starrocks, that could be an option
>>> :
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
>>> > >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
>>> > >>
>>> > >>
>>> > >> [1]
>>> > >>
>>> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
>>> > >>
>>> > >> Best,
>>> > >> Prashant Singh
>>> > >>
>>> > >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org>
>>> wrote:
>>> > >>
>>> > >>> I feel like if we want to support updating/adding a representation
>>> for a
>>> > >>> specific engine, the right way is to allow 'ALTER VIEW' to update
>>> or add
>>> > >>> the current engine's representation.
>>> > >>>
>>> > >>> I don't think we can expect an engine to add or update other
>>> engines'
>>> > >>> representations because that would potentially require parsing a
>>> different
>>> > >>> SQL syntax than is native to the engine.
>>> > >>>
>>> > >>> For the example you gave where there's an existing representation
>>> in
>>> > >>> Trino, then using `ALTER VIEW . . .` would just set the spark
>>> > >>> representation (as opposed to a `REPLACE` or `CREATE` that would
>>> create a
>>> > >>> new representation with only the spake dialect).
>>> > >>>
>>> > >>> I don't like the idea of adding properties to toggle behavior
>>> though as
>>> > >>> that makes something declarative subject to side-effects of the
>>> > >>> configuration/session.
>>> > >>>
>>> > >>> -Dan
>>> > >>>
>>> > >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
>>> > >>> dev@iceberg.apache.org> wrote:
>>> > >>>
>>> > >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as
>>> much as
>>> > >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW
>>> > >>>> VIEWS` (which just calls the underlying List Views API), so they
>>> know that
>>> > >>>> they don't need to create a view as much as alter it.
>>> > >>>>
>>> > >>>> The Java implementation
>>> > >>>> <
>>> https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130>
>>> surfaces
>>> > >>>> the first representation when a matching one doesn't exist. This
>>> assumes
>>> > >>>> representations are more-or-less compatible between engines,
>>> which isn't
>>> > >>>> the case.
>>> > >>>>
>>> > >>>> I'm concerned about the user journey where a user attempts to
>>> query a
>>> > >>>> view and receives an error message stating they cannot query it
>>> due to a
>>> > >>>> wrong engine or invalid SQL syntax (because their engine is using
>>> the wrong
>>> > >>>> representation). At this point, their only current recourse is to
>>> use the
>>> > >>>> REST endpoints directly to add their new representation.
>>> > >>>>
>>> > >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
>>> > >>>>
>>> > >>>>> I think the intent was for this to happen through API
>>> integration.
>>> > >>>>> What is the user interaction that you're trying to achieve? Do
>>> you want
>>> > >>>>> each `CREATE VIEW` call to automatically append a new
>>> representation?
>>> > >>>>>
>>> > >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
>>> > >>>>> dev@iceberg.apache.org> wrote:
>>> > >>>>>
>>> > >>>>>> Hello,
>>> > >>>>>>
>>> > >>>>>> The view spec allows creating a single view with multiple
>>> > >>>>>> engine-specific representations. This allows a single view to
>>> be queried by
>>> > >>>>>> multiple engines, each using its own specific SQL dialect.
>>> > >>>>>>
>>> > >>>>>> In practice, this is difficult to achieve. Users can use the
>>> > >>>>>> `replace.drop-dialect.allowed` view property to allow the
>>> replacing of an
>>> > >>>>>> existing representation. However, there's no way to append a
>>> representation
>>> > >>>>>> (by creating a new ViewVersion with the existing
>>> representations and a new
>>> > >>>>>> one). This means that a direct REST API call is the only way to
>>> create a
>>> > >>>>>> View with multiple representations.
>>> > >>>>>>
>>> > >>>>>> I'd like to propose the creation of a new view property called
>>> > >>>>>> `replace.append-dialect.allowed` that allows users to append a
>>> new
>>> > >>>>>> representation to an existing view.
>>> > >>>>>>
>>> > >>>>>> PR: https://github.com/apache/iceberg/pull/17930
>>> > >>>>>>
>>> > >>>>>> Please take a look. I'd love to hear from the community whether
>>> there
>>> > >>>>>> is value in this use case.
>>> > >>>>>>
>>> > >>>>>> Thanks!
>>> > >>>>>>
>>> > >>>>>> -- Alex Stephen
>>> > >>>>>>
>>> > >>>>>
>>> >
>>>
>>
Re: Views with Multiple Representations
Posted by Ajantha Bhat <aj...@gmail.com>.
This discussion feels very similar to the earlier thread that I opened on
view interoperability a long time ago.
One of the takeaways there was that we wanted to avoid each engine manually
implementing or maintaining dialect translations. Instead, the preference
was to have a centralized translation layer (at the time we discussed
exposing that through PyIceberg) rather than pushing that responsibility
into every engine. Sharing the previous discussion for context, as it
covers many of the same trade-offs:
https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81
<https://lists.apache.org/thread/k6szpr5smyrh37sy563xpgjor4g6pr81?utm_source=chatgpt.com>
- Ajantha
On Tue, Sep 8, 2026 at 12:09 PM Péter Váry <pe...@gmail.com>
wrote:
> In my opinion, the ideal experience is for users to define both SQL
> statements when creating the view. This avoids any period during which the
> view is only partially defined and allows the entire workflow to be handled
> through a single authoring tool.
>
> That said, the Iceberg Java API is available, and engines are free to
> choose which capabilities they support and how they expose them to users.
>
> Talat Uyarer <ta...@apache.org> ezt írta (időpont: 2026. szept. 7., H,
> 17:18):
>
>> Hi Péter,
>>
>> I think a SQL-only workflow is closer than it looks. a Spark user doesn't
>> need Spark to parse the Flink SQL, they need to read it, and showing
>> another engine's SQL as text is engine-agnostic. What's missing is
>> presentation, not translation.
>>
>> The workflow I'd propose:
>>
>> 1. Flink: CREATE VIEW v AS SELECT ... — version 1, with a `flink`
>> representation.
>> 2. Spark user: DESCRIBE EXTENDED VIEW— engines should display *all*
>> representations with their dialect labels. The strings are already in the
>> metadata, so this is purely a display convention, no spec change. This
>> answers "how can the Spark user know the Flink SQL".
>> 3. The user reads the Flink SQL and authors the Spark equivalent.
>> 4. Spark: ALTER VIEW v AS SELECT ... — upserts the `spark` representation
>> and carries the `flink` one forward unchanged. Issuing that statement *is*
>> the consistency assertion — the "user authority" you described, made
>> concrete. The version summary already records engine name/version, so we
>> get per-version provenance for free.
>> 5. On read, each engine executes exactly its own dialect.
>>
>> The rule that makes this flag-free is, I think, just a crisper version of
>> what Dan
>> said:
>>
>> ALTER VIEW ... AS = update/add my engine's representation,
>> preserve the rest
>> CREATE OR REPLACE VIEW = redefine the view; only my representation
>> remains
>>
>> Two intents, two existing syntaxes, no properties, no new grammar.
>>
>> When someone genuinely changes the view's meaning with CREATE OR REPLACE,
>> the other representations are gone, the next read from another engine fails
>> fast, and that engine's user re-adds their dialect via steps 2–4. The
>> failure is the signal, no cross-engine coordination protocol needed.
>>
>> That only holds if engines actually fail fast, which is Alex's fallback
>> point: today sqlFor() silently returns the "closest" representation, which
>> is exactly how an engine ends up executing another engine's SQL. That
>> leniency was defensible while appending a dialect was impossible; with this
>> workflow it loses its justification. #17874 adds an opt-in
>> view-dialect-strict to Flink as a first step, and ideally the error message
>> itself teaches the workflow: "View v has no 'flink'
>> representation (found: spark). Use SHOW CREATE VIEW to see the existing
>> SQL and ALTER VIEW to add a flink representation."
>>
>> Talat
>>
>>
>> On 2026/09/07 14:38:10 Péter Váry wrote:
>> > Do I understand correctly that the expected behavior for the user if
>> they
>> > want to create a view with Flink and Spark representations is:
>> >
>> > 1. CREATE VIEW my_view AS SELECT ... - in Flink
>> > 2. ALTER VIEW my_view AS SELECT ... - in Spark
>> >
>> > How can a Spark user know the Flink SQL the user used to create the
>> view?
>> >
>> > I'm trying to put together a workflow for creating interoperable views
>> and
>> > using SQL doesn't seem viable, so as Ryan mentioned the only option is
>> > Iceberg API for now.
>> >
>> >
>> > Péter Váry <pe...@gmail.com> ezt írta (időpont: 2026.
>> szept.
>> > 5., Szo, 9:16):
>> >
>> > > Thanks Alex for bringing this up.
>> > >
>> > > We had a very similar discussion with Guo when he wanted to add a
>> Flink
>> > > representation to a view:
>> https://github.com/apache/iceberg/issues/15296.
>> > > Talat also has a fresh PR for this issue in Flink:
>> > > https://github.com/apache/iceberg/pull/17874
>> > >
>> > > My main concern is consistency. How can Flink be sure that the Flink
>> SQL
>> > > returns the same results as the already existing Spark SQL.
>> > > In the end we accept the users authority to state that the Flink SQL
>> is
>> > > consistent with the existing Spark SQL. This means we accept that
>> they are
>> > > able to parse both, and make decisions.
>> > > I do think we should allow a user to update one or more of the SQLs
>> > > without needing extra flags.
>> > >
>> > >
>> > >
>> > > On Sat, Sep 5, 2026, 04:43 Prashant Singh <pr...@gmail.com>
>> > > wrote:
>> > >
>> > >> Thanks for starting the thread Alex,
>> > >>
>> > >> I agree with Dan, an engine should only update / add their own
>> dialect
>> > >> and not make guarantees about other engine dialects without passing
>> through
>> > >> the other engine's compiler.
>> > >> I would recommend ALTER too, I know engines such as starrocks have
>> came
>> > >> up with grammar [1] to be more specific that one is adding /
>> modifying the
>> > >> dialect, given the dialect is of starrocks, that could be an option :
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
>> > >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
>> > >>
>> > >>
>> > >> [1]
>> > >>
>> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
>> > >>
>> > >> Best,
>> > >> Prashant Singh
>> > >>
>> > >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org>
>> wrote:
>> > >>
>> > >>> I feel like if we want to support updating/adding a representation
>> for a
>> > >>> specific engine, the right way is to allow 'ALTER VIEW' to update
>> or add
>> > >>> the current engine's representation.
>> > >>>
>> > >>> I don't think we can expect an engine to add or update other
>> engines'
>> > >>> representations because that would potentially require parsing a
>> different
>> > >>> SQL syntax than is native to the engine.
>> > >>>
>> > >>> For the example you gave where there's an existing representation
>> in
>> > >>> Trino, then using `ALTER VIEW . . .` would just set the spark
>> > >>> representation (as opposed to a `REPLACE` or `CREATE` that would
>> create a
>> > >>> new representation with only the spake dialect).
>> > >>>
>> > >>> I don't like the idea of adding properties to toggle behavior
>> though as
>> > >>> that makes something declarative subject to side-effects of the
>> > >>> configuration/session.
>> > >>>
>> > >>> -Dan
>> > >>>
>> > >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
>> > >>> dev@iceberg.apache.org> wrote:
>> > >>>
>> > >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as much
>> as
>> > >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW
>> > >>>> VIEWS` (which just calls the underlying List Views API), so they
>> know that
>> > >>>> they don't need to create a view as much as alter it.
>> > >>>>
>> > >>>> The Java implementation
>> > >>>> <
>> https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130>
>> surfaces
>> > >>>> the first representation when a matching one doesn't exist. This
>> assumes
>> > >>>> representations are more-or-less compatible between engines, which
>> isn't
>> > >>>> the case.
>> > >>>>
>> > >>>> I'm concerned about the user journey where a user attempts to
>> query a
>> > >>>> view and receives an error message stating they cannot query it
>> due to a
>> > >>>> wrong engine or invalid SQL syntax (because their engine is using
>> the wrong
>> > >>>> representation). At this point, their only current recourse is to
>> use the
>> > >>>> REST endpoints directly to add their new representation.
>> > >>>>
>> > >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
>> > >>>>
>> > >>>>> I think the intent was for this to happen through API integration.
>> > >>>>> What is the user interaction that you're trying to achieve? Do
>> you want
>> > >>>>> each `CREATE VIEW` call to automatically append a new
>> representation?
>> > >>>>>
>> > >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
>> > >>>>> dev@iceberg.apache.org> wrote:
>> > >>>>>
>> > >>>>>> Hello,
>> > >>>>>>
>> > >>>>>> The view spec allows creating a single view with multiple
>> > >>>>>> engine-specific representations. This allows a single view to be
>> queried by
>> > >>>>>> multiple engines, each using its own specific SQL dialect.
>> > >>>>>>
>> > >>>>>> In practice, this is difficult to achieve. Users can use the
>> > >>>>>> `replace.drop-dialect.allowed` view property to allow the
>> replacing of an
>> > >>>>>> existing representation. However, there's no way to append a
>> representation
>> > >>>>>> (by creating a new ViewVersion with the existing representations
>> and a new
>> > >>>>>> one). This means that a direct REST API call is the only way to
>> create a
>> > >>>>>> View with multiple representations.
>> > >>>>>>
>> > >>>>>> I'd like to propose the creation of a new view property called
>> > >>>>>> `replace.append-dialect.allowed` that allows users to append a
>> new
>> > >>>>>> representation to an existing view.
>> > >>>>>>
>> > >>>>>> PR: https://github.com/apache/iceberg/pull/17930
>> > >>>>>>
>> > >>>>>> Please take a look. I'd love to hear from the community whether
>> there
>> > >>>>>> is value in this use case.
>> > >>>>>>
>> > >>>>>> Thanks!
>> > >>>>>>
>> > >>>>>> -- Alex Stephen
>> > >>>>>>
>> > >>>>>
>> >
>>
>
Re: Views with Multiple Representations
Posted by Péter Váry <pe...@gmail.com>.
In my opinion, the ideal experience is for users to define both SQL
statements when creating the view. This avoids any period during which the
view is only partially defined and allows the entire workflow to be handled
through a single authoring tool.
That said, the Iceberg Java API is available, and engines are free to
choose which capabilities they support and how they expose them to users.
Talat Uyarer <ta...@apache.org> ezt írta (időpont: 2026. szept. 7., H,
17:18):
> Hi Péter,
>
> I think a SQL-only workflow is closer than it looks. a Spark user doesn't
> need Spark to parse the Flink SQL, they need to read it, and showing
> another engine's SQL as text is engine-agnostic. What's missing is
> presentation, not translation.
>
> The workflow I'd propose:
>
> 1. Flink: CREATE VIEW v AS SELECT ... — version 1, with a `flink`
> representation.
> 2. Spark user: DESCRIBE EXTENDED VIEW— engines should display *all*
> representations with their dialect labels. The strings are already in the
> metadata, so this is purely a display convention, no spec change. This
> answers "how can the Spark user know the Flink SQL".
> 3. The user reads the Flink SQL and authors the Spark equivalent.
> 4. Spark: ALTER VIEW v AS SELECT ... — upserts the `spark` representation
> and carries the `flink` one forward unchanged. Issuing that statement *is*
> the consistency assertion — the "user authority" you described, made
> concrete. The version summary already records engine name/version, so we
> get per-version provenance for free.
> 5. On read, each engine executes exactly its own dialect.
>
> The rule that makes this flag-free is, I think, just a crisper version of
> what Dan
> said:
>
> ALTER VIEW ... AS = update/add my engine's representation,
> preserve the rest
> CREATE OR REPLACE VIEW = redefine the view; only my representation
> remains
>
> Two intents, two existing syntaxes, no properties, no new grammar.
>
> When someone genuinely changes the view's meaning with CREATE OR REPLACE,
> the other representations are gone, the next read from another engine fails
> fast, and that engine's user re-adds their dialect via steps 2–4. The
> failure is the signal, no cross-engine coordination protocol needed.
>
> That only holds if engines actually fail fast, which is Alex's fallback
> point: today sqlFor() silently returns the "closest" representation, which
> is exactly how an engine ends up executing another engine's SQL. That
> leniency was defensible while appending a dialect was impossible; with this
> workflow it loses its justification. #17874 adds an opt-in
> view-dialect-strict to Flink as a first step, and ideally the error message
> itself teaches the workflow: "View v has no 'flink'
> representation (found: spark). Use SHOW CREATE VIEW to see the existing
> SQL and ALTER VIEW to add a flink representation."
>
> Talat
>
>
> On 2026/09/07 14:38:10 Péter Váry wrote:
> > Do I understand correctly that the expected behavior for the user if they
> > want to create a view with Flink and Spark representations is:
> >
> > 1. CREATE VIEW my_view AS SELECT ... - in Flink
> > 2. ALTER VIEW my_view AS SELECT ... - in Spark
> >
> > How can a Spark user know the Flink SQL the user used to create the view?
> >
> > I'm trying to put together a workflow for creating interoperable views
> and
> > using SQL doesn't seem viable, so as Ryan mentioned the only option is
> > Iceberg API for now.
> >
> >
> > Péter Váry <pe...@gmail.com> ezt írta (időpont: 2026. szept.
> > 5., Szo, 9:16):
> >
> > > Thanks Alex for bringing this up.
> > >
> > > We had a very similar discussion with Guo when he wanted to add a Flink
> > > representation to a view:
> https://github.com/apache/iceberg/issues/15296.
> > > Talat also has a fresh PR for this issue in Flink:
> > > https://github.com/apache/iceberg/pull/17874
> > >
> > > My main concern is consistency. How can Flink be sure that the Flink
> SQL
> > > returns the same results as the already existing Spark SQL.
> > > In the end we accept the users authority to state that the Flink SQL is
> > > consistent with the existing Spark SQL. This means we accept that they
> are
> > > able to parse both, and make decisions.
> > > I do think we should allow a user to update one or more of the SQLs
> > > without needing extra flags.
> > >
> > >
> > >
> > > On Sat, Sep 5, 2026, 04:43 Prashant Singh <pr...@gmail.com>
> > > wrote:
> > >
> > >> Thanks for starting the thread Alex,
> > >>
> > >> I agree with Dan, an engine should only update / add their own dialect
> > >> and not make guarantees about other engine dialects without passing
> through
> > >> the other engine's compiler.
> > >> I would recommend ALTER too, I know engines such as starrocks have
> came
> > >> up with grammar [1] to be more specific that one is adding /
> modifying the
> > >> dialect, given the dialect is of starrocks, that could be an option :
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
> > >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
> > >>
> > >>
> > >> [1]
> > >>
> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
> > >>
> > >> Best,
> > >> Prashant Singh
> > >>
> > >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org>
> wrote:
> > >>
> > >>> I feel like if we want to support updating/adding a representation
> for a
> > >>> specific engine, the right way is to allow 'ALTER VIEW' to update or
> add
> > >>> the current engine's representation.
> > >>>
> > >>> I don't think we can expect an engine to add or update other engines'
> > >>> representations because that would potentially require parsing a
> different
> > >>> SQL syntax than is native to the engine.
> > >>>
> > >>> For the example you gave where there's an existing representation in
> > >>> Trino, then using `ALTER VIEW . . .` would just set the spark
> > >>> representation (as opposed to a `REPLACE` or `CREATE` that would
> create a
> > >>> new representation with only the spake dialect).
> > >>>
> > >>> I don't like the idea of adding properties to toggle behavior though
> as
> > >>> that makes something declarative subject to side-effects of the
> > >>> configuration/session.
> > >>>
> > >>> -Dan
> > >>>
> > >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
> > >>> dev@iceberg.apache.org> wrote:
> > >>>
> > >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as much
> as
> > >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW
> > >>>> VIEWS` (which just calls the underlying List Views API), so they
> know that
> > >>>> they don't need to create a view as much as alter it.
> > >>>>
> > >>>> The Java implementation
> > >>>> <
> https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130>
> surfaces
> > >>>> the first representation when a matching one doesn't exist. This
> assumes
> > >>>> representations are more-or-less compatible between engines, which
> isn't
> > >>>> the case.
> > >>>>
> > >>>> I'm concerned about the user journey where a user attempts to query
> a
> > >>>> view and receives an error message stating they cannot query it due
> to a
> > >>>> wrong engine or invalid SQL syntax (because their engine is using
> the wrong
> > >>>> representation). At this point, their only current recourse is to
> use the
> > >>>> REST endpoints directly to add their new representation.
> > >>>>
> > >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
> > >>>>
> > >>>>> I think the intent was for this to happen through API integration.
> > >>>>> What is the user interaction that you're trying to achieve? Do you
> want
> > >>>>> each `CREATE VIEW` call to automatically append a new
> representation?
> > >>>>>
> > >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
> > >>>>> dev@iceberg.apache.org> wrote:
> > >>>>>
> > >>>>>> Hello,
> > >>>>>>
> > >>>>>> The view spec allows creating a single view with multiple
> > >>>>>> engine-specific representations. This allows a single view to be
> queried by
> > >>>>>> multiple engines, each using its own specific SQL dialect.
> > >>>>>>
> > >>>>>> In practice, this is difficult to achieve. Users can use the
> > >>>>>> `replace.drop-dialect.allowed` view property to allow the
> replacing of an
> > >>>>>> existing representation. However, there's no way to append a
> representation
> > >>>>>> (by creating a new ViewVersion with the existing representations
> and a new
> > >>>>>> one). This means that a direct REST API call is the only way to
> create a
> > >>>>>> View with multiple representations.
> > >>>>>>
> > >>>>>> I'd like to propose the creation of a new view property called
> > >>>>>> `replace.append-dialect.allowed` that allows users to append a new
> > >>>>>> representation to an existing view.
> > >>>>>>
> > >>>>>> PR: https://github.com/apache/iceberg/pull/17930
> > >>>>>>
> > >>>>>> Please take a look. I'd love to hear from the community whether
> there
> > >>>>>> is value in this use case.
> > >>>>>>
> > >>>>>> Thanks!
> > >>>>>>
> > >>>>>> -- Alex Stephen
> > >>>>>>
> > >>>>>
> >
>
Re: Views with Multiple Representations
Posted by Talat Uyarer <ta...@apache.org>.
Hi Péter,
I think a SQL-only workflow is closer than it looks. a Spark user doesn't need Spark to parse the Flink SQL, they need to read it, and showing another engine's SQL as text is engine-agnostic. What's missing is presentation, not translation.
The workflow I'd propose:
1. Flink: CREATE VIEW v AS SELECT ... — version 1, with a `flink` representation.
2. Spark user: DESCRIBE EXTENDED VIEW— engines should display *all* representations with their dialect labels. The strings are already in the metadata, so this is purely a display convention, no spec change. This answers "how can the Spark user know the Flink SQL".
3. The user reads the Flink SQL and authors the Spark equivalent.
4. Spark: ALTER VIEW v AS SELECT ... — upserts the `spark` representation and carries the `flink` one forward unchanged. Issuing that statement *is* the consistency assertion — the "user authority" you described, made concrete. The version summary already records engine name/version, so we get per-version provenance for free.
5. On read, each engine executes exactly its own dialect.
The rule that makes this flag-free is, I think, just a crisper version of what Dan
said:
ALTER VIEW ... AS = update/add my engine's representation, preserve the rest
CREATE OR REPLACE VIEW = redefine the view; only my representation remains
Two intents, two existing syntaxes, no properties, no new grammar.
When someone genuinely changes the view's meaning with CREATE OR REPLACE, the other representations are gone, the next read from another engine fails fast, and that engine's user re-adds their dialect via steps 2–4. The failure is the signal, no cross-engine coordination protocol needed.
That only holds if engines actually fail fast, which is Alex's fallback point: today sqlFor() silently returns the "closest" representation, which is exactly how an engine ends up executing another engine's SQL. That leniency was defensible while appending a dialect was impossible; with this workflow it loses its justification. #17874 adds an opt-in view-dialect-strict to Flink as a first step, and ideally the error message itself teaches the workflow: "View v has no 'flink'
representation (found: spark). Use SHOW CREATE VIEW to see the existing SQL and ALTER VIEW to add a flink representation."
Talat
On 2026/09/07 14:38:10 Péter Váry wrote:
> Do I understand correctly that the expected behavior for the user if they
> want to create a view with Flink and Spark representations is:
>
> 1. CREATE VIEW my_view AS SELECT ... - in Flink
> 2. ALTER VIEW my_view AS SELECT ... - in Spark
>
> How can a Spark user know the Flink SQL the user used to create the view?
>
> I'm trying to put together a workflow for creating interoperable views and
> using SQL doesn't seem viable, so as Ryan mentioned the only option is
> Iceberg API for now.
>
>
> Péter Váry <pe...@gmail.com> ezt írta (időpont: 2026. szept.
> 5., Szo, 9:16):
>
> > Thanks Alex for bringing this up.
> >
> > We had a very similar discussion with Guo when he wanted to add a Flink
> > representation to a view: https://github.com/apache/iceberg/issues/15296.
> > Talat also has a fresh PR for this issue in Flink:
> > https://github.com/apache/iceberg/pull/17874
> >
> > My main concern is consistency. How can Flink be sure that the Flink SQL
> > returns the same results as the already existing Spark SQL.
> > In the end we accept the users authority to state that the Flink SQL is
> > consistent with the existing Spark SQL. This means we accept that they are
> > able to parse both, and make decisions.
> > I do think we should allow a user to update one or more of the SQLs
> > without needing extra flags.
> >
> >
> >
> > On Sat, Sep 5, 2026, 04:43 Prashant Singh <pr...@gmail.com>
> > wrote:
> >
> >> Thanks for starting the thread Alex,
> >>
> >> I agree with Dan, an engine should only update / add their own dialect
> >> and not make guarantees about other engine dialects without passing through
> >> the other engine's compiler.
> >> I would recommend ALTER too, I know engines such as starrocks have came
> >> up with grammar [1] to be more specific that one is adding / modifying the
> >> dialect, given the dialect is of starrocks, that could be an option :
> >>
> >>
> >>
> >>
> >>
> >>
> >> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
> >> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
> >>
> >>
> >> [1]
> >> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
> >>
> >> Best,
> >> Prashant Singh
> >>
> >> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org> wrote:
> >>
> >>> I feel like if we want to support updating/adding a representation for a
> >>> specific engine, the right way is to allow 'ALTER VIEW' to update or add
> >>> the current engine's representation.
> >>>
> >>> I don't think we can expect an engine to add or update other engines'
> >>> representations because that would potentially require parsing a different
> >>> SQL syntax than is native to the engine.
> >>>
> >>> For the example you gave where there's an existing representation in
> >>> Trino, then using `ALTER VIEW . . .` would just set the spark
> >>> representation (as opposed to a `REPLACE` or `CREATE` that would create a
> >>> new representation with only the spake dialect).
> >>>
> >>> I don't like the idea of adding properties to toggle behavior though as
> >>> that makes something declarative subject to side-effects of the
> >>> configuration/session.
> >>>
> >>> -Dan
> >>>
> >>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
> >>> dev@iceberg.apache.org> wrote:
> >>>
> >>>> I'm not sure if `CREATE VIEW` would be the correct choice, as much as
> >>>> `ALTER VIEW`. The user can see that a view exists through `SHOW
> >>>> VIEWS` (which just calls the underlying List Views API), so they know that
> >>>> they don't need to create a view as much as alter it.
> >>>>
> >>>> The Java implementation
> >>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> surfaces
> >>>> the first representation when a matching one doesn't exist. This assumes
> >>>> representations are more-or-less compatible between engines, which isn't
> >>>> the case.
> >>>>
> >>>> I'm concerned about the user journey where a user attempts to query a
> >>>> view and receives an error message stating they cannot query it due to a
> >>>> wrong engine or invalid SQL syntax (because their engine is using the wrong
> >>>> representation). At this point, their only current recourse is to use the
> >>>> REST endpoints directly to add their new representation.
> >>>>
> >>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
> >>>>
> >>>>> I think the intent was for this to happen through API integration.
> >>>>> What is the user interaction that you're trying to achieve? Do you want
> >>>>> each `CREATE VIEW` call to automatically append a new representation?
> >>>>>
> >>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
> >>>>> dev@iceberg.apache.org> wrote:
> >>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> The view spec allows creating a single view with multiple
> >>>>>> engine-specific representations. This allows a single view to be queried by
> >>>>>> multiple engines, each using its own specific SQL dialect.
> >>>>>>
> >>>>>> In practice, this is difficult to achieve. Users can use the
> >>>>>> `replace.drop-dialect.allowed` view property to allow the replacing of an
> >>>>>> existing representation. However, there's no way to append a representation
> >>>>>> (by creating a new ViewVersion with the existing representations and a new
> >>>>>> one). This means that a direct REST API call is the only way to create a
> >>>>>> View with multiple representations.
> >>>>>>
> >>>>>> I'd like to propose the creation of a new view property called
> >>>>>> `replace.append-dialect.allowed` that allows users to append a new
> >>>>>> representation to an existing view.
> >>>>>>
> >>>>>> PR: https://github.com/apache/iceberg/pull/17930
> >>>>>>
> >>>>>> Please take a look. I'd love to hear from the community whether there
> >>>>>> is value in this use case.
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>> -- Alex Stephen
> >>>>>>
> >>>>>
>
Re: Views with Multiple Representations
Posted by Péter Váry <pe...@gmail.com>.
Do I understand correctly that the expected behavior for the user if they
want to create a view with Flink and Spark representations is:
1. CREATE VIEW my_view AS SELECT ... - in Flink
2. ALTER VIEW my_view AS SELECT ... - in Spark
How can a Spark user know the Flink SQL the user used to create the view?
I'm trying to put together a workflow for creating interoperable views and
using SQL doesn't seem viable, so as Ryan mentioned the only option is
Iceberg API for now.
Péter Váry <pe...@gmail.com> ezt írta (időpont: 2026. szept.
5., Szo, 9:16):
> Thanks Alex for bringing this up.
>
> We had a very similar discussion with Guo when he wanted to add a Flink
> representation to a view: https://github.com/apache/iceberg/issues/15296.
> Talat also has a fresh PR for this issue in Flink:
> https://github.com/apache/iceberg/pull/17874
>
> My main concern is consistency. How can Flink be sure that the Flink SQL
> returns the same results as the already existing Spark SQL.
> In the end we accept the users authority to state that the Flink SQL is
> consistent with the existing Spark SQL. This means we accept that they are
> able to parse both, and make decisions.
> I do think we should allow a user to update one or more of the SQLs
> without needing extra flags.
>
>
>
> On Sat, Sep 5, 2026, 04:43 Prashant Singh <pr...@gmail.com>
> wrote:
>
>> Thanks for starting the thread Alex,
>>
>> I agree with Dan, an engine should only update / add their own dialect
>> and not make guarantees about other engine dialects without passing through
>> the other engine's compiler.
>> I would recommend ALTER too, I know engines such as starrocks have came
>> up with grammar [1] to be more specific that one is adding / modifying the
>> dialect, given the dialect is of starrocks, that could be an option :
>>
>>
>>
>>
>>
>>
>> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
>> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
>>
>>
>> [1]
>> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
>>
>> Best,
>> Prashant Singh
>>
>> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org> wrote:
>>
>>> I feel like if we want to support updating/adding a representation for a
>>> specific engine, the right way is to allow 'ALTER VIEW' to update or add
>>> the current engine's representation.
>>>
>>> I don't think we can expect an engine to add or update other engines'
>>> representations because that would potentially require parsing a different
>>> SQL syntax than is native to the engine.
>>>
>>> For the example you gave where there's an existing representation in
>>> Trino, then using `ALTER VIEW . . .` would just set the spark
>>> representation (as opposed to a `REPLACE` or `CREATE` that would create a
>>> new representation with only the spake dialect).
>>>
>>> I don't like the idea of adding properties to toggle behavior though as
>>> that makes something declarative subject to side-effects of the
>>> configuration/session.
>>>
>>> -Dan
>>>
>>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
>>> dev@iceberg.apache.org> wrote:
>>>
>>>> I'm not sure if `CREATE VIEW` would be the correct choice, as much as
>>>> `ALTER VIEW`. The user can see that a view exists through `SHOW
>>>> VIEWS` (which just calls the underlying List Views API), so they know that
>>>> they don't need to create a view as much as alter it.
>>>>
>>>> The Java implementation
>>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> surfaces
>>>> the first representation when a matching one doesn't exist. This assumes
>>>> representations are more-or-less compatible between engines, which isn't
>>>> the case.
>>>>
>>>> I'm concerned about the user journey where a user attempts to query a
>>>> view and receives an error message stating they cannot query it due to a
>>>> wrong engine or invalid SQL syntax (because their engine is using the wrong
>>>> representation). At this point, their only current recourse is to use the
>>>> REST endpoints directly to add their new representation.
>>>>
>>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
>>>>
>>>>> I think the intent was for this to happen through API integration.
>>>>> What is the user interaction that you're trying to achieve? Do you want
>>>>> each `CREATE VIEW` call to automatically append a new representation?
>>>>>
>>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
>>>>> dev@iceberg.apache.org> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> The view spec allows creating a single view with multiple
>>>>>> engine-specific representations. This allows a single view to be queried by
>>>>>> multiple engines, each using its own specific SQL dialect.
>>>>>>
>>>>>> In practice, this is difficult to achieve. Users can use the
>>>>>> `replace.drop-dialect.allowed` view property to allow the replacing of an
>>>>>> existing representation. However, there's no way to append a representation
>>>>>> (by creating a new ViewVersion with the existing representations and a new
>>>>>> one). This means that a direct REST API call is the only way to create a
>>>>>> View with multiple representations.
>>>>>>
>>>>>> I'd like to propose the creation of a new view property called
>>>>>> `replace.append-dialect.allowed` that allows users to append a new
>>>>>> representation to an existing view.
>>>>>>
>>>>>> PR: https://github.com/apache/iceberg/pull/17930
>>>>>>
>>>>>> Please take a look. I'd love to hear from the community whether there
>>>>>> is value in this use case.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> -- Alex Stephen
>>>>>>
>>>>>
Re: Views with Multiple Representations
Posted by Péter Váry <pe...@gmail.com>.
Thanks Alex for bringing this up.
We had a very similar discussion with Guo when he wanted to add a Flink
representation to a view: https://github.com/apache/iceberg/issues/15296.
Talat also has a fresh PR for this issue in Flink:
https://github.com/apache/iceberg/pull/17874
My main concern is consistency. How can Flink be sure that the Flink SQL
returns the same results as the already existing Spark SQL.
In the end we accept the users authority to state that the Flink SQL is
consistent with the existing Spark SQL. This means we accept that they are
able to parse both, and make decisions.
I do think we should allow a user to update one or more of the SQLs without
needing extra flags.
On Sat, Sep 5, 2026, 04:43 Prashant Singh <pr...@gmail.com> wrote:
> Thanks for starting the thread Alex,
>
> I agree with Dan, an engine should only update / add their own dialect and
> not make guarantees about other engine dialects without passing through the
> other engine's compiler.
> I would recommend ALTER too, I know engines such as starrocks have came
> up with grammar [1] to be more specific that one is adding / modifying the
> dialect, given the dialect is of starrocks, that could be an option :
>
>
>
>
>
>
> *ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
> <column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
>
>
> [1]
> https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
>
> Best,
> Prashant Singh
>
> On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org> wrote:
>
>> I feel like if we want to support updating/adding a representation for a
>> specific engine, the right way is to allow 'ALTER VIEW' to update or add
>> the current engine's representation.
>>
>> I don't think we can expect an engine to add or update other engines'
>> representations because that would potentially require parsing a different
>> SQL syntax than is native to the engine.
>>
>> For the example you gave where there's an existing representation in
>> Trino, then using `ALTER VIEW . . .` would just set the spark
>> representation (as opposed to a `REPLACE` or `CREATE` that would create a
>> new representation with only the spake dialect).
>>
>> I don't like the idea of adding properties to toggle behavior though as
>> that makes something declarative subject to side-effects of the
>> configuration/session.
>>
>> -Dan
>>
>> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
>> dev@iceberg.apache.org> wrote:
>>
>>> I'm not sure if `CREATE VIEW` would be the correct choice, as much as
>>> `ALTER VIEW`. The user can see that a view exists through `SHOW VIEWS`
>>> (which just calls the underlying List Views API), so they know that they
>>> don't need to create a view as much as alter it.
>>>
>>> The Java implementation
>>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> surfaces
>>> the first representation when a matching one doesn't exist. This assumes
>>> representations are more-or-less compatible between engines, which isn't
>>> the case.
>>>
>>> I'm concerned about the user journey where a user attempts to query a
>>> view and receives an error message stating they cannot query it due to a
>>> wrong engine or invalid SQL syntax (because their engine is using the wrong
>>> representation). At this point, their only current recourse is to use the
>>> REST endpoints directly to add their new representation.
>>>
>>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
>>>
>>>> I think the intent was for this to happen through API integration. What
>>>> is the user interaction that you're trying to achieve? Do you want each
>>>> `CREATE VIEW` call to automatically append a new representation?
>>>>
>>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
>>>> dev@iceberg.apache.org> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> The view spec allows creating a single view with multiple
>>>>> engine-specific representations. This allows a single view to be queried by
>>>>> multiple engines, each using its own specific SQL dialect.
>>>>>
>>>>> In practice, this is difficult to achieve. Users can use the
>>>>> `replace.drop-dialect.allowed` view property to allow the replacing of an
>>>>> existing representation. However, there's no way to append a representation
>>>>> (by creating a new ViewVersion with the existing representations and a new
>>>>> one). This means that a direct REST API call is the only way to create a
>>>>> View with multiple representations.
>>>>>
>>>>> I'd like to propose the creation of a new view property called
>>>>> `replace.append-dialect.allowed` that allows users to append a new
>>>>> representation to an existing view.
>>>>>
>>>>> PR: https://github.com/apache/iceberg/pull/17930
>>>>>
>>>>> Please take a look. I'd love to hear from the community whether there
>>>>> is value in this use case.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> -- Alex Stephen
>>>>>
>>>>
Re: Views with Multiple Representations
Posted by Prashant Singh <pr...@gmail.com>.
Thanks for starting the thread Alex,
I agree with Dan, an engine should only update / add their own dialect and
not make guarantees about other engine dialects without passing through the
other engine's compiler.
I would recommend ALTER too, I know engines such as starrocks have came up
with grammar [1] to be more specific that one is adding / modifying the
dialect, given the dialect is of starrocks, that could be an option :
*ALTER VIEW [<catalog>.<database>.]<view_name>( <column_name> [,
<column_name>]){ ADD | MODIFY } DIALECT<query_statement>*
[1]
https://docs.starrocks.io/docs/data_source/catalog/iceberg/DDL/#alter-view-to-update-starrocks-dialect
Best,
Prashant Singh
On Fri, Sep 4, 2026 at 4:33 PM Daniel Weeks <dw...@apache.org> wrote:
> I feel like if we want to support updating/adding a representation for a
> specific engine, the right way is to allow 'ALTER VIEW' to update or add
> the current engine's representation.
>
> I don't think we can expect an engine to add or update other engines'
> representations because that would potentially require parsing a different
> SQL syntax than is native to the engine.
>
> For the example you gave where there's an existing representation in
> Trino, then using `ALTER VIEW . . .` would just set the spark
> representation (as opposed to a `REPLACE` or `CREATE` that would create a
> new representation with only the spake dialect).
>
> I don't like the idea of adding properties to toggle behavior though as
> that makes something declarative subject to side-effects of the
> configuration/session.
>
> -Dan
>
> On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <
> dev@iceberg.apache.org> wrote:
>
>> I'm not sure if `CREATE VIEW` would be the correct choice, as much as
>> `ALTER VIEW`. The user can see that a view exists through `SHOW VIEWS`
>> (which just calls the underlying List Views API), so they know that they
>> don't need to create a view as much as alter it.
>>
>> The Java implementation
>> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> surfaces
>> the first representation when a matching one doesn't exist. This assumes
>> representations are more-or-less compatible between engines, which isn't
>> the case.
>>
>> I'm concerned about the user journey where a user attempts to query a
>> view and receives an error message stating they cannot query it due to a
>> wrong engine or invalid SQL syntax (because their engine is using the wrong
>> representation). At this point, their only current recourse is to use the
>> REST endpoints directly to add their new representation.
>>
>> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
>>
>>> I think the intent was for this to happen through API integration. What
>>> is the user interaction that you're trying to achieve? Do you want each
>>> `CREATE VIEW` call to automatically append a new representation?
>>>
>>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
>>> dev@iceberg.apache.org> wrote:
>>>
>>>> Hello,
>>>>
>>>> The view spec allows creating a single view with multiple
>>>> engine-specific representations. This allows a single view to be queried by
>>>> multiple engines, each using its own specific SQL dialect.
>>>>
>>>> In practice, this is difficult to achieve. Users can use the
>>>> `replace.drop-dialect.allowed` view property to allow the replacing of an
>>>> existing representation. However, there's no way to append a representation
>>>> (by creating a new ViewVersion with the existing representations and a new
>>>> one). This means that a direct REST API call is the only way to create a
>>>> View with multiple representations.
>>>>
>>>> I'd like to propose the creation of a new view property called
>>>> `replace.append-dialect.allowed` that allows users to append a new
>>>> representation to an existing view.
>>>>
>>>> PR: https://github.com/apache/iceberg/pull/17930
>>>>
>>>> Please take a look. I'd love to hear from the community whether there
>>>> is value in this use case.
>>>>
>>>> Thanks!
>>>>
>>>> -- Alex Stephen
>>>>
>>>
Re: Views with Multiple Representations
Posted by Daniel Weeks <dw...@apache.org>.
I feel like if we want to support updating/adding a representation for a
specific engine, the right way is to allow 'ALTER VIEW' to update or add
the current engine's representation.
I don't think we can expect an engine to add or update other engines'
representations because that would potentially require parsing a different
SQL syntax than is native to the engine.
For the example you gave where there's an existing representation in
Trino, then using `ALTER VIEW . . .` would just set the spark
representation (as opposed to a `REPLACE` or `CREATE` that would create a
new representation with only the spake dialect).
I don't like the idea of adding properties to toggle behavior though as
that makes something declarative subject to side-effects of the
configuration/session.
-Dan
On Fri, Sep 4, 2026 at 3:37 PM Alex Stephen via dev <de...@iceberg.apache.org>
wrote:
> I'm not sure if `CREATE VIEW` would be the correct choice, as much as
> `ALTER VIEW`. The user can see that a view exists through `SHOW VIEWS`
> (which just calls the underlying List Views API), so they know that they
> don't need to create a view as much as alter it.
>
> The Java implementation
> <https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130> surfaces
> the first representation when a matching one doesn't exist. This assumes
> representations are more-or-less compatible between engines, which isn't
> the case.
>
> I'm concerned about the user journey where a user attempts to query a view
> and receives an error message stating they cannot query it due to a wrong
> engine or invalid SQL syntax (because their engine is using the wrong
> representation). At this point, their only current recourse is to use the
> REST endpoints directly to add their new representation.
>
> On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
>
>> I think the intent was for this to happen through API integration. What
>> is the user interaction that you're trying to achieve? Do you want each
>> `CREATE VIEW` call to automatically append a new representation?
>>
>> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
>> dev@iceberg.apache.org> wrote:
>>
>>> Hello,
>>>
>>> The view spec allows creating a single view with multiple
>>> engine-specific representations. This allows a single view to be queried by
>>> multiple engines, each using its own specific SQL dialect.
>>>
>>> In practice, this is difficult to achieve. Users can use the
>>> `replace.drop-dialect.allowed` view property to allow the replacing of an
>>> existing representation. However, there's no way to append a representation
>>> (by creating a new ViewVersion with the existing representations and a new
>>> one). This means that a direct REST API call is the only way to create a
>>> View with multiple representations.
>>>
>>> I'd like to propose the creation of a new view property called
>>> `replace.append-dialect.allowed` that allows users to append a new
>>> representation to an existing view.
>>>
>>> PR: https://github.com/apache/iceberg/pull/17930
>>>
>>> Please take a look. I'd love to hear from the community whether there is
>>> value in this use case.
>>>
>>> Thanks!
>>>
>>> -- Alex Stephen
>>>
>>
Re: Views with Multiple Representations
Posted by Alex Stephen via dev <de...@iceberg.apache.org>.
I'm not sure if `CREATE VIEW` would be the correct choice, as much as
`ALTER VIEW`. The user can see that a view exists through `SHOW VIEWS`
(which just calls the underlying List Views API), so they know that they
don't need to create a view as much as alter it.
The Java implementation
<https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/view/BaseView.java#L114-L130>
surfaces
the first representation when a matching one doesn't exist. This assumes
representations are more-or-less compatible between engines, which isn't
the case.
I'm concerned about the user journey where a user attempts to query a view
and receives an error message stating they cannot query it due to a wrong
engine or invalid SQL syntax (because their engine is using the wrong
representation). At this point, their only current recourse is to use the
REST endpoints directly to add their new representation.
On Fri, Sep 4, 2026 at 3:01 PM Ryan Blue <bl...@apache.org> wrote:
> I think the intent was for this to happen through API integration. What is
> the user interaction that you're trying to achieve? Do you want each
> `CREATE VIEW` call to automatically append a new representation?
>
> On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <
> dev@iceberg.apache.org> wrote:
>
>> Hello,
>>
>> The view spec allows creating a single view with multiple engine-specific
>> representations. This allows a single view to be queried by multiple
>> engines, each using its own specific SQL dialect.
>>
>> In practice, this is difficult to achieve. Users can use the
>> `replace.drop-dialect.allowed` view property to allow the replacing of an
>> existing representation. However, there's no way to append a representation
>> (by creating a new ViewVersion with the existing representations and a new
>> one). This means that a direct REST API call is the only way to create a
>> View with multiple representations.
>>
>> I'd like to propose the creation of a new view property called
>> `replace.append-dialect.allowed` that allows users to append a new
>> representation to an existing view.
>>
>> PR: https://github.com/apache/iceberg/pull/17930
>>
>> Please take a look. I'd love to hear from the community whether there is
>> value in this use case.
>>
>> Thanks!
>>
>> -- Alex Stephen
>>
>
Re: Views with Multiple Representations
Posted by Ryan Blue <bl...@apache.org>.
I think the intent was for this to happen through API integration. What is
the user interaction that you're trying to achieve? Do you want each
`CREATE VIEW` call to automatically append a new representation?
On Fri, Sep 4, 2026 at 2:32 PM Alex Stephen via dev <de...@iceberg.apache.org>
wrote:
> Hello,
>
> The view spec allows creating a single view with multiple engine-specific
> representations. This allows a single view to be queried by multiple
> engines, each using its own specific SQL dialect.
>
> In practice, this is difficult to achieve. Users can use the
> `replace.drop-dialect.allowed` view property to allow the replacing of an
> existing representation. However, there's no way to append a representation
> (by creating a new ViewVersion with the existing representations and a new
> one). This means that a direct REST API call is the only way to create a
> View with multiple representations.
>
> I'd like to propose the creation of a new view property called
> `replace.append-dialect.allowed` that allows users to append a new
> representation to an existing view.
>
> PR: https://github.com/apache/iceberg/pull/17930
>
> Please take a look. I'd love to hear from the community whether there is
> value in this use case.
>
> Thanks!
>
> -- Alex Stephen
>