SupportsReadingMetadata flow between table transformations

[Yuval Itzchakov <yu...@gmail.com>]

Hi,
I am looking for a way to propagate a column all the way from the source to
the sink, without the knowledge of whoever is applying transformations on
the tables.

I looked into SupportsReadingMetadata, and saw that it is able to "tag"
tables with metadata. I was wondering, does this tag flow between the
different projections and joins the table goes through? If not, what could
be alternatives to achieve flowing some column A between all table
transformations such that I can provide some "metadata" context that is
coming in from the source itself?
-- 
Best Regards,
Yuval Itzchakov.

Re: SupportsReadingMetadata flow between table transformations

[Hang Ruan <ru...@gmail.com>]

Hi, Yuval.

`SupportsReadingMetadata` make connectors be able to append metadata
columns to the schema of table sources.  But one operator does not know the
columns map to which column in the source.
We could try to generate this information when parsing Flink sql. But it is
hard for DataStream API to generate this information.

Best,
Hang

Yuval Itzchakov <yu...@gmail.com> 于2023年5月30日周二 12:27写道：

>
> Hi,
> I am looking for a way to propagate a column all the way from the source
> to the sink, without the knowledge of whoever is applying transformations
> on the tables.
>
> I looked into SupportsReadingMetadata, and saw that it is able to "tag"
> tables with metadata. I was wondering, does this tag flow between the
> different projections and joins the table goes through? If not, what could
> be alternatives to achieve flowing some column A between all table
> transformations such that I can provide some "metadata" context that is
> coming in from the source itself?
> --
> Best Regards,
> Yuval Itzchakov.
>