FlinkSource to read iceberg table in Batch mode

[Chetas Joshi <ch...@gmail.com>]

Hello,

I am using iceberg-flink-runtime lib (1.17-1.4.0) and running the following
code to read an iceberg table in BATCH mode.

var source = FlinkSource
    .forRowData()
    .streaming(false)
    .env(execEnv)
    .tableLoader(tableLoader)
    .limit((long) operation.getLimit())
    .filters(buildFilterExpression(operation))
    .build();

var stream = source.map(rowDataMapper).name(operation.getName());

I am running into the following exception even though streaming = false.

Detected an UNBOUNDED source with the 'execution.runtime-mode' set to
'BATCH'. This combination is not allowed, please set the
'execution.runtime-mode' to STREAMING or AUTOMATIC

Would appreciate any pointers here.

Thank you
Chetas

Re: FlinkSource to read iceberg table in Batch mode

[Péter Váry <pe...@gmail.com>]

Hi Chetas,

How sure are you that your job has no other unbounded source?

I have seen this working in several cases, but if you could help with
providing a sort executable example, I would like to take a look.

Thanks,
Peter

On Wed, Mar 13, 2024, 20:55 Chetas Joshi <ch...@gmail.com> wrote:

> Hello,
>
> I am using iceberg-flink-runtime lib (1.17-1.4.0) and running the
> following code to read an iceberg table in BATCH mode.
>
> var source = FlinkSource
>     .forRowData()
>     .streaming(false)
>     .env(execEnv)
>     .tableLoader(tableLoader)
>     .limit((long) operation.getLimit())
>     .filters(buildFilterExpression(operation))
>     .build();
>
> var stream = source.map(rowDataMapper).name(operation.getName());
>
> I am running into the following exception even though streaming = false.
>
> Detected an UNBOUNDED source with the 'execution.runtime-mode' set to
> 'BATCH'. This combination is not allowed, please set the
> 'execution.runtime-mode' to STREAMING or AUTOMATIC
>
> Would appreciate any pointers here.
>
> Thank you
> Chetas
>
>
>