DataSet - Broadcast set in output format

[bastien dine <ba...@gmail.com>]

Hello,

I would like to use a broadcast variable in my outputformat (to pass some
information, and control execution flow)
How would I do it ?
.output does not have a .withBroadcast function as it does not extends
SingleInputUdfOperator


------------------

Bastien DINE
Data Architect / Software Engineer / Sysadmin
bastiendine.io

Re: DataSet - Broadcast set in output format

[Till Rohrmann <tr...@apache.org>]

Hi Bastien,

the OutputFormat specifies how a given record is written to an external
system. The DataSink using these formats do not support using broadcast
variables. This is currently a limitation of Flink.

What you could do is to introduce a mapper before your sink which enriches
the records with respect to the broadcast variable. The OutputFormat could
then react to this additional information.

Cheers,
Till

On Thu, Nov 22, 2018 at 2:57 PM bastien dine <ba...@gmail.com> wrote:

> Hello,
>
> I would like to use a broadcast variable in my outputformat (to pass some
> information, and control execution flow)
> How would I do it ?
> .output does not have a .withBroadcast function as it does not extends
> SingleInputUdfOperator
>
>
> ------------------
>
> Bastien DINE
> Data Architect / Software Engineer / Sysadmin
> bastiendine.io
>