Best practices for assigning operator UIDs

[Jared Stehler <ja...@intellifylearning.com>]

I have some confusion around how best to assign UIDs to operators. The documentation states simply that they are important, but stops short of recommending what if any stateful information should go into the name. For example, if the same code is used to create two separate job graphs, should the operator UIDs contain information specific to each job instance?

Example: for a kafka source, should the UID contain the subscribed topic name / pattern?

Example: if I have a custom mapping function which reshapes a JSON field based on an expression, should that expression (or a hash thereof) be contained in that operator’s UID?

Basically, is the UID similar to overriding hashcode and equals for POJOs, or is it more like overriding serializableVersionUID?


--
Jared Stehler
Chief Architect - Intellify Learning
o: 617.701.6330 x703

Re: Best practices for assigning operator UIDs

[Gyula Fóra <gy...@gmail.com>]

Hi Jared,

The only thing that matters is that UIDs are unique within one JobGraph.
Its completely fine to use the same uids in two separate jobs.


Beyond this I would go with simple uids that dont contain parts of the
logic, because maybe you want to change the logic (expression, or add new
topics) and still keep the state.

Hope this helps :)
Gyula

On Sun, Jul 9, 2017, 18:36 Jared Stehler <
jared.stehler@intellifylearning.com> wrote:

> I have some confusion around how best to assign UIDs to operators. The
> documentation states simply that they are important, but stops short of
> recommending what if any stateful information should go into the name. For
> example, if the same code is used to create two separate job graphs, should
> the operator UIDs contain information specific to each job instance?
>
> Example: for a kafka source, should the UID contain the subscribed topic
> name / pattern?
>
> Example: if I have a custom mapping function which reshapes a JSON field
> based on an expression, should that expression (or a hash thereof) be
> contained in that operator’s UID?
>
> Basically, is the UID similar to overriding hashcode and equals for POJOs,
> or is it more like overriding serializableVersionUID?
>
>
> --
> Jared Stehler
> Chief Architect - Intellify Learning
> o: 617.701.6330 x703
>
>
>
>