Status type mismatch between different API calls

[Randal Moore <rd...@gmail.com>]

I'm using dataflow. Found what seems to me to be "usage problem" with the
available APIs.
When I submit a job to the dataflow runner, I get back a
DataflowPipelineJob or its superclass, PipelineResult which provides me the
status of the job - as an enumerated type. But if I use a DataflowClient to
inquire about the status of my job
  public Job getJob(@Nonnull String jobId) throws IOException {
I get back a string. Why are these different? I would think this should be
the same data type. Short of that, is there something that can translate
between them? A list of strings strings that dataflow client might return?

I am wrapping dataflow jobs with a higher level service that would like to
translate all dataflow client status to its own. Therefore the enum is much
nicer than the string.

Am I overlooking something?
randy

Re: Status type mismatch between different API calls

[Randal Moore <rd...@gmail.com>]

Thanks!

On Fri, Feb 2, 2018 at 11:50 AM Lukasz Cwik <lc...@google.com> wrote:

> The DataflowClient is an auto-generated client for the Dataflow service.
> When your interacting with the Dataflow runner, your interacting with a
> library written by humans which wraps the auto generated client.
>
> MonitoringUtil.toState() converts that string to the set of enums your
> familiar with:
>
> https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/util/MonitoringUtil.java#L222
>
> On Fri, Feb 2, 2018 at 8:40 AM, Randal Moore <rd...@gmail.com> wrote:
>
>> I'm using dataflow. Found what seems to me to be "usage problem" with the
>> available APIs.
>> When I submit a job to the dataflow runner, I get back a
>> DataflowPipelineJob or its superclass, PipelineResult which provides me
>> the status of the job - as an enumerated type. But if I use a
>> DataflowClient to inquire about the status of my job
>>   public Job getJob(@Nonnull String jobId) throws IOException {
>> I get back a string. Why are these different? I would think this should
>> be the same data type. Short of that, is there something that can translate
>> between them? A list of strings strings that dataflow client might return?
>>
>> I am wrapping dataflow jobs with a higher level service that would like
>> to translate all dataflow client status to its own. Therefore the enum is
>> much nicer than the string.
>>
>> Am I overlooking something?
>> randy
>>
>
>

Re: Status type mismatch between different API calls

[Lukasz Cwik <lc...@google.com>]

The DataflowClient is an auto-generated client for the Dataflow service.
When your interacting with the Dataflow runner, your interacting with a
library written by humans which wraps the auto generated client.

MonitoringUtil.toState() converts that string to the set of enums your
familiar with:
https://github.com/apache/beam/blob/master/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/util/MonitoringUtil.java#L222

On Fri, Feb 2, 2018 at 8:40 AM, Randal Moore <rd...@gmail.com> wrote:

> I'm using dataflow. Found what seems to me to be "usage problem" with the
> available APIs.
> When I submit a job to the dataflow runner, I get back a
> DataflowPipelineJob or its superclass, PipelineResult which provides me
> the status of the job - as an enumerated type. But if I use a
> DataflowClient to inquire about the status of my job
>   public Job getJob(@Nonnull String jobId) throws IOException {
> I get back a string. Why are these different? I would think this should be
> the same data type. Short of that, is there something that can translate
> between them? A list of strings strings that dataflow client might return?
>
> I am wrapping dataflow jobs with a higher level service that would like to
> translate all dataflow client status to its own. Therefore the enum is much
> nicer than the string.
>
> Am I overlooking something?
> randy
>