[jira] [Created] (BEAM-1801) default_job_name can generate names not accepted by DataFlow

["Dennis Docter (JIRA)" <ji...@apache.org>]

Dennis Docter created BEAM-1801:
-----------------------------------

             Summary: default_job_name can generate names not accepted by DataFlow
                 Key: BEAM-1801
                 URL: https://issues.apache.org/jira/browse/BEAM-1801
             Project: Beam
          Issue Type: Bug
          Components: sdk-py
            Reporter: Dennis Docter
            Assignee: Ahmet Altay
            Priority: Trivial


The default job name generated by: 

https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py#L288

is partially derived from the os username of the executing user. These may contain characters not accepted by Dataflow, resulting in errors like:

apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
(a1b878f3562c0e6d): Error processing pipeline. Causes: (a1b878f3562c04ae): Prefix for cluster 'beamapp-dennis.docter-032-03231324-1edc-harness' should match '[a-z]([-a-z0-9]{0,61}[a-z0-9])?'. This probably means the joblabel is invalid.

To solve this issue, sanitise the username to only container alphanumeric characters and dashes.

Also there seems to be no length restriction and dataflow imposes a 63 character length limit in the above case. Limiting on length substantially shorter than that to allow for postfixes (like -harness in this case) may be wise.




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)