[jira] [Created] (BEAM-3584) Java dataflow job fails with 2.3.0 RC1, due to missing worker image

["Alan Myrvold (JIRA)" <ji...@apache.org>]

Alan Myrvold created BEAM-3584:
----------------------------------

             Summary: Java dataflow job fails with 2.3.0 RC1, due to missing worker image
                 Key: BEAM-3584
                 URL: https://issues.apache.org/jira/browse/BEAM-3584
             Project: Beam
          Issue Type: Bug
          Components: examples-java
    Affects Versions: 2.3.0
            Reporter: Alan Myrvold
            Assignee: Alan Myrvold


Dataflow java jobs currently fail due to a missing docker image when using 2.3.0 RC1. This is not a bug in the SDK, the worker image needs to be published by google. I will be coordinating the worker image publication.

# Update to your own project and bucket.

GCS_BUCKET=my-cloud-storage-bucket

GCP_PROJECT=my-cloud-project

mkdir -p ~/try2.3.0/.m2/repository

cd ~/try2.3.0/.m2/repository

wget [https://repository.apache.org/content/repositories/orgapachebeam-1026/archetype-catalog.xml]

cd ~/try2.3.0

echo "<settings><localRepository>$PWD/.m2/repository</localRepository><profiles><profile><id>testrel</id><repositories><repository><id>testrel</id><url>[https://repository.apache.org/content/repositories/orgapachebeam-1026]</url></repository></repositories></profile></profiles></settings>" > settings.xml

mvn archetype:generate -DarchetypeCatalog=local  -DarchetypeGroupId=org.apache.beam 
 -DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples       -DarchetypeVersion=2.3.0      -DgroupId=org.example       -DartifactId=word-count-beam       -Dversion="0.1"       -Dpackage=org.apache.beam.examples -s settings.xml

# answer yes

cd word-count-beam

mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount        -Dexec.args="--runner=DataflowRunner --project=alan-jenkins-test  --gcpTempLocation=gs://${GCS_BUCKET}/tmp  --inputFile=gs://apache-beam-samples/shakespeare/* --output=gs://${GCP_PROJECT}e/counts"   -s ../settings.xml -Ptestrel,dataflow-runner

 

Dataflow logs contain:

E  Handler for GET /v1.27/images/dataflow.gcr.io/v1beta3/beam-java-batch:beam-2.3.0/json returned error: No such image: dataflow.gcr.io/v1beta3/beam-java-batch:beam-2.3.0 
E  PullImage "dataflow.gcr.io/v1beta3/beam-java-batch:beam-2.3.0" from image service failed: rpc error: code = 2 desc = Error: Status 405 trying to pull repository v1beta3/beam-java-batch: "v1 Registry API is disabled. If you are not explicitly using the v1 Registry API, it is possible your v2 image could not be found. Verify that your image is available, or retry with `dockerd --disable-legacy-registry`. See https://cloud.google.com/container-registry/docs/support/deprecation-notices" 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)