You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@beam.apache.org by Shuangyin Ge <go...@gmail.com> on 2017/05/22 01:03:42 UTC

Getting "Unable to find registrar for c" error

Hello team,

Not sure if any can share some lights. I am getting "Unable to find
registrar for c" error while running

$ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
     -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner

following the Java Quickstart for Beam https://beam.apache.org/get-
started/quickstart-java/

Complete output is shown below:

[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model
for org.example:word-count-beam:jar:0.1
[WARNING] 'build.plugins.plugin.version' for
org.apache.maven.plugins:maven-jar-plugin
is missing. @ line 80, column 15
[WARNING]
[WARNING] It is highly recommended to fix these problems because they
threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support
building such malformed projects.
[WARNING]
[INFO]
[INFO] ------------------------------------------------------------
------------
[INFO] Building word-count-beam 0.1
[INFO] ------------------------------------------------------------
------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
word-count-beam ---
[WARNING] Using platform encoding (Cp1252 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory C:\Users\ges\Workspace\word-
count-beam\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
word-count-beam ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding Cp1252,
i.e. build is platform dependent!
[INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-
count-beam\target\classes
[INFO]
[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam ---
[WARNING]
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
java.lang.IllegalStateException: Unable to find registrar for c
at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.
waitUntilFinish(DirectRunner.java:322)
at org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.
waitUntilFinish(DirectRunner.java:292)
at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:200)
at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:63)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
at org.apache.beam.examples.WordCount.main(WordCount.java:184)
... 6 more
Caused by: java.lang.IllegalStateException: Unable to find registrar for c
at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(
FileSystems.java:447)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:174)
at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(
FileBasedSink.java:905)
at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.
processElement(WriteFiles.java:376)
[INFO] ------------------------------------------------------------
------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------
------------
[INFO] Total time: 10.279 s
[INFO] Finished at: 2017-05-22T08:56:16+08:00
[INFO] Final Memory: 32M/450M
[INFO] ------------------------------------------------------------
------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
(default-cli) on project word-count-beam: An exception occured while
executing the Java class. null: InvocationTargetException:
java.lang.IllegalStateException:
Unable to find registrar for c -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/
MojoExecutionException

Pom.xml is as below

<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.example</groupId>
<artifactId>word-count-beam</artifactId>
<version>0.1</version>

<packaging>jar</packaging>

<properties>
<beam.version>2.0.0</beam.version>
<surefire-plugin.version>2.20</surefire-plugin.version>
</properties>

<repositories>
<repository>
<id>apache.snapshots</id>
<name>Apache Development Snapshot Repository</name>
<url>https://repository.apache.org/content/repositories/snapshots/</url>
<releases>
<enabled>false</enabled>
</releases>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.5.1</version>
<configuration>
<source>1.7</source>
<target>1.7</target>
</configuration>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
<configuration>
<parallel>all</parallel>
<threadCount>4</threadCount>
<redirectTestOutputToFile>true</redirectTestOutputToFile>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.20</version>
</dependency>
</dependencies>
</plugin>

<!-- Ensure that the Maven jar plugin runs before the Maven
shade plugin by listing the plugin higher within the file. -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
</plugin>

<!--
Configures `mvn package` to produce a bundled jar ("fat jar") for runners
that require this for job submission to a cluster.
-->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/LICENSE</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.
ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>

<pluginManagement>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<configuration>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
</configuration>
</plugin>
</plugins>
</pluginManagement>
</build>

<profiles>
<profile>
<id>direct-runner</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<!-- Makes the DirectRunner available when running a pipeline. -->
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-direct-java</artifactId>
<version>2.0.0</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</profile>

<profile>
<id>apex-runner</id>
<!-- Makes the ApexRunner available when running a pipeline. -->
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-apex</artifactId>
<version>${beam.version}</version>
<scope>runtime</scope>
</dependency>
<!--
Apex depends on httpclient version 4.3.5, project has a transitive
dependency to httpclient 4.0.1 from
google-http-client. Apex dependency version being specified explicitly so
that it gets picked up. This
can be removed when the project no longer has a dependency on a different
httpclient version.
-->
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.3.5</version>
<scope>runtime</scope>
<exclusions>
<exclusion>
<groupId>commons-codec</groupId>
<artifactId>commons-codec</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencies>
</profile>

<profile>
<id>dataflow-runner</id>
<!-- Makes the DataflowRunner available when running a pipeline. -->
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>${beam.version}</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</profile>

<profile>
<id>flink-runner</id>
<!-- Makes the FlinkRunner available when running a pipeline. -->
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-flink_2.10</artifactId>
<version>${beam.version}</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</profile>

<profile>
<id>spark-runner</id>
<!-- Makes the SparkRunner available when running a pipeline. Additionally,
overrides some Spark dependencies to Beam-compatible versions. -->
<dependencies>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-spark</artifactId>
<version>${beam.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
<version>${beam.version}</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-streaming_2.10</artifactId>
<version>1.6.2</version>
<scope>runtime</scope>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>jul-to-slf4j</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.module</groupId>
<artifactId>jackson-module-scala_2.10</artifactId>
<version>2.8.8</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</profile>
</profiles>

<dependencies>
<!-- Adds a dependency on the Beam SDK. -->
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-core</artifactId>
<version>${beam.version}</version>
</dependency>

<!-- Adds a dependency on the Beam Google Cloud Platform IO module. -->
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
<version>${beam.version}</version>
</dependency>

<!-- Dependencies below this line are specific dependencies needed by the
examples code. -->
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.22.0</version>
<exclusions>
<!-- Exclude an old version of guava that is being pulled
in by a transitive dependency of google-api-client -->
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava-jdk5</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-bigquery</artifactId>
<version>v2-rev295-1.22.0</version>
<exclusions>
<!-- Exclude an old version of guava that is being pulled
in by a transitive dependency of google-api-client -->
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava-jdk5</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>com.google.http-client</groupId>
<artifactId>google-http-client</artifactId>
<version>1.22.0</version>
<exclusions>
<!-- Exclude an old version of guava that is being pulled
in by a transitive dependency of google-api-client -->
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava-jdk5</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-pubsub</artifactId>
<version>v1-rev10-1.22.0</version>
<exclusions>
<!-- Exclude an old version of guava that is being pulled
in by a transitive dependency of google-api-client -->
<exclusion>
<groupId>com.google.guava</groupId>
<artifactId>guava-jdk5</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>2.4</version>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>20.0</version>
</dependency>

<!-- Add slf4j API frontend binding with JUL backend -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.14</version>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>1.7.14</version>
<!-- When loaded at runtime this will wire up slf4j to the JUL backend -->
<scope>runtime</scope>
</dependency>

<!-- Hamcrest and JUnit are required dependencies of PAssert,
which is used in the main code of DebuggingWordCount example. -->
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest-all</artifactId>
<version>1.3</version>
</dependency>

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>

<!-- The DirectRunner is needed for unit tests. -->
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-direct-java</artifactId>
<version>${beam.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>



Thanks in advance.

Re: Getting "Unable to find registrar for c" error

Posted by Lukasz Cwik <lc...@google.com>.

I'm all out of suggestions at this point beyond doing one of the following:
* contributing a fix
* up vote and watch the windows integration issue:
https://issues.apache.org/jira/browse/BEAM-2299

On Thu, May 25, 2017 at 1:48 AM, Shuangyin Ge <go...@gmail.com> wrote:

> BTW, what I see in the folder structure after running the below command,
>
> mvn compile exec:java -P direct-runner -D exec.mainClass=org.apache.beam.examples.WordCount
> -D exec.args="--inputFile=pom.xml --output=counts"
>
> is
>
> C:\Users\ges\Workspace\word-count-beam>dir
>  Volume in drive C has no label.
>  Volume Serial Number is 5244-E4AC
>
>  Directory of C:\Users\ges\Workspace\word-count-beam
>
> 05/25/2017  04:48 PM    <DIR>          .
> 05/25/2017  04:48 PM    <DIR>          ..
> 05/25/2017  04:04 PM    <DIR>          .temp-beam-2017-05-145_08-04-47-1
> 05/25/2017  04:48 PM    <DIR>          .temp-beam-2017-05-145_08-47-59-1
> 05/22/2017  08:56 AM             4,231 debug.txt
> 05/22/2017  08:14 AM            12,658 pom.xml
> 05/22/2017  07:55 AM    <DIR>          src
> 05/25/2017  04:04 PM    <DIR>          target
>                2 File(s)         16,889 bytes
>                6 Dir(s)  38,775,607,296 bytes free
>
> Let me know if that is needed.
>
> Thanks,
> Silas Ge
>
> 2017-05-25 16:08 GMT+08:00 Shuangyin Ge <go...@gmail.com>:
>
>> I used "mvn compile exec:java -P direct-runner -D
>> exec.mainClass=org.apache.beam.examples.WordCount -D
>> exec.args="--inputFile=C%3A/users/ges/workspace/word-count-beam/pom.xml
>> --output=counts"" however got a "NOT FOUND" error. It didn't create any
>> temp files yet.
>>
>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>> (default-cli) on project word-count-beam: An exception occured while
>> executing the Java class. null: InvocationTargetException:
>> java.lang.IllegalArgumentException: Error matching the pattern or glob
>> C%3A/users/ges/workspace/word-count-beam/pom.xml: status NOT_FOUND ->
>> [Help 1]
>>
>> I do believe without specifying the URI the compile works on the most
>> parts -
>>
>> mvn compile exec:java -P direct-runner -D exec.mainClass=org.apache.beam.examples.WordCount
>> -D exec.args="--inputFile=pom.xml --output=counts"
>>
>> Failed with
>>
>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>> (default-cli) on project word-count-beam: An exception occured while
>> executing the Java class. null: InvocationTargetException:
>> java.lang.IllegalStateException: Unable to find registrar for c -> [Help
>> 1]
>>
>> Is there a way to know which "c" beams tried to read but failed to find
>> registrar? The reason I ask is because from what I can see, the compile and
>> execution definitely already passed the input stage. The content of the
>> file "pom.xml" had already been stored and counted in the temp files
>> generated. Is it possible that Beams was reading some other "c"s somewhere
>> else and reported this error?
>>
>> Thanks,
>> Silas Ge
>>
>>
>>
>> 2017-05-23 22:41 GMT+08:00 Lukasz Cwik <lc...@google.com>:
>>
>>> The paths are URI-like so "C:/path/to/file.txt" resolves to a URI with a
>>> scheme of "C", an authority of "path" and a path of "/to/file.txt" which is
>>> likely to never work since no filesystem handler will be bound to the
>>> scheme "C".
>>>
>>> Try "file:///C%3A/Users/ges/Workspace/word-count-beampom.xml"
>>> (% is the way to escape reserved characters in URIs and 3A is the hex
>>> for ":")
>>>
>>>
>>> On Tue, May 23, 2017 at 2:43 AM, Shuangyin Ge <go...@gmail.com>
>>> wrote:
>>>
>>>> When I simply used "C:/Users/ges/Workspace/word-count-beam/pom.xml" as
>>>> input file, I am pretty sure that it has gone farther than using URI
>>>> "file:///", because it has already generated the temp pipeline files and
>>>> wordcount. Just the compile failed with "Unable to find registrar for c".
>>>> To my gut this is a different issue than not able to locate the input file.
>>>>
>>>> Thanks,
>>>> Silas Ge
>>>>
>>>> 2017-05-23 17:31 GMT+08:00 Shuangyin Ge <go...@gmail.com>:
>>>>
>>>>> Hello Luke,
>>>>>
>>>>> I tried with "file:///C:/Users/ges/Workspace/word-count-beampom.xml"
>>>>> or "C:/Users/ges/Workspace/word-count-beampom.xml" as input file as
>>>>> seen in https://issues.apache.org/jira/browse/BEAM-2298, however both
>>>>> failed.
>>>>>
>>>>> "file:///C:/Users/ges/Workspace/word-count-beampom.xml" failed
>>>>> with Illegal char <:> at index 4
>>>>> "C:/Users/ges/Workspace/word-count-beampom.xml" failed with "Unable
>>>>> to find registrar for c"
>>>>>
>>>>> Any ideas?
>>>>>
>>>>> Thanks,
>>>>> Silas Ge
>>>>>
>>>>> 2017-05-22 23:28 GMT+08:00 Lukasz Cwik <lc...@google.com>:
>>>>>
>>>>>> This is a known problem with how the local file system is being
>>>>>> interacted with as is evident by the similar test failures when running the
>>>>>> Apache Beam unit tests on Windows:
>>>>>> https://issues.apache.org/jira/browse/BEAM-2299
>>>>>>
>>>>>> I would suggest trying to use "file:///c:/my/path/to/file.txt"
>>>>>>
>>>>>> There is a general proposal to change how we work with resource paths:
>>>>>> https://issues.apache.org/jira/browse/BEAM-2283
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <go...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hello team,
>>>>>>>
>>>>>>> Not sure if any can share some lights. I am getting "Unable to find
>>>>>>> registrar for c" error while running
>>>>>>>
>>>>>>> $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>>>>>>>      -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner
>>>>>>>
>>>>>>> following the Java Quickstart for Beam https://beam.apache.org/g
>>>>>>> et-started/quickstart-java/
>>>>>>>
>>>>>>> Complete output is shown below:
>>>>>>>
>>>>>>> [INFO] Scanning for projects...
>>>>>>> [WARNING]
>>>>>>> [WARNING] Some problems were encountered while building the
>>>>>>> effective model for org.example:word-count-beam:jar:0.1
>>>>>>> [WARNING] 'build.plugins.plugin.version' for
>>>>>>> org.apache.maven.plugins:maven-jar-plugin is missing. @ line 80,
>>>>>>> column 15
>>>>>>> [WARNING]
>>>>>>> [WARNING] It is highly recommended to fix these problems because
>>>>>>> they threaten the stability of your build.
>>>>>>> [WARNING]
>>>>>>> [WARNING] For this reason, future Maven versions might no longer
>>>>>>> support building such malformed projects.
>>>>>>> [WARNING]
>>>>>>> [INFO]
>>>>>>> [INFO] ------------------------------------------------------------
>>>>>>> ------------
>>>>>>> [INFO] Building word-count-beam 0.1
>>>>>>> [INFO] ------------------------------------------------------------
>>>>>>> ------------
>>>>>>> [INFO]
>>>>>>> [INFO] --- maven-resources-plugin:2.6:resources (default-resources)
>>>>>>> @ word-count-beam ---
>>>>>>> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered
>>>>>>> resources, i.e. build is platform dependent!
>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>> C:\Users\ges\Workspace\word-count-beam\src\main\resources
>>>>>>> [INFO]
>>>>>>> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
>>>>>>> word-count-beam ---
>>>>>>> [INFO] Changes detected - recompiling the module!
>>>>>>> [WARNING] File encoding has not been set, using platform encoding
>>>>>>> Cp1252, i.e. build is platform dependent!
>>>>>>> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co
>>>>>>> unt-beam\target\classes
>>>>>>> [INFO]
>>>>>>> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @
>>>>>>> word-count-beam ---
>>>>>>> [WARNING]
>>>>>>> java.lang.reflect.InvocationTargetException
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>>>>>> ssorImpl.java:62)
>>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>>>>>> thodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>>>> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
>>>>>>> java.lang.IllegalStateException: Unable to find registrar for c
>>>>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>>>>> sult.waitUntilFinish(DirectRunner.java:322)
>>>>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>>>>> sult.waitUntilFinish(DirectRunner.java:292)
>>>>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>>>>> .java:200)
>>>>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>>>>> .java:63)
>>>>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
>>>>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
>>>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:184)
>>>>>>> ... 6 more
>>>>>>> Caused by: java.lang.IllegalStateException: Unable to find
>>>>>>> registrar for c
>>>>>>> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil
>>>>>>> eSystems.java:447)
>>>>>>> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
>>>>>>> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystem
>>>>>>> s.java:174)
>>>>>>> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
>>>>>>> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase
>>>>>>> dSink.java:905)
>>>>>>> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces
>>>>>>> sElement(WriteFiles.java:376)
>>>>>>> [INFO] ------------------------------------------------------------
>>>>>>> ------------
>>>>>>> [INFO] BUILD FAILURE
>>>>>>> [INFO] ------------------------------------------------------------
>>>>>>> ------------
>>>>>>> [INFO] Total time: 10.279 s
>>>>>>> [INFO] Finished at: 2017-05-22T08:56:16+08:00
>>>>>>> [INFO] Final Memory: 32M/450M
>>>>>>> [INFO] ------------------------------------------------------------
>>>>>>> ------------
>>>>>>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>>>>>>> (default-cli) on project word-count-beam: An exception occured while
>>>>>>> executing the Java class. null: InvocationTargetException:
>>>>>>> java.lang.IllegalStateException: Unable to find registrar for c ->
>>>>>>> [Help 1]
>>>>>>> [ERROR]
>>>>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>>>>>> the -e switch.
>>>>>>> [ERROR] Re-run Maven using the -X switch to enable full debug
>>>>>>> logging.
>>>>>>> [ERROR]
>>>>>>> [ERROR] For more information about the errors and possible
>>>>>>> solutions, please read the following articles:
>>>>>>> [ERROR] [Help 1] http://cwiki.apache.org/con
>>>>>>> fluence/display/MAVEN/MojoExecutionException
>>>>>>>
>>>>>>> Pom.xml is as below
>>>>>>>
>>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>>> <!--
>>>>>>> Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>>> contributor license agreements. See the NOTICE file distributed with
>>>>>>> this work for additional information regarding copyright ownership.
>>>>>>> The ASF licenses this file to You under the Apache License, Version
>>>>>>> 2.0
>>>>>>> (the "License"); you may not use this file except in compliance with
>>>>>>> the License. You may obtain a copy of the License at
>>>>>>>
>>>>>>> http://www.apache.org/licenses/LICENSE-2.0
>>>>>>>
>>>>>>> Unless required by applicable law or agreed to in writing, software
>>>>>>> distributed under the License is distributed on an "AS IS" BASIS,
>>>>>>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>>> implied.
>>>>>>> See the License for the specific language governing permissions and
>>>>>>> limitations under the License.
>>>>>>> -->
>>>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0"
>>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>>>>>>> http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>>>
>>>>>>> <groupId>org.example</groupId>
>>>>>>> <artifactId>word-count-beam</artifactId>
>>>>>>> <version>0.1</version>
>>>>>>>
>>>>>>> <packaging>jar</packaging>
>>>>>>>
>>>>>>> <properties>
>>>>>>> <beam.version>2.0.0</beam.version>
>>>>>>> <surefire-plugin.version>2.20</surefire-plugin.version>
>>>>>>> </properties>
>>>>>>>
>>>>>>> <repositories>
>>>>>>> <repository>
>>>>>>> <id>apache.snapshots</id>
>>>>>>> <name>Apache Development Snapshot Repository</name>
>>>>>>> <url>https://repository.apache.org/content/repositories/snapshots/</
>>>>>>> url>
>>>>>>> <releases>
>>>>>>> <enabled>false</enabled>
>>>>>>> </releases>
>>>>>>> <snapshots>
>>>>>>> <enabled>true</enabled>
>>>>>>> </snapshots>
>>>>>>> </repository>
>>>>>>> </repositories>
>>>>>>>
>>>>>>> <build>
>>>>>>> <plugins>
>>>>>>> <plugin>
>>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>>> <artifactId>maven-compiler-plugin</artifactId>
>>>>>>> <version>3.5.1</version>
>>>>>>> <configuration>
>>>>>>> <source>1.7</source>
>>>>>>> <target>1.7</target>
>>>>>>> </configuration>
>>>>>>> </plugin>
>>>>>>>
>>>>>>> <plugin>
>>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>>> <artifactId>maven-surefire-plugin</artifactId>
>>>>>>> <version>2.20</version>
>>>>>>> <configuration>
>>>>>>> <parallel>all</parallel>
>>>>>>> <threadCount>4</threadCount>
>>>>>>> <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>>>>>> </configuration>
>>>>>>> <dependencies>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.maven.surefire</groupId>
>>>>>>> <artifactId>surefire-junit47</artifactId>
>>>>>>> <version>2.20</version>
>>>>>>> </dependency>
>>>>>>> </dependencies>
>>>>>>> </plugin>
>>>>>>>
>>>>>>> <!-- Ensure that the Maven jar plugin runs before the Maven
>>>>>>> shade plugin by listing the plugin higher within the file. -->
>>>>>>> <plugin>
>>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>>> <artifactId>maven-jar-plugin</artifactId>
>>>>>>> </plugin>
>>>>>>>
>>>>>>> <!--
>>>>>>> Configures `mvn package` to produce a bundled jar ("fat jar") for
>>>>>>> runners
>>>>>>> that require this for job submission to a cluster.
>>>>>>> -->
>>>>>>> <plugin>
>>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>>> <artifactId>maven-shade-plugin</artifactId>
>>>>>>> <version>3.0.0</version>
>>>>>>> <executions>
>>>>>>> <execution>
>>>>>>> <phase>package</phase>
>>>>>>> <goals>
>>>>>>> <goal>shade</goal>
>>>>>>> </goals>
>>>>>>> <configuration>
>>>>>>> <filters>
>>>>>>> <filter>
>>>>>>> <artifact>*:*</artifact>
>>>>>>> <excludes>
>>>>>>> <exclude>META-INF/LICENSE</exclude>
>>>>>>> <exclude>META-INF/*.SF</exclude>
>>>>>>> <exclude>META-INF/*.DSA</exclude>
>>>>>>> <exclude>META-INF/*.RSA</exclude>
>>>>>>> </excludes>
>>>>>>> </filter>
>>>>>>> </filters>
>>>>>>> <transformers>
>>>>>>> <transformer implementation="org.apache.mav
>>>>>>> en.plugins.shade.resource.ServicesResourceTransformer"/>
>>>>>>> </transformers>
>>>>>>> </configuration>
>>>>>>> </execution>
>>>>>>> </executions>
>>>>>>> </plugin>
>>>>>>> </plugins>
>>>>>>>
>>>>>>> <pluginManagement>
>>>>>>> <plugins>
>>>>>>> <plugin>
>>>>>>> <groupId>org.codehaus.mojo</groupId>
>>>>>>> <artifactId>exec-maven-plugin</artifactId>
>>>>>>> <version>1.4.0</version>
>>>>>>> <configuration>
>>>>>>> <cleanupDaemonThreads>false</cleanupDaemonThreads>
>>>>>>> </configuration>
>>>>>>> </plugin>
>>>>>>> </plugins>
>>>>>>> </pluginManagement>
>>>>>>> </build>
>>>>>>>
>>>>>>> <profiles>
>>>>>>> <profile>
>>>>>>> <id>direct-runner</id>
>>>>>>> <activation>
>>>>>>> <activeByDefault>true</activeByDefault>
>>>>>>> </activation>
>>>>>>> <!-- Makes the DirectRunner available when running a pipeline. -->
>>>>>>> <dependencies>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>>>>> <version>2.0.0</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>> </dependencies>
>>>>>>> </profile>
>>>>>>>
>>>>>>> <profile>
>>>>>>> <id>apex-runner</id>
>>>>>>> <!-- Makes the ApexRunner available when running a pipeline. -->
>>>>>>> <dependencies>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-runners-apex</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>> <!--
>>>>>>> Apex depends on httpclient version 4.3.5, project has a transitive
>>>>>>> dependency to httpclient 4.0.1 from
>>>>>>> google-http-client. Apex dependency version being specified
>>>>>>> explicitly so that it gets picked up. This
>>>>>>> can be removed when the project no longer has a dependency on a
>>>>>>> different httpclient version.
>>>>>>> -->
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.httpcomponents</groupId>
>>>>>>> <artifactId>httpclient</artifactId>
>>>>>>> <version>4.3.5</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> <exclusions>
>>>>>>> <exclusion>
>>>>>>> <groupId>commons-codec</groupId>
>>>>>>> <artifactId>commons-codec</artifactId>
>>>>>>> </exclusion>
>>>>>>> </exclusions>
>>>>>>> </dependency>
>>>>>>> </dependencies>
>>>>>>> </profile>
>>>>>>>
>>>>>>> <profile>
>>>>>>> <id>dataflow-runner</id>
>>>>>>> <!-- Makes the DataflowRunner available when running a pipeline. -->
>>>>>>> <dependencies>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>> </dependencies>
>>>>>>> </profile>
>>>>>>>
>>>>>>> <profile>
>>>>>>> <id>flink-runner</id>
>>>>>>> <!-- Makes the FlinkRunner available when running a pipeline. -->
>>>>>>> <dependencies>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-runners-flink_2.10</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>> </dependencies>
>>>>>>> </profile>
>>>>>>>
>>>>>>> <profile>
>>>>>>> <id>spark-runner</id>
>>>>>>> <!-- Makes the SparkRunner available when running a pipeline.
>>>>>>> Additionally,
>>>>>>> overrides some Spark dependencies to Beam-compatible versions. -->
>>>>>>> <dependencies>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-runners-spark</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.spark</groupId>
>>>>>>> <artifactId>spark-streaming_2.10</artifactId>
>>>>>>> <version>1.6.2</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> <exclusions>
>>>>>>> <exclusion>
>>>>>>> <groupId>org.slf4j</groupId>
>>>>>>> <artifactId>jul-to-slf4j</artifactId>
>>>>>>> </exclusion>
>>>>>>> </exclusions>
>>>>>>> </dependency>
>>>>>>> <dependency>
>>>>>>> <groupId>com.fasterxml.jackson.module</groupId>
>>>>>>> <artifactId>jackson-module-scala_2.10</artifactId>
>>>>>>> <version>2.8.8</version>
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>> </dependencies>
>>>>>>> </profile>
>>>>>>> </profiles>
>>>>>>>
>>>>>>> <dependencies>
>>>>>>> <!-- Adds a dependency on the Beam SDK. -->
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-sdks-java-core</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <!-- Adds a dependency on the Beam Google Cloud Platform IO module.
>>>>>>> -->
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <!-- Dependencies below this line are specific dependencies needed
>>>>>>> by the examples code. -->
>>>>>>> <dependency>
>>>>>>> <groupId>com.google.api-client</groupId>
>>>>>>> <artifactId>google-api-client</artifactId>
>>>>>>> <version>1.22.0</version>
>>>>>>> <exclusions>
>>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>>> in by a transitive dependency of google-api-client -->
>>>>>>> <exclusion>
>>>>>>> <groupId>com.google.guava</groupId>
>>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>>> </exclusion>
>>>>>>> </exclusions>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>com.google.apis</groupId>
>>>>>>> <artifactId>google-api-services-bigquery</artifactId>
>>>>>>> <version>v2-rev295-1.22.0</version>
>>>>>>> <exclusions>
>>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>>> in by a transitive dependency of google-api-client -->
>>>>>>> <exclusion>
>>>>>>> <groupId>com.google.guava</groupId>
>>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>>> </exclusion>
>>>>>>> </exclusions>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>com.google.http-client</groupId>
>>>>>>> <artifactId>google-http-client</artifactId>
>>>>>>> <version>1.22.0</version>
>>>>>>> <exclusions>
>>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>>> in by a transitive dependency of google-api-client -->
>>>>>>> <exclusion>
>>>>>>> <groupId>com.google.guava</groupId>
>>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>>> </exclusion>
>>>>>>> </exclusions>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>com.google.apis</groupId>
>>>>>>> <artifactId>google-api-services-pubsub</artifactId>
>>>>>>> <version>v1-rev10-1.22.0</version>
>>>>>>> <exclusions>
>>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>>> in by a transitive dependency of google-api-client -->
>>>>>>> <exclusion>
>>>>>>> <groupId>com.google.guava</groupId>
>>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>>> </exclusion>
>>>>>>> </exclusions>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>joda-time</groupId>
>>>>>>> <artifactId>joda-time</artifactId>
>>>>>>> <version>2.4</version>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>com.google.guava</groupId>
>>>>>>> <artifactId>guava</artifactId>
>>>>>>> <version>20.0</version>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <!-- Add slf4j API frontend binding with JUL backend -->
>>>>>>> <dependency>
>>>>>>> <groupId>org.slf4j</groupId>
>>>>>>> <artifactId>slf4j-api</artifactId>
>>>>>>> <version>1.7.14</version>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>org.slf4j</groupId>
>>>>>>> <artifactId>slf4j-jdk14</artifactId>
>>>>>>> <version>1.7.14</version>
>>>>>>> <!-- When loaded at runtime this will wire up slf4j to the JUL
>>>>>>> backend -->
>>>>>>> <scope>runtime</scope>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <!-- Hamcrest and JUnit are required dependencies of PAssert,
>>>>>>> which is used in the main code of DebuggingWordCount example. -->
>>>>>>> <dependency>
>>>>>>> <groupId>org.hamcrest</groupId>
>>>>>>> <artifactId>hamcrest-all</artifactId>
>>>>>>> <version>1.3</version>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <dependency>
>>>>>>> <groupId>junit</groupId>
>>>>>>> <artifactId>junit</artifactId>
>>>>>>> <version>4.12</version>
>>>>>>> </dependency>
>>>>>>>
>>>>>>> <!-- The DirectRunner is needed for unit tests. -->
>>>>>>> <dependency>
>>>>>>> <groupId>org.apache.beam</groupId>
>>>>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>>>>> <version>${beam.version}</version>
>>>>>>> <scope>test</scope>
>>>>>>> </dependency>
>>>>>>> </dependencies>
>>>>>>> </project>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks in advance.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Getting "Unable to find registrar for c" error

Posted by Shuangyin Ge <go...@gmail.com>.

BTW, what I see in the folder structure after running the below command,

mvn compile exec:java -P direct-runner -D
exec.mainClass=org.apache.beam.examples.WordCount -D
exec.args="--inputFile=pom.xml --output=counts"

is

C:\Users\ges\Workspace\word-count-beam>dir
 Volume in drive C has no label.
 Volume Serial Number is 5244-E4AC

 Directory of C:\Users\ges\Workspace\word-count-beam

05/25/2017  04:48 PM    <DIR>          .
05/25/2017  04:48 PM    <DIR>          ..
05/25/2017  04:04 PM    <DIR>          .temp-beam-2017-05-145_08-04-47-1
05/25/2017  04:48 PM    <DIR>          .temp-beam-2017-05-145_08-47-59-1
05/22/2017  08:56 AM             4,231 debug.txt
05/22/2017  08:14 AM            12,658 pom.xml
05/22/2017  07:55 AM    <DIR>          src
05/25/2017  04:04 PM    <DIR>          target
               2 File(s)         16,889 bytes
               6 Dir(s)  38,775,607,296 bytes free

Let me know if that is needed.

Thanks,
Silas Ge

2017-05-25 16:08 GMT+08:00 Shuangyin Ge <go...@gmail.com>:

> I used "mvn compile exec:java -P direct-runner -D
> exec.mainClass=org.apache.beam.examples.WordCount -D
> exec.args="--inputFile=C%3A/users/ges/workspace/word-count-beam/pom.xml
> --output=counts"" however got a "NOT FOUND" error. It didn't create any
> temp files yet.
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
> (default-cli) on project word-count-beam: An exception occured while
> executing the Java class. null: InvocationTargetException: java.lang.IllegalArgumentException:
> Error matching the pattern or glob C%3A/users/ges/workspace/word-count-beam/pom.xml:
> status NOT_FOUND -> [Help 1]
>
> I do believe without specifying the URI the compile works on the most
> parts -
>
> mvn compile exec:java -P direct-runner -D exec.mainClass=org.apache.beam.examples.WordCount
> -D exec.args="--inputFile=pom.xml --output=counts"
>
> Failed with
>
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
> (default-cli) on project word-count-beam: An exception occured while
> executing the Java class. null: InvocationTargetException: java.lang.IllegalStateException:
> Unable to find registrar for c -> [Help 1]
>
> Is there a way to know which "c" beams tried to read but failed to find
> registrar? The reason I ask is because from what I can see, the compile and
> execution definitely already passed the input stage. The content of the
> file "pom.xml" had already been stored and counted in the temp files
> generated. Is it possible that Beams was reading some other "c"s somewhere
> else and reported this error?
>
> Thanks,
> Silas Ge
>
>
>
> 2017-05-23 22:41 GMT+08:00 Lukasz Cwik <lc...@google.com>:
>
>> The paths are URI-like so "C:/path/to/file.txt" resolves to a URI with a
>> scheme of "C", an authority of "path" and a path of "/to/file.txt" which is
>> likely to never work since no filesystem handler will be bound to the
>> scheme "C".
>>
>> Try "file:///C%3A/Users/ges/Workspace/word-count-beampom.xml"
>> (% is the way to escape reserved characters in URIs and 3A is the hex for
>> ":")
>>
>>
>> On Tue, May 23, 2017 at 2:43 AM, Shuangyin Ge <go...@gmail.com> wrote:
>>
>>> When I simply used "C:/Users/ges/Workspace/word-count-beam/pom.xml" as
>>> input file, I am pretty sure that it has gone farther than using URI
>>> "file:///", because it has already generated the temp pipeline files and
>>> wordcount. Just the compile failed with "Unable to find registrar for c".
>>> To my gut this is a different issue than not able to locate the input file.
>>>
>>> Thanks,
>>> Silas Ge
>>>
>>> 2017-05-23 17:31 GMT+08:00 Shuangyin Ge <go...@gmail.com>:
>>>
>>>> Hello Luke,
>>>>
>>>> I tried with "file:///C:/Users/ges/Workspace/word-count-beampom.xml"
>>>> or "C:/Users/ges/Workspace/word-count-beampom.xml" as input file as
>>>> seen in https://issues.apache.org/jira/browse/BEAM-2298, however both
>>>> failed.
>>>>
>>>> "file:///C:/Users/ges/Workspace/word-count-beampom.xml" failed
>>>> with Illegal char <:> at index 4
>>>> "C:/Users/ges/Workspace/word-count-beampom.xml" failed with "Unable to
>>>> find registrar for c"
>>>>
>>>> Any ideas?
>>>>
>>>> Thanks,
>>>> Silas Ge
>>>>
>>>> 2017-05-22 23:28 GMT+08:00 Lukasz Cwik <lc...@google.com>:
>>>>
>>>>> This is a known problem with how the local file system is being
>>>>> interacted with as is evident by the similar test failures when running the
>>>>> Apache Beam unit tests on Windows:
>>>>> https://issues.apache.org/jira/browse/BEAM-2299
>>>>>
>>>>> I would suggest trying to use "file:///c:/my/path/to/file.txt"
>>>>>
>>>>> There is a general proposal to change how we work with resource paths:
>>>>> https://issues.apache.org/jira/browse/BEAM-2283
>>>>>
>>>>>
>>>>>
>>>>> On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <go...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello team,
>>>>>>
>>>>>> Not sure if any can share some lights. I am getting "Unable to find
>>>>>> registrar for c" error while running
>>>>>>
>>>>>> $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>>>>>>      -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner
>>>>>>
>>>>>> following the Java Quickstart for Beam https://beam.apache.org/g
>>>>>> et-started/quickstart-java/
>>>>>>
>>>>>> Complete output is shown below:
>>>>>>
>>>>>> [INFO] Scanning for projects...
>>>>>> [WARNING]
>>>>>> [WARNING] Some problems were encountered while building the effective
>>>>>> model for org.example:word-count-beam:jar:0.1
>>>>>> [WARNING] 'build.plugins.plugin.version' for
>>>>>> org.apache.maven.plugins:maven-jar-plugin is missing. @ line 80,
>>>>>> column 15
>>>>>> [WARNING]
>>>>>> [WARNING] It is highly recommended to fix these problems because they
>>>>>> threaten the stability of your build.
>>>>>> [WARNING]
>>>>>> [WARNING] For this reason, future Maven versions might no longer
>>>>>> support building such malformed projects.
>>>>>> [WARNING]
>>>>>> [INFO]
>>>>>> [INFO] ------------------------------------------------------------
>>>>>> ------------
>>>>>> [INFO] Building word-count-beam 0.1
>>>>>> [INFO] ------------------------------------------------------------
>>>>>> ------------
>>>>>> [INFO]
>>>>>> [INFO] --- maven-resources-plugin:2.6:resources (default-resources)
>>>>>> @ word-count-beam ---
>>>>>> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered
>>>>>> resources, i.e. build is platform dependent!
>>>>>> [INFO] skip non existing resourceDirectory
>>>>>> C:\Users\ges\Workspace\word-count-beam\src\main\resources
>>>>>> [INFO]
>>>>>> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
>>>>>> word-count-beam ---
>>>>>> [INFO] Changes detected - recompiling the module!
>>>>>> [WARNING] File encoding has not been set, using platform encoding
>>>>>> Cp1252, i.e. build is platform dependent!
>>>>>> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co
>>>>>> unt-beam\target\classes
>>>>>> [INFO]
>>>>>> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @
>>>>>> word-count-beam ---
>>>>>> [WARNING]
>>>>>> java.lang.reflect.InvocationTargetException
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>>>>> ssorImpl.java:62)
>>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>>>>> thodAccessorImpl.java:43)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>>> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
>>>>>> java.lang.IllegalStateException: Unable to find registrar for c
>>>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>>>> sult.waitUntilFinish(DirectRunner.java:322)
>>>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>>>> sult.waitUntilFinish(DirectRunner.java:292)
>>>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>>>> .java:200)
>>>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>>>> .java:63)
>>>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
>>>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
>>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:184)
>>>>>> ... 6 more
>>>>>> Caused by: java.lang.IllegalStateException: Unable to find registrar
>>>>>> for c
>>>>>> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil
>>>>>> eSystems.java:447)
>>>>>> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
>>>>>> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystem
>>>>>> s.java:174)
>>>>>> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
>>>>>> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase
>>>>>> dSink.java:905)
>>>>>> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces
>>>>>> sElement(WriteFiles.java:376)
>>>>>> [INFO] ------------------------------------------------------------
>>>>>> ------------
>>>>>> [INFO] BUILD FAILURE
>>>>>> [INFO] ------------------------------------------------------------
>>>>>> ------------
>>>>>> [INFO] Total time: 10.279 s
>>>>>> [INFO] Finished at: 2017-05-22T08:56:16+08:00
>>>>>> [INFO] Final Memory: 32M/450M
>>>>>> [INFO] ------------------------------------------------------------
>>>>>> ------------
>>>>>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>>>>>> (default-cli) on project word-count-beam: An exception occured while
>>>>>> executing the Java class. null: InvocationTargetException:
>>>>>> java.lang.IllegalStateException: Unable to find registrar for c ->
>>>>>> [Help 1]
>>>>>> [ERROR]
>>>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>>>>> the -e switch.
>>>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>>>>> [ERROR]
>>>>>> [ERROR] For more information about the errors and possible solutions,
>>>>>> please read the following articles:
>>>>>> [ERROR] [Help 1] http://cwiki.apache.org/con
>>>>>> fluence/display/MAVEN/MojoExecutionException
>>>>>>
>>>>>> Pom.xml is as below
>>>>>>
>>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>>> <!--
>>>>>> Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> contributor license agreements. See the NOTICE file distributed with
>>>>>> this work for additional information regarding copyright ownership.
>>>>>> The ASF licenses this file to You under the Apache License, Version
>>>>>> 2.0
>>>>>> (the "License"); you may not use this file except in compliance with
>>>>>> the License. You may obtain a copy of the License at
>>>>>>
>>>>>> http://www.apache.org/licenses/LICENSE-2.0
>>>>>>
>>>>>> Unless required by applicable law or agreed to in writing, software
>>>>>> distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>>> implied.
>>>>>> See the License for the specific language governing permissions and
>>>>>> limitations under the License.
>>>>>> -->
>>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0"
>>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>>>>>> http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>>
>>>>>> <groupId>org.example</groupId>
>>>>>> <artifactId>word-count-beam</artifactId>
>>>>>> <version>0.1</version>
>>>>>>
>>>>>> <packaging>jar</packaging>
>>>>>>
>>>>>> <properties>
>>>>>> <beam.version>2.0.0</beam.version>
>>>>>> <surefire-plugin.version>2.20</surefire-plugin.version>
>>>>>> </properties>
>>>>>>
>>>>>> <repositories>
>>>>>> <repository>
>>>>>> <id>apache.snapshots</id>
>>>>>> <name>Apache Development Snapshot Repository</name>
>>>>>> <url>https://repository.apache.org/content/repositories/snapshots/</
>>>>>> url>
>>>>>> <releases>
>>>>>> <enabled>false</enabled>
>>>>>> </releases>
>>>>>> <snapshots>
>>>>>> <enabled>true</enabled>
>>>>>> </snapshots>
>>>>>> </repository>
>>>>>> </repositories>
>>>>>>
>>>>>> <build>
>>>>>> <plugins>
>>>>>> <plugin>
>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>> <artifactId>maven-compiler-plugin</artifactId>
>>>>>> <version>3.5.1</version>
>>>>>> <configuration>
>>>>>> <source>1.7</source>
>>>>>> <target>1.7</target>
>>>>>> </configuration>
>>>>>> </plugin>
>>>>>>
>>>>>> <plugin>
>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>> <artifactId>maven-surefire-plugin</artifactId>
>>>>>> <version>2.20</version>
>>>>>> <configuration>
>>>>>> <parallel>all</parallel>
>>>>>> <threadCount>4</threadCount>
>>>>>> <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>>>>> </configuration>
>>>>>> <dependencies>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.maven.surefire</groupId>
>>>>>> <artifactId>surefire-junit47</artifactId>
>>>>>> <version>2.20</version>
>>>>>> </dependency>
>>>>>> </dependencies>
>>>>>> </plugin>
>>>>>>
>>>>>> <!-- Ensure that the Maven jar plugin runs before the Maven
>>>>>> shade plugin by listing the plugin higher within the file. -->
>>>>>> <plugin>
>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>> <artifactId>maven-jar-plugin</artifactId>
>>>>>> </plugin>
>>>>>>
>>>>>> <!--
>>>>>> Configures `mvn package` to produce a bundled jar ("fat jar") for
>>>>>> runners
>>>>>> that require this for job submission to a cluster.
>>>>>> -->
>>>>>> <plugin>
>>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>>> <artifactId>maven-shade-plugin</artifactId>
>>>>>> <version>3.0.0</version>
>>>>>> <executions>
>>>>>> <execution>
>>>>>> <phase>package</phase>
>>>>>> <goals>
>>>>>> <goal>shade</goal>
>>>>>> </goals>
>>>>>> <configuration>
>>>>>> <filters>
>>>>>> <filter>
>>>>>> <artifact>*:*</artifact>
>>>>>> <excludes>
>>>>>> <exclude>META-INF/LICENSE</exclude>
>>>>>> <exclude>META-INF/*.SF</exclude>
>>>>>> <exclude>META-INF/*.DSA</exclude>
>>>>>> <exclude>META-INF/*.RSA</exclude>
>>>>>> </excludes>
>>>>>> </filter>
>>>>>> </filters>
>>>>>> <transformers>
>>>>>> <transformer implementation="org.apache.mav
>>>>>> en.plugins.shade.resource.ServicesResourceTransformer"/>
>>>>>> </transformers>
>>>>>> </configuration>
>>>>>> </execution>
>>>>>> </executions>
>>>>>> </plugin>
>>>>>> </plugins>
>>>>>>
>>>>>> <pluginManagement>
>>>>>> <plugins>
>>>>>> <plugin>
>>>>>> <groupId>org.codehaus.mojo</groupId>
>>>>>> <artifactId>exec-maven-plugin</artifactId>
>>>>>> <version>1.4.0</version>
>>>>>> <configuration>
>>>>>> <cleanupDaemonThreads>false</cleanupDaemonThreads>
>>>>>> </configuration>
>>>>>> </plugin>
>>>>>> </plugins>
>>>>>> </pluginManagement>
>>>>>> </build>
>>>>>>
>>>>>> <profiles>
>>>>>> <profile>
>>>>>> <id>direct-runner</id>
>>>>>> <activation>
>>>>>> <activeByDefault>true</activeByDefault>
>>>>>> </activation>
>>>>>> <!-- Makes the DirectRunner available when running a pipeline. -->
>>>>>> <dependencies>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>>>> <version>2.0.0</version>
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>> </dependencies>
>>>>>> </profile>
>>>>>>
>>>>>> <profile>
>>>>>> <id>apex-runner</id>
>>>>>> <!-- Makes the ApexRunner available when running a pipeline. -->
>>>>>> <dependencies>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-runners-apex</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>> <!--
>>>>>> Apex depends on httpclient version 4.3.5, project has a transitive
>>>>>> dependency to httpclient 4.0.1 from
>>>>>> google-http-client. Apex dependency version being specified
>>>>>> explicitly so that it gets picked up. This
>>>>>> can be removed when the project no longer has a dependency on a
>>>>>> different httpclient version.
>>>>>> -->
>>>>>> <dependency>
>>>>>> <groupId>org.apache.httpcomponents</groupId>
>>>>>> <artifactId>httpclient</artifactId>
>>>>>> <version>4.3.5</version>
>>>>>> <scope>runtime</scope>
>>>>>> <exclusions>
>>>>>> <exclusion>
>>>>>> <groupId>commons-codec</groupId>
>>>>>> <artifactId>commons-codec</artifactId>
>>>>>> </exclusion>
>>>>>> </exclusions>
>>>>>> </dependency>
>>>>>> </dependencies>
>>>>>> </profile>
>>>>>>
>>>>>> <profile>
>>>>>> <id>dataflow-runner</id>
>>>>>> <!-- Makes the DataflowRunner available when running a pipeline. -->
>>>>>> <dependencies>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>> </dependencies>
>>>>>> </profile>
>>>>>>
>>>>>> <profile>
>>>>>> <id>flink-runner</id>
>>>>>> <!-- Makes the FlinkRunner available when running a pipeline. -->
>>>>>> <dependencies>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-runners-flink_2.10</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>> </dependencies>
>>>>>> </profile>
>>>>>>
>>>>>> <profile>
>>>>>> <id>spark-runner</id>
>>>>>> <!-- Makes the SparkRunner available when running a pipeline.
>>>>>> Additionally,
>>>>>> overrides some Spark dependencies to Beam-compatible versions. -->
>>>>>> <dependencies>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-runners-spark</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>> <dependency>
>>>>>> <groupId>org.apache.spark</groupId>
>>>>>> <artifactId>spark-streaming_2.10</artifactId>
>>>>>> <version>1.6.2</version>
>>>>>> <scope>runtime</scope>
>>>>>> <exclusions>
>>>>>> <exclusion>
>>>>>> <groupId>org.slf4j</groupId>
>>>>>> <artifactId>jul-to-slf4j</artifactId>
>>>>>> </exclusion>
>>>>>> </exclusions>
>>>>>> </dependency>
>>>>>> <dependency>
>>>>>> <groupId>com.fasterxml.jackson.module</groupId>
>>>>>> <artifactId>jackson-module-scala_2.10</artifactId>
>>>>>> <version>2.8.8</version>
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>> </dependencies>
>>>>>> </profile>
>>>>>> </profiles>
>>>>>>
>>>>>> <dependencies>
>>>>>> <!-- Adds a dependency on the Beam SDK. -->
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-sdks-java-core</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> </dependency>
>>>>>>
>>>>>> <!-- Adds a dependency on the Beam Google Cloud Platform IO module.
>>>>>> -->
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> </dependency>
>>>>>>
>>>>>> <!-- Dependencies below this line are specific dependencies needed by
>>>>>> the examples code. -->
>>>>>> <dependency>
>>>>>> <groupId>com.google.api-client</groupId>
>>>>>> <artifactId>google-api-client</artifactId>
>>>>>> <version>1.22.0</version>
>>>>>> <exclusions>
>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>> in by a transitive dependency of google-api-client -->
>>>>>> <exclusion>
>>>>>> <groupId>com.google.guava</groupId>
>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>> </exclusion>
>>>>>> </exclusions>
>>>>>> </dependency>
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>com.google.apis</groupId>
>>>>>> <artifactId>google-api-services-bigquery</artifactId>
>>>>>> <version>v2-rev295-1.22.0</version>
>>>>>> <exclusions>
>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>> in by a transitive dependency of google-api-client -->
>>>>>> <exclusion>
>>>>>> <groupId>com.google.guava</groupId>
>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>> </exclusion>
>>>>>> </exclusions>
>>>>>> </dependency>
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>com.google.http-client</groupId>
>>>>>> <artifactId>google-http-client</artifactId>
>>>>>> <version>1.22.0</version>
>>>>>> <exclusions>
>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>> in by a transitive dependency of google-api-client -->
>>>>>> <exclusion>
>>>>>> <groupId>com.google.guava</groupId>
>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>> </exclusion>
>>>>>> </exclusions>
>>>>>> </dependency>
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>com.google.apis</groupId>
>>>>>> <artifactId>google-api-services-pubsub</artifactId>
>>>>>> <version>v1-rev10-1.22.0</version>
>>>>>> <exclusions>
>>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>>> in by a transitive dependency of google-api-client -->
>>>>>> <exclusion>
>>>>>> <groupId>com.google.guava</groupId>
>>>>>> <artifactId>guava-jdk5</artifactId>
>>>>>> </exclusion>
>>>>>> </exclusions>
>>>>>> </dependency>
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>joda-time</groupId>
>>>>>> <artifactId>joda-time</artifactId>
>>>>>> <version>2.4</version>
>>>>>> </dependency>
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>com.google.guava</groupId>
>>>>>> <artifactId>guava</artifactId>
>>>>>> <version>20.0</version>
>>>>>> </dependency>
>>>>>>
>>>>>> <!-- Add slf4j API frontend binding with JUL backend -->
>>>>>> <dependency>
>>>>>> <groupId>org.slf4j</groupId>
>>>>>> <artifactId>slf4j-api</artifactId>
>>>>>> <version>1.7.14</version>
>>>>>> </dependency>
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>org.slf4j</groupId>
>>>>>> <artifactId>slf4j-jdk14</artifactId>
>>>>>> <version>1.7.14</version>
>>>>>> <!-- When loaded at runtime this will wire up slf4j to the JUL
>>>>>> backend -->
>>>>>> <scope>runtime</scope>
>>>>>> </dependency>
>>>>>>
>>>>>> <!-- Hamcrest and JUnit are required dependencies of PAssert,
>>>>>> which is used in the main code of DebuggingWordCount example. -->
>>>>>> <dependency>
>>>>>> <groupId>org.hamcrest</groupId>
>>>>>> <artifactId>hamcrest-all</artifactId>
>>>>>> <version>1.3</version>
>>>>>> </dependency>
>>>>>>
>>>>>> <dependency>
>>>>>> <groupId>junit</groupId>
>>>>>> <artifactId>junit</artifactId>
>>>>>> <version>4.12</version>
>>>>>> </dependency>
>>>>>>
>>>>>> <!-- The DirectRunner is needed for unit tests. -->
>>>>>> <dependency>
>>>>>> <groupId>org.apache.beam</groupId>
>>>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>>>> <version>${beam.version}</version>
>>>>>> <scope>test</scope>
>>>>>> </dependency>
>>>>>> </dependencies>
>>>>>> </project>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks in advance.
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>

Re: Getting "Unable to find registrar for c" error

Posted by Shuangyin Ge <go...@gmail.com>.

I used "mvn compile exec:java -P direct-runner -D
exec.mainClass=org.apache.beam.examples.WordCount -D
exec.args="--inputFile=C%3A/users/ges/workspace/word-count-beam/pom.xml
--output=counts"" however got a "NOT FOUND" error. It didn't create any
temp files yet.

[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
word-count-beam: An exception occured while executing the Java class. null:
InvocationTargetException: java.lang.IllegalArgumentException: Error
matching the pattern or glob
C%3A/users/ges/workspace/word-count-beam/pom.xml: status NOT_FOUND -> [Help
1]

I do believe without specifying the URI the compile works on the most parts
-

mvn compile exec:java -P direct-runner -D
exec.mainClass=org.apache.beam.examples.WordCount -D
exec.args="--inputFile=pom.xml --output=counts"

Failed with

[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
word-count-beam: An exception occured while executing the Java class. null:
InvocationTargetException: java.lang.IllegalStateException: Unable to find
registrar for c -> [Help 1]

Is there a way to know which "c" beams tried to read but failed to find
registrar? The reason I ask is because from what I can see, the compile and
execution definitely already passed the input stage. The content of the
file "pom.xml" had already been stored and counted in the temp files
generated. Is it possible that Beams was reading some other "c"s somewhere
else and reported this error?

Thanks,
Silas Ge



2017-05-23 22:41 GMT+08:00 Lukasz Cwik <lc...@google.com>:

> The paths are URI-like so "C:/path/to/file.txt" resolves to a URI with a
> scheme of "C", an authority of "path" and a path of "/to/file.txt" which is
> likely to never work since no filesystem handler will be bound to the
> scheme "C".
>
> Try "file:///C%3A/Users/ges/Workspace/word-count-beampom.xml"
> (% is the way to escape reserved characters in URIs and 3A is the hex for
> ":")
>
>
> On Tue, May 23, 2017 at 2:43 AM, Shuangyin Ge <go...@gmail.com> wrote:
>
>> When I simply used "C:/Users/ges/Workspace/word-count-beam/pom.xml" as
>> input file, I am pretty sure that it has gone farther than using URI
>> "file:///", because it has already generated the temp pipeline files and
>> wordcount. Just the compile failed with "Unable to find registrar for c".
>> To my gut this is a different issue than not able to locate the input file.
>>
>> Thanks,
>> Silas Ge
>>
>> 2017-05-23 17:31 GMT+08:00 Shuangyin Ge <go...@gmail.com>:
>>
>>> Hello Luke,
>>>
>>> I tried with "file:///C:/Users/ges/Workspace/word-count-beampom.xml" or
>>> "C:/Users/ges/Workspace/word-count-beampom.xml" as input file as seen
>>> in https://issues.apache.org/jira/browse/BEAM-2298, however both failed.
>>>
>>> "file:///C:/Users/ges/Workspace/word-count-beampom.xml" failed
>>> with Illegal char <:> at index 4
>>> "C:/Users/ges/Workspace/word-count-beampom.xml" failed with "Unable to
>>> find registrar for c"
>>>
>>> Any ideas?
>>>
>>> Thanks,
>>> Silas Ge
>>>
>>> 2017-05-22 23:28 GMT+08:00 Lukasz Cwik <lc...@google.com>:
>>>
>>>> This is a known problem with how the local file system is being
>>>> interacted with as is evident by the similar test failures when running the
>>>> Apache Beam unit tests on Windows:
>>>> https://issues.apache.org/jira/browse/BEAM-2299
>>>>
>>>> I would suggest trying to use "file:///c:/my/path/to/file.txt"
>>>>
>>>> There is a general proposal to change how we work with resource paths:
>>>> https://issues.apache.org/jira/browse/BEAM-2283
>>>>
>>>>
>>>>
>>>> On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <go...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hello team,
>>>>>
>>>>> Not sure if any can share some lights. I am getting "Unable to find
>>>>> registrar for c" error while running
>>>>>
>>>>> $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>>>>>      -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner
>>>>>
>>>>> following the Java Quickstart for Beam https://beam.apache.org/g
>>>>> et-started/quickstart-java/
>>>>>
>>>>> Complete output is shown below:
>>>>>
>>>>> [INFO] Scanning for projects...
>>>>> [WARNING]
>>>>> [WARNING] Some problems were encountered while building the effective
>>>>> model for org.example:word-count-beam:jar:0.1
>>>>> [WARNING] 'build.plugins.plugin.version' for
>>>>> org.apache.maven.plugins:maven-jar-plugin is missing. @ line 80,
>>>>> column 15
>>>>> [WARNING]
>>>>> [WARNING] It is highly recommended to fix these problems because they
>>>>> threaten the stability of your build.
>>>>> [WARNING]
>>>>> [WARNING] For this reason, future Maven versions might no longer
>>>>> support building such malformed projects.
>>>>> [WARNING]
>>>>> [INFO]
>>>>> [INFO] ------------------------------------------------------------
>>>>> ------------
>>>>> [INFO] Building word-count-beam 0.1
>>>>> [INFO] ------------------------------------------------------------
>>>>> ------------
>>>>> [INFO]
>>>>> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
>>>>> word-count-beam ---
>>>>> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered
>>>>> resources, i.e. build is platform dependent!
>>>>> [INFO] skip non existing resourceDirectory
>>>>> C:\Users\ges\Workspace\word-count-beam\src\main\resources
>>>>> [INFO]
>>>>> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
>>>>> word-count-beam ---
>>>>> [INFO] Changes detected - recompiling the module!
>>>>> [WARNING] File encoding has not been set, using platform encoding
>>>>> Cp1252, i.e. build is platform dependent!
>>>>> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co
>>>>> unt-beam\target\classes
>>>>> [INFO]
>>>>> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @
>>>>> word-count-beam ---
>>>>> [WARNING]
>>>>> java.lang.reflect.InvocationTargetException
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>>>> ssorImpl.java:62)
>>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>>>> thodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>> at java.lang.Thread.run(Thread.java:748)
>>>>> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
>>>>> java.lang.IllegalStateException: Unable to find registrar for c
>>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>>> sult.waitUntilFinish(DirectRunner.java:322)
>>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>>> sult.waitUntilFinish(DirectRunner.java:292)
>>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>>> .java:200)
>>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>>> .java:63)
>>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
>>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
>>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:184)
>>>>> ... 6 more
>>>>> Caused by: java.lang.IllegalStateException: Unable to find registrar
>>>>> for c
>>>>> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil
>>>>> eSystems.java:447)
>>>>> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
>>>>> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystem
>>>>> s.java:174)
>>>>> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
>>>>> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase
>>>>> dSink.java:905)
>>>>> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces
>>>>> sElement(WriteFiles.java:376)
>>>>> [INFO] ------------------------------------------------------------
>>>>> ------------
>>>>> [INFO] BUILD FAILURE
>>>>> [INFO] ------------------------------------------------------------
>>>>> ------------
>>>>> [INFO] Total time: 10.279 s
>>>>> [INFO] Finished at: 2017-05-22T08:56:16+08:00
>>>>> [INFO] Final Memory: 32M/450M
>>>>> [INFO] ------------------------------------------------------------
>>>>> ------------
>>>>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>>>>> (default-cli) on project word-count-beam: An exception occured while
>>>>> executing the Java class. null: InvocationTargetException:
>>>>> java.lang.IllegalStateException: Unable to find registrar for c ->
>>>>> [Help 1]
>>>>> [ERROR]
>>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>>>> the -e switch.
>>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>>>> [ERROR]
>>>>> [ERROR] For more information about the errors and possible solutions,
>>>>> please read the following articles:
>>>>> [ERROR] [Help 1] http://cwiki.apache.org/con
>>>>> fluence/display/MAVEN/MojoExecutionException
>>>>>
>>>>> Pom.xml is as below
>>>>>
>>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>>> <!--
>>>>> Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> contributor license agreements. See the NOTICE file distributed with
>>>>> this work for additional information regarding copyright ownership.
>>>>> The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> (the "License"); you may not use this file except in compliance with
>>>>> the License. You may obtain a copy of the License at
>>>>>
>>>>> http://www.apache.org/licenses/LICENSE-2.0
>>>>>
>>>>> Unless required by applicable law or agreed to in writing, software
>>>>> distributed under the License is distributed on an "AS IS" BASIS,
>>>>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
>>>>> implied.
>>>>> See the License for the specific language governing permissions and
>>>>> limitations under the License.
>>>>> -->
>>>>> <project xmlns="http://maven.apache.org/POM/4.0.0"
>>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>>>>> http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>>> <modelVersion>4.0.0</modelVersion>
>>>>>
>>>>> <groupId>org.example</groupId>
>>>>> <artifactId>word-count-beam</artifactId>
>>>>> <version>0.1</version>
>>>>>
>>>>> <packaging>jar</packaging>
>>>>>
>>>>> <properties>
>>>>> <beam.version>2.0.0</beam.version>
>>>>> <surefire-plugin.version>2.20</surefire-plugin.version>
>>>>> </properties>
>>>>>
>>>>> <repositories>
>>>>> <repository>
>>>>> <id>apache.snapshots</id>
>>>>> <name>Apache Development Snapshot Repository</name>
>>>>> <url>https://repository.apache.org/content/repositories/snapshots/</
>>>>> url>
>>>>> <releases>
>>>>> <enabled>false</enabled>
>>>>> </releases>
>>>>> <snapshots>
>>>>> <enabled>true</enabled>
>>>>> </snapshots>
>>>>> </repository>
>>>>> </repositories>
>>>>>
>>>>> <build>
>>>>> <plugins>
>>>>> <plugin>
>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>> <artifactId>maven-compiler-plugin</artifactId>
>>>>> <version>3.5.1</version>
>>>>> <configuration>
>>>>> <source>1.7</source>
>>>>> <target>1.7</target>
>>>>> </configuration>
>>>>> </plugin>
>>>>>
>>>>> <plugin>
>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>> <artifactId>maven-surefire-plugin</artifactId>
>>>>> <version>2.20</version>
>>>>> <configuration>
>>>>> <parallel>all</parallel>
>>>>> <threadCount>4</threadCount>
>>>>> <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>>>> </configuration>
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.maven.surefire</groupId>
>>>>> <artifactId>surefire-junit47</artifactId>
>>>>> <version>2.20</version>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </plugin>
>>>>>
>>>>> <!-- Ensure that the Maven jar plugin runs before the Maven
>>>>> shade plugin by listing the plugin higher within the file. -->
>>>>> <plugin>
>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>> <artifactId>maven-jar-plugin</artifactId>
>>>>> </plugin>
>>>>>
>>>>> <!--
>>>>> Configures `mvn package` to produce a bundled jar ("fat jar") for
>>>>> runners
>>>>> that require this for job submission to a cluster.
>>>>> -->
>>>>> <plugin>
>>>>> <groupId>org.apache.maven.plugins</groupId>
>>>>> <artifactId>maven-shade-plugin</artifactId>
>>>>> <version>3.0.0</version>
>>>>> <executions>
>>>>> <execution>
>>>>> <phase>package</phase>
>>>>> <goals>
>>>>> <goal>shade</goal>
>>>>> </goals>
>>>>> <configuration>
>>>>> <filters>
>>>>> <filter>
>>>>> <artifact>*:*</artifact>
>>>>> <excludes>
>>>>> <exclude>META-INF/LICENSE</exclude>
>>>>> <exclude>META-INF/*.SF</exclude>
>>>>> <exclude>META-INF/*.DSA</exclude>
>>>>> <exclude>META-INF/*.RSA</exclude>
>>>>> </excludes>
>>>>> </filter>
>>>>> </filters>
>>>>> <transformers>
>>>>> <transformer implementation="org.apache.mav
>>>>> en.plugins.shade.resource.ServicesResourceTransformer"/>
>>>>> </transformers>
>>>>> </configuration>
>>>>> </execution>
>>>>> </executions>
>>>>> </plugin>
>>>>> </plugins>
>>>>>
>>>>> <pluginManagement>
>>>>> <plugins>
>>>>> <plugin>
>>>>> <groupId>org.codehaus.mojo</groupId>
>>>>> <artifactId>exec-maven-plugin</artifactId>
>>>>> <version>1.4.0</version>
>>>>> <configuration>
>>>>> <cleanupDaemonThreads>false</cleanupDaemonThreads>
>>>>> </configuration>
>>>>> </plugin>
>>>>> </plugins>
>>>>> </pluginManagement>
>>>>> </build>
>>>>>
>>>>> <profiles>
>>>>> <profile>
>>>>> <id>direct-runner</id>
>>>>> <activation>
>>>>> <activeByDefault>true</activeByDefault>
>>>>> </activation>
>>>>> <!-- Makes the DirectRunner available when running a pipeline. -->
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>>> <version>2.0.0</version>
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </profile>
>>>>>
>>>>> <profile>
>>>>> <id>apex-runner</id>
>>>>> <!-- Makes the ApexRunner available when running a pipeline. -->
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-runners-apex</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>> <!--
>>>>> Apex depends on httpclient version 4.3.5, project has a transitive
>>>>> dependency to httpclient 4.0.1 from
>>>>> google-http-client. Apex dependency version being specified explicitly
>>>>> so that it gets picked up. This
>>>>> can be removed when the project no longer has a dependency on a
>>>>> different httpclient version.
>>>>> -->
>>>>> <dependency>
>>>>> <groupId>org.apache.httpcomponents</groupId>
>>>>> <artifactId>httpclient</artifactId>
>>>>> <version>4.3.5</version>
>>>>> <scope>runtime</scope>
>>>>> <exclusions>
>>>>> <exclusion>
>>>>> <groupId>commons-codec</groupId>
>>>>> <artifactId>commons-codec</artifactId>
>>>>> </exclusion>
>>>>> </exclusions>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </profile>
>>>>>
>>>>> <profile>
>>>>> <id>dataflow-runner</id>
>>>>> <!-- Makes the DataflowRunner available when running a pipeline. -->
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </profile>
>>>>>
>>>>> <profile>
>>>>> <id>flink-runner</id>
>>>>> <!-- Makes the FlinkRunner available when running a pipeline. -->
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-runners-flink_2.10</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </profile>
>>>>>
>>>>> <profile>
>>>>> <id>spark-runner</id>
>>>>> <!-- Makes the SparkRunner available when running a pipeline.
>>>>> Additionally,
>>>>> overrides some Spark dependencies to Beam-compatible versions. -->
>>>>> <dependencies>
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-runners-spark</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>org.apache.spark</groupId>
>>>>> <artifactId>spark-streaming_2.10</artifactId>
>>>>> <version>1.6.2</version>
>>>>> <scope>runtime</scope>
>>>>> <exclusions>
>>>>> <exclusion>
>>>>> <groupId>org.slf4j</groupId>
>>>>> <artifactId>jul-to-slf4j</artifactId>
>>>>> </exclusion>
>>>>> </exclusions>
>>>>> </dependency>
>>>>> <dependency>
>>>>> <groupId>com.fasterxml.jackson.module</groupId>
>>>>> <artifactId>jackson-module-scala_2.10</artifactId>
>>>>> <version>2.8.8</version>
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </profile>
>>>>> </profiles>
>>>>>
>>>>> <dependencies>
>>>>> <!-- Adds a dependency on the Beam SDK. -->
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-sdks-java-core</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> </dependency>
>>>>>
>>>>> <!-- Adds a dependency on the Beam Google Cloud Platform IO module. -->
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> </dependency>
>>>>>
>>>>> <!-- Dependencies below this line are specific dependencies needed by
>>>>> the examples code. -->
>>>>> <dependency>
>>>>> <groupId>com.google.api-client</groupId>
>>>>> <artifactId>google-api-client</artifactId>
>>>>> <version>1.22.0</version>
>>>>> <exclusions>
>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>> in by a transitive dependency of google-api-client -->
>>>>> <exclusion>
>>>>> <groupId>com.google.guava</groupId>
>>>>> <artifactId>guava-jdk5</artifactId>
>>>>> </exclusion>
>>>>> </exclusions>
>>>>> </dependency>
>>>>>
>>>>> <dependency>
>>>>> <groupId>com.google.apis</groupId>
>>>>> <artifactId>google-api-services-bigquery</artifactId>
>>>>> <version>v2-rev295-1.22.0</version>
>>>>> <exclusions>
>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>> in by a transitive dependency of google-api-client -->
>>>>> <exclusion>
>>>>> <groupId>com.google.guava</groupId>
>>>>> <artifactId>guava-jdk5</artifactId>
>>>>> </exclusion>
>>>>> </exclusions>
>>>>> </dependency>
>>>>>
>>>>> <dependency>
>>>>> <groupId>com.google.http-client</groupId>
>>>>> <artifactId>google-http-client</artifactId>
>>>>> <version>1.22.0</version>
>>>>> <exclusions>
>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>> in by a transitive dependency of google-api-client -->
>>>>> <exclusion>
>>>>> <groupId>com.google.guava</groupId>
>>>>> <artifactId>guava-jdk5</artifactId>
>>>>> </exclusion>
>>>>> </exclusions>
>>>>> </dependency>
>>>>>
>>>>> <dependency>
>>>>> <groupId>com.google.apis</groupId>
>>>>> <artifactId>google-api-services-pubsub</artifactId>
>>>>> <version>v1-rev10-1.22.0</version>
>>>>> <exclusions>
>>>>> <!-- Exclude an old version of guava that is being pulled
>>>>> in by a transitive dependency of google-api-client -->
>>>>> <exclusion>
>>>>> <groupId>com.google.guava</groupId>
>>>>> <artifactId>guava-jdk5</artifactId>
>>>>> </exclusion>
>>>>> </exclusions>
>>>>> </dependency>
>>>>>
>>>>> <dependency>
>>>>> <groupId>joda-time</groupId>
>>>>> <artifactId>joda-time</artifactId>
>>>>> <version>2.4</version>
>>>>> </dependency>
>>>>>
>>>>> <dependency>
>>>>> <groupId>com.google.guava</groupId>
>>>>> <artifactId>guava</artifactId>
>>>>> <version>20.0</version>
>>>>> </dependency>
>>>>>
>>>>> <!-- Add slf4j API frontend binding with JUL backend -->
>>>>> <dependency>
>>>>> <groupId>org.slf4j</groupId>
>>>>> <artifactId>slf4j-api</artifactId>
>>>>> <version>1.7.14</version>
>>>>> </dependency>
>>>>>
>>>>> <dependency>
>>>>> <groupId>org.slf4j</groupId>
>>>>> <artifactId>slf4j-jdk14</artifactId>
>>>>> <version>1.7.14</version>
>>>>> <!-- When loaded at runtime this will wire up slf4j to the JUL backend
>>>>> -->
>>>>> <scope>runtime</scope>
>>>>> </dependency>
>>>>>
>>>>> <!-- Hamcrest and JUnit are required dependencies of PAssert,
>>>>> which is used in the main code of DebuggingWordCount example. -->
>>>>> <dependency>
>>>>> <groupId>org.hamcrest</groupId>
>>>>> <artifactId>hamcrest-all</artifactId>
>>>>> <version>1.3</version>
>>>>> </dependency>
>>>>>
>>>>> <dependency>
>>>>> <groupId>junit</groupId>
>>>>> <artifactId>junit</artifactId>
>>>>> <version>4.12</version>
>>>>> </dependency>
>>>>>
>>>>> <!-- The DirectRunner is needed for unit tests. -->
>>>>> <dependency>
>>>>> <groupId>org.apache.beam</groupId>
>>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>>> <version>${beam.version}</version>
>>>>> <scope>test</scope>
>>>>> </dependency>
>>>>> </dependencies>
>>>>> </project>
>>>>>
>>>>>
>>>>>
>>>>> Thanks in advance.
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Getting "Unable to find registrar for c" error

Posted by Lukasz Cwik <lc...@google.com>.

The paths are URI-like so "C:/path/to/file.txt" resolves to a URI with a
scheme of "C", an authority of "path" and a path of "/to/file.txt" which is
likely to never work since no filesystem handler will be bound to the
scheme "C".

Try "file:///C%3A/Users/ges/Workspace/word-count-beampom.xml"
(% is the way to escape reserved characters in URIs and 3A is the hex for
":")


On Tue, May 23, 2017 at 2:43 AM, Shuangyin Ge <go...@gmail.com> wrote:

> When I simply used "C:/Users/ges/Workspace/word-count-beam/pom.xml" as
> input file, I am pretty sure that it has gone farther than using URI
> "file:///", because it has already generated the temp pipeline files and
> wordcount. Just the compile failed with "Unable to find registrar for c".
> To my gut this is a different issue than not able to locate the input file.
>
> Thanks,
> Silas Ge
>
> 2017-05-23 17:31 GMT+08:00 Shuangyin Ge <go...@gmail.com>:
>
>> Hello Luke,
>>
>> I tried with "file:///C:/Users/ges/Workspace/word-count-beampom.xml" or
>> "C:/Users/ges/Workspace/word-count-beampom.xml" as input file as seen in
>> https://issues.apache.org/jira/browse/BEAM-2298, however both failed.
>>
>> "file:///C:/Users/ges/Workspace/word-count-beampom.xml" failed
>> with Illegal char <:> at index 4
>> "C:/Users/ges/Workspace/word-count-beampom.xml" failed with "Unable to
>> find registrar for c"
>>
>> Any ideas?
>>
>> Thanks,
>> Silas Ge
>>
>> 2017-05-22 23:28 GMT+08:00 Lukasz Cwik <lc...@google.com>:
>>
>>> This is a known problem with how the local file system is being
>>> interacted with as is evident by the similar test failures when running the
>>> Apache Beam unit tests on Windows:
>>> https://issues.apache.org/jira/browse/BEAM-2299
>>>
>>> I would suggest trying to use "file:///c:/my/path/to/file.txt"
>>>
>>> There is a general proposal to change how we work with resource paths:
>>> https://issues.apache.org/jira/browse/BEAM-2283
>>>
>>>
>>>
>>> On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <go...@gmail.com>
>>> wrote:
>>>
>>>> Hello team,
>>>>
>>>> Not sure if any can share some lights. I am getting "Unable to find
>>>> registrar for c" error while running
>>>>
>>>> $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>>>>      -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner
>>>>
>>>> following the Java Quickstart for Beam https://beam.apache.org/g
>>>> et-started/quickstart-java/
>>>>
>>>> Complete output is shown below:
>>>>
>>>> [INFO] Scanning for projects...
>>>> [WARNING]
>>>> [WARNING] Some problems were encountered while building the effective
>>>> model for org.example:word-count-beam:jar:0.1
>>>> [WARNING] 'build.plugins.plugin.version' for
>>>> org.apache.maven.plugins:maven-jar-plugin is missing. @ line 80,
>>>> column 15
>>>> [WARNING]
>>>> [WARNING] It is highly recommended to fix these problems because they
>>>> threaten the stability of your build.
>>>> [WARNING]
>>>> [WARNING] For this reason, future Maven versions might no longer
>>>> support building such malformed projects.
>>>> [WARNING]
>>>> [INFO]
>>>> [INFO] ------------------------------------------------------------
>>>> ------------
>>>> [INFO] Building word-count-beam 0.1
>>>> [INFO] ------------------------------------------------------------
>>>> ------------
>>>> [INFO]
>>>> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
>>>> word-count-beam ---
>>>> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered
>>>> resources, i.e. build is platform dependent!
>>>> [INFO] skip non existing resourceDirectory
>>>> C:\Users\ges\Workspace\word-count-beam\src\main\resources
>>>> [INFO]
>>>> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
>>>> word-count-beam ---
>>>> [INFO] Changes detected - recompiling the module!
>>>> [WARNING] File encoding has not been set, using platform encoding
>>>> Cp1252, i.e. build is platform dependent!
>>>> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co
>>>> unt-beam\target\classes
>>>> [INFO]
>>>> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam
>>>> ---
>>>> [WARNING]
>>>> java.lang.reflect.InvocationTargetException
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>>> ssorImpl.java:62)
>>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>>> thodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>> at java.lang.Thread.run(Thread.java:748)
>>>> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
>>>> java.lang.IllegalStateException: Unable to find registrar for c
>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>> sult.waitUntilFinish(DirectRunner.java:322)
>>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>>> sult.waitUntilFinish(DirectRunner.java:292)
>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>> .java:200)
>>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>>> .java:63)
>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
>>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
>>>> at org.apache.beam.examples.WordCount.main(WordCount.java:184)
>>>> ... 6 more
>>>> Caused by: java.lang.IllegalStateException: Unable to find registrar
>>>> for c
>>>> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil
>>>> eSystems.java:447)
>>>> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
>>>> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystem
>>>> s.java:174)
>>>> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
>>>> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase
>>>> dSink.java:905)
>>>> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces
>>>> sElement(WriteFiles.java:376)
>>>> [INFO] ------------------------------------------------------------
>>>> ------------
>>>> [INFO] BUILD FAILURE
>>>> [INFO] ------------------------------------------------------------
>>>> ------------
>>>> [INFO] Total time: 10.279 s
>>>> [INFO] Finished at: 2017-05-22T08:56:16+08:00
>>>> [INFO] Final Memory: 32M/450M
>>>> [INFO] ------------------------------------------------------------
>>>> ------------
>>>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>>>> (default-cli) on project word-count-beam: An exception occured while
>>>> executing the Java class. null: InvocationTargetException:
>>>> java.lang.IllegalStateException: Unable to find registrar for c ->
>>>> [Help 1]
>>>> [ERROR]
>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>>>> the -e switch.
>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>>> [ERROR]
>>>> [ERROR] For more information about the errors and possible solutions,
>>>> please read the following articles:
>>>> [ERROR] [Help 1] http://cwiki.apache.org/con
>>>> fluence/display/MAVEN/MojoExecutionException
>>>>
>>>> Pom.xml is as below
>>>>
>>>> <?xml version="1.0" encoding="UTF-8"?>
>>>> <!--
>>>> Licensed to the Apache Software Foundation (ASF) under one or more
>>>> contributor license agreements. See the NOTICE file distributed with
>>>> this work for additional information regarding copyright ownership.
>>>> The ASF licenses this file to You under the Apache License, Version 2.0
>>>> (the "License"); you may not use this file except in compliance with
>>>> the License. You may obtain a copy of the License at
>>>>
>>>> http://www.apache.org/licenses/LICENSE-2.0
>>>>
>>>> Unless required by applicable law or agreed to in writing, software
>>>> distributed under the License is distributed on an "AS IS" BASIS,
>>>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> See the License for the specific language governing permissions and
>>>> limitations under the License.
>>>> -->
>>>> <project xmlns="http://maven.apache.org/POM/4.0.0"
>>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>>>> http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>>> <modelVersion>4.0.0</modelVersion>
>>>>
>>>> <groupId>org.example</groupId>
>>>> <artifactId>word-count-beam</artifactId>
>>>> <version>0.1</version>
>>>>
>>>> <packaging>jar</packaging>
>>>>
>>>> <properties>
>>>> <beam.version>2.0.0</beam.version>
>>>> <surefire-plugin.version>2.20</surefire-plugin.version>
>>>> </properties>
>>>>
>>>> <repositories>
>>>> <repository>
>>>> <id>apache.snapshots</id>
>>>> <name>Apache Development Snapshot Repository</name>
>>>> <url>https://repository.apache.org/content/repositories/snapshots/</url
>>>> >
>>>> <releases>
>>>> <enabled>false</enabled>
>>>> </releases>
>>>> <snapshots>
>>>> <enabled>true</enabled>
>>>> </snapshots>
>>>> </repository>
>>>> </repositories>
>>>>
>>>> <build>
>>>> <plugins>
>>>> <plugin>
>>>> <groupId>org.apache.maven.plugins</groupId>
>>>> <artifactId>maven-compiler-plugin</artifactId>
>>>> <version>3.5.1</version>
>>>> <configuration>
>>>> <source>1.7</source>
>>>> <target>1.7</target>
>>>> </configuration>
>>>> </plugin>
>>>>
>>>> <plugin>
>>>> <groupId>org.apache.maven.plugins</groupId>
>>>> <artifactId>maven-surefire-plugin</artifactId>
>>>> <version>2.20</version>
>>>> <configuration>
>>>> <parallel>all</parallel>
>>>> <threadCount>4</threadCount>
>>>> <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>>> </configuration>
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.maven.surefire</groupId>
>>>> <artifactId>surefire-junit47</artifactId>
>>>> <version>2.20</version>
>>>> </dependency>
>>>> </dependencies>
>>>> </plugin>
>>>>
>>>> <!-- Ensure that the Maven jar plugin runs before the Maven
>>>> shade plugin by listing the plugin higher within the file. -->
>>>> <plugin>
>>>> <groupId>org.apache.maven.plugins</groupId>
>>>> <artifactId>maven-jar-plugin</artifactId>
>>>> </plugin>
>>>>
>>>> <!--
>>>> Configures `mvn package` to produce a bundled jar ("fat jar") for
>>>> runners
>>>> that require this for job submission to a cluster.
>>>> -->
>>>> <plugin>
>>>> <groupId>org.apache.maven.plugins</groupId>
>>>> <artifactId>maven-shade-plugin</artifactId>
>>>> <version>3.0.0</version>
>>>> <executions>
>>>> <execution>
>>>> <phase>package</phase>
>>>> <goals>
>>>> <goal>shade</goal>
>>>> </goals>
>>>> <configuration>
>>>> <filters>
>>>> <filter>
>>>> <artifact>*:*</artifact>
>>>> <excludes>
>>>> <exclude>META-INF/LICENSE</exclude>
>>>> <exclude>META-INF/*.SF</exclude>
>>>> <exclude>META-INF/*.DSA</exclude>
>>>> <exclude>META-INF/*.RSA</exclude>
>>>> </excludes>
>>>> </filter>
>>>> </filters>
>>>> <transformers>
>>>> <transformer implementation="org.apache.mav
>>>> en.plugins.shade.resource.ServicesResourceTransformer"/>
>>>> </transformers>
>>>> </configuration>
>>>> </execution>
>>>> </executions>
>>>> </plugin>
>>>> </plugins>
>>>>
>>>> <pluginManagement>
>>>> <plugins>
>>>> <plugin>
>>>> <groupId>org.codehaus.mojo</groupId>
>>>> <artifactId>exec-maven-plugin</artifactId>
>>>> <version>1.4.0</version>
>>>> <configuration>
>>>> <cleanupDaemonThreads>false</cleanupDaemonThreads>
>>>> </configuration>
>>>> </plugin>
>>>> </plugins>
>>>> </pluginManagement>
>>>> </build>
>>>>
>>>> <profiles>
>>>> <profile>
>>>> <id>direct-runner</id>
>>>> <activation>
>>>> <activeByDefault>true</activeByDefault>
>>>> </activation>
>>>> <!-- Makes the DirectRunner available when running a pipeline. -->
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>> <version>2.0.0</version>
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>> </dependencies>
>>>> </profile>
>>>>
>>>> <profile>
>>>> <id>apex-runner</id>
>>>> <!-- Makes the ApexRunner available when running a pipeline. -->
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-runners-apex</artifactId>
>>>> <version>${beam.version}</version>
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>> <!--
>>>> Apex depends on httpclient version 4.3.5, project has a transitive
>>>> dependency to httpclient 4.0.1 from
>>>> google-http-client. Apex dependency version being specified explicitly
>>>> so that it gets picked up. This
>>>> can be removed when the project no longer has a dependency on a
>>>> different httpclient version.
>>>> -->
>>>> <dependency>
>>>> <groupId>org.apache.httpcomponents</groupId>
>>>> <artifactId>httpclient</artifactId>
>>>> <version>4.3.5</version>
>>>> <scope>runtime</scope>
>>>> <exclusions>
>>>> <exclusion>
>>>> <groupId>commons-codec</groupId>
>>>> <artifactId>commons-codec</artifactId>
>>>> </exclusion>
>>>> </exclusions>
>>>> </dependency>
>>>> </dependencies>
>>>> </profile>
>>>>
>>>> <profile>
>>>> <id>dataflow-runner</id>
>>>> <!-- Makes the DataflowRunner available when running a pipeline. -->
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
>>>> <version>${beam.version}</version>
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>> </dependencies>
>>>> </profile>
>>>>
>>>> <profile>
>>>> <id>flink-runner</id>
>>>> <!-- Makes the FlinkRunner available when running a pipeline. -->
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-runners-flink_2.10</artifactId>
>>>> <version>${beam.version}</version>
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>> </dependencies>
>>>> </profile>
>>>>
>>>> <profile>
>>>> <id>spark-runner</id>
>>>> <!-- Makes the SparkRunner available when running a pipeline.
>>>> Additionally,
>>>> overrides some Spark dependencies to Beam-compatible versions. -->
>>>> <dependencies>
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-runners-spark</artifactId>
>>>> <version>${beam.version}</version>
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
>>>> <version>${beam.version}</version>
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>org.apache.spark</groupId>
>>>> <artifactId>spark-streaming_2.10</artifactId>
>>>> <version>1.6.2</version>
>>>> <scope>runtime</scope>
>>>> <exclusions>
>>>> <exclusion>
>>>> <groupId>org.slf4j</groupId>
>>>> <artifactId>jul-to-slf4j</artifactId>
>>>> </exclusion>
>>>> </exclusions>
>>>> </dependency>
>>>> <dependency>
>>>> <groupId>com.fasterxml.jackson.module</groupId>
>>>> <artifactId>jackson-module-scala_2.10</artifactId>
>>>> <version>2.8.8</version>
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>> </dependencies>
>>>> </profile>
>>>> </profiles>
>>>>
>>>> <dependencies>
>>>> <!-- Adds a dependency on the Beam SDK. -->
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-sdks-java-core</artifactId>
>>>> <version>${beam.version}</version>
>>>> </dependency>
>>>>
>>>> <!-- Adds a dependency on the Beam Google Cloud Platform IO module. -->
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
>>>> <version>${beam.version}</version>
>>>> </dependency>
>>>>
>>>> <!-- Dependencies below this line are specific dependencies needed by
>>>> the examples code. -->
>>>> <dependency>
>>>> <groupId>com.google.api-client</groupId>
>>>> <artifactId>google-api-client</artifactId>
>>>> <version>1.22.0</version>
>>>> <exclusions>
>>>> <!-- Exclude an old version of guava that is being pulled
>>>> in by a transitive dependency of google-api-client -->
>>>> <exclusion>
>>>> <groupId>com.google.guava</groupId>
>>>> <artifactId>guava-jdk5</artifactId>
>>>> </exclusion>
>>>> </exclusions>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>> <groupId>com.google.apis</groupId>
>>>> <artifactId>google-api-services-bigquery</artifactId>
>>>> <version>v2-rev295-1.22.0</version>
>>>> <exclusions>
>>>> <!-- Exclude an old version of guava that is being pulled
>>>> in by a transitive dependency of google-api-client -->
>>>> <exclusion>
>>>> <groupId>com.google.guava</groupId>
>>>> <artifactId>guava-jdk5</artifactId>
>>>> </exclusion>
>>>> </exclusions>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>> <groupId>com.google.http-client</groupId>
>>>> <artifactId>google-http-client</artifactId>
>>>> <version>1.22.0</version>
>>>> <exclusions>
>>>> <!-- Exclude an old version of guava that is being pulled
>>>> in by a transitive dependency of google-api-client -->
>>>> <exclusion>
>>>> <groupId>com.google.guava</groupId>
>>>> <artifactId>guava-jdk5</artifactId>
>>>> </exclusion>
>>>> </exclusions>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>> <groupId>com.google.apis</groupId>
>>>> <artifactId>google-api-services-pubsub</artifactId>
>>>> <version>v1-rev10-1.22.0</version>
>>>> <exclusions>
>>>> <!-- Exclude an old version of guava that is being pulled
>>>> in by a transitive dependency of google-api-client -->
>>>> <exclusion>
>>>> <groupId>com.google.guava</groupId>
>>>> <artifactId>guava-jdk5</artifactId>
>>>> </exclusion>
>>>> </exclusions>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>> <groupId>joda-time</groupId>
>>>> <artifactId>joda-time</artifactId>
>>>> <version>2.4</version>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>> <groupId>com.google.guava</groupId>
>>>> <artifactId>guava</artifactId>
>>>> <version>20.0</version>
>>>> </dependency>
>>>>
>>>> <!-- Add slf4j API frontend binding with JUL backend -->
>>>> <dependency>
>>>> <groupId>org.slf4j</groupId>
>>>> <artifactId>slf4j-api</artifactId>
>>>> <version>1.7.14</version>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>> <groupId>org.slf4j</groupId>
>>>> <artifactId>slf4j-jdk14</artifactId>
>>>> <version>1.7.14</version>
>>>> <!-- When loaded at runtime this will wire up slf4j to the JUL backend
>>>> -->
>>>> <scope>runtime</scope>
>>>> </dependency>
>>>>
>>>> <!-- Hamcrest and JUnit are required dependencies of PAssert,
>>>> which is used in the main code of DebuggingWordCount example. -->
>>>> <dependency>
>>>> <groupId>org.hamcrest</groupId>
>>>> <artifactId>hamcrest-all</artifactId>
>>>> <version>1.3</version>
>>>> </dependency>
>>>>
>>>> <dependency>
>>>> <groupId>junit</groupId>
>>>> <artifactId>junit</artifactId>
>>>> <version>4.12</version>
>>>> </dependency>
>>>>
>>>> <!-- The DirectRunner is needed for unit tests. -->
>>>> <dependency>
>>>> <groupId>org.apache.beam</groupId>
>>>> <artifactId>beam-runners-direct-java</artifactId>
>>>> <version>${beam.version}</version>
>>>> <scope>test</scope>
>>>> </dependency>
>>>> </dependencies>
>>>> </project>
>>>>
>>>>
>>>>
>>>> Thanks in advance.
>>>>
>>>
>>>
>>
>

Re: Getting "Unable to find registrar for c" error

Posted by Shuangyin Ge <go...@gmail.com>.

When I simply used "C:/Users/ges/Workspace/word-count-beam/pom.xml" as
input file, I am pretty sure that it has gone farther than using URI
"file:///", because it has already generated the temp pipeline files and
wordcount. Just the compile failed with "Unable to find registrar for c".
To my gut this is a different issue than not able to locate the input file.

Thanks,
Silas Ge

2017-05-23 17:31 GMT+08:00 Shuangyin Ge <go...@gmail.com>:

> Hello Luke,
>
> I tried with "file:///C:/Users/ges/Workspace/word-count-beampom.xml" or
> "C:/Users/ges/Workspace/word-count-beampom.xml" as input file as seen in
> https://issues.apache.org/jira/browse/BEAM-2298, however both failed.
>
> "file:///C:/Users/ges/Workspace/word-count-beampom.xml" failed
> with Illegal char <:> at index 4
> "C:/Users/ges/Workspace/word-count-beampom.xml" failed with "Unable to
> find registrar for c"
>
> Any ideas?
>
> Thanks,
> Silas Ge
>
> 2017-05-22 23:28 GMT+08:00 Lukasz Cwik <lc...@google.com>:
>
>> This is a known problem with how the local file system is being
>> interacted with as is evident by the similar test failures when running the
>> Apache Beam unit tests on Windows:
>> https://issues.apache.org/jira/browse/BEAM-2299
>>
>> I would suggest trying to use "file:///c:/my/path/to/file.txt"
>>
>> There is a general proposal to change how we work with resource paths:
>> https://issues.apache.org/jira/browse/BEAM-2283
>>
>>
>>
>> On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <go...@gmail.com> wrote:
>>
>>> Hello team,
>>>
>>> Not sure if any can share some lights. I am getting "Unable to find
>>> registrar for c" error while running
>>>
>>> $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>>>      -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner
>>>
>>> following the Java Quickstart for Beam https://beam.apache.org/g
>>> et-started/quickstart-java/
>>>
>>> Complete output is shown below:
>>>
>>> [INFO] Scanning for projects...
>>> [WARNING]
>>> [WARNING] Some problems were encountered while building the effective
>>> model for org.example:word-count-beam:jar:0.1
>>> [WARNING] 'build.plugins.plugin.version' for
>>> org.apache.maven.plugins:maven-jar-plugin is missing. @ line 80, column
>>> 15
>>> [WARNING]
>>> [WARNING] It is highly recommended to fix these problems because they
>>> threaten the stability of your build.
>>> [WARNING]
>>> [WARNING] For this reason, future Maven versions might no longer support
>>> building such malformed projects.
>>> [WARNING]
>>> [INFO]
>>> [INFO] ------------------------------------------------------------
>>> ------------
>>> [INFO] Building word-count-beam 0.1
>>> [INFO] ------------------------------------------------------------
>>> ------------
>>> [INFO]
>>> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
>>> word-count-beam ---
>>> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered
>>> resources, i.e. build is platform dependent!
>>> [INFO] skip non existing resourceDirectory C:\Users\ges\Workspace\word-co
>>> unt-beam\src\main\resources
>>> [INFO]
>>> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
>>> word-count-beam ---
>>> [INFO] Changes detected - recompiling the module!
>>> [WARNING] File encoding has not been set, using platform encoding
>>> Cp1252, i.e. build is platform dependent!
>>> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co
>>> unt-beam\target\classes
>>> [INFO]
>>> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam
>>> ---
>>> [WARNING]
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>>> ssorImpl.java:62)
>>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>>> thodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:498)
>>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>> at java.lang.Thread.run(Thread.java:748)
>>> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
>>> java.lang.IllegalStateException: Unable to find registrar for c
>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>> sult.waitUntilFinish(DirectRunner.java:322)
>>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>>> sult.waitUntilFinish(DirectRunner.java:292)
>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner
>>> .java:200)
>>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:63)
>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
>>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
>>> at org.apache.beam.examples.WordCount.main(WordCount.java:184)
>>> ... 6 more
>>> Caused by: java.lang.IllegalStateException: Unable to find registrar
>>> for c
>>> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil
>>> eSystems.java:447)
>>> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
>>> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystem
>>> s.java:174)
>>> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
>>> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase
>>> dSink.java:905)
>>> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces
>>> sElement(WriteFiles.java:376)
>>> [INFO] ------------------------------------------------------------
>>> ------------
>>> [INFO] BUILD FAILURE
>>> [INFO] ------------------------------------------------------------
>>> ------------
>>> [INFO] Total time: 10.279 s
>>> [INFO] Finished at: 2017-05-22T08:56:16+08:00
>>> [INFO] Final Memory: 32M/450M
>>> [INFO] ------------------------------------------------------------
>>> ------------
>>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>>> (default-cli) on project word-count-beam: An exception occured while
>>> executing the Java class. null: InvocationTargetException:
>>> java.lang.IllegalStateException: Unable to find registrar for c ->
>>> [Help 1]
>>> [ERROR]
>>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>>> -e switch.
>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>> [ERROR]
>>> [ERROR] For more information about the errors and possible solutions,
>>> please read the following articles:
>>> [ERROR] [Help 1] http://cwiki.apache.org/con
>>> fluence/display/MAVEN/MojoExecutionException
>>>
>>> Pom.xml is as below
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <!--
>>> Licensed to the Apache Software Foundation (ASF) under one or more
>>> contributor license agreements. See the NOTICE file distributed with
>>> this work for additional information regarding copyright ownership.
>>> The ASF licenses this file to You under the Apache License, Version 2.0
>>> (the "License"); you may not use this file except in compliance with
>>> the License. You may obtain a copy of the License at
>>>
>>> http://www.apache.org/licenses/LICENSE-2.0
>>>
>>> Unless required by applicable law or agreed to in writing, software
>>> distributed under the License is distributed on an "AS IS" BASIS,
>>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>> See the License for the specific language governing permissions and
>>> limitations under the License.
>>> -->
>>> <project xmlns="http://maven.apache.org/POM/4.0.0"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>>> http://maven.apache.org/xsd/maven-4.0.0.xsd">
>>> <modelVersion>4.0.0</modelVersion>
>>>
>>> <groupId>org.example</groupId>
>>> <artifactId>word-count-beam</artifactId>
>>> <version>0.1</version>
>>>
>>> <packaging>jar</packaging>
>>>
>>> <properties>
>>> <beam.version>2.0.0</beam.version>
>>> <surefire-plugin.version>2.20</surefire-plugin.version>
>>> </properties>
>>>
>>> <repositories>
>>> <repository>
>>> <id>apache.snapshots</id>
>>> <name>Apache Development Snapshot Repository</name>
>>> <url>https://repository.apache.org/content/repositories/snapshots/</url>
>>> <releases>
>>> <enabled>false</enabled>
>>> </releases>
>>> <snapshots>
>>> <enabled>true</enabled>
>>> </snapshots>
>>> </repository>
>>> </repositories>
>>>
>>> <build>
>>> <plugins>
>>> <plugin>
>>> <groupId>org.apache.maven.plugins</groupId>
>>> <artifactId>maven-compiler-plugin</artifactId>
>>> <version>3.5.1</version>
>>> <configuration>
>>> <source>1.7</source>
>>> <target>1.7</target>
>>> </configuration>
>>> </plugin>
>>>
>>> <plugin>
>>> <groupId>org.apache.maven.plugins</groupId>
>>> <artifactId>maven-surefire-plugin</artifactId>
>>> <version>2.20</version>
>>> <configuration>
>>> <parallel>all</parallel>
>>> <threadCount>4</threadCount>
>>> <redirectTestOutputToFile>true</redirectTestOutputToFile>
>>> </configuration>
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.maven.surefire</groupId>
>>> <artifactId>surefire-junit47</artifactId>
>>> <version>2.20</version>
>>> </dependency>
>>> </dependencies>
>>> </plugin>
>>>
>>> <!-- Ensure that the Maven jar plugin runs before the Maven
>>> shade plugin by listing the plugin higher within the file. -->
>>> <plugin>
>>> <groupId>org.apache.maven.plugins</groupId>
>>> <artifactId>maven-jar-plugin</artifactId>
>>> </plugin>
>>>
>>> <!--
>>> Configures `mvn package` to produce a bundled jar ("fat jar") for runners
>>> that require this for job submission to a cluster.
>>> -->
>>> <plugin>
>>> <groupId>org.apache.maven.plugins</groupId>
>>> <artifactId>maven-shade-plugin</artifactId>
>>> <version>3.0.0</version>
>>> <executions>
>>> <execution>
>>> <phase>package</phase>
>>> <goals>
>>> <goal>shade</goal>
>>> </goals>
>>> <configuration>
>>> <filters>
>>> <filter>
>>> <artifact>*:*</artifact>
>>> <excludes>
>>> <exclude>META-INF/LICENSE</exclude>
>>> <exclude>META-INF/*.SF</exclude>
>>> <exclude>META-INF/*.DSA</exclude>
>>> <exclude>META-INF/*.RSA</exclude>
>>> </excludes>
>>> </filter>
>>> </filters>
>>> <transformers>
>>> <transformer implementation="org.apache.mav
>>> en.plugins.shade.resource.ServicesResourceTransformer"/>
>>> </transformers>
>>> </configuration>
>>> </execution>
>>> </executions>
>>> </plugin>
>>> </plugins>
>>>
>>> <pluginManagement>
>>> <plugins>
>>> <plugin>
>>> <groupId>org.codehaus.mojo</groupId>
>>> <artifactId>exec-maven-plugin</artifactId>
>>> <version>1.4.0</version>
>>> <configuration>
>>> <cleanupDaemonThreads>false</cleanupDaemonThreads>
>>> </configuration>
>>> </plugin>
>>> </plugins>
>>> </pluginManagement>
>>> </build>
>>>
>>> <profiles>
>>> <profile>
>>> <id>direct-runner</id>
>>> <activation>
>>> <activeByDefault>true</activeByDefault>
>>> </activation>
>>> <!-- Makes the DirectRunner available when running a pipeline. -->
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-runners-direct-java</artifactId>
>>> <version>2.0.0</version>
>>> <scope>runtime</scope>
>>> </dependency>
>>> </dependencies>
>>> </profile>
>>>
>>> <profile>
>>> <id>apex-runner</id>
>>> <!-- Makes the ApexRunner available when running a pipeline. -->
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-runners-apex</artifactId>
>>> <version>${beam.version}</version>
>>> <scope>runtime</scope>
>>> </dependency>
>>> <!--
>>> Apex depends on httpclient version 4.3.5, project has a transitive
>>> dependency to httpclient 4.0.1 from
>>> google-http-client. Apex dependency version being specified explicitly
>>> so that it gets picked up. This
>>> can be removed when the project no longer has a dependency on a
>>> different httpclient version.
>>> -->
>>> <dependency>
>>> <groupId>org.apache.httpcomponents</groupId>
>>> <artifactId>httpclient</artifactId>
>>> <version>4.3.5</version>
>>> <scope>runtime</scope>
>>> <exclusions>
>>> <exclusion>
>>> <groupId>commons-codec</groupId>
>>> <artifactId>commons-codec</artifactId>
>>> </exclusion>
>>> </exclusions>
>>> </dependency>
>>> </dependencies>
>>> </profile>
>>>
>>> <profile>
>>> <id>dataflow-runner</id>
>>> <!-- Makes the DataflowRunner available when running a pipeline. -->
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
>>> <version>${beam.version}</version>
>>> <scope>runtime</scope>
>>> </dependency>
>>> </dependencies>
>>> </profile>
>>>
>>> <profile>
>>> <id>flink-runner</id>
>>> <!-- Makes the FlinkRunner available when running a pipeline. -->
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-runners-flink_2.10</artifactId>
>>> <version>${beam.version}</version>
>>> <scope>runtime</scope>
>>> </dependency>
>>> </dependencies>
>>> </profile>
>>>
>>> <profile>
>>> <id>spark-runner</id>
>>> <!-- Makes the SparkRunner available when running a pipeline.
>>> Additionally,
>>> overrides some Spark dependencies to Beam-compatible versions. -->
>>> <dependencies>
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-runners-spark</artifactId>
>>> <version>${beam.version}</version>
>>> <scope>runtime</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
>>> <version>${beam.version}</version>
>>> <scope>runtime</scope>
>>> </dependency>
>>> <dependency>
>>> <groupId>org.apache.spark</groupId>
>>> <artifactId>spark-streaming_2.10</artifactId>
>>> <version>1.6.2</version>
>>> <scope>runtime</scope>
>>> <exclusions>
>>> <exclusion>
>>> <groupId>org.slf4j</groupId>
>>> <artifactId>jul-to-slf4j</artifactId>
>>> </exclusion>
>>> </exclusions>
>>> </dependency>
>>> <dependency>
>>> <groupId>com.fasterxml.jackson.module</groupId>
>>> <artifactId>jackson-module-scala_2.10</artifactId>
>>> <version>2.8.8</version>
>>> <scope>runtime</scope>
>>> </dependency>
>>> </dependencies>
>>> </profile>
>>> </profiles>
>>>
>>> <dependencies>
>>> <!-- Adds a dependency on the Beam SDK. -->
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-sdks-java-core</artifactId>
>>> <version>${beam.version}</version>
>>> </dependency>
>>>
>>> <!-- Adds a dependency on the Beam Google Cloud Platform IO module. -->
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
>>> <version>${beam.version}</version>
>>> </dependency>
>>>
>>> <!-- Dependencies below this line are specific dependencies needed by
>>> the examples code. -->
>>> <dependency>
>>> <groupId>com.google.api-client</groupId>
>>> <artifactId>google-api-client</artifactId>
>>> <version>1.22.0</version>
>>> <exclusions>
>>> <!-- Exclude an old version of guava that is being pulled
>>> in by a transitive dependency of google-api-client -->
>>> <exclusion>
>>> <groupId>com.google.guava</groupId>
>>> <artifactId>guava-jdk5</artifactId>
>>> </exclusion>
>>> </exclusions>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>com.google.apis</groupId>
>>> <artifactId>google-api-services-bigquery</artifactId>
>>> <version>v2-rev295-1.22.0</version>
>>> <exclusions>
>>> <!-- Exclude an old version of guava that is being pulled
>>> in by a transitive dependency of google-api-client -->
>>> <exclusion>
>>> <groupId>com.google.guava</groupId>
>>> <artifactId>guava-jdk5</artifactId>
>>> </exclusion>
>>> </exclusions>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>com.google.http-client</groupId>
>>> <artifactId>google-http-client</artifactId>
>>> <version>1.22.0</version>
>>> <exclusions>
>>> <!-- Exclude an old version of guava that is being pulled
>>> in by a transitive dependency of google-api-client -->
>>> <exclusion>
>>> <groupId>com.google.guava</groupId>
>>> <artifactId>guava-jdk5</artifactId>
>>> </exclusion>
>>> </exclusions>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>com.google.apis</groupId>
>>> <artifactId>google-api-services-pubsub</artifactId>
>>> <version>v1-rev10-1.22.0</version>
>>> <exclusions>
>>> <!-- Exclude an old version of guava that is being pulled
>>> in by a transitive dependency of google-api-client -->
>>> <exclusion>
>>> <groupId>com.google.guava</groupId>
>>> <artifactId>guava-jdk5</artifactId>
>>> </exclusion>
>>> </exclusions>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>joda-time</groupId>
>>> <artifactId>joda-time</artifactId>
>>> <version>2.4</version>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>com.google.guava</groupId>
>>> <artifactId>guava</artifactId>
>>> <version>20.0</version>
>>> </dependency>
>>>
>>> <!-- Add slf4j API frontend binding with JUL backend -->
>>> <dependency>
>>> <groupId>org.slf4j</groupId>
>>> <artifactId>slf4j-api</artifactId>
>>> <version>1.7.14</version>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>org.slf4j</groupId>
>>> <artifactId>slf4j-jdk14</artifactId>
>>> <version>1.7.14</version>
>>> <!-- When loaded at runtime this will wire up slf4j to the JUL backend
>>> -->
>>> <scope>runtime</scope>
>>> </dependency>
>>>
>>> <!-- Hamcrest and JUnit are required dependencies of PAssert,
>>> which is used in the main code of DebuggingWordCount example. -->
>>> <dependency>
>>> <groupId>org.hamcrest</groupId>
>>> <artifactId>hamcrest-all</artifactId>
>>> <version>1.3</version>
>>> </dependency>
>>>
>>> <dependency>
>>> <groupId>junit</groupId>
>>> <artifactId>junit</artifactId>
>>> <version>4.12</version>
>>> </dependency>
>>>
>>> <!-- The DirectRunner is needed for unit tests. -->
>>> <dependency>
>>> <groupId>org.apache.beam</groupId>
>>> <artifactId>beam-runners-direct-java</artifactId>
>>> <version>${beam.version}</version>
>>> <scope>test</scope>
>>> </dependency>
>>> </dependencies>
>>> </project>
>>>
>>>
>>>
>>> Thanks in advance.
>>>
>>
>>
>

Re: Getting "Unable to find registrar for c" error

Posted by Shuangyin Ge <go...@gmail.com>.

Hello Luke,

I tried with "file:///C:/Users/ges/Workspace/word-count-beampom.xml" or
"C:/Users/ges/Workspace/word-count-beampom.xml" as input file as seen in
https://issues.apache.org/jira/browse/BEAM-2298, however both failed.

"file:///C:/Users/ges/Workspace/word-count-beampom.xml" failed with Illegal
char <:> at index 4
"C:/Users/ges/Workspace/word-count-beampom.xml" failed with "Unable to find
registrar for c"

Any ideas?

Thanks,
Silas Ge

2017-05-22 23:28 GMT+08:00 Lukasz Cwik <lc...@google.com>:

> This is a known problem with how the local file system is being interacted
> with as is evident by the similar test failures when running the Apache
> Beam unit tests on Windows:
> https://issues.apache.org/jira/browse/BEAM-2299
>
> I would suggest trying to use "file:///c:/my/path/to/file.txt"
>
> There is a general proposal to change how we work with resource paths:
> https://issues.apache.org/jira/browse/BEAM-2283
>
>
>
> On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <go...@gmail.com> wrote:
>
>> Hello team,
>>
>> Not sure if any can share some lights. I am getting "Unable to find
>> registrar for c" error while running
>>
>> $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>>      -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner
>>
>> following the Java Quickstart for Beam https://beam.apache.org/g
>> et-started/quickstart-java/
>>
>> Complete output is shown below:
>>
>> [INFO] Scanning for projects...
>> [WARNING]
>> [WARNING] Some problems were encountered while building the effective
>> model for org.example:word-count-beam:jar:0.1
>> [WARNING] 'build.plugins.plugin.version' for
>> org.apache.maven.plugins:maven-jar-plugin is missing. @ line 80, column
>> 15
>> [WARNING]
>> [WARNING] It is highly recommended to fix these problems because they
>> threaten the stability of your build.
>> [WARNING]
>> [WARNING] For this reason, future Maven versions might no longer support
>> building such malformed projects.
>> [WARNING]
>> [INFO]
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Building word-count-beam 0.1
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO]
>> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
>> word-count-beam ---
>> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered
>> resources, i.e. build is platform dependent!
>> [INFO] skip non existing resourceDirectory C:\Users\ges\Workspace\word-co
>> unt-beam\src\main\resources
>> [INFO]
>> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
>> word-count-beam ---
>> [INFO] Changes detected - recompiling the module!
>> [WARNING] File encoding has not been set, using platform encoding Cp1252,
>> i.e. build is platform dependent!
>> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co
>> unt-beam\target\classes
>> [INFO]
>> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam
>> ---
>> [WARNING]
>> java.lang.reflect.InvocationTargetException
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
>> thodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>> at java.lang.Thread.run(Thread.java:748)
>> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
>> java.lang.IllegalStateException: Unable to find registrar for c
>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>> sult.waitUntilFinish(DirectRunner.java:322)
>> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
>> sult.waitUntilFinish(DirectRunner.java:292)
>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:200)
>> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:63)
>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
>> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
>> at org.apache.beam.examples.WordCount.main(WordCount.java:184)
>> ... 6 more
>> Caused by: java.lang.IllegalStateException: Unable to find registrar for
>> c
>> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil
>> eSystems.java:447)
>> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
>> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystem
>> s.java:174)
>> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
>> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase
>> dSink.java:905)
>> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces
>> sElement(WriteFiles.java:376)
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] BUILD FAILURE
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Total time: 10.279 s
>> [INFO] Finished at: 2017-05-22T08:56:16+08:00
>> [INFO] Final Memory: 32M/450M
>> [INFO] ------------------------------------------------------------
>> ------------
>> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
>> (default-cli) on project word-count-beam: An exception occured while
>> executing the Java class. null: InvocationTargetException:
>> java.lang.IllegalStateException: Unable to find registrar for c -> [Help
>> 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>> -e switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1] http://cwiki.apache.org/con
>> fluence/display/MAVEN/MojoExecutionException
>>
>> Pom.xml is as below
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <!--
>> Licensed to the Apache Software Foundation (ASF) under one or more
>> contributor license agreements. See the NOTICE file distributed with
>> this work for additional information regarding copyright ownership.
>> The ASF licenses this file to You under the Apache License, Version 2.0
>> (the "License"); you may not use this file except in compliance with
>> the License. You may obtain a copy of the License at
>>
>> http://www.apache.org/licenses/LICENSE-2.0
>>
>> Unless required by applicable law or agreed to in writing, software
>> distributed under the License is distributed on an "AS IS" BASIS,
>> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> See the License for the specific language governing permissions and
>> limitations under the License.
>> -->
>> <project xmlns="http://maven.apache.org/POM/4.0.0"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
>> http://maven.apache.org/xsd/maven-4.0.0.xsd">
>> <modelVersion>4.0.0</modelVersion>
>>
>> <groupId>org.example</groupId>
>> <artifactId>word-count-beam</artifactId>
>> <version>0.1</version>
>>
>> <packaging>jar</packaging>
>>
>> <properties>
>> <beam.version>2.0.0</beam.version>
>> <surefire-plugin.version>2.20</surefire-plugin.version>
>> </properties>
>>
>> <repositories>
>> <repository>
>> <id>apache.snapshots</id>
>> <name>Apache Development Snapshot Repository</name>
>> <url>https://repository.apache.org/content/repositories/snapshots/</url>
>> <releases>
>> <enabled>false</enabled>
>> </releases>
>> <snapshots>
>> <enabled>true</enabled>
>> </snapshots>
>> </repository>
>> </repositories>
>>
>> <build>
>> <plugins>
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-compiler-plugin</artifactId>
>> <version>3.5.1</version>
>> <configuration>
>> <source>1.7</source>
>> <target>1.7</target>
>> </configuration>
>> </plugin>
>>
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-surefire-plugin</artifactId>
>> <version>2.20</version>
>> <configuration>
>> <parallel>all</parallel>
>> <threadCount>4</threadCount>
>> <redirectTestOutputToFile>true</redirectTestOutputToFile>
>> </configuration>
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.maven.surefire</groupId>
>> <artifactId>surefire-junit47</artifactId>
>> <version>2.20</version>
>> </dependency>
>> </dependencies>
>> </plugin>
>>
>> <!-- Ensure that the Maven jar plugin runs before the Maven
>> shade plugin by listing the plugin higher within the file. -->
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-jar-plugin</artifactId>
>> </plugin>
>>
>> <!--
>> Configures `mvn package` to produce a bundled jar ("fat jar") for runners
>> that require this for job submission to a cluster.
>> -->
>> <plugin>
>> <groupId>org.apache.maven.plugins</groupId>
>> <artifactId>maven-shade-plugin</artifactId>
>> <version>3.0.0</version>
>> <executions>
>> <execution>
>> <phase>package</phase>
>> <goals>
>> <goal>shade</goal>
>> </goals>
>> <configuration>
>> <filters>
>> <filter>
>> <artifact>*:*</artifact>
>> <excludes>
>> <exclude>META-INF/LICENSE</exclude>
>> <exclude>META-INF/*.SF</exclude>
>> <exclude>META-INF/*.DSA</exclude>
>> <exclude>META-INF/*.RSA</exclude>
>> </excludes>
>> </filter>
>> </filters>
>> <transformers>
>> <transformer implementation="org.apache.maven.plugins.shade.resource.Serv
>> icesResourceTransformer"/>
>> </transformers>
>> </configuration>
>> </execution>
>> </executions>
>> </plugin>
>> </plugins>
>>
>> <pluginManagement>
>> <plugins>
>> <plugin>
>> <groupId>org.codehaus.mojo</groupId>
>> <artifactId>exec-maven-plugin</artifactId>
>> <version>1.4.0</version>
>> <configuration>
>> <cleanupDaemonThreads>false</cleanupDaemonThreads>
>> </configuration>
>> </plugin>
>> </plugins>
>> </pluginManagement>
>> </build>
>>
>> <profiles>
>> <profile>
>> <id>direct-runner</id>
>> <activation>
>> <activeByDefault>true</activeByDefault>
>> </activation>
>> <!-- Makes the DirectRunner available when running a pipeline. -->
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-runners-direct-java</artifactId>
>> <version>2.0.0</version>
>> <scope>runtime</scope>
>> </dependency>
>> </dependencies>
>> </profile>
>>
>> <profile>
>> <id>apex-runner</id>
>> <!-- Makes the ApexRunner available when running a pipeline. -->
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-runners-apex</artifactId>
>> <version>${beam.version}</version>
>> <scope>runtime</scope>
>> </dependency>
>> <!--
>> Apex depends on httpclient version 4.3.5, project has a transitive
>> dependency to httpclient 4.0.1 from
>> google-http-client. Apex dependency version being specified explicitly so
>> that it gets picked up. This
>> can be removed when the project no longer has a dependency on a different
>> httpclient version.
>> -->
>> <dependency>
>> <groupId>org.apache.httpcomponents</groupId>
>> <artifactId>httpclient</artifactId>
>> <version>4.3.5</version>
>> <scope>runtime</scope>
>> <exclusions>
>> <exclusion>
>> <groupId>commons-codec</groupId>
>> <artifactId>commons-codec</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>> </dependencies>
>> </profile>
>>
>> <profile>
>> <id>dataflow-runner</id>
>> <!-- Makes the DataflowRunner available when running a pipeline. -->
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
>> <version>${beam.version}</version>
>> <scope>runtime</scope>
>> </dependency>
>> </dependencies>
>> </profile>
>>
>> <profile>
>> <id>flink-runner</id>
>> <!-- Makes the FlinkRunner available when running a pipeline. -->
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-runners-flink_2.10</artifactId>
>> <version>${beam.version}</version>
>> <scope>runtime</scope>
>> </dependency>
>> </dependencies>
>> </profile>
>>
>> <profile>
>> <id>spark-runner</id>
>> <!-- Makes the SparkRunner available when running a pipeline.
>> Additionally,
>> overrides some Spark dependencies to Beam-compatible versions. -->
>> <dependencies>
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-runners-spark</artifactId>
>> <version>${beam.version}</version>
>> <scope>runtime</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
>> <version>${beam.version}</version>
>> <scope>runtime</scope>
>> </dependency>
>> <dependency>
>> <groupId>org.apache.spark</groupId>
>> <artifactId>spark-streaming_2.10</artifactId>
>> <version>1.6.2</version>
>> <scope>runtime</scope>
>> <exclusions>
>> <exclusion>
>> <groupId>org.slf4j</groupId>
>> <artifactId>jul-to-slf4j</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>> <dependency>
>> <groupId>com.fasterxml.jackson.module</groupId>
>> <artifactId>jackson-module-scala_2.10</artifactId>
>> <version>2.8.8</version>
>> <scope>runtime</scope>
>> </dependency>
>> </dependencies>
>> </profile>
>> </profiles>
>>
>> <dependencies>
>> <!-- Adds a dependency on the Beam SDK. -->
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-sdks-java-core</artifactId>
>> <version>${beam.version}</version>
>> </dependency>
>>
>> <!-- Adds a dependency on the Beam Google Cloud Platform IO module. -->
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
>> <version>${beam.version}</version>
>> </dependency>
>>
>> <!-- Dependencies below this line are specific dependencies needed by the
>> examples code. -->
>> <dependency>
>> <groupId>com.google.api-client</groupId>
>> <artifactId>google-api-client</artifactId>
>> <version>1.22.0</version>
>> <exclusions>
>> <!-- Exclude an old version of guava that is being pulled
>> in by a transitive dependency of google-api-client -->
>> <exclusion>
>> <groupId>com.google.guava</groupId>
>> <artifactId>guava-jdk5</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>>
>> <dependency>
>> <groupId>com.google.apis</groupId>
>> <artifactId>google-api-services-bigquery</artifactId>
>> <version>v2-rev295-1.22.0</version>
>> <exclusions>
>> <!-- Exclude an old version of guava that is being pulled
>> in by a transitive dependency of google-api-client -->
>> <exclusion>
>> <groupId>com.google.guava</groupId>
>> <artifactId>guava-jdk5</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>>
>> <dependency>
>> <groupId>com.google.http-client</groupId>
>> <artifactId>google-http-client</artifactId>
>> <version>1.22.0</version>
>> <exclusions>
>> <!-- Exclude an old version of guava that is being pulled
>> in by a transitive dependency of google-api-client -->
>> <exclusion>
>> <groupId>com.google.guava</groupId>
>> <artifactId>guava-jdk5</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>>
>> <dependency>
>> <groupId>com.google.apis</groupId>
>> <artifactId>google-api-services-pubsub</artifactId>
>> <version>v1-rev10-1.22.0</version>
>> <exclusions>
>> <!-- Exclude an old version of guava that is being pulled
>> in by a transitive dependency of google-api-client -->
>> <exclusion>
>> <groupId>com.google.guava</groupId>
>> <artifactId>guava-jdk5</artifactId>
>> </exclusion>
>> </exclusions>
>> </dependency>
>>
>> <dependency>
>> <groupId>joda-time</groupId>
>> <artifactId>joda-time</artifactId>
>> <version>2.4</version>
>> </dependency>
>>
>> <dependency>
>> <groupId>com.google.guava</groupId>
>> <artifactId>guava</artifactId>
>> <version>20.0</version>
>> </dependency>
>>
>> <!-- Add slf4j API frontend binding with JUL backend -->
>> <dependency>
>> <groupId>org.slf4j</groupId>
>> <artifactId>slf4j-api</artifactId>
>> <version>1.7.14</version>
>> </dependency>
>>
>> <dependency>
>> <groupId>org.slf4j</groupId>
>> <artifactId>slf4j-jdk14</artifactId>
>> <version>1.7.14</version>
>> <!-- When loaded at runtime this will wire up slf4j to the JUL backend -->
>> <scope>runtime</scope>
>> </dependency>
>>
>> <!-- Hamcrest and JUnit are required dependencies of PAssert,
>> which is used in the main code of DebuggingWordCount example. -->
>> <dependency>
>> <groupId>org.hamcrest</groupId>
>> <artifactId>hamcrest-all</artifactId>
>> <version>1.3</version>
>> </dependency>
>>
>> <dependency>
>> <groupId>junit</groupId>
>> <artifactId>junit</artifactId>
>> <version>4.12</version>
>> </dependency>
>>
>> <!-- The DirectRunner is needed for unit tests. -->
>> <dependency>
>> <groupId>org.apache.beam</groupId>
>> <artifactId>beam-runners-direct-java</artifactId>
>> <version>${beam.version}</version>
>> <scope>test</scope>
>> </dependency>
>> </dependencies>
>> </project>
>>
>>
>>
>> Thanks in advance.
>>
>
>

Re: Getting "Unable to find registrar for c" error

Posted by Lukasz Cwik <lc...@google.com>.

This is a known problem with how the local file system is being interacted
with as is evident by the similar test failures when running the Apache
Beam unit tests on Windows:
https://issues.apache.org/jira/browse/BEAM-2299

I would suggest trying to use "file:///c:/my/path/to/file.txt"

There is a general proposal to change how we work with resource paths:
https://issues.apache.org/jira/browse/BEAM-2283



On Sun, May 21, 2017 at 6:03 PM, Shuangyin Ge <go...@gmail.com> wrote:

> Hello team,
>
> Not sure if any can share some lights. I am getting "Unable to find
> registrar for c" error while running
>
> $ mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount \
>      -Dexec.args="--inputFile=pom.xml --output=counts" -Pdirect-runner
>
> following the Java Quickstart for Beam https://beam.apache.org/
> get-started/quickstart-java/
>
> Complete output is shown below:
>
> [INFO] Scanning for projects...
> [WARNING]
> [WARNING] Some problems were encountered while building the effective
> model for org.example:word-count-beam:jar:0.1
> [WARNING] 'build.plugins.plugin.version' for org.apache.maven.plugins:maven-jar-plugin
> is missing. @ line 80, column 15
> [WARNING]
> [WARNING] It is highly recommended to fix these problems because they
> threaten the stability of your build.
> [WARNING]
> [WARNING] For this reason, future Maven versions might no longer support
> building such malformed projects.
> [WARNING]
> [INFO]
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Building word-count-beam 0.1
> [INFO] ------------------------------------------------------------
> ------------
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
> word-count-beam ---
> [WARNING] Using platform encoding (Cp1252 actually) to copy filtered
> resources, i.e. build is platform dependent!
> [INFO] skip non existing resourceDirectory C:\Users\ges\Workspace\word-co
> unt-beam\src\main\resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @
> word-count-beam ---
> [INFO] Changes detected - recompiling the module!
> [WARNING] File encoding has not been set, using platform encoding Cp1252,
> i.e. build is platform dependent!
> [INFO] Compiling 10 source files to C:\Users\ges\Workspace\word-co
> unt-beam\target\classes
> [INFO]
> [INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @ word-count-beam ---
> [WARNING]
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> ssorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
> thodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.beam.sdk.Pipeline$PipelineExecutionException:
> java.lang.IllegalStateException: Unable to find registrar for c
> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
> sult.waitUntilFinish(DirectRunner.java:322)
> at org.apache.beam.runners.direct.DirectRunner$DirectPipelineRe
> sult.waitUntilFinish(DirectRunner.java:292)
> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:200)
> at org.apache.beam.runners.direct.DirectRunner.run(DirectRunner.java:63)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:295)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:281)
> at org.apache.beam.examples.WordCount.main(WordCount.java:184)
> ... 6 more
> Caused by: java.lang.IllegalStateException: Unable to find registrar for c
> at org.apache.beam.sdk.io.FileSystems.getFileSystemInternal(Fil
> eSystems.java:447)
> at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:111)
> at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:174)
> at org.apache.beam.sdk.io.FileSystems.delete(FileSystems.java:321)
> at org.apache.beam.sdk.io.FileBasedSink$Writer.cleanup(FileBase
> dSink.java:905)
> at org.apache.beam.sdk.io.WriteFiles$WriteShardedBundles.proces
> sElement(WriteFiles.java:376)
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Total time: 10.279 s
> [INFO] Finished at: 2017-05-22T08:56:16+08:00
> [INFO] Final Memory: 32M/450M
> [INFO] ------------------------------------------------------------
> ------------
> [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java
> (default-cli) on project word-count-beam: An exception occured while
> executing the Java class. null: InvocationTargetException:
> java.lang.IllegalStateException: Unable to find registrar for c -> [Help
> 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExec
> utionException
>
> Pom.xml is as below
>
> <?xml version="1.0" encoding="UTF-8"?>
> <!--
> Licensed to the Apache Software Foundation (ASF) under one or more
> contributor license agreements. See the NOTICE file distributed with
> this work for additional information regarding copyright ownership.
> The ASF licenses this file to You under the Apache License, Version 2.0
> (the "License"); you may not use this file except in compliance with
> the License. You may obtain a copy of the License at
>
> http://www.apache.org/licenses/LICENSE-2.0
>
> Unless required by applicable law or agreed to in writing, software
> distributed under the License is distributed on an "AS IS" BASIS,
> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> See the License for the specific language governing permissions and
> limitations under the License.
> -->
> <project xmlns="http://maven.apache.org/POM/4.0.0"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/xsd/maven-4.0.0.xsd">
> <modelVersion>4.0.0</modelVersion>
>
> <groupId>org.example</groupId>
> <artifactId>word-count-beam</artifactId>
> <version>0.1</version>
>
> <packaging>jar</packaging>
>
> <properties>
> <beam.version>2.0.0</beam.version>
> <surefire-plugin.version>2.20</surefire-plugin.version>
> </properties>
>
> <repositories>
> <repository>
> <id>apache.snapshots</id>
> <name>Apache Development Snapshot Repository</name>
> <url>https://repository.apache.org/content/repositories/snapshots/</url>
> <releases>
> <enabled>false</enabled>
> </releases>
> <snapshots>
> <enabled>true</enabled>
> </snapshots>
> </repository>
> </repositories>
>
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-compiler-plugin</artifactId>
> <version>3.5.1</version>
> <configuration>
> <source>1.7</source>
> <target>1.7</target>
> </configuration>
> </plugin>
>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>2.20</version>
> <configuration>
> <parallel>all</parallel>
> <threadCount>4</threadCount>
> <redirectTestOutputToFile>true</redirectTestOutputToFile>
> </configuration>
> <dependencies>
> <dependency>
> <groupId>org.apache.maven.surefire</groupId>
> <artifactId>surefire-junit47</artifactId>
> <version>2.20</version>
> </dependency>
> </dependencies>
> </plugin>
>
> <!-- Ensure that the Maven jar plugin runs before the Maven
> shade plugin by listing the plugin higher within the file. -->
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-jar-plugin</artifactId>
> </plugin>
>
> <!--
> Configures `mvn package` to produce a bundled jar ("fat jar") for runners
> that require this for job submission to a cluster.
> -->
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-shade-plugin</artifactId>
> <version>3.0.0</version>
> <executions>
> <execution>
> <phase>package</phase>
> <goals>
> <goal>shade</goal>
> </goals>
> <configuration>
> <filters>
> <filter>
> <artifact>*:*</artifact>
> <excludes>
> <exclude>META-INF/LICENSE</exclude>
> <exclude>META-INF/*.SF</exclude>
> <exclude>META-INF/*.DSA</exclude>
> <exclude>META-INF/*.RSA</exclude>
> </excludes>
> </filter>
> </filters>
> <transformers>
> <transformer implementation="org.apache.maven.plugins.shade.resource.Serv
> icesResourceTransformer"/>
> </transformers>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
>
> <pluginManagement>
> <plugins>
> <plugin>
> <groupId>org.codehaus.mojo</groupId>
> <artifactId>exec-maven-plugin</artifactId>
> <version>1.4.0</version>
> <configuration>
> <cleanupDaemonThreads>false</cleanupDaemonThreads>
> </configuration>
> </plugin>
> </plugins>
> </pluginManagement>
> </build>
>
> <profiles>
> <profile>
> <id>direct-runner</id>
> <activation>
> <activeByDefault>true</activeByDefault>
> </activation>
> <!-- Makes the DirectRunner available when running a pipeline. -->
> <dependencies>
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-runners-direct-java</artifactId>
> <version>2.0.0</version>
> <scope>runtime</scope>
> </dependency>
> </dependencies>
> </profile>
>
> <profile>
> <id>apex-runner</id>
> <!-- Makes the ApexRunner available when running a pipeline. -->
> <dependencies>
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-runners-apex</artifactId>
> <version>${beam.version}</version>
> <scope>runtime</scope>
> </dependency>
> <!--
> Apex depends on httpclient version 4.3.5, project has a transitive
> dependency to httpclient 4.0.1 from
> google-http-client. Apex dependency version being specified explicitly so
> that it gets picked up. This
> can be removed when the project no longer has a dependency on a different
> httpclient version.
> -->
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpclient</artifactId>
> <version>4.3.5</version>
> <scope>runtime</scope>
> <exclusions>
> <exclusion>
> <groupId>commons-codec</groupId>
> <artifactId>commons-codec</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> </dependencies>
> </profile>
>
> <profile>
> <id>dataflow-runner</id>
> <!-- Makes the DataflowRunner available when running a pipeline. -->
> <dependencies>
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
> <version>${beam.version}</version>
> <scope>runtime</scope>
> </dependency>
> </dependencies>
> </profile>
>
> <profile>
> <id>flink-runner</id>
> <!-- Makes the FlinkRunner available when running a pipeline. -->
> <dependencies>
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-runners-flink_2.10</artifactId>
> <version>${beam.version}</version>
> <scope>runtime</scope>
> </dependency>
> </dependencies>
> </profile>
>
> <profile>
> <id>spark-runner</id>
> <!-- Makes the SparkRunner available when running a pipeline. Additionally,
> overrides some Spark dependencies to Beam-compatible versions. -->
> <dependencies>
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-runners-spark</artifactId>
> <version>${beam.version}</version>
> <scope>runtime</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-sdks-java-io-hadoop-file-system</artifactId>
> <version>${beam.version}</version>
> <scope>runtime</scope>
> </dependency>
> <dependency>
> <groupId>org.apache.spark</groupId>
> <artifactId>spark-streaming_2.10</artifactId>
> <version>1.6.2</version>
> <scope>runtime</scope>
> <exclusions>
> <exclusion>
> <groupId>org.slf4j</groupId>
> <artifactId>jul-to-slf4j</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> <dependency>
> <groupId>com.fasterxml.jackson.module</groupId>
> <artifactId>jackson-module-scala_2.10</artifactId>
> <version>2.8.8</version>
> <scope>runtime</scope>
> </dependency>
> </dependencies>
> </profile>
> </profiles>
>
> <dependencies>
> <!-- Adds a dependency on the Beam SDK. -->
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-sdks-java-core</artifactId>
> <version>${beam.version}</version>
> </dependency>
>
> <!-- Adds a dependency on the Beam Google Cloud Platform IO module. -->
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-sdks-java-io-google-cloud-platform</artifactId>
> <version>${beam.version}</version>
> </dependency>
>
> <!-- Dependencies below this line are specific dependencies needed by the
> examples code. -->
> <dependency>
> <groupId>com.google.api-client</groupId>
> <artifactId>google-api-client</artifactId>
> <version>1.22.0</version>
> <exclusions>
> <!-- Exclude an old version of guava that is being pulled
> in by a transitive dependency of google-api-client -->
> <exclusion>
> <groupId>com.google.guava</groupId>
> <artifactId>guava-jdk5</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
>
> <dependency>
> <groupId>com.google.apis</groupId>
> <artifactId>google-api-services-bigquery</artifactId>
> <version>v2-rev295-1.22.0</version>
> <exclusions>
> <!-- Exclude an old version of guava that is being pulled
> in by a transitive dependency of google-api-client -->
> <exclusion>
> <groupId>com.google.guava</groupId>
> <artifactId>guava-jdk5</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
>
> <dependency>
> <groupId>com.google.http-client</groupId>
> <artifactId>google-http-client</artifactId>
> <version>1.22.0</version>
> <exclusions>
> <!-- Exclude an old version of guava that is being pulled
> in by a transitive dependency of google-api-client -->
> <exclusion>
> <groupId>com.google.guava</groupId>
> <artifactId>guava-jdk5</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
>
> <dependency>
> <groupId>com.google.apis</groupId>
> <artifactId>google-api-services-pubsub</artifactId>
> <version>v1-rev10-1.22.0</version>
> <exclusions>
> <!-- Exclude an old version of guava that is being pulled
> in by a transitive dependency of google-api-client -->
> <exclusion>
> <groupId>com.google.guava</groupId>
> <artifactId>guava-jdk5</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
>
> <dependency>
> <groupId>joda-time</groupId>
> <artifactId>joda-time</artifactId>
> <version>2.4</version>
> </dependency>
>
> <dependency>
> <groupId>com.google.guava</groupId>
> <artifactId>guava</artifactId>
> <version>20.0</version>
> </dependency>
>
> <!-- Add slf4j API frontend binding with JUL backend -->
> <dependency>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-api</artifactId>
> <version>1.7.14</version>
> </dependency>
>
> <dependency>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-jdk14</artifactId>
> <version>1.7.14</version>
> <!-- When loaded at runtime this will wire up slf4j to the JUL backend -->
> <scope>runtime</scope>
> </dependency>
>
> <!-- Hamcrest and JUnit are required dependencies of PAssert,
> which is used in the main code of DebuggingWordCount example. -->
> <dependency>
> <groupId>org.hamcrest</groupId>
> <artifactId>hamcrest-all</artifactId>
> <version>1.3</version>
> </dependency>
>
> <dependency>
> <groupId>junit</groupId>
> <artifactId>junit</artifactId>
> <version>4.12</version>
> </dependency>
>
> <!-- The DirectRunner is needed for unit tests. -->
> <dependency>
> <groupId>org.apache.beam</groupId>
> <artifactId>beam-runners-direct-java</artifactId>
> <version>${beam.version}</version>
> <scope>test</scope>
> </dependency>
> </dependencies>
> </project>
>
>
>
> Thanks in advance.
>