You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@beam.apache.org by "Chaoran Yu (JIRA)" <ji...@apache.org> on 2017/05/02 23:23:04 UTC

[jira] [Commented] (BEAM-302) Add Scio Scala DSL to Beam

    [ https://issues.apache.org/jira/browse/BEAM-302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15993999#comment-15993999 ] 

Chaoran Yu commented on BEAM-302:
---------------------------------

[~neville]
I tried the latest SNAPSHOT version of scio but it's still not working with Spark runner out of the box. For example, running sbt "scio-examples/run-main com.spotify.scio.examples.WordCount --runner=SparkRunner --input=README.md --output=wc" gave me the following exception:
java.lang.IllegalArgumentException: Unknown 'runner' specified 'SparkRunner', supported pipeline runners [DataflowRunner, DirectRunner]
Caused by: java.lang.ClassNotFoundException: SparkRunner
The same thing happened in the scio repl. Looking at the code, more things need to be done to integrate with Spark/Flink etc.

> Add Scio Scala DSL to Beam
> --------------------------
>
>                 Key: BEAM-302
>                 URL: https://issues.apache.org/jira/browse/BEAM-302
>             Project: Beam
>          Issue Type: Wish
>          Components: sdk-ideas
>            Reporter: Jean-Baptiste Onofré
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)