You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@crunch.apache.org by "Josh Wills (JIRA)" <ji...@apache.org> on 2015/01/31 18:03:35 UTC

[jira] [Updated] (CRUNCH-496) Scrunch on Spark

     [ https://issues.apache.org/jira/browse/CRUNCH-496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Josh Wills updated CRUNCH-496:
------------------------------
    Attachment: CRUNCH-496.patch

I got Scrunch running on top of the SparkPipeline last night after making a small hack to the Spark closure serializer so that it would properly load serialized primitive types (float, int, byte, etc.), which weren't working for some reason.

Will likely iterate on this one a bit more, but feedback is welcome.

> Scrunch on Spark
> ----------------
>
>                 Key: CRUNCH-496
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-496
>             Project: Crunch
>          Issue Type: New Feature
>          Components: Scrunch
>            Reporter: Josh Wills
>         Attachments: CRUNCH-496.patch
>
>
> Add an implementation of Scrunch (specifically, a subclass of o.a.c.scrunch.Pipeline) that wraps SparkPipeline.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)