You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "David Whiting (JIRA)" <ji...@apache.org> on 2017/08/04 11:42:02 UTC

[jira] [Created] (CRUNCH-651) (WIP) Kotlin API for Crunch

David Whiting created CRUNCH-651:
------------------------------------

             Summary: (WIP) Kotlin API for Crunch
                 Key: CRUNCH-651
                 URL: https://issues.apache.org/jira/browse/CRUNCH-651
             Project: Crunch
          Issue Type: Improvement
            Reporter: David Whiting
            Assignee: David Whiting
            Priority: Minor
             Fix For: 1.0.0
         Attachments: 0001-WIP-Kotlin-API-for-Crunch.patch

Motivation: Kotlin has recently gained a lot of popularity and people are really starting to talk about it and see its merits; and language-wise it should be perfect for writing data-transformation code as it has better support for functional programming and useful type stuff than Java, without the performance overhead and implicit mysteries of Scala.

Attached is a patch for a proof-of-concept Kotlin API for Crunch. It makes heavy use of inlined functions so that the compiled bytecode ends up looking a lot like standard Crunch code (I ran it through a decompiler), and should be just as performant.

It also uses reified type parameters to construct PTypes automatically from TypeReferences, so in most cases it's not necessary to provide a PType for transformations.

See the example job and unit tests to get a quick idea of what kind of thing is possible.

I've decided to share an unfinished patch with you all so that I can get some feedback if we think this is a useful addition to the Crunch project at all, and whether the general approach seems sensible.

Still left TODO (if we decide to finish this and add it):

* PGroupedTable API
* Extensible PType inference (I'm thinking something like register<MyType>(pTypeForMyType))
* Support for data classes via kotlin-reflect
* Fill in any other gaps (mapWithContext, counters, ... ?)







--
This message was sent by Atlassian JIRA
(v6.4.14#64029)