[jira] [Commented] (FLINK-2315) Hadoop Writables cannot exploit implementing NormalizableKey

["Martin Junghanns (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-2315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15063684#comment-15063684 ] 

Martin Junghanns commented on FLINK-2315:
-----------------------------------------

Hi, is this hard to fix? I wanted to apply that exact pattern at our id type [1] which implements {{WritableComparable}}. I just read your answer on tera-sort [2] and now I am unsure if it was wrong to assume, that implementing {{Writable}} instead of using basic types has no effect on performance?

Another example is here [3]. It holds a {{byte[]}} to store a property value and is used in Tuples and Pojos (which are used in DataSets). Do you think this is slower compared to the byte-wise comparison directly inside Flink's managed memory?

Thanks!

[1] https://github.com/dbs-leipzig/gradoop/blob/master/gradoop-core/src/main/java/org/gradoop/model/impl/id/GradoopId.java
[2] http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/TeraSort-on-Flink-and-Spark-td1915.html
[3] https://github.com/dbs-leipzig/gradoop/blob/master/gradoop-core/src/main/java/org/gradoop/model/impl/properties/PropertyValue.java

> Hadoop Writables cannot exploit implementing NormalizableKey
> ------------------------------------------------------------
>
>                 Key: FLINK-2315
>                 URL: https://issues.apache.org/jira/browse/FLINK-2315
>             Project: Flink
>          Issue Type: Bug
>          Components: Java API
>    Affects Versions: 0.9
>            Reporter: Stephan Ewen
>             Fix For: 1.0.0
>
>
> When one implements a type that extends {{hadoop.io.Writable}} and it implements {{NormalizableKey}}, this is still not exploited.
> The Writable comparator fails to recognize that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)