[jira] [Commented] (SAMZA-762) java arrays not usable as keys anymore

["József Márton Jung (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726987#comment-14726987 ] 

József Márton Jung commented on SAMZA-762:
------------------------------------------

Hi,

{quote}
In 0.8 it was possible to configure your partitioner with systems.kafka.producer.partitioner.class and select for example the default murmur2 based partitioner of kafka.
But this is also not possible anymore.
{quote}
It will be possible again in Kafka 8.3 once it gets released. Relevant KIP and JIRA ticket can be found [here|https://cwiki.apache.org/confluence/display/KAFKA/KIP-22+-+Expose+a+Partitioner+interface+in+the+new+producer] and [here|https://issues.apache.org/jira/browse/KAFKA-2091].

As for the partitioner, I'll try to expose an interface, so the users can implement their own partitioner.

> java arrays not usable as keys anymore
> --------------------------------------
>
>                 Key: SAMZA-762
>                 URL: https://issues.apache.org/jira/browse/SAMZA-762
>             Project: Samza
>          Issue Type: Bug
>          Components: kafka
>    Affects Versions: 0.9.0
>            Reporter: Steven Aerts
>
> Since the integration of the new kafka producer into samza, the partitioner is defined like [this|https://github.com/apache/samza/blob/0.9.0/samza-kafka/src/main/scala/org/apache/samza/util/KafkaUtil.scala#L49]:
> {code}
> return abs(envelope.getPartitionKey.hashCode()) % numPartitions
> {code}
> This is problematic for java arrays, as their hash-code is independent of their content (unlike in scala).  So a specific array based key is randomly assigned to a partition.
> In 0.8 it was possible to configure your partitioner with {{systems.kafka.producer.partitioner.class}} and select for example the default murmur2 based partitioner of kafka.
> But this is also not possible anymore.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)