You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@kafka.apache.org by 男Danngo <ra...@gmail.com> on 2017/06/25 13:21:27 UTC

Kafka Connect Transformations - tranformations for value broken on tombsone events

Hi all,

I'm trying to use the transformation for Kafka Connect and running into issues.


The tranformation configuration is:

-----------------

"transforms": "GetAfter",

"transforms.GetAfter.type":
"org.apache.kafka.connect.transforms.ExtractField$Value",

"transforms.GetAfter.field": "after",

-----------------

And I got the following errors occasionally：

-----------------

org.apache.kafka.connect.errors.DataException: Only Map objects
supported in absence of schema for [field extraction], found: null
	at org.apache.kafka.connect.transforms.util.Requirements.requireMap(Requirements.java:38)
	at org.apache.kafka.connect.transforms.ExtractField.apply(ExtractField.java:57)
	at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:39)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.convertMessages(WorkerSinkTask.java:408)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:249)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:179)
	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:139)
	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:182)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

-----------------

Seems that tombstone events break the transformation, but after
checking the source code for Transformations on Value
(ExtractField$Value, ValueToKey, MaskField$Value, ReplaceField$Value),
none of them handles tombstone events explicitly, none of them work
through Tombstone events neither.

So, is this a bug? Or is there any configuration that handles
tombstone events? Am I missing something?

Re: Kafka Connect Transformations - tranformations for value broken on tombsone events

Posted by Randall Hauch <rh...@gmail.com>.
Would you mind logging an issue against the Kafka Connect component in our
JIRA? https://issues.apache.org/jira/projects/KAFKA

Thanks!

On Sun, Jun 25, 2017 at 8:21 AM, 男Danngo <ra...@gmail.com> wrote:

> Hi all,
>
> I'm trying to use the transformation for Kafka Connect and running into
> issues.
>
>
> The tranformation configuration is:
>
> -----------------
>
> "transforms": "GetAfter",
>
> "transforms.GetAfter.type":
> "org.apache.kafka.connect.transforms.ExtractField$Value",
>
> "transforms.GetAfter.field": "after",
>
> -----------------
>
> And I got the following errors occasionally：
>
> -----------------
>
> org.apache.kafka.connect.errors.DataException: Only Map objects
> supported in absence of schema for [field extraction], found: null
>         at org.apache.kafka.connect.transforms.util.Requirements.
> requireMap(Requirements.java:38)
>         at org.apache.kafka.connect.transforms.ExtractField.apply(
> ExtractField.java:57)
>         at org.apache.kafka.connect.runtime.TransformationChain.
> apply(TransformationChain.java:39)
>         at org.apache.kafka.connect.runtime.WorkerSinkTask.
> convertMessages(WorkerSinkTask.java:408)
>         at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(
> WorkerSinkTask.java:249)
>         at org.apache.kafka.connect.runtime.WorkerSinkTask.
> iteration(WorkerSinkTask.java:179)
>         at org.apache.kafka.connect.runtime.WorkerSinkTask.
> execute(WorkerSinkTask.java:148)
>         at org.apache.kafka.connect.runtime.WorkerTask.doRun(
> WorkerTask.java:139)
>         at org.apache.kafka.connect.runtime.WorkerTask.run(
> WorkerTask.java:182)
>         at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
>
> -----------------
>
> Seems that tombstone events break the transformation, but after
> checking the source code for Transformations on Value
> (ExtractField$Value, ValueToKey, MaskField$Value, ReplaceField$Value),
> none of them handles tombstone events explicitly, none of them work
> through Tombstone events neither.
>
> So, is this a bug? Or is there any configuration that handles
> tombstone events? Am I missing something?
>