table_name is null with cdc from mysql

[gywndi <gy...@gmail.com>]

Hi~ 

I set with mysql 5.7.16(percona) and testing..
(In fact, minimal format is not suitable with NiFi, so I changed to binlog
image to "FULL")

But I updated 1 rows.. there's no database and table_name (also table_id),
when I read "Provenance Event
". (Only updated data is seen..) 

Is this common? I want to make a new script to send to another system. But,
I can't see the table_name.. 
Reply me please.
(Do you have any good reference?)

Thanks.
============================

{ "type" : "update",
  "timestamp" : 1509739175000,
  "binlog_filename" : "mysql-binary.000001",
  "binlog_position" : 5645,
  "database" : NULL,
  "table_name" : NULL,
  "table_id" : NULL,
  "columns" : [ 
  { "id" : 1, "last_value" : 2603098, "value" : 2603098 }, 
  { "id" : 2, "last_value" : "22b847352243de264f3542d059c5f94a", "value" :
"d32edc9ec1db8c286941cd8bd7162ae2" } 
]}



--
Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/

Re: table_name is null with cdc from mysql

[Matt Burgess <ma...@apache.org>]

Gywndi,

This is definitely something that should be improved, I have written
NIFI-4572 [1] to cover the improvement. The short story is that the
processor currently only stores the database/table information in a
DistributedMapCache, so if you want those fields populated, you'll
want to configure a DistributedMapCacheServer controller service, and
also a DistributedMapCacheClient to connect to it. Then you specify
that client in your CaptureChangeMySQL processor, and you will see
database, table, and column information in the outgoing CDC events.


Regards,
Matt

[1] https://issues.apache.org/jira/browse/NIFI-4572

On Mon, Nov 6, 2017 at 6:21 AM, gywndi <gy...@gmail.com> wrote:
> Hi~
>
> I set with mysql 5.7.16(percona) and testing..
> (In fact, minimal format is not suitable with NiFi, so I changed to binlog
> image to "FULL")
>
> But I updated 1 rows.. there's no database and table_name (also table_id),
> when I read "Provenance Event
> ". (Only updated data is seen..)
>
> Is this common? I want to make a new script to send to another system. But,
> I can't see the table_name..
> Reply me please.
> (Do you have any good reference?)
>
> Thanks.
> ============================
>
> { "type" : "update",
>   "timestamp" : 1509739175000,
>   "binlog_filename" : "mysql-binary.000001",
>   "binlog_position" : 5645,
>   "database" : NULL,
>   "table_name" : NULL,
>   "table_id" : NULL,
>   "columns" : [
>   { "id" : 1, "last_value" : 2603098, "value" : 2603098 },
>   { "id" : 2, "last_value" : "22b847352243de264f3542d059c5f94a", "value" :
> "d32edc9ec1db8c286941cd8bd7162ae2" }
> ]}
>
>
>
> --
> Sent from: http://apache-nifi-developer-list.39713.n7.nabble.com/