[jira] [Commented] (FLUME-2256) Generic JDBC Sink

["Viktor Trako (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLUME-2256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010917#comment-14010917 ] 

Viktor Trako commented on FLUME-2256:
-------------------------------------

Where is this at the moment - has anyone tested this patch further?

Currently using it in a staging environment and its working beautifully for writing events to MySql server. The dependencies required for this patch are working in harmony with my current agent requirements.

Dissapointed this didnt make its way through to 1.5 release.

Viktor

> Generic JDBC Sink
> -----------------
>
>                 Key: FLUME-2256
>                 URL: https://issues.apache.org/jira/browse/FLUME-2256
>             Project: Flume
>          Issue Type: New Feature
>            Reporter: Jeremy Karlson
>            Assignee: Jeremy Karlson
>         Attachments: FLUME-2256.diff
>
>
> I've been working on a generic JDBC sink.  It needs a bit more testing, but I think it's ready for review and feedback.  I have not yet updated the Flume documentation, but I can / will if people are happy with this.
> Since the config file is how you’d interact with it, here’s a working example from my source tree:
> {code}
> a.sinks.k.type=jdbc
> a.sinks.k.channel=c
> a.sinks.k.driver=com.mysql.jdbc.Driver
> a.sinks.k.url=jdbc:mysql://localhost:8889/flume
> a.sinks.k.user=username
> a.sinks.k.password=password
> a.sinks.k.batchSize=100
> a.sinks.k.sql=insert into twitter (body, timestamp) values (${body:string}, ${header.timestamp:long})
> {code}
> The interesting part is the SQL statement.  You can put anything you want in there - it will get converted to a prepared statement on execution.  The Ant-ish tokens get parsed and replaced with parameters at startup.
> The tokens are three part.  For example, in:
> {code}
> ${body:string(UTF-8)}
> {code}
> The first is a place in the event to get the value from (“body”, “header.foo”, or “custom”).  The second part ("string") is a type identifier that converts into an appropriate JDBC parameter.  The third part (“UTF-8") is a configuration string for that type, if needed.  As for types, so far I’ve defined:
> body: string (with optional charset encoding), bytearray
> header: string, long, int, float, double, date (with mandatory date format and optional timezone)
> Additionally, if none of those make you happy you can define you own parameter converters:
> {code}
> ${custom:com.company.foo.MyConverter(optionaltextconfig)}
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)