[hdfs2 producer] dynamically defined path to a target file

[Mikhail Pryakhin <m....@gmail.com>]

Hi guys!

I'm currently working with HDFS2 producer and I've got a case when I need to dynamically specify path to a target file.
Let's say my path to a destination file consists of two parts: The first one is a static path and the second one is dynamically defined path which depends on realtime processing context.
I know that it is possible to setup a file name dynamically by specifying the FILE_NAME header value. This option almost works for me but at the same time it introduces another problem - setting up this header makes hdfs producer create a new file on each write. I propose to use FILE_PATH header to be able to dynamically specify path to the file. I can create a patch and tests for this case. What do you think about this?

Regards,
Mikhail

Re: [hdfs2 producer] dynamically defined path to a target file

[Grzegorz Grzybek <gr...@gmail.com>]

Hi Mikhail

Great - we love contributions - please send a patch or even create Pull
Request at https://github.com/apache/camel/pulls

I can check this as well.

best regards
Grzegorz Grzybek

2015-05-26 13:05 GMT+02:00 Mikhail Pryakhin <m....@gmail.com>:

> Hi guys!
>
> I'm currently working with HDFS2 producer and I've got a case when I need
> to dynamically specify path to a target file.
> Let's say my path to a destination file consists of two parts: The first
> one is a static path and the second one is dynamically defined path which
> depends on realtime processing context.
> I know that it is possible to setup a file name dynamically by specifying
> the FILE_NAME header value. This option almost works for me but at the same
> time it introduces another problem - setting up this header makes hdfs
> producer create a new file on each write. I propose to use FILE_PATH header
> to be able to dynamically specify path to the file. I can create a patch
> and tests for this case. What do you think about this?
>
> Regards,
> Mikhail
>
>
>
>
>
>
>