Multi data able joining

[Ravisankar Mani <rr...@gmail.com>]

Hi All,

I need some clarification about some workflow cases using apache NiFi tool.
Please find the following scenarios.





Get multi table from several datasource and to form a single table



I have 3 tables from different data sources , one from MySQL, one from REST
API and another from Excel. I have collected a tables from the different
data sources and need to move a single table(all three into one) to SQL
server

1.In this case, I did move to different tables to SQL server(mean separate
table for each datasource table). But I need to merge into one(Join the
multi tables) and move to a single table

2.Is there any default processors available to join multiple tables?



Regards,

Ravisankar

Re: Multi data able joining

[Bryan Bende <bb...@gmail.com>]

Hello,

I'm not sure if this will work depending how large the tables are, but
since you were already able to move to the data into three separate
tables...

Could you then do an ExecuteSQL processor that used a select query that
joined the three tables together, so the results coming out of the DB are
already joined? then insert those results into a new table?

Joining data streams is usually done in stream processing framework like
Flink, Spark streaming, or Storm.

Another option might be to prepare the data in NiFi and deliver it to one
of those stream processing systems,
do the join there and send the results back to NiFi to be inserted into the
new table.

Thanks,

Bryan


On Wed, May 4, 2016 at 3:22 AM, Ravisankar Mani <rr...@gmail.com> wrote:

> Hi All,
>
> I need some clarification about some workflow cases using apache NiFi
> tool. Please find the following scenarios.
>
>
>
>
>
> Get multi table from several datasource and to form a single table
>
>
>
> I have 3 tables from different data sources , one from MySQL, one from
> REST API and another from Excel. I have collected a tables from the
> different data sources and need to move a single table(all three into one)
> to SQL server
>
> 1.In this case, I did move to different tables to SQL server(mean separate
> table for each datasource table). But I need to merge into one(Join the
> multi tables) and move to a single table
>
> 2.Is there any default processors available to join multiple tables?
>
>
>
> Regards,
>
> Ravisankar
>
>