You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Labard <da...@at-consulting.ru> on 2016/08/01 12:45:10 UTC

Re: Load data from Hadoop

Hi Vladimir,

I have remote data source which produces files with data once a day. I load
this data to hadoop cluster.
I must provide this data to users, and also I need to process this data for
a long period (90 days from current moment) and make cache with integrated
data.

Users want to get raw and processed data by SQL queries to Ignite.

I will be glad to know any thoughts or information on my questions.

Labard.




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Load-data-from-Hadoop-tp6544p6655.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Load data from Hadoop

Posted by Jörn Franke <jo...@gmail.com>.
Hmm this would require more details. You can, for example, use ignite as a HDFS cache for hive and hive (minimum 1.2) +tez +Orc as the SQL layer. This is probably one of the most fastest way currently available. However, this depends on your use case. 

> On 01 Aug 2016, at 14:45, Labard <da...@at-consulting.ru> wrote:
> 
> Hi Vladimir,
> 
> I have remote data source which produces files with data once a day. I load
> this data to hadoop cluster.
> I must provide this data to users, and also I need to process this data for
> a long period (90 days from current moment) and make cache with integrated
> data.
> 
> Users want to get raw and processed data by SQL queries to Ignite.
> 
> I will be glad to know any thoughts or information on my questions.
> 
> Labard.
> 
> 
> 
> 
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Load-data-from-Hadoop-tp6544p6655.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.