You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Labard <da...@at-consulting.ru> on 2016/08/01 12:45:10 UTC
Re: Load data from Hadoop
Hi Vladimir,
I have remote data source which produces files with data once a day. I load
this data to hadoop cluster.
I must provide this data to users, and also I need to process this data for
a long period (90 days from current moment) and make cache with integrated
data.
Users want to get raw and processed data by SQL queries to Ignite.
I will be glad to know any thoughts or information on my questions.
Labard.
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Load-data-from-Hadoop-tp6544p6655.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Load data from Hadoop
Posted by Jörn Franke <jo...@gmail.com>.
Hmm this would require more details. You can, for example, use ignite as a HDFS cache for hive and hive (minimum 1.2) +tez +Orc as the SQL layer. This is probably one of the most fastest way currently available. However, this depends on your use case.
> On 01 Aug 2016, at 14:45, Labard <da...@at-consulting.ru> wrote:
>
> Hi Vladimir,
>
> I have remote data source which produces files with data once a day. I load
> this data to hadoop cluster.
> I must provide this data to users, and also I need to process this data for
> a long period (90 days from current moment) and make cache with integrated
> data.
>
> Users want to get raw and processed data by SQL queries to Ignite.
>
> I will be glad to know any thoughts or information on my questions.
>
> Labard.
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Load-data-from-Hadoop-tp6544p6655.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.