Loading data from Hive Table into Ignite Cache

[ravi <me...@gmail.com>]

Hi,
  We have the OLAP application which aggregates the data on configured
interval(day/week) and loads the data into hive tables(HDFS-SNAPPY-ORC File
format). We have the reporting engine which will query the data from these
tables using Hive/Spark SQL.
   Is it possible to load the data from Hive(Once it reaches to Hive Table)
to Ignite cache in a faster manner(Hive as CacheJDBCStore or using Data
Streamer API's or some other means), so that our reporting engine can run
ignite sql and get the benefit of indexing and other ignite native benefits
and respond faster?.
  We are not in favor of using IGFS and run Hive QL using Tez+LLAP+ORC. We
wanted to run native Ignite SQL.
  So please suggest any other alternate approach
  
Regards
Ravi
  



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Loading-data-from-Hive-Table-into-Ignite-Cache-tp16367.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Loading data from Hive Table into Ignite Cache

[vkulichenko <va...@gmail.com>]

Ravi,

It supports anything that can be accessed via JDBC.

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Loading-data-from-Hive-Table-into-Ignite-Cache-tp16367p16387.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Loading data from Hive Table into Ignite Cache

[ravi <me...@gmail.com>]

Hi,
 Thanks for the reply. Using Automatic RDBMS Configuration, Does ignite web
console supports loading the data from Hive Database(No SQL DB- using Hive
Server2 Driver?).
 Can you share some links if any such a sample is available?

Regards
Ravi




--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Loading-data-from-Hive-Table-into-Ignite-Cache-tp16367p16374.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Loading data from Hive Table into Ignite Cache

[vkulichenko <va...@gmail.com>]

Hi Ravi,

Sure you can. You can use both CacheStore and data streamer for this, as
discussed here: https://apacheignite.readme.io/docs/data-loading

If you can connect with JDBC driver, you can also use automatic persistence
(probably the best way for you case):
https://apacheignite-tools.readme.io/docs/automatic-rdbms-integration

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Loading-data-from-Hive-Table-into-Ignite-Cache-tp16367p16372.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.