You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Clay Teahouse <cl...@gmail.com> on 2019/02/19 03:49:06 UTC

Populating tables via IgniteDataStreamer

Hello All,

Can someone share an example of utilizing IgniteDataStreamer  API to
populate a table created via a DDL?

thank you,
Clay

Re: Populating tables via IgniteDataStreamer

Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!

I don't think there is this particular example but you can frankenstain it
from existing ones. If you have any troubles then you can share your
project for us to see.

Otherwise you can indeed use SET STREAMING and avoid trouble.

There is extra time and space considerations when using secondary indexes
but otherwise there is no overhead.

Regards,
-- 
Ilya Kasnacheev


вт, 19 февр. 2019 г. в 13:44, Clay Teahouse <cl...@gmail.com>:

> Thank you for the replies.
> Any example, with using a cache template for creating the tables and using
> the streamer API to load the table?
> Also, one more thing: is there a performance penalty for using the
> relational structure imposed on the data, as opposed to manipulating the
> java objects directly? If no, what is the most optimized way of populating
> and accessing the tables?
>
> thank you again
> Clay
>
> On Tue, Feb 19, 2019 at 3:11 AM Stephen Darlington <
> stephen.darlington@gridgain.com> wrote:
>
>> Ignite comes with a nice sample for using the Data Streamer API:
>> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/streaming/StreamTransformerExample.java
>>
>>
>> There’s nothing really “special” about caches/tables created using DDL
>> rather than one of the other methods.
>>
>> Regards,
>> Stephen
>>
>> On 19 Feb 2019, at 03:49, Clay Teahouse <cl...@gmail.com> wrote:
>>
>> Hello All,
>>
>> Can someone share an example of utilizing IgniteDataStreamer  API to
>> populate a table created via a DDL?
>>
>> thank you,
>> Clay
>>
>>
>>
>>

Re: Populating tables via IgniteDataStreamer

Posted by Clay Teahouse <cl...@gmail.com>.
Thank you for the replies.
Any example, with using a cache template for creating the tables and using
the streamer API to load the table?
Also, one more thing: is there a performance penalty for using the
relational structure imposed on the data, as opposed to manipulating the
java objects directly? If no, what is the most optimized way of populating
and accessing the tables?

thank you again
Clay

On Tue, Feb 19, 2019 at 3:11 AM Stephen Darlington <
stephen.darlington@gridgain.com> wrote:

> Ignite comes with a nice sample for using the Data Streamer API:
> https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/streaming/StreamTransformerExample.java
>
>
> There’s nothing really “special” about caches/tables created using DDL
> rather than one of the other methods.
>
> Regards,
> Stephen
>
> On 19 Feb 2019, at 03:49, Clay Teahouse <cl...@gmail.com> wrote:
>
> Hello All,
>
> Can someone share an example of utilizing IgniteDataStreamer  API to
> populate a table created via a DDL?
>
> thank you,
> Clay
>
>
>
>

Re: Populating tables via IgniteDataStreamer

Posted by Stephen Darlington <st...@gridgain.com>.
Ignite comes with a nice sample for using the Data Streamer API: https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/streaming/StreamTransformerExample.java <https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/streaming/StreamTransformerExample.java> 

There’s nothing really “special” about caches/tables created using DDL rather than one of the other methods.

Regards,
Stephen

> On 19 Feb 2019, at 03:49, Clay Teahouse <cl...@gmail.com> wrote:
> 
> Hello All,
> 
> Can someone share an example of utilizing IgniteDataStreamer  API to populate a table created via a DDL?
> 
> thank you,
> Clay
>

Re: Populating tables via IgniteDataStreamer

Posted by joseheitor <jo...@heitorprojects.com>.
Hi Clay,

With JDBC, I have used the following successfully:

  dbConnection = HikariCPDatasource.getConnection();
  dbConnection.setSchema("public");

  *dbConnection.createStatement().execute("ALTER TABLE tablename
NOLOGGING");*
  *dbConnection.createStatement().execute("SET STREAMING ON");*

  for(...) { ** execute insert statements ** }

  *dbConnection.createStatement().execute("SET STREAMING OFF");*
  *dbConnection.createStatement().execute("ALTER TABLE tablename LOGGING");*

  dbConnection.close();

Hope this helps,
Jose



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/