You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by narges saleh <sn...@gmail.com> on 2020/04/13 02:51:09 UTC
JDBC Connection Pooling
Hi All,
Do I need to create a JDBC connection pool for my SQL queries, in
particular the inserts via the ignite data streamer?
I am not able to find a consistent answer to this question.
thanks.
Re: JDBC Connection Pooling
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
In streaming mode, INSERTs should not block, so you may reuse the
connection. Still, it would not hurt to benchmark both approaches, if you
can.
Regards,
--
Ilya Kasnacheev
сб, 2 мая 2020 г. в 01:42, narges saleh <sn...@gmail.com>:
> Hi All,
> If I use client connection configuration to set the number of threads for
> a JDBC connection, and use the connection with multiple insert statements
> (with streaming set to true), to multiple different caches, are the inserts
> stacked because the connection is shared? Should I create multiple JDBC
> connections and switch them across the statements?
> thanks.
>
> On Thu, Apr 16, 2020 at 12:33 PM Evgenii Zhuravlev <
> e.zhuravlev.wk@gmail.com> wrote:
>
>> As I said, if you use only DataStreamer, without jdbc, just a plain,
>> key-value IgniteDataStreamer, then, you should have only one instance per
>> cache. It will give you the better performance. This one streamer can be
>> used from multiple threads.
>>
>> чт, 16 апр. 2020 г. в 09:54, narges saleh <sn...@gmail.com>:
>>
>>> I am sorry for mixing these two up.
>>> I am asking if I were to use binaryobject builder with datastreamer in
>>> place on jdbc connection would/should I create a pool of the streamer
>>> objects. From your answers, the answer seems to be yes. thank you.
>>>
>>> On Thu, Apr 16, 2020 at 9:07 AM Evgenii Zhuravlev <
>>> e.zhuravlev.wk@gmail.com> wrote:
>>>
>>>> You said that you use Binary Object Builder, so, I thought that you use
>>>> key value API and data streamers. I don't really understand now you use
>>>> BinaryObjectBuilder with thin JDBC client.
>>>>
>>>> >What if I have a persistent connection that sends data continuously?
>>>> Should I hold on the instance of the streamer (for a particular cache), or
>>>> recreate a new one once a new load of data arrives?
>>>> If you're loading data continuously, it makes sense to store data
>>>> streamer instance somewhere and just reuse it, avoiding recreating it each
>>>> time. I
>>>>
>>>> >Are you saying have the data streamed to the streamer via multiple
>>>> connections, across multiple threads?
>>>> If you use just a simple IgniteDataStreamer, you can use it from
>>>> multiple threads(use addData from multiple threads) to increase the
>>>> throughput.
>>>> Evgenii
>>>>
>>>> ср, 15 апр. 2020 г. в 12:07, narges saleh <sn...@gmail.com>:
>>>>
>>>>> Hello Evgenii,
>>>>>
>>>>> I am not sure what you mean by reuse a data streamer from multiple
>>>>> threads. I have data being constantly "streamed" to the streamer via a
>>>>> connection. Are you saying have the data streamed to the streamer via
>>>>> multiple connections, across multiple threads?
>>>>> What if I have a persistent connection that sends data continuously?
>>>>> Should I hold on the instance of the streamer (for a particular cache), or
>>>>> recreate a new one once a new load of data arrives?
>>>>>
>>>>> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev <
>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>
>>>>>> > Should I create a pool of data streamers (a few for each cache)?
>>>>>> If you use just KV API, it's better to have only one data streamer
>>>>>> per cache and reuse it from multiple threads - it will give you the best
>>>>>> performance.
>>>>>>
>>>>>> Evgenii
>>>>>>
>>>>>> ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
>>>>>>
>>>>>>> Please note that in my case, the streamers are running on the server
>>>>>>> side (as part of different services).
>>>>>>>
>>>>>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So, in effect, I'll be having a pool of streamers, right?
>>>>>>>> Would this still be the case if I am using BinaryObjectBuilder to
>>>>>>>> build objects to stream the data to a few caches? Should I create a pool of
>>>>>>>> data streamers (a few for each cache)?
>>>>>>>> I don't want to have to create a new object builder and data
>>>>>>>> streamer if I am inserting to the same cache over and over.
>>>>>>>>
>>>>>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> For each connection, on node side will be created its own
>>>>>>>>> datastreamer. I think it makes sense to try pooling for data load, but you
>>>>>>>>> will need to measure everything, since the pool size depends on the lot of
>>>>>>>>> things
>>>>>>>>>
>>>>>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Yes, Evgenii.
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>>>>>>>
>>>>>>>>>>> Evgenii
>>>>>>>>>>>
>>>>>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <snarges124@gmail.com
>>>>>>>>>>> >:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>>>>>>>
>>>>>>>>>>>> I read somewhere that jdbc datasource is available via Ignite
>>>>>>>>>>>> JDBC, (which should provide connection pooling).
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <
>>>>>>>>>>>> alexanderkor12@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> At this point you need to implement connection pooling
>>>>>>>>>>>>> yourself.
>>>>>>>>>>>>> Use
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>>>>>>>> to specify number of threads Ignite creates to service
>>>>>>>>>>>>> connection
>>>>>>>>>>>>> requests.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Each new connection will be handled by a separate thread
>>>>>>>>>>>>> inside
>>>>>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>>>>>>>
>>>>>>>>>>>>> ClientConnectorConfiguration is set inside
>>>>>>>>>>>>> IgniteConfiguration:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>>>>>>>
>>>>>>>>>>>>> More info:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks, Alex
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>>>>>
>>>>>>>>>>>>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
Hi All,
If I use client connection configuration to set the number of threads for
a JDBC connection, and use the connection with multiple insert statements
(with streaming set to true), to multiple different caches, are the inserts
stacked because the connection is shared? Should I create multiple JDBC
connections and switch them across the statements?
thanks.
On Thu, Apr 16, 2020 at 12:33 PM Evgenii Zhuravlev <e....@gmail.com>
wrote:
> As I said, if you use only DataStreamer, without jdbc, just a plain,
> key-value IgniteDataStreamer, then, you should have only one instance per
> cache. It will give you the better performance. This one streamer can be
> used from multiple threads.
>
> чт, 16 апр. 2020 г. в 09:54, narges saleh <sn...@gmail.com>:
>
>> I am sorry for mixing these two up.
>> I am asking if I were to use binaryobject builder with datastreamer in
>> place on jdbc connection would/should I create a pool of the streamer
>> objects. From your answers, the answer seems to be yes. thank you.
>>
>> On Thu, Apr 16, 2020 at 9:07 AM Evgenii Zhuravlev <
>> e.zhuravlev.wk@gmail.com> wrote:
>>
>>> You said that you use Binary Object Builder, so, I thought that you use
>>> key value API and data streamers. I don't really understand now you use
>>> BinaryObjectBuilder with thin JDBC client.
>>>
>>> >What if I have a persistent connection that sends data continuously?
>>> Should I hold on the instance of the streamer (for a particular cache), or
>>> recreate a new one once a new load of data arrives?
>>> If you're loading data continuously, it makes sense to store data
>>> streamer instance somewhere and just reuse it, avoiding recreating it each
>>> time. I
>>>
>>> >Are you saying have the data streamed to the streamer via multiple
>>> connections, across multiple threads?
>>> If you use just a simple IgniteDataStreamer, you can use it from
>>> multiple threads(use addData from multiple threads) to increase the
>>> throughput.
>>> Evgenii
>>>
>>> ср, 15 апр. 2020 г. в 12:07, narges saleh <sn...@gmail.com>:
>>>
>>>> Hello Evgenii,
>>>>
>>>> I am not sure what you mean by reuse a data streamer from multiple
>>>> threads. I have data being constantly "streamed" to the streamer via a
>>>> connection. Are you saying have the data streamed to the streamer via
>>>> multiple connections, across multiple threads?
>>>> What if I have a persistent connection that sends data continuously?
>>>> Should I hold on the instance of the streamer (for a particular cache), or
>>>> recreate a new one once a new load of data arrives?
>>>>
>>>> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev <
>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>
>>>>> > Should I create a pool of data streamers (a few for each cache)?
>>>>> If you use just KV API, it's better to have only one data streamer per
>>>>> cache and reuse it from multiple threads - it will give you the best
>>>>> performance.
>>>>>
>>>>> Evgenii
>>>>>
>>>>> ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
>>>>>
>>>>>> Please note that in my case, the streamers are running on the server
>>>>>> side (as part of different services).
>>>>>>
>>>>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> So, in effect, I'll be having a pool of streamers, right?
>>>>>>> Would this still be the case if I am using BinaryObjectBuilder to
>>>>>>> build objects to stream the data to a few caches? Should I create a pool of
>>>>>>> data streamers (a few for each cache)?
>>>>>>> I don't want to have to create a new object builder and data
>>>>>>> streamer if I am inserting to the same cache over and over.
>>>>>>>
>>>>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>
>>>>>>>> For each connection, on node side will be created its own
>>>>>>>> datastreamer. I think it makes sense to try pooling for data load, but you
>>>>>>>> will need to measure everything, since the pool size depends on the lot of
>>>>>>>> things
>>>>>>>>
>>>>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>>>>>>
>>>>>>>>> Yes, Evgenii.
>>>>>>>>>
>>>>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>>>>>>
>>>>>>>>>> Evgenii
>>>>>>>>>>
>>>>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>>>>>>
>>>>>>>>>>> I read somewhere that jdbc datasource is available via Ignite
>>>>>>>>>>> JDBC, (which should provide connection pooling).
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <
>>>>>>>>>>> alexanderkor12@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> At this point you need to implement connection pooling
>>>>>>>>>>>> yourself.
>>>>>>>>>>>> Use
>>>>>>>>>>>>
>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>>>>>>> to specify number of threads Ignite creates to service
>>>>>>>>>>>> connection
>>>>>>>>>>>> requests.
>>>>>>>>>>>>
>>>>>>>>>>>> Each new connection will be handled by a separate thread
>>>>>>>>>>>> inside
>>>>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>>>>>>
>>>>>>>>>>>> ClientConnectorConfiguration is set inside
>>>>>>>>>>>> IgniteConfiguration:
>>>>>>>>>>>>
>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>>>>>>
>>>>>>>>>>>> More info:
>>>>>>>>>>>>
>>>>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, Alex
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>>>>
>>>>>>>>>>>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
Hi
On Thu, Apr 16, 2020 at 12:33 PM Evgenii Zhuravlev <e....@gmail.com>
wrote:
> As I said, if you use only DataStreamer, without jdbc, just a plain,
> key-value IgniteDataStreamer, then, you should have only one instance per
> cache. It will give you the better performance. This one streamer can be
> used from multiple threads.
>
> чт, 16 апр. 2020 г. в 09:54, narges saleh <sn...@gmail.com>:
>
>> I am sorry for mixing these two up.
>> I am asking if I were to use binaryobject builder with datastreamer in
>> place on jdbc connection would/should I create a pool of the streamer
>> objects. From your answers, the answer seems to be yes. thank you.
>>
>> On Thu, Apr 16, 2020 at 9:07 AM Evgenii Zhuravlev <
>> e.zhuravlev.wk@gmail.com> wrote:
>>
>>> You said that you use Binary Object Builder, so, I thought that you use
>>> key value API and data streamers. I don't really understand now you use
>>> BinaryObjectBuilder with thin JDBC client.
>>>
>>> >What if I have a persistent connection that sends data continuously?
>>> Should I hold on the instance of the streamer (for a particular cache), or
>>> recreate a new one once a new load of data arrives?
>>> If you're loading data continuously, it makes sense to store data
>>> streamer instance somewhere and just reuse it, avoiding recreating it each
>>> time. I
>>>
>>> >Are you saying have the data streamed to the streamer via multiple
>>> connections, across multiple threads?
>>> If you use just a simple IgniteDataStreamer, you can use it from
>>> multiple threads(use addData from multiple threads) to increase the
>>> throughput.
>>> Evgenii
>>>
>>> ср, 15 апр. 2020 г. в 12:07, narges saleh <sn...@gmail.com>:
>>>
>>>> Hello Evgenii,
>>>>
>>>> I am not sure what you mean by reuse a data streamer from multiple
>>>> threads. I have data being constantly "streamed" to the streamer via a
>>>> connection. Are you saying have the data streamed to the streamer via
>>>> multiple connections, across multiple threads?
>>>> What if I have a persistent connection that sends data continuously?
>>>> Should I hold on the instance of the streamer (for a particular cache), or
>>>> recreate a new one once a new load of data arrives?
>>>>
>>>> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev <
>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>
>>>>> > Should I create a pool of data streamers (a few for each cache)?
>>>>> If you use just KV API, it's better to have only one data streamer per
>>>>> cache and reuse it from multiple threads - it will give you the best
>>>>> performance.
>>>>>
>>>>> Evgenii
>>>>>
>>>>> ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
>>>>>
>>>>>> Please note that in my case, the streamers are running on the server
>>>>>> side (as part of different services).
>>>>>>
>>>>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> So, in effect, I'll be having a pool of streamers, right?
>>>>>>> Would this still be the case if I am using BinaryObjectBuilder to
>>>>>>> build objects to stream the data to a few caches? Should I create a pool of
>>>>>>> data streamers (a few for each cache)?
>>>>>>> I don't want to have to create a new object builder and data
>>>>>>> streamer if I am inserting to the same cache over and over.
>>>>>>>
>>>>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>
>>>>>>>> For each connection, on node side will be created its own
>>>>>>>> datastreamer. I think it makes sense to try pooling for data load, but you
>>>>>>>> will need to measure everything, since the pool size depends on the lot of
>>>>>>>> things
>>>>>>>>
>>>>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>>>>>>
>>>>>>>>> Yes, Evgenii.
>>>>>>>>>
>>>>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>>>>>>
>>>>>>>>>> Evgenii
>>>>>>>>>>
>>>>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>>>>>>
>>>>>>>>>>> I read somewhere that jdbc datasource is available via Ignite
>>>>>>>>>>> JDBC, (which should provide connection pooling).
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <
>>>>>>>>>>> alexanderkor12@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> At this point you need to implement connection pooling
>>>>>>>>>>>> yourself.
>>>>>>>>>>>> Use
>>>>>>>>>>>>
>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>>>>>>> to specify number of threads Ignite creates to service
>>>>>>>>>>>> connection
>>>>>>>>>>>> requests.
>>>>>>>>>>>>
>>>>>>>>>>>> Each new connection will be handled by a separate thread
>>>>>>>>>>>> inside
>>>>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>>>>>>
>>>>>>>>>>>> ClientConnectorConfiguration is set inside
>>>>>>>>>>>> IgniteConfiguration:
>>>>>>>>>>>>
>>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>>>>>>
>>>>>>>>>>>> More info:
>>>>>>>>>>>>
>>>>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, Alex
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>>>>
>>>>>>>>>>>
Re: JDBC Connection Pooling
Posted by Evgenii Zhuravlev <e....@gmail.com>.
As I said, if you use only DataStreamer, without jdbc, just a plain,
key-value IgniteDataStreamer, then, you should have only one instance per
cache. It will give you the better performance. This one streamer can be
used from multiple threads.
чт, 16 апр. 2020 г. в 09:54, narges saleh <sn...@gmail.com>:
> I am sorry for mixing these two up.
> I am asking if I were to use binaryobject builder with datastreamer in
> place on jdbc connection would/should I create a pool of the streamer
> objects. From your answers, the answer seems to be yes. thank you.
>
> On Thu, Apr 16, 2020 at 9:07 AM Evgenii Zhuravlev <
> e.zhuravlev.wk@gmail.com> wrote:
>
>> You said that you use Binary Object Builder, so, I thought that you use
>> key value API and data streamers. I don't really understand now you use
>> BinaryObjectBuilder with thin JDBC client.
>>
>> >What if I have a persistent connection that sends data continuously?
>> Should I hold on the instance of the streamer (for a particular cache), or
>> recreate a new one once a new load of data arrives?
>> If you're loading data continuously, it makes sense to store data
>> streamer instance somewhere and just reuse it, avoiding recreating it each
>> time. I
>>
>> >Are you saying have the data streamed to the streamer via multiple
>> connections, across multiple threads?
>> If you use just a simple IgniteDataStreamer, you can use it from multiple
>> threads(use addData from multiple threads) to increase the throughput.
>> Evgenii
>>
>> ср, 15 апр. 2020 г. в 12:07, narges saleh <sn...@gmail.com>:
>>
>>> Hello Evgenii,
>>>
>>> I am not sure what you mean by reuse a data streamer from multiple
>>> threads. I have data being constantly "streamed" to the streamer via a
>>> connection. Are you saying have the data streamed to the streamer via
>>> multiple connections, across multiple threads?
>>> What if I have a persistent connection that sends data continuously?
>>> Should I hold on the instance of the streamer (for a particular cache), or
>>> recreate a new one once a new load of data arrives?
>>>
>>> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev <
>>> e.zhuravlev.wk@gmail.com> wrote:
>>>
>>>> > Should I create a pool of data streamers (a few for each cache)?
>>>> If you use just KV API, it's better to have only one data streamer per
>>>> cache and reuse it from multiple threads - it will give you the best
>>>> performance.
>>>>
>>>> Evgenii
>>>>
>>>> ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
>>>>
>>>>> Please note that in my case, the streamers are running on the server
>>>>> side (as part of different services).
>>>>>
>>>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> So, in effect, I'll be having a pool of streamers, right?
>>>>>> Would this still be the case if I am using BinaryObjectBuilder to
>>>>>> build objects to stream the data to a few caches? Should I create a pool of
>>>>>> data streamers (a few for each cache)?
>>>>>> I don't want to have to create a new object builder and data streamer
>>>>>> if I am inserting to the same cache over and over.
>>>>>>
>>>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>
>>>>>>> For each connection, on node side will be created its own
>>>>>>> datastreamer. I think it makes sense to try pooling for data load, but you
>>>>>>> will need to measure everything, since the pool size depends on the lot of
>>>>>>> things
>>>>>>>
>>>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>>>>>
>>>>>>>> Yes, Evgenii.
>>>>>>>>
>>>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>>>>>
>>>>>>>>> Evgenii
>>>>>>>>>
>>>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>>>>>
>>>>>>>>>> I read somewhere that jdbc datasource is available via Ignite
>>>>>>>>>> JDBC, (which should provide connection pooling).
>>>>>>>>>>
>>>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <
>>>>>>>>>> alexanderkor12@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> At this point you need to implement connection pooling
>>>>>>>>>>> yourself.
>>>>>>>>>>> Use
>>>>>>>>>>>
>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>>>>>> to specify number of threads Ignite creates to service
>>>>>>>>>>> connection
>>>>>>>>>>> requests.
>>>>>>>>>>>
>>>>>>>>>>> Each new connection will be handled by a separate thread inside
>>>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>>>>>
>>>>>>>>>>> ClientConnectorConfiguration is set inside
>>>>>>>>>>> IgniteConfiguration:
>>>>>>>>>>>
>>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>>>>>
>>>>>>>>>>> More info:
>>>>>>>>>>>
>>>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>>>>>
>>>>>>>>>>> Thanks, Alex
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>>>
>>>>>>>>>>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
I am sorry for mixing these two up.
I am asking if I were to use binaryobject builder with datastreamer in
place on jdbc connection would/should I create a pool of the streamer
objects. From your answers, the answer seems to be yes. thank you.
On Thu, Apr 16, 2020 at 9:07 AM Evgenii Zhuravlev <e....@gmail.com>
wrote:
> You said that you use Binary Object Builder, so, I thought that you use
> key value API and data streamers. I don't really understand now you use
> BinaryObjectBuilder with thin JDBC client.
>
> >What if I have a persistent connection that sends data continuously?
> Should I hold on the instance of the streamer (for a particular cache), or
> recreate a new one once a new load of data arrives?
> If you're loading data continuously, it makes sense to store data streamer
> instance somewhere and just reuse it, avoiding recreating it each time. I
>
> >Are you saying have the data streamed to the streamer via multiple
> connections, across multiple threads?
> If you use just a simple IgniteDataStreamer, you can use it from multiple
> threads(use addData from multiple threads) to increase the throughput.
> Evgenii
>
> ср, 15 апр. 2020 г. в 12:07, narges saleh <sn...@gmail.com>:
>
>> Hello Evgenii,
>>
>> I am not sure what you mean by reuse a data streamer from multiple
>> threads. I have data being constantly "streamed" to the streamer via a
>> connection. Are you saying have the data streamed to the streamer via
>> multiple connections, across multiple threads?
>> What if I have a persistent connection that sends data continuously?
>> Should I hold on the instance of the streamer (for a particular cache), or
>> recreate a new one once a new load of data arrives?
>>
>> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev <
>> e.zhuravlev.wk@gmail.com> wrote:
>>
>>> > Should I create a pool of data streamers (a few for each cache)?
>>> If you use just KV API, it's better to have only one data streamer per
>>> cache and reuse it from multiple threads - it will give you the best
>>> performance.
>>>
>>> Evgenii
>>>
>>> ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
>>>
>>>> Please note that in my case, the streamers are running on the server
>>>> side (as part of different services).
>>>>
>>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com>
>>>> wrote:
>>>>
>>>>> So, in effect, I'll be having a pool of streamers, right?
>>>>> Would this still be the case if I am using BinaryObjectBuilder to
>>>>> build objects to stream the data to a few caches? Should I create a pool of
>>>>> data streamers (a few for each cache)?
>>>>> I don't want to have to create a new object builder and data streamer
>>>>> if I am inserting to the same cache over and over.
>>>>>
>>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>
>>>>>> For each connection, on node side will be created its own
>>>>>> datastreamer. I think it makes sense to try pooling for data load, but you
>>>>>> will need to measure everything, since the pool size depends on the lot of
>>>>>> things
>>>>>>
>>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>>>>
>>>>>>> Yes, Evgenii.
>>>>>>>
>>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>>>>
>>>>>>>> Evgenii
>>>>>>>>
>>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>>>>>
>>>>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>>>>
>>>>>>>>> I read somewhere that jdbc datasource is available via Ignite
>>>>>>>>> JDBC, (which should provide connection pooling).
>>>>>>>>>
>>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <
>>>>>>>>> alexanderkor12@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> At this point you need to implement connection pooling yourself.
>>>>>>>>>> Use
>>>>>>>>>>
>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>>>>> to specify number of threads Ignite creates to service
>>>>>>>>>> connection
>>>>>>>>>> requests.
>>>>>>>>>>
>>>>>>>>>> Each new connection will be handled by a separate thread inside
>>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>>>>
>>>>>>>>>> ClientConnectorConfiguration is set inside
>>>>>>>>>> IgniteConfiguration:
>>>>>>>>>>
>>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>>>>
>>>>>>>>>> More info:
>>>>>>>>>>
>>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>>>>
>>>>>>>>>> Thanks, Alex
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>>
>>>>>>>>>
Re: JDBC Connection Pooling
Posted by Evgenii Zhuravlev <e....@gmail.com>.
You said that you use Binary Object Builder, so, I thought that you use key
value API and data streamers. I don't really understand now you use
BinaryObjectBuilder with thin JDBC client.
>What if I have a persistent connection that sends data continuously?
Should I hold on the instance of the streamer (for a particular cache), or
recreate a new one once a new load of data arrives?
If you're loading data continuously, it makes sense to store data streamer
instance somewhere and just reuse it, avoiding recreating it each time. I
>Are you saying have the data streamed to the streamer via multiple
connections, across multiple threads?
If you use just a simple IgniteDataStreamer, you can use it from multiple
threads(use addData from multiple threads) to increase the throughput.
Evgenii
ср, 15 апр. 2020 г. в 12:07, narges saleh <sn...@gmail.com>:
> Hello Evgenii,
>
> I am not sure what you mean by reuse a data streamer from multiple
> threads. I have data being constantly "streamed" to the streamer via a
> connection. Are you saying have the data streamed to the streamer via
> multiple connections, across multiple threads?
> What if I have a persistent connection that sends data continuously?
> Should I hold on the instance of the streamer (for a particular cache), or
> recreate a new one once a new load of data arrives?
>
> On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev <
> e.zhuravlev.wk@gmail.com> wrote:
>
>> > Should I create a pool of data streamers (a few for each cache)?
>> If you use just KV API, it's better to have only one data streamer per
>> cache and reuse it from multiple threads - it will give you the best
>> performance.
>>
>> Evgenii
>>
>> ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
>>
>>> Please note that in my case, the streamers are running on the server
>>> side (as part of different services).
>>>
>>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com>
>>> wrote:
>>>
>>>> So, in effect, I'll be having a pool of streamers, right?
>>>> Would this still be the case if I am using BinaryObjectBuilder to build
>>>> objects to stream the data to a few caches? Should I create a pool of data
>>>> streamers (a few for each cache)?
>>>> I don't want to have to create a new object builder and data streamer
>>>> if I am inserting to the same cache over and over.
>>>>
>>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>
>>>>> For each connection, on node side will be created its own
>>>>> datastreamer. I think it makes sense to try pooling for data load, but you
>>>>> will need to measure everything, since the pool size depends on the lot of
>>>>> things
>>>>>
>>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>>>
>>>>>> Yes, Evgenii.
>>>>>>
>>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>>>
>>>>>>> Evgenii
>>>>>>>
>>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>>>>
>>>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>>>
>>>>>>>> I read somewhere that jdbc datasource is available via Ignite JDBC,
>>>>>>>> (which should provide connection pooling).
>>>>>>>>
>>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> At this point you need to implement connection pooling yourself.
>>>>>>>>> Use
>>>>>>>>>
>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>>>> to specify number of threads Ignite creates to service connection
>>>>>>>>> requests.
>>>>>>>>>
>>>>>>>>> Each new connection will be handled by a separate thread inside
>>>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>>>
>>>>>>>>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>>>>>>>>
>>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>>>
>>>>>>>>> More info:
>>>>>>>>>
>>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>>>
>>>>>>>>> Thanks, Alex
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>>
>>>>>>>>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
Hello Evgenii,
I am not sure what you mean by reuse a data streamer from multiple
threads. I have data being constantly "streamed" to the streamer via a
connection. Are you saying have the data streamed to the streamer via
multiple connections, across multiple threads?
What if I have a persistent connection that sends data continuously? Should
I hold on the instance of the streamer (for a particular cache), or
recreate a new one once a new load of data arrives?
On Wed, Apr 15, 2020 at 1:17 PM Evgenii Zhuravlev <e....@gmail.com>
wrote:
> > Should I create a pool of data streamers (a few for each cache)?
> If you use just KV API, it's better to have only one data streamer per
> cache and reuse it from multiple threads - it will give you the best
> performance.
>
> Evgenii
>
> ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
>
>> Please note that in my case, the streamers are running on the server side
>> (as part of different services).
>>
>> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com>
>> wrote:
>>
>>> So, in effect, I'll be having a pool of streamers, right?
>>> Would this still be the case if I am using BinaryObjectBuilder to build
>>> objects to stream the data to a few caches? Should I create a pool of data
>>> streamers (a few for each cache)?
>>> I don't want to have to create a new object builder and data streamer if
>>> I am inserting to the same cache over and over.
>>>
>>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>>> e.zhuravlev.wk@gmail.com> wrote:
>>>
>>>> For each connection, on node side will be created its own datastreamer.
>>>> I think it makes sense to try pooling for data load, but you will need to
>>>> measure everything, since the pool size depends on the lot of things
>>>>
>>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>>
>>>>> Yes, Evgenii.
>>>>>
>>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>>
>>>>>> Evgenii
>>>>>>
>>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>>>
>>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>>
>>>>>>> I read somewhere that jdbc datasource is available via Ignite JDBC,
>>>>>>> (which should provide connection pooling).
>>>>>>>
>>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> At this point you need to implement connection pooling yourself.
>>>>>>>> Use
>>>>>>>>
>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>>> to specify number of threads Ignite creates to service connection
>>>>>>>> requests.
>>>>>>>>
>>>>>>>> Each new connection will be handled by a separate thread inside
>>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>>
>>>>>>>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>>>>>>>
>>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>>
>>>>>>>> More info:
>>>>>>>>
>>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>>
>>>>>>>> Thanks, Alex
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>>
>>>>>>>
Re: JDBC Connection Pooling
Posted by Evgenii Zhuravlev <e....@gmail.com>.
> Should I create a pool of data streamers (a few for each cache)?
If you use just KV API, it's better to have only one data streamer per
cache and reuse it from multiple threads - it will give you the best
performance.
Evgenii
ср, 15 апр. 2020 г. в 04:53, narges saleh <sn...@gmail.com>:
> Please note that in my case, the streamers are running on the server side
> (as part of different services).
>
> On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com> wrote:
>
>> So, in effect, I'll be having a pool of streamers, right?
>> Would this still be the case if I am using BinaryObjectBuilder to build
>> objects to stream the data to a few caches? Should I create a pool of data
>> streamers (a few for each cache)?
>> I don't want to have to create a new object builder and data streamer if
>> I am inserting to the same cache over and over.
>>
>> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
>> e.zhuravlev.wk@gmail.com> wrote:
>>
>>> For each connection, on node side will be created its own datastreamer.
>>> I think it makes sense to try pooling for data load, but you will need to
>>> measure everything, since the pool size depends on the lot of things
>>>
>>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>>
>>>> Yes, Evgenii.
>>>>
>>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>>> e.zhuravlev.wk@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>>
>>>>> Evgenii
>>>>>
>>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>>
>>>>>> Thanks Alex. I will study the links you provided.
>>>>>>
>>>>>> I read somewhere that jdbc datasource is available via Ignite JDBC,
>>>>>> (which should provide connection pooling).
>>>>>>
>>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> At this point you need to implement connection pooling yourself.
>>>>>>> Use
>>>>>>>
>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>>> to specify number of threads Ignite creates to service connection
>>>>>>> requests.
>>>>>>>
>>>>>>> Each new connection will be handled by a separate thread inside
>>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>>
>>>>>>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>>>>>>
>>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>>
>>>>>>> More info:
>>>>>>>
>>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>>
>>>>>>> Thanks, Alex
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>>
>>>>>>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
Please note that in my case, the streamers are running on the server side
(as part of different services).
On Wed, Apr 15, 2020 at 6:46 AM narges saleh <sn...@gmail.com> wrote:
> So, in effect, I'll be having a pool of streamers, right?
> Would this still be the case if I am using BinaryObjectBuilder to build
> objects to stream the data to a few caches? Should I create a pool of data
> streamers (a few for each cache)?
> I don't want to have to create a new object builder and data streamer if I
> am inserting to the same cache over and over.
>
> On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <
> e.zhuravlev.wk@gmail.com> wrote:
>
>> For each connection, on node side will be created its own datastreamer. I
>> think it makes sense to try pooling for data load, but you will need to
>> measure everything, since the pool size depends on the lot of things
>>
>> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>>
>>> Yes, Evgenii.
>>>
>>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>>> e.zhuravlev.wk@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Do you use STREAMING MODE for thin JDBC driver?
>>>>
>>>> Evgenii
>>>>
>>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>>
>>>>> Thanks Alex. I will study the links you provided.
>>>>>
>>>>> I read somewhere that jdbc datasource is available via Ignite JDBC,
>>>>> (which should provide connection pooling).
>>>>>
>>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>> At this point you need to implement connection pooling yourself.
>>>>>> Use
>>>>>>
>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>>> to specify number of threads Ignite creates to service connection
>>>>>> requests.
>>>>>>
>>>>>> Each new connection will be handled by a separate thread inside
>>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>>
>>>>>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>>>>>
>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>>
>>>>>> More info:
>>>>>>
>>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>>
>>>>>> Thanks, Alex
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>>
>>>>>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
So, in effect, I'll be having a pool of streamers, right?
Would this still be the case if I am using BinaryObjectBuilder to build
objects to stream the data to a few caches? Should I create a pool of data
streamers (a few for each cache)?
I don't want to have to create a new object builder and data streamer if I
am inserting to the same cache over and over.
On Tue, Apr 14, 2020 at 11:56 AM Evgenii Zhuravlev <e....@gmail.com>
wrote:
> For each connection, on node side will be created its own datastreamer. I
> think it makes sense to try pooling for data load, but you will need to
> measure everything, since the pool size depends on the lot of things
>
> вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
>
>> Yes, Evgenii.
>>
>> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
>> e.zhuravlev.wk@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Do you use STREAMING MODE for thin JDBC driver?
>>>
>>> Evgenii
>>>
>>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>>
>>>> Thanks Alex. I will study the links you provided.
>>>>
>>>> I read somewhere that jdbc datasource is available via Ignite JDBC,
>>>> (which should provide connection pooling).
>>>>
>>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> At this point you need to implement connection pooling yourself.
>>>>> Use
>>>>>
>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>>> to specify number of threads Ignite creates to service connection
>>>>> requests.
>>>>>
>>>>> Each new connection will be handled by a separate thread inside
>>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>>
>>>>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>>>>
>>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>>
>>>>> More info:
>>>>>
>>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>>
>>>>> Thanks, Alex
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>>
>>>>
Re: JDBC Connection Pooling
Posted by Evgenii Zhuravlev <e....@gmail.com>.
For each connection, on node side will be created its own datastreamer. I
think it makes sense to try pooling for data load, but you will need to
measure everything, since the pool size depends on the lot of things
вт, 14 апр. 2020 г. в 07:31, narges saleh <sn...@gmail.com>:
> Yes, Evgenii.
>
> On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <
> e.zhuravlev.wk@gmail.com> wrote:
>
>> Hi,
>>
>> Do you use STREAMING MODE for thin JDBC driver?
>>
>> Evgenii
>>
>> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>>
>>> Thanks Alex. I will study the links you provided.
>>>
>>> I read somewhere that jdbc datasource is available via Ignite JDBC,
>>> (which should provide connection pooling).
>>>
>>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>> At this point you need to implement connection pooling yourself.
>>>> Use
>>>>
>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>>> to specify number of threads Ignite creates to service connection
>>>> requests.
>>>>
>>>> Each new connection will be handled by a separate thread inside
>>>> Ignite(maxed out a threadPoolSize - as described above)
>>>>
>>>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>>>
>>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>>
>>>> More info:
>>>>
>>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>>
>>>> Thanks, Alex
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>>
>>>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
Yes, Evgenii.
On Mon, Apr 13, 2020 at 10:06 PM Evgenii Zhuravlev <e....@gmail.com>
wrote:
> Hi,
>
> Do you use STREAMING MODE for thin JDBC driver?
>
> Evgenii
>
> пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
>
>> Thanks Alex. I will study the links you provided.
>>
>> I read somewhere that jdbc datasource is available via Ignite JDBC,
>> (which should provide connection pooling).
>>
>> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
>> wrote:
>>
>>> Hi,
>>> At this point you need to implement connection pooling yourself.
>>> Use
>>>
>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>>> to specify number of threads Ignite creates to service connection
>>> requests.
>>>
>>> Each new connection will be handled by a separate thread inside
>>> Ignite(maxed out a threadPoolSize - as described above)
>>>
>>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>>
>>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>>
>>> More info:
>>>
>>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>>
>>> Thanks, Alex
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>
Re: JDBC Connection Pooling
Posted by Evgenii Zhuravlev <e....@gmail.com>.
Hi,
Do you use STREAMING MODE for thin JDBC driver?
Evgenii
пн, 13 апр. 2020 г. в 19:33, narges saleh <sn...@gmail.com>:
> Thanks Alex. I will study the links you provided.
>
> I read somewhere that jdbc datasource is available via Ignite JDBC, (which
> should provide connection pooling).
>
> On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com>
> wrote:
>
>> Hi,
>> At this point you need to implement connection pooling yourself.
>> Use
>>
>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
>> to specify number of threads Ignite creates to service connection
>> requests.
>>
>> Each new connection will be handled by a separate thread inside
>> Ignite(maxed out a threadPoolSize - as described above)
>>
>> ClientConnectorConfiguration is set inside IgniteConfiguration:
>>
>> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>>
>> More info:
>>
>> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>>
>> Thanks, Alex
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
Re: JDBC Connection Pooling
Posted by narges saleh <sn...@gmail.com>.
Thanks Alex. I will study the links you provided.
I read somewhere that jdbc datasource is available via Ignite JDBC, (which
should provide connection pooling).
On Mon, Apr 13, 2020 at 12:31 PM akorensh <al...@gmail.com> wrote:
> Hi,
> At this point you need to implement connection pooling yourself.
> Use
>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
> to specify number of threads Ignite creates to service connection
> requests.
>
> Each new connection will be handled by a separate thread inside
> Ignite(maxed out a threadPoolSize - as described above)
>
> ClientConnectorConfiguration is set inside IgniteConfiguration:
>
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
>
> More info:
>
> https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
>
> Thanks, Alex
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: JDBC Connection Pooling
Posted by akorensh <al...@gmail.com>.
Hi,
At this point you need to implement connection pooling yourself.
Use
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html#setThreadPoolSize-int-
to specify number of threads Ignite creates to service connection
requests.
Each new connection will be handled by a separate thread inside
Ignite(maxed out a threadPoolSize - as described above)
ClientConnectorConfiguration is set inside IgniteConfiguration:
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/IgniteConfiguration.html#setClientConnectorConfiguration-org.apache.ignite.configuration.ClientConnectorConfiguration-
More info:
https://www.gridgain.com/docs/latest/developers-guide/SQL/JDBC/jdbc-driver#cluster-configuration
Thanks, Alex
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/