You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Anil <an...@gmail.com> on 2016/10/23 06:26:03 UTC
Ignite Jdbc connection
HI,
Does ignite jdbc connection is fault tolerant ? as it is distributed
cluster and any node can go down at any point of time.
and Does it support connection pool ? as each connection is starting ignite
with client mode true.
Thanks for your clarifications.
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
Thank you Manu. This is really helpful.
On 24 October 2016 at 20:07, Manu <ma...@hotmail.com> wrote:
> If you use ignite jdbc driver, to ensure that you always get a valid ignite
> instance before call a ignite operation I recommend to use a datasource
> implementation that validates connection before calls and create new ones
> otherwise.
>
> For common operations with a ignite instance, I use this method to ensure a
> *good* ignite instance and don´t waits or control reconnection... maybe
> there are some other mechanisms... but who cares? ;)
>
> public Ignite getIgnite() {
> if (this.ignite!=null){
> try{
> //ensure this ignite instance is STARTED
> and connected
> this.ignite.getOrCreateCache("default");
> }catch (IllegalStateException e){
> this.ignite=null;
> }catch (IgniteClientDisconnectedException cause) {
> this.ignite=null;
> }catch (CacheException e) {
> if (e.getCause() instanceof
> IgniteClientDisconnectedException) {
> this.ignite=null;
> }else if (e.getCause() instanceof
> IgniteClientDisconnectedCheckedException) {
> this.ignite=null;
> }else{
> throw e;
> }
> }
> }
> if (this.ignite==null){
> this.createIgniteInstance();
> }
> return ignite;
> }
>
> also you can wait for reconnection using this catch block instead of
> above... but as I said... who cares?... sometimes reconnection waits are
> not
> desirable...
> [...]
> try{
> //ensure this ignite instance is STARTED
> and connected
> this.ignite.getOrCreateCache("default");
> }catch (IllegalStateException e){
> this.ignite=null;
> }catch (IgniteClientDisconnectedException cause) {
> LOG.warn("Client disconnected from cluster.
> Waiting for reconnect...");
> cause.reconnectFuture().get(); // Wait for
> reconnect.
> }catch (CacheException e) {
> if (e.getCause() instanceof
> IgniteClientDisconnectedException) {
> LOG.warn("Client disconnected from
> cluster. Waiting for reconnect...");
> IgniteClientDisconnectedException cause =
> (IgniteClientDisconnectedException)e.getCause();
> cause.reconnectFuture().get(); // Wait for
> reconnect.
> }else if (e.getCause() instanceof
> IgniteClientDisconnectedCheckedException) {
> LOG.warn("Client disconnected from
> cluster. Waiting for reconnect...");
> IgniteClientDisconnectedCheckedException
> cause =
> (IgniteClientDisconnectedCheckedException)e.getCause();
> cause.reconnectFuture().get(); // Wait for
> reconnect.
> }else{
> throw e;
> }
> }
> [...]
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-Jdbc-connection-tp8431p8441.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
HI Andrey,
Thanks for your response. #2 answered from other answers
You are right. i created only one connection and it looks good. thanks.
On 11 November 2016 at 16:59, Andrey Gura <ag...@apache.org> wrote:
> Hi,
>
>
> 1. Ignite client node is thread-safe and you can create multiple
> statements in order to query execution. So, from my point of view, you
> should close connection when finish all your queries.
> 2. Could you please clarify your question?
> 3. I don't think that pooling is required.
> 4. Ignite client will try to reconnect to the Ignite cluster in case of
> server node fails. All you need is proper IP finder configuration.
>
>
> On Thu, Nov 10, 2016 at 5:01 PM, Anil <an...@gmail.com> wrote:
>
>> Any help in understanding below ?
>>
>> On 10 November 2016 at 16:31, Anil <an...@gmail.com> wrote:
>>
>>> I have couple of questions on ignite jdbc connection. Could you please
>>> clarify ?
>>>
>>> 1. Should connection be closed like other jdbc db connection ? - I see
>>> connection close is shutdown of ignite client node.
>>> 2. Connection objects are not getting released and all connections are
>>> busy ?
>>> 3. Connection pool is really required for ignite client ? i hope one
>>> ignite connection can handle number of queries in parallel.
>>> 4. What is the recommended configuration for ignite client to support
>>> failover ?
>>>
>>> Thanks.
>>>
>>
>>
>
Re: Ignite Jdbc connection
Posted by Andrey Gura <ag...@apache.org>.
Hi,
1. Ignite client node is thread-safe and you can create multiple statements
in order to query execution. So, from my point of view, you should close
connection when finish all your queries.
2. Could you please clarify your question?
3. I don't think that pooling is required.
4. Ignite client will try to reconnect to the Ignite cluster in case of
server node fails. All you need is proper IP finder configuration.
On Thu, Nov 10, 2016 at 5:01 PM, Anil <an...@gmail.com> wrote:
> Any help in understanding below ?
>
> On 10 November 2016 at 16:31, Anil <an...@gmail.com> wrote:
>
>> I have couple of questions on ignite jdbc connection. Could you please
>> clarify ?
>>
>> 1. Should connection be closed like other jdbc db connection ? - I see
>> connection close is shutdown of ignite client node.
>> 2. Connection objects are not getting released and all connections are
>> busy ?
>> 3. Connection pool is really required for ignite client ? i hope one
>> ignite connection can handle number of queries in parallel.
>> 4. What is the recommended configuration for ignite client to support
>> failover ?
>>
>> Thanks.
>>
>
>
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
Any help in understanding below ?
On 10 November 2016 at 16:31, Anil <an...@gmail.com> wrote:
> I have couple of questions on ignite jdbc connection. Could you please
> clarify ?
>
> 1. Should connection be closed like other jdbc db connection ? - I see
> connection close is shutdown of ignite client node.
> 2. Connection objects are not getting released and all connections are
> busy ?
> 3. Connection pool is really required for ignite client ? i hope one
> ignite connection can handle number of queries in parallel.
> 4. What is the recommended configuration for ignite client to support
> failover ?
>
> Thanks.
>
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
I have couple of questions on ignite jdbc connection. Could you please
clarify ?
1. Should connection be closed like other jdbc db connection ? - I see
connection close is shutdown of ignite client node.
2. Connection objects are not getting released and all connections are busy
?
3. Connection pool is really required for ignite client ? i hope one ignite
connection can handle number of queries in parallel.
4. What is the recommended configuration for ignite client to support
failover ?
Thanks.
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
i see it is because of client timeout. and has been resolved. thanks.
On 6 November 2016 at 00:36, Anil <an...@gmail.com> wrote:
> Hi Manu and All,
>
> Ignite jdbc connection is very slow for very first time even with data
> source.
>
> Consecutive queries are very fast. queries with 1 mins time duration
> becoming slow.
>
> PoolProperties p = new PoolProperties();
> p.setUrl("jdbc:ignite:cfg://cache=TEST_CACHE@file:" +
> System.getProperties().getProperty("ignite.config.file"));
> p.setDriverClassName("org.apache.ignite.IgniteJdbcDriver");
> p.setMaxActive(20);
> p.setInitialSize(5);
> p.setMaxWait(5000);
> p.setMinIdle(5);
> p.setMaxIdle(10);
> p.setTestOnBorrow(true);
> p.setTestWhileIdle(true);
> p.setTestOnReturn(true);
> p.setTimeBetweenEvictionRunsMillis(60000);
> p.setMinEvictableIdleTimeMillis(120000);
> p.setMaxAge(1500000);
> p.setRemoveAbandoned(true);
> p.setRemoveAbandonedTimeout(300);
> p.setLogAbandoned(true);
> p.setFairQueue(true);
> p.setValidationQuery("select count(*) from \"TEST_CACHE\".Person
> limit 1");
> p.setValidationInterval(3000);
> ds = new DataSource();
> ds.setPoolProperties(p);
> anyone facing the similar problem ?
>
> Thanks.
>
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
Hi Manu and All,
Ignite jdbc connection is very slow for very first time even with data
source.
Consecutive queries are very fast. queries with 1 mins time duration
becoming slow.
PoolProperties p = new PoolProperties();
p.setUrl("jdbc:ignite:cfg://cache=TEST_CACHE@file:" +
System.getProperties().getProperty("ignite.config.file"));
p.setDriverClassName("org.apache.ignite.IgniteJdbcDriver");
p.setMaxActive(20);
p.setInitialSize(5);
p.setMaxWait(5000);
p.setMinIdle(5);
p.setMaxIdle(10);
p.setTestOnBorrow(true);
p.setTestWhileIdle(true);
p.setTestOnReturn(true);
p.setTimeBetweenEvictionRunsMillis(60000);
p.setMinEvictableIdleTimeMillis(120000);
p.setMaxAge(1500000);
p.setRemoveAbandoned(true);
p.setRemoveAbandonedTimeout(300);
p.setLogAbandoned(true);
p.setFairQueue(true);
p.setValidationQuery("select count(*) from \"TEST_CACHE\".Person
limit 1");
p.setValidationInterval(3000);
ds = new DataSource();
ds.setPoolProperties(p);
anyone facing the similar problem ?
Thanks.
Re: Ignite Jdbc connection
Posted by Manu <ma...@hotmail.com>.
If you use ignite jdbc driver, to ensure that you always get a valid ignite
instance before call a ignite operation I recommend to use a datasource
implementation that validates connection before calls and create new ones
otherwise.
For common operations with a ignite instance, I use this method to ensure a
*good* ignite instance and don´t waits or control reconnection... maybe
there are some other mechanisms... but who cares? ;)
public Ignite getIgnite() {
if (this.ignite!=null){
try{
//ensure this ignite instance is STARTED and connected
this.ignite.getOrCreateCache("default");
}catch (IllegalStateException e){
this.ignite=null;
}catch (IgniteClientDisconnectedException cause) {
this.ignite=null;
}catch (CacheException e) {
if (e.getCause() instanceof IgniteClientDisconnectedException) {
this.ignite=null;
}else if (e.getCause() instanceof
IgniteClientDisconnectedCheckedException) {
this.ignite=null;
}else{
throw e;
}
}
}
if (this.ignite==null){
this.createIgniteInstance();
}
return ignite;
}
also you can wait for reconnection using this catch block instead of
above... but as I said... who cares?... sometimes reconnection waits are not
desirable...
[...]
try{
//ensure this ignite instance is STARTED and connected
this.ignite.getOrCreateCache("default");
}catch (IllegalStateException e){
this.ignite=null;
}catch (IgniteClientDisconnectedException cause) {
LOG.warn("Client disconnected from cluster. Waiting for reconnect...");
cause.reconnectFuture().get(); // Wait for reconnect.
}catch (CacheException e) {
if (e.getCause() instanceof IgniteClientDisconnectedException) {
LOG.warn("Client disconnected from cluster. Waiting for reconnect...");
IgniteClientDisconnectedException cause =
(IgniteClientDisconnectedException)e.getCause();
cause.reconnectFuture().get(); // Wait for reconnect.
}else if (e.getCause() instanceof
IgniteClientDisconnectedCheckedException) {
LOG.warn("Client disconnected from cluster. Waiting for reconnect...");
IgniteClientDisconnectedCheckedException cause =
(IgniteClientDisconnectedCheckedException)e.getCause();
cause.reconnectFuture().get(); // Wait for reconnect.
}else{
throw e;
}
}
[...]
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Jdbc-connection-tp8431p8441.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Jdbc connection
Posted by Manu <ma...@hotmail.com>.
You are right, if connection is closed due to cluster *client* node
disconnection, client will automatically recreate connection using discovery
configuration. Pool is also supported, but N pooled instances of
org.apache.ignite.internal.jdbc2.JdbcConnection for same url on same java VM
will use same and unique ignite instance...
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Jdbc-connection-tp8431p8440.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
typo correction.
Thanks Manu.
>
> if i understand it correctly, if connection is closed due to cluster node
> failure, client will automatically recreate connection using discovery
> configuration.
>
> and *jdbc connection does support connection pool*.
>
> thanks for your help.
>
>
>
>
>
> On 24 October 2016 at 18:12, Manu <ma...@hotmail.com> wrote:
>
>> Hi,
>>
>> as you know, org.apache.ignite.internal.jdbc2.JdbcConnection is an
>> implementation of java.sql.Connection, works always on client mode (this
>> flag is hardcoded to true when load xml configuration passed on connection
>> url) and works on read mode (only select). On same java VM instance,
>> connection (ignite instance) is cached internally in JdbcConnection by
>> url,
>> so for same connection (type, path, collocation...) you only have (and
>> need)
>> one ignite instance. For more info check this
>> https://apacheignite.readme.io/docs/jdbc-driver
>> <https://apacheignite.readme.io/docs/jdbc-driver>
>>
>> As a java.sql.Connection, you could use a javax.sql.DataSource
>> implementation to manage it and checks connection status (validation
>> query)
>> etc, but you don't need a pool, for example:
>>
>> <spring:bean id="collocatedDbcpIgniteDataGridDataSource"
>> destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
>> <spring:property name="driverClassName"
>> value="org.apache.ignite.IgniteJdbcDriver"/>
>> <spring:property name="url"
>> value="jdbc:ignite:cfg://cache=default:collocated=true:local=false@ignite
>> /data_grid/ignite-client.xml"/>
>> <spring:property name="validationQuery" value="select
>> count(*) from
>> "default".string limit 1"/>
>> <spring:property name="removeAbandoned" value="true"/>
>> <spring:property name="removeAbandonedTimeout"
>> value="300"/>
>> <spring:property name="logAbandoned" value="true"/>
>> </spring:bean>
>>
>>
>> [...]
>> This is client ignite configuration with default cache (dummy, without
>> data,
>> only used to validate client connection) used on url of
>> collocatedDbcpIgniteDataGridDataSource
>>
>> <bean id="ignite.client.default.cfg.jdbc"
>> class="org.apache.ignite.configuration.IgniteConfiguration">
>> <property name="clientMode" value="true" />
>> <property name="peerClassLoadingEnabled" value="false" />
>> <property name="metricsLogFrequency" value="0"/>
>> </property>
>> <property name="cacheConfiguration">
>> <list>
>> <bean class="org.apache.ignite.confi
>> guration.CacheConfiguration">
>> <property name="name"
>> value="<b>default*" />
>> <property name="cacheMode"
>> value="PARTITIONED" />
>> <property name="indexedTypes">
>> <array>
>>
>> <value>java.lang.String</value>
>>
>> <value>java.lang.String</value>
>> </array>
>> </property>
>> </bean>
>> [...]
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/Ignite-Jdbc-connection-tp8431p8436.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
Re: Ignite Jdbc connection
Posted by Anil <an...@gmail.com>.
Thanks Manu.
if i understand it correctly, if connection is closed due to cluster node
failure, client will automatically recreate connection using discovery
configuration.
and jdbc connection does not support connection pool.
thanks for your help.
On 24 October 2016 at 18:12, Manu <ma...@hotmail.com> wrote:
> Hi,
>
> as you know, org.apache.ignite.internal.jdbc2.JdbcConnection is an
> implementation of java.sql.Connection, works always on client mode (this
> flag is hardcoded to true when load xml configuration passed on connection
> url) and works on read mode (only select). On same java VM instance,
> connection (ignite instance) is cached internally in JdbcConnection by url,
> so for same connection (type, path, collocation...) you only have (and
> need)
> one ignite instance. For more info check this
> https://apacheignite.readme.io/docs/jdbc-driver
> <https://apacheignite.readme.io/docs/jdbc-driver>
>
> As a java.sql.Connection, you could use a javax.sql.DataSource
> implementation to manage it and checks connection status (validation query)
> etc, but you don't need a pool, for example:
>
> <spring:bean id="collocatedDbcpIgniteDataGridDataSource"
> destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
> <spring:property name="driverClassName"
> value="org.apache.ignite.IgniteJdbcDriver"/>
> <spring:property name="url"
> value="jdbc:ignite:cfg://cache=default:collocated=true:local=false@ignite
> /data_grid/ignite-client.xml"/>
> <spring:property name="validationQuery" value="select
> count(*) from
> "default".string limit 1"/>
> <spring:property name="removeAbandoned" value="true"/>
> <spring:property name="removeAbandonedTimeout"
> value="300"/>
> <spring:property name="logAbandoned" value="true"/>
> </spring:bean>
>
>
> [...]
> This is client ignite configuration with default cache (dummy, without
> data,
> only used to validate client connection) used on url of
> collocatedDbcpIgniteDataGridDataSource
>
> <bean id="ignite.client.default.cfg.jdbc"
> class="org.apache.ignite.configuration.IgniteConfiguration">
> <property name="clientMode" value="true" />
> <property name="peerClassLoadingEnabled" value="false" />
> <property name="metricsLogFrequency" value="0"/>
> </property>
> <property name="cacheConfiguration">
> <list>
> <bean class="org.apache.ignite.
> configuration.CacheConfiguration">
> <property name="name"
> value="<b>default*" />
> <property name="cacheMode"
> value="PARTITIONED" />
> <property name="indexedTypes">
> <array>
>
> <value>java.lang.String</value>
>
> <value>java.lang.String</value>
> </array>
> </property>
> </bean>
> [...]
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Ignite-Jdbc-connection-tp8431p8436.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: Ignite Jdbc connection
Posted by Manu <ma...@hotmail.com>.
Hi,
as you know, org.apache.ignite.internal.jdbc2.JdbcConnection is an
implementation of java.sql.Connection, works always on client mode (this
flag is hardcoded to true when load xml configuration passed on connection
url) and works on read mode (only select). On same java VM instance,
connection (ignite instance) is cached internally in JdbcConnection by url,
so for same connection (type, path, collocation...) you only have (and need)
one ignite instance. For more info check this
https://apacheignite.readme.io/docs/jdbc-driver
<https://apacheignite.readme.io/docs/jdbc-driver>
As a java.sql.Connection, you could use a javax.sql.DataSource
implementation to manage it and checks connection status (validation query)
etc, but you don't need a pool, for example:
<spring:bean id="collocatedDbcpIgniteDataGridDataSource"
destroy-method="close" class="org.apache.commons.dbcp.BasicDataSource">
<spring:property name="driverClassName"
value="org.apache.ignite.IgniteJdbcDriver"/>
<spring:property name="url"
value="jdbc:ignite:cfg://cache=default:collocated=true:local=false@ignite/data_grid/ignite-client.xml"/>
<spring:property name="validationQuery" value="select count(*) from
"default".string limit 1"/>
<spring:property name="removeAbandoned" value="true"/>
<spring:property name="removeAbandonedTimeout" value="300"/>
<spring:property name="logAbandoned" value="true"/>
</spring:bean>
[...]
This is client ignite configuration with default cache (dummy, without data,
only used to validate client connection) used on url of
collocatedDbcpIgniteDataGridDataSource
<bean id="ignite.client.default.cfg.jdbc"
class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="clientMode" value="true" />
<property name="peerClassLoadingEnabled" value="false" />
<property name="metricsLogFrequency" value="0"/>
</property>
<property name="cacheConfiguration">
<list>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<property name="name" value="<b>default*" />
<property name="cacheMode" value="PARTITIONED" />
<property name="indexedTypes">
<array>
<value>java.lang.String</value>
<value>java.lang.String</value>
</array>
</property>
</bean>
[...]
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Ignite-Jdbc-connection-tp8431p8436.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.