You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Jacob Johansen <jo...@gmail.com> on 2016/06/08 13:49:43 UTC
Re: Too many connections error while using storm-JDBC
In the HikariCPConnectionProvider you need to set the connection pool
smaller or in your MySQL DB configure, increase the number of
available connections.
Jacob Johansen
On Wed, Jun 8, 2016 at 6:06 AM, Alex Chew <al...@msn.com> wrote:
> Hi all,
>
>
> I got Too Many Connections error while running my topology in test
> environment. I am checking below codes,
>
>
> //prepare JDBC configuration
> Map jdbcConfigMap = Maps.newHashMap();
> jdbcConfigMap.put("dataSourceClassName",
> prop.getProperty("mysql.dataSource.className"));//com.mysql.jdbc.jdbc2.optional.MysqlDataSource
> jdbcConfigMap.put("dataSource.url",
> prop.getProperty("mysql.url"));//jdbc:mysql://localhost/test
> jdbcConfigMap.put("dataSource.user",
> prop.getProperty("mysql.user"));//root
> jdbcConfigMap.put("dataSource.password",
> prop.getProperty("mysql.password"));//password
> config.put(JDBC_CONF, jdbcConfigMap);
> this.connectionProvider = new
> HikariCPConnectionProvider(jdbcConfigMap);
>
> This will create a new HikariCPConnectionProvider each Spout/Bolt task. By
> default each connectionProvider will initial 10 connections. Totally there
> have 48 tasks. This will create 480 connections while MySQL max-conenctions
> is 400. I am going to change this, but I don't know how to force all tasks
> running on my 4 supervisor nodes to use the same dataSource?
>
>
> Thanks in advance.
>
> Alex
>
>
>
回复: Too many connections error while using storm-JDBC
Posted by Alex Chew <al...@msn.com>.
Thanks a lot. because topologies are running in seperated JVMs I'll let them use different datasources. I think I can change DB pool configuration to reduce connections for a single dataProvider. I can also let those tasks running in the same JVM share one datasource.
Best Regards,
Alex
-------- 原始邮件 --------
主题：Re: Too many connections error while using storm-JDBC
发件人：Alberto S鉶 Marcos
收件人：user@storm.apache.org
抄送：
I dont think is possible to use the same datasource across topologies/bolts
how would u share such a resource across multiple jvms and threads?
On Wed, Jun 8, 2016 at 2:49 PM, Jacob Johansen <jo...@gmail.com>> wrote:
In the HikariCPConnectionProvider you need to set the connection pool
smaller or in your MySQL DB configure, increase the number of
available connections.
Jacob Johansen
On Wed, Jun 8, 2016 at 6:06 AM, Alex Chew <al...@msn.com>> wrote:
> Hi all,
>
>
> I got Too Many Connections error while running my topology in test
> environment. I am checking below codes,
>
>
> //prepare JDBC configuration
> Map jdbcConfigMap = Maps.newHashMap();
> jdbcConfigMap.put("dataSourceClassName",
> prop.getProperty("mysql.dataSource.className"));//com.mysql.jdbc.jdbc2.optional.MysqlDataSource
> jdbcConfigMap.put("dataSource.url",
> prop.getProperty("mysql.url"));//jdbc:mysql://localhost/test
> jdbcConfigMap.put("dataSource.user",
> prop.getProperty("mysql.user"));//root
> jdbcConfigMap.put("dataSource.password",
> prop.getProperty("mysql.password"));//password
> config.put(JDBC_CONF, jdbcConfigMap);
> this.connectionProvider = new
> HikariCPConnectionProvider(jdbcConfigMap);
>
> This will create a new HikariCPConnectionProvider each Spout/Bolt task. By
> default each connectionProvider will initial 10 connections. Totally there
> have 48 tasks. This will create 480 connections while MySQL max-conenctions
> is 400. I am going to change this, but I don't know how to force all tasks
> running on my 4 supervisor nodes to use the same dataSource?
>
>
> Thanks in advance.
>
> Alex
>
>
>
Re: Too many connections error while using storm-JDBC
Posted by Alberto São Marcos <al...@gmail.com>.
I dont think is possible to use the same datasource across topologies/bolts
how would u share such a resource across multiple jvms and threads?
On Wed, Jun 8, 2016 at 2:49 PM, Jacob Johansen <jo...@gmail.com>
wrote:
> In the HikariCPConnectionProvider you need to set the connection pool
> smaller or in your MySQL DB configure, increase the number of
> available connections.
> Jacob Johansen
>
>
> On Wed, Jun 8, 2016 at 6:06 AM, Alex Chew <al...@msn.com> wrote:
> > Hi all,
> >
> >
> > I got Too Many Connections error while running my topology in test
> > environment. I am checking below codes,
> >
> >
> > //prepare JDBC configuration
> > Map jdbcConfigMap = Maps.newHashMap();
> > jdbcConfigMap.put("dataSourceClassName",
> >
> prop.getProperty("mysql.dataSource.className"));//com.mysql.jdbc.jdbc2.optional.MysqlDataSource
> > jdbcConfigMap.put("dataSource.url",
> > prop.getProperty("mysql.url"));//jdbc:mysql://localhost/test
> > jdbcConfigMap.put("dataSource.user",
> > prop.getProperty("mysql.user"));//root
> > jdbcConfigMap.put("dataSource.password",
> > prop.getProperty("mysql.password"));//password
> > config.put(JDBC_CONF, jdbcConfigMap);
> > this.connectionProvider = new
> > HikariCPConnectionProvider(jdbcConfigMap);
> >
> > This will create a new HikariCPConnectionProvider each Spout/Bolt task.
> By
> > default each connectionProvider will initial 10 connections. Totally
> there
> > have 48 tasks. This will create 480 connections while MySQL
> max-conenctions
> > is 400. I am going to change this, but I don't know how to force all
> tasks
> > running on my 4 supervisor nodes to use the same dataSource?
> >
> >
> > Thanks in advance.
> >
> > Alex
> >
> >
> >
>