You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by Parth-Brahmbhatt <gi...@git.apache.org> on 2015/03/07 03:48:14 UTC
[GitHub] storm pull request: STORM-699: storm-jdbc should support custom in...
GitHub user Parth-Brahmbhatt opened a pull request:
https://github.com/apache/storm/pull/458
STORM-699: storm-jdbc should support custom insert queries.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Parth-Brahmbhatt/incubator-storm STORM-699
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/458.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #458
----
commit 3cad1c71bc274d097bfeb6691145bbc31e8e8a91
Author: Parth Brahmbhatt <br...@gmail.com>
Date: 2015-03-07T02:46:10Z
STORM-699: storm-jdbc should support custom insert queries.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-699: storm-jdbc should support custom in...
Posted by Parth-Brahmbhatt <gi...@git.apache.org>.
Github user Parth-Brahmbhatt commented on a diff in the pull request:
https://github.com/apache/storm/pull/458#discussion_r26232904
--- Diff: external/storm-jdbc/README.md ---
@@ -13,14 +13,39 @@ public interface JdbcMapper extends Serializable {
}
```
-The `getColumns()` method defines how a storm tuple maps to a list of columns representing a row in a database.
+The `getColumns()` method defines how a storm tuple maps to a list of columns representing a row in a database.
+**The order of the returned list is important. The place holders in the supplied queries are resolved in the same order as returned list.**
+For example if the user supplied insert query is `insert into table user(user_id, user_name, create_date) values (?,?, now())` the 1st item
--- End diff --
I had an extra "table" in it, fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-699: storm-jdbc should support custom in...
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on the pull request:
https://github.com/apache/storm/pull/458#issuecomment-77799714
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-699: storm-jdbc should support custom in...
Posted by Parth-Brahmbhatt <gi...@git.apache.org>.
Github user Parth-Brahmbhatt commented on a diff in the pull request:
https://github.com/apache/storm/pull/458#discussion_r26232936
--- Diff: external/storm-jdbc/README.md ---
@@ -38,34 +63,30 @@ hikariConfigMap.put("dataSource.password","password");
String tableName = "user_details";
JdbcMapper simpleJdbcMapper = new SimpleJdbcMapper(tableName, map);
```
-The mapper initialized in the example above assumes a storm tuple has value for all the columns.
-If your storm tuple only has fields for a subset of columns i.e. if some of the columns in your table have default values
-and you want to only insert values for columns with no default values you can enforce the behavior by initializing the
-`SimpleJdbcMapper` with explicit columnschema. For example, if you have a user_details table
-`create table if not exists user_details (user_id integer, user_name varchar(100), dept_name varchar(100), create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP);`
-In this table the create_time column has a default value. To ensure only the columns with no default values are inserted
-you can initialize the `jdbcMapper` as below:
+The mapper initialized in the example above assumes a storm tuple has value for all the columns of the table you intend to insert data into and its `getColumn`
+method will return the columns in the order in which Jdbc connection instance's `connection.getMetaData().getColumns();` method returns them.
+**If you specified your own insert query to `JdbcInsertBolt` you must initialize `SimpleJdbcMapper` with explicit columnschema such that the schema has columns in the same order as your insert queries.**
+For example if your insert query is `Insert into user (user_id, user_name) values (?,?)` then your `SimpleJdbcMapper` should be initialized with the following statements:
```java
List<Column> columnSchema = Lists.newArrayList(
new Column("user_id", java.sql.Types.INTEGER),
- new Column("user_name", java.sql.Types.VARCHAR));
- JdbcMapper simpleJdbcMapper = new SimpleJdbcMapper(columnSchema);
+ new Column("user_name", java.sql.Types.VARCHAR),
--- End diff --
missed a closing bracket and ; fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-699: storm-jdbc should support custom in...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/458
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-699: storm-jdbc should support custom in...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/458#discussion_r26190228
--- Diff: external/storm-jdbc/README.md ---
@@ -38,34 +63,30 @@ hikariConfigMap.put("dataSource.password","password");
String tableName = "user_details";
JdbcMapper simpleJdbcMapper = new SimpleJdbcMapper(tableName, map);
```
-The mapper initialized in the example above assumes a storm tuple has value for all the columns.
-If your storm tuple only has fields for a subset of columns i.e. if some of the columns in your table have default values
-and you want to only insert values for columns with no default values you can enforce the behavior by initializing the
-`SimpleJdbcMapper` with explicit columnschema. For example, if you have a user_details table
-`create table if not exists user_details (user_id integer, user_name varchar(100), dept_name varchar(100), create_time TIMESTAMP DEFAULT CURRENT_TIMESTAMP);`
-In this table the create_time column has a default value. To ensure only the columns with no default values are inserted
-you can initialize the `jdbcMapper` as below:
+The mapper initialized in the example above assumes a storm tuple has value for all the columns of the table you intend to insert data into and its `getColumn`
+method will return the columns in the order in which Jdbc connection instance's `connection.getMetaData().getColumns();` method returns them.
+**If you specified your own insert query to `JdbcInsertBolt` you must initialize `SimpleJdbcMapper` with explicit columnschema such that the schema has columns in the same order as your insert queries.**
+For example if your insert query is `Insert into user (user_id, user_name) values (?,?)` then your `SimpleJdbcMapper` should be initialized with the following statements:
```java
List<Column> columnSchema = Lists.newArrayList(
new Column("user_id", java.sql.Types.INTEGER),
- new Column("user_name", java.sql.Types.VARCHAR));
- JdbcMapper simpleJdbcMapper = new SimpleJdbcMapper(columnSchema);
+ new Column("user_name", java.sql.Types.VARCHAR),
--- End diff --
Codes in comments seems broken. Could you have a look?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: STORM-699: storm-jdbc should support custom in...
Posted by HeartSaVioR <gi...@git.apache.org>.
Github user HeartSaVioR commented on a diff in the pull request:
https://github.com/apache/storm/pull/458#discussion_r26190281
--- Diff: external/storm-jdbc/README.md ---
@@ -13,14 +13,39 @@ public interface JdbcMapper extends Serializable {
}
```
-The `getColumns()` method defines how a storm tuple maps to a list of columns representing a row in a database.
+The `getColumns()` method defines how a storm tuple maps to a list of columns representing a row in a database.
+**The order of the returned list is important. The place holders in the supplied queries are resolved in the same order as returned list.**
+For example if the user supplied insert query is `insert into table user(user_id, user_name, create_date) values (?,?, now())` the 1st item
--- End diff --
Could you check query again? Seems like it's strange.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---