You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "cswangzheng (via GitHub)" <gi...@apache.org> on 2023/03/29 01:57:25 UTC
[GitHub] [incubator-seatunnel] cswangzheng opened a new pull request, #4439: [Bug][Connector][JDBC] jdbc source partition column support decimal type
cswangzheng opened a new pull request, #4439:
URL: https://github.com/apache/incubator-seatunnel/pull/4439
To resolve first issue in https://github.com/apache/incubator-seatunnel/issues/4437
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] EricJoy2048 commented on a diff in pull request #4439: [Bug][Connector][JDBC] jdbc source partition column support decimal type
Posted by "EricJoy2048 (via GitHub)" <gi...@apache.org>.
EricJoy2048 commented on code in PR #4439:
URL: https://github.com/apache/incubator-seatunnel/pull/4439#discussion_r1155069239
##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSource.java:
##########
@@ -236,6 +237,6 @@ private PartitionParameter initPartitionParameterAndExtendSql(Connection connect
}
private boolean isNumericType(SeaTunnelDataType<?> type) {
- return type.equals(BasicType.INT_TYPE) || type.equals(BasicType.LONG_TYPE);
+ return type.equals(BasicType.INT_TYPE) || type.equals(BasicType.LONG_TYPE) || type instanceof DecimalType;
Review Comment:
> DecimalType.equals(type) and decimal may cause some exception for spilt, because how solve the float data?
I agree with you. The `max` and `min` type is long. It may case exception.
```
long max = Long.MAX_VALUE;
long min = Long.MIN_VALUE;
if (jdbcSourceConfig.getPartitionLowerBound().isPresent()
&& jdbcSourceConfig.getPartitionUpperBound().isPresent()) {
max = jdbcSourceConfig.getPartitionUpperBound().get();
min = jdbcSourceConfig.getPartitionLowerBound().get();
return new PartitionParameter(
columnName, min, max, jdbcSourceConfig.getPartitionNumber().orElse(null));
}
try (ResultSet rs =
connection
.createStatement()
.executeQuery(
String.format(
"SELECT MAX(%s),MIN(%s) " + "FROM (%s) tt",
columnName, columnName, query))) {
if (rs.next()) {
max =
jdbcSourceConfig.getPartitionUpperBound().isPresent()
? jdbcSourceConfig.getPartitionUpperBound().get()
: Long.parseLong(rs.getString(1));
min =
jdbcSourceConfig.getPartitionLowerBound().isPresent()
? jdbcSourceConfig.getPartitionLowerBound().get()
: Long.parseLong(rs.getString(2));
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] cswangzheng commented on a diff in pull request #4439: [Bug][Connector][JDBC] jdbc source partition column support decimal type
Posted by "cswangzheng (via GitHub)" <gi...@apache.org>.
cswangzheng commented on code in PR #4439:
URL: https://github.com/apache/incubator-seatunnel/pull/4439#discussion_r1160396644
##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSource.java:
##########
@@ -236,6 +237,6 @@ private PartitionParameter initPartitionParameterAndExtendSql(Connection connect
}
private boolean isNumericType(SeaTunnelDataType<?> type) {
- return type.equals(BasicType.INT_TYPE) || type.equals(BasicType.LONG_TYPE);
+ return type.equals(BasicType.INT_TYPE) || type.equals(BasicType.LONG_TYPE) || type instanceof DecimalType;
Review Comment:
but in most case, like in Oracle, the default number type's precision is 38， which will map to Decimal type in seatunnel. It should be better to throw exception when init split param than reject split in the first time. isNumericType will also throw exception, but it is much earlier.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] laglangyue commented on a diff in pull request #4439: [Bug][Connector][JDBC] jdbc source partition column support decimal type
Posted by "laglangyue (via GitHub)" <gi...@apache.org>.
laglangyue commented on code in PR #4439:
URL: https://github.com/apache/incubator-seatunnel/pull/4439#discussion_r1153396524
##########
seatunnel-connectors-v2/connector-jdbc/src/main/java/org/apache/seatunnel/connectors/seatunnel/jdbc/source/JdbcSource.java:
##########
@@ -236,6 +237,6 @@ private PartitionParameter initPartitionParameterAndExtendSql(Connection connect
}
private boolean isNumericType(SeaTunnelDataType<?> type) {
- return type.equals(BasicType.INT_TYPE) || type.equals(BasicType.LONG_TYPE);
+ return type.equals(BasicType.INT_TYPE) || type.equals(BasicType.LONG_TYPE) || type instanceof DecimalType;
Review Comment:
DecimalType.equals(type)
and decimal may cause some exception for spilt, because how solve the float data?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] cswangzheng closed pull request #4439: [Bug][Connector][JDBC] jdbc source partition column support decimal type
Posted by "cswangzheng (via GitHub)" <gi...@apache.org>.
cswangzheng closed pull request #4439: [Bug][Connector][JDBC] jdbc source partition column support decimal type
URL: https://github.com/apache/incubator-seatunnel/pull/4439
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org