You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/08/19 12:51:50 UTC
[GitHub] [shardingsphere] sandynz opened a new issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
sandynz opened a new issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942
## Bug Report
### Which version of ShardingSphere did you use?
5.0.0-RC1-SNAPSHOT
### Which project did you use? ShardingSphere-JDBC or ShardingSphere-Proxy?
ShardingSphere-Proxy
### Expected behavior
Sql could be executed correctly, just like MySQL 5.7 server.
### Actual behavior
Exception thrown:
```
Exception in thread "main" java.sql.SQLException: Statement parameter 5 not set.
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:965)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:898)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:887)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:861)
at com.mysql.jdbc.ServerPreparedStatement.serverExecute(ServerPreparedStatement.java:1174)
at com.mysql.jdbc.ServerPreparedStatement.executeInternal(ServerPreparedStatement.java:787)
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2133)
at com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2067)
at com.mysql.jdbc.PreparedStatement.executeLargeUpdate(PreparedStatement.java:5175)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2052)
```
### Reason analyze (If you can)
```
mysql-connector-java-5.1.48
ServerPreparedStatement.java, serverPrepare(String sql)
Buffer prepareResultPacket = mysql.sendCommand(MysqlDefs.COM_PREPARE, sql, null, false, characterEncoding, 0);
if (this.connection.versionMeetsMinimum(4, 1, 1)) {
// 4.1.1 and newer use the first byte as an 'ok' or 'error' flag, so move the buffer pointer past it to start reading the statement id.
prepareResultPacket.setPosition(1);
} else {
// 4.1.0 doesn't use the first byte as an 'ok' or 'error' flag
prepareResultPacket.setPosition(0);
}
this.serverStatementId = prepareResultPacket.readLong();
this.fieldCount = prepareResultPacket.readInt();
this.parameterCount = prepareResultPacket.readInt();
this.parameterBindings = new BindValue[this.parameterCount];
this.parameterCount is 5, which is from proxy server side, but it's 4 in fact
```
```
org.apache.shardingsphere.proxy.frontend.mysql.command.query.binary.prepare.MySQLComStmtPrepareExecutor.java, execute()
SQLStatement sqlStatement = schema.getRuntimeContext().getSqlParserEngine().parse(packet.getSql(), true);
if (!MySQLComStmtPrepareChecker.isStatementAllowed(sqlStatement)) {
result.add(new MySQLErrPacket(++currentSequenceId, MySQLServerErrorCode.ER_UNSUPPORTED_PS));
return result;
}
int parametersCount = sqlStatement.getParameterCount();
parametersCount is 5 after parsing
```
### Steps to reproduce the behavior, such as: SQL to execute, sharding rule configuration, when exception occur etc.
Run sql via jdbc:
```
jdbcUrl:
jdbc:mysql://127.0.0.1:3307/sharding_db?serverTimezone=UTC&useSSL=false&useServerPrepStmts=true&cachePrepStmts=true
try (PreparedStatement statement = connection.prepareStatement(
"insert into t_order (order_id, user_id, status) values (?, ?, ?) ON DUPLICATE KEY UPDATE status = ?")) {
statement.setInt(1, orderId);
statement.setInt(2, userId);
statement.setString(3, status);
statement.setString(4, "DUPLICATED");
statement.executeUpdate();
}
```
### Example codes for reproduce this issue (such as a github link).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] sandynz commented on issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
Posted by GitBox <gi...@apache.org>.
sandynz commented on issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942#issuecomment-677351537
>
>
> @sandynz Hi, Can you provide your table schema and sharding configuration? I have used the master branch to test, the `INSERT INTO ... ON DUPLICATE KEY UPDATE ...` statement can be executed normally.
Hi @strongduanmu , I tested it on lastest master branch again, commit: bfb1486549e9908f8784210fe2100f1c5fdd212d (2020-08-19), run `org.apache.shardingsphere.proxy.Bootstrap`, it doesn't work.
My config-sharding.yaml
```
schemaName: sharding_db
dataSourceCommon:
username: root
password: test
connectionTimeoutMilliseconds: 30000
idleTimeoutMilliseconds: 60000
maxLifetimeMilliseconds: 1800000
maxPoolSize: 50
minPoolSize: 1
maintenanceIntervalMilliseconds: 30000
dataSources:
ds:
url: jdbc:mysql://localhost:3306/demo_ds_1?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8
rules:
- !SHARDING
tables:
t_order:
actualDataNodes: ds.t_order_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_inline
keyGenerateStrategy:
column: order_id
keyGeneratorName: snowflake
t_order_item:
actualDataNodes: ds.t_order_item_${0..1}
tableStrategy:
standard:
shardingColumn: order_id
shardingAlgorithmName: t_order_item_inline
keyGenerateStrategy:
column: order_item_id
keyGeneratorName: snowflake
bindingTables:
- t_order,t_order_item
broadcastTables:
- t_address
shardingAlgorithms:
t_order_inline:
type: INLINE
props:
algorithm.expression: t_order_${order_id % 2}
t_order_item_inline:
type: INLINE
props:
algorithm.expression: t_order_item_${order_id % 2}
keyGenerators:
snowflake:
type: SNOWFLAKE
props:
worker.id: 123
```
My database:
```
mysql> use demo_ds_1
Database changed
CREATE TABLE IF NOT EXISTS t_address (address_id BIGINT NOT NULL, address_name VARCHAR(100) NOT NULL, PRIMARY KEY (address_id));
CREATE TABLE IF NOT EXISTS `t_order_0` (`order_id` INT NOT NULL, `user_id` INT NOT NULL, `status` VARCHAR(45) NULL, PRIMARY KEY (`order_id`));
CREATE TABLE IF NOT EXISTS `t_order_1` (`order_id` INT NOT NULL, `user_id` INT NOT NULL, `status` VARCHAR(45) NULL, PRIMARY KEY (`order_id`));
CREATE TABLE IF NOT EXISTS `t_order_item_0` (`order_item_id` INT NOT NULL, `order_id` INT NOT NULL, `user_id` INT NOT NULL, `status` VARCHAR(45) NULL, PRIMARY KEY (`order_item_id`));
CREATE TABLE IF NOT EXISTS `t_order_item_1` (`order_item_id` INT NOT NULL, `order_id` INT NOT NULL, `user_id` INT NOT NULL, `status` VARCHAR(45) NULL, PRIMARY KEY (`order_item_id`));
mysql> show tables;
+---------------------+
| Tables_in_demo_ds_1 |
+---------------------+
| t_address |
| t_order_0 |
| t_order_1 |
| t_order_item_0 |
| t_order_item_1 |
+---------------------+
5 rows in set (0.00 sec)
```
My client test code snippet:
```
public static void main(String[] args) throws Exception {
final boolean useServerPrepStmts = true;
try (Connection connection = DriverManager.getConnection("jdbc:mysql://127.0.0.1:3307/sharding_db?serverTimezone=UTC&useSSL=false" + (useServerPrepStmts ? "&useServerPrepStmts=true&cachePrepStmts=true" : ""), "root", "root")) {
int orderId = 1, userId = 2;
String status = "PAID";
try (PreparedStatement statement = connection.prepareStatement("insert into t_order (order_id, user_id, status) values (?, ?, ?) ON DUPLICATE KEY UPDATE status = ?")) {
statement.setInt(1, orderId);
statement.setInt(2, userId);
statement.setString(3, status);
statement.setString(4, "DUPLICATED");
statement.executeUpdate();
}
}
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu edited a comment on issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
Posted by GitBox <gi...@apache.org>.
strongduanmu edited a comment on issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942#issuecomment-677740258
@sandynz Thank you for the example, and I have reproduced this exception. The reason for this problem is that the value in the `on duplicate key` statement is parsed twice, therefore the `parametersCount` is greater than the actual value by 1.
```java
@Override
public ASTNode visitOnDuplicateKeyClause(final OnDuplicateKeyClauseContext ctx) {
Collection<AssignmentSegment> columns = new LinkedList<>();
for (AssignmentContext each : ctx.assignment()) {
columns.add((AssignmentSegment) visit(each));
visit(each.assignmentValue());
}
return new OnDuplicateKeyColumnsSegment(ctx.getStart().getStartIndex(), ctx.getStop().getStopIndex(), columns);
}
```
I will fix this problem as soon as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu edited a comment on issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
Posted by GitBox <gi...@apache.org>.
strongduanmu edited a comment on issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942#issuecomment-677740258
@sandynz Thank you for the example, and I have reproduced this exception. The reason for this problem is that the value in the `on duplicate key statement` is parsed twice, therefore the `parametersCount` is greater than the actual value by 1.
```java
@Override
public ASTNode visitOnDuplicateKeyClause(final OnDuplicateKeyClauseContext ctx) {
Collection<AssignmentSegment> columns = new LinkedList<>();
for (AssignmentContext each : ctx.assignment()) {
columns.add((AssignmentSegment) visit(each));
visit(each.assignmentValue());
}
return new OnDuplicateKeyColumnsSegment(ctx.getStart().getStartIndex(), ctx.getStop().getStopIndex(), columns);
}
```
I will fix this problem as soon as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942#issuecomment-676856813
@sandynz Hi, Can you provide your table schema and sharding configuration? I have used the master branch to test, the `INSERT INTO ... ON DUPLICATE KEY UPDATE ...` statement can be executed normally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] tristaZero closed issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
Posted by GitBox <gi...@apache.org>.
tristaZero closed issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942#issuecomment-677454763
@sandynz Thank you for your reply, I will test it again!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [shardingsphere] strongduanmu commented on issue #6942: Parameterized ON DUPLICATE KEY UPDATE in insert statement doesn't work on MySQL proxy
Posted by GitBox <gi...@apache.org>.
strongduanmu commented on issue #6942:
URL: https://github.com/apache/shardingsphere/issues/6942#issuecomment-677740258
@sandynz Thank you for the example, and I have reproduced this exception. The reason for this problem is that the value in the `on duplicate key statement` is parsed twice, therefore the `parametersCount` is greater than the actual value by 1.
I will fix this problem as soon as possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org