You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "makssent (via GitHub)" <gi...@apache.org> on 2026/10/29 13:28:36 UTC
[I] Firebird Proxy closes the connection after a batch message without an open batch [shardingsphere]
makssent opened a new issue, #40399:
URL: https://github.com/apache/shardingsphere/issues/40399
### Before reporting a bug
- [x] I have searched open and closed [GitHub issues](https://github.com/apache/shardingsphere/issues).
- [x] I have read the [ShardingSphere documentation](https://shardingsphere.apache.org/document/current/en/overview).
### ShardingSphere version
master, commit [ab8624a369dac2a9a84762a24e1f078a57ee75e6](https://github.com/apache/shardingsphere/commit/ab8624a369dac2a9a84762a24e1f078a57ee75e6)
### ShardingSphere project
ShardingSphere-Proxy
### Expected behavior
After rejecting a message sent without an open batch, the proxy should keep the connection and transaction usable, as the Firebird server does.
### Actual behavior
After a batch is released, sending a separate [`op_batch_msg`](https://github.com/FirebirdSQL/firebird/blob/v5.0.4/src/remote/server/server.cpp#L3667-L3682) returns `isc_bad_batch_handle` on both sides, but the proxy closes the connection and loses the uncommitted transaction.
| Step | Firebird server | Proxy |
|---|---|---|
| Create and release a batch | Succeeds | Succeeds |
| Send message ID 701 without a batch | `335545159 isc_bad_batch_handle`, invalid batch handle | `335545159 isc_bad_batch_handle`, invalid batch handle |
| Synchronize the batch | Succeeds | No response, connection closed |
| Execute a new batch with ID 702 and insert ID 709 | Succeeds | Cannot execute |
| Commit and read the rows directly | Rows 702 and 709 are saved | The transaction is lost and neither row is saved |
### Reason analysis
- The server checks for an open batch in [`batch_msg`](https://github.com/FirebirdSQL/firebird/blob/v5.0.4/src/remote/server/server.cpp#L3667-L3682) and retains the connection after the error in this scenario.
- To determine the message length, [`FirebirdBatchMessageCommandPacket#getLength`](https://github.com/apache/shardingsphere/blob/ab8624a369dac2a9a84762a24e1f078a57ee75e6/database/protocol/dialect/firebird/src/main/java/org/apache/shardingsphere/database/protocol/firebird/packet/command/query/batch/FirebirdBatchMessageCommandPacket.java#L67-L76) looks up the batch and throws an invalid-handle exception when it is absent. [`FirebirdPacketCodecEngine#decode`](https://github.com/apache/shardingsphere/blob/ab8624a369dac2a9a84762a24e1f078a57ee75e6/database/protocol/dialect/firebird/src/main/java/org/apache/shardingsphere/database/protocol/firebird/codec/FirebirdPacketCodecEngine.java#L78-L86) sends the error and closes the channel.
### Steps to reproduce
#### Environment
- A Firebird 5 compatible server.
- Client: Jaybird 6.0.7.
#### Steps
1. Create the table on the database server:
```sql
CREATE TABLE T_PLAIN (ID INTEGER NOT NULL PRIMARY KEY, NAME VARCHAR(20));
```
2. Start ShardingSphere-Proxy with this configuration:
```yaml
# global.yaml
authority:
users:
- user: root@%
password: root
privilege:
type: ALL_PERMITTED
props:
proxy-frontend-database-protocol-type: Firebird
```
```yaml
# database-firebird.yaml
databaseName: firebird_db
dataSources:
ds_0:
url: jdbc:firebirdsql://<host>/<database file>
username: <user>
password: <password>
rules:
- !SINGLE
tables:
- "*.*"
```
3. Set the system properties `server.url`, `server.user`, `server.password`, `proxy.url`, `proxy.user`, and `proxy.password` to the respective connection values. Run the example once with the `server` argument and once with the `proxy` argument. Each run deletes IDs 700 through 709 before starting the transaction. Compare the responses and rows read directly from the server.
The example creates a batch with BLR `05 02 04 00 02 00 08 00 07 00 ff 4c`, message length 6, and parameter buffer `01 02 04 00 00 00 01 00 00 00`, then releases it before sending the message.
### Example code
```java
import org.firebirdsql.gds.ISCConstants;
import org.firebirdsql.gds.JaybirdErrorCodes;
import org.firebirdsql.gds.TransactionParameterBuffer;
import org.firebirdsql.gds.impl.wire.XdrInputStream;
import org.firebirdsql.gds.impl.wire.XdrOutputStream;
import org.firebirdsql.gds.ng.FbExceptionBuilder;
import org.firebirdsql.gds.ng.FbStatement;
import org.firebirdsql.gds.ng.FbTransaction;
import org.firebirdsql.gds.ng.LockCloseable;
import org.firebirdsql.gds.ng.fields.RowValue;
import org.firebirdsql.gds.ng.wire.FbWireDatabase;
import org.firebirdsql.gds.ng.wire.TransmitAction;
import org.firebirdsql.jaybird.fb.constants.TpbItems;
import org.firebirdsql.jdbc.FirebirdConnection;
import java.io.ByteArrayOutputStream;
import java.io.EOFException;
import java.io.IOException;
import java.lang.reflect.Field;
import java.net.SocketException;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.ArrayList;
import java.util.HexFormat;
import java.util.List;
import java.util.Properties;
public class BatchMsgWithoutBatch {
static final String SERVER_URL = "" + System.getProperty("server.url", "<server JDBC URL>");
static final String PROXY_URL = System.getProperty("proxy.url", "<proxy JDBC URL>");
static final String BLR_INT = "05020400020008000700ff4c";
static final String PB = "01020400000001000000";
static boolean closed;
public static void main(String[] args) throws Exception {
java.util.logging.LogManager.getLogManager().reset();
boolean server = 0 < args.length && "server".equals(args[0]);
Properties props = new Properties();
props.setProperty("user", server ? System.getProperty("server.user", "<server user>") : System.getProperty("proxy.user", "<proxy user>"));
props.setProperty("password", server ? System.getProperty("server.password", "<server password>") : System.getProperty("proxy.password", "<proxy password>"));
props.setProperty("encoding", "UTF8");
props.setProperty("wireCrypt", "DISABLED");
try (Connection connection = DriverManager.getConnection(server ? SERVER_URL : PROXY_URL, props)) {
FbWireDatabase db = (FbWireDatabase) connection.unwrap(FirebirdConnection.class).getFbDatabase();
run(connection, db);
} catch (SQLException | IOException ex) {
System.out.println("connection -> " + error(ex));
}
readDirectly("SELECT ID, NAME FROM T_PLAIN WHERE ID BETWEEN 700 AND 709 ORDER BY ID");
}
static void run(Connection connection, FbWireDatabase db) throws Exception {
execute(connection, "DELETE FROM T_PLAIN WHERE ID BETWEEN 700 AND 709");
FbTransaction tx = db.startTransaction(tpb(db));
FbStatement statement = db.createStatement(tx);
statement.prepare("INSERT INTO T_PLAIN(ID) VALUES (?)");
int h = statement.getHandle();
send(db, out -> {
batchCreate(out, h, BLR_INT, 6, PB);
sync(out);
});
System.out.println("op_batch_create (create batch) -> " + read(db));
System.out.println("op_batch_sync -> " + read(db));
send(db, out -> {
handleOp(out, 102, h); // op_batch_rls
sync(out);
});
System.out.println("op_batch_rls (release batch) -> " + read(db));
System.out.println("op_batch_sync -> " + read(db));
send(db, out -> {
batchMsg(out, h, 1, xdr(0, 701));
sync(out);
});
System.out.println("op_batch_msg (message ID = 701 without batch) -> " + read(db));
System.out.println("op_batch_sync -> " + read(db));
send(db, out -> {
batchCreate(out, h, BLR_INT, 6, PB);
batchMsg(out, h, 1, xdr(0, 702));
batchExec(out, h, tx.getHandle());
});
System.out.println("op_batch_create (new batch) -> " + read(db));
System.out.println("op_batch_msg (message ID = 702) -> " + read(db));
System.out.println("op_batch_exec (execute batch) -> " + read(db));
send(db, out -> {
handleOp(out, 102, h); // op_batch_rls
sync(out);
});
System.out.println("op_batch_rls (release batch) -> " + read(db));
System.out.println("op_batch_sync -> " + read(db));
step("close statement", statement::close);
step("INSERT INTO T_PLAIN(ID, NAME) VALUES (709, 'after')", () -> {
FbStatement insert = db.createStatement(tx);
insert.prepare("INSERT INTO T_PLAIN(ID, NAME) VALUES (709, 'after')");
insert.execute(RowValue.EMPTY_ROW_VALUE);
insert.close();
});
step("COMMIT", tx::commit);
}
static TransactionParameterBuffer tpb(FbWireDatabase db) {
TransactionParameterBuffer result = db.createTransactionParameterBuffer();
result.addArgument(TpbItems.isc_tpb_read_committed);
result.addArgument(TpbItems.isc_tpb_rec_version);
result.addArgument(TpbItems.isc_tpb_write);
result.addArgument(TpbItems.isc_tpb_nowait);
return result;
}
static void execute(Connection connection, String sql) throws SQLException {
try (Statement statement = connection.createStatement()) {
statement.executeUpdate(sql);
}
}
static void send(FbWireDatabase db, TransmitAction action) throws SQLException, IOException {
if (closed) {
throw new EOFException("connection closed");
}
try (LockCloseable ignored = db.withLock()) {
db.completeDeferredActions();
db.getXdrStreamAccess().withTransmitLock(out -> {
action.transmit(out);
out.flush();
});
}
}
interface Action {
void run() throws Exception;
}
static void step(String label, Action action) {
try {
action.run();
System.out.println(label + " -> success");
} catch (Exception ex) {
System.out.println(label + " -> " + error(ex));
}
}
record Response(int op, int object, long blobId, byte[] data, String text) {
@Override
public String toString() {
return text;
}
}
static Response read(FbWireDatabase db) throws SQLException, IOException {
if (closed) {
return new Response(-1, 0, 0, new byte[0], "no response, connection closed");
}
try {
XdrInputStream in = db.getXdrStreamAccess().getXdrIn();
int op;
do {
op = in.readInt();
} while (op == 71); // op_dummy
if (op == 103) { // op_batch_cs
return new Response(op, 0, 0, new byte[0], batchCompletion(in));
}
if (op != 9) {
return new Response(op, 0, 0, new byte[0], "packet " + op + " instead of op_response");
}
int object = in.readInt();
long blobId = in.readLong();
byte[] data = in.readBuffer();
return new Response(op, object, blobId, data, status(in));
} catch (EOFException | SocketException ex) {
closed = true;
return new Response(-1, 0, 0, new byte[0], "no response, connection closed");
}
}
static String batchCompletion(XdrInputStream in) throws IOException {
in.readInt(); // p_batch_statement
int records = in.readInt();
int updates = in.readInt();
int vectors = in.readInt();
int errors = in.readInt();
List<String> counts = new ArrayList<>();
for (int i = 0; i < updates; i++) {
counts.add(String.valueOf(in.readInt()));
}
StringBuilder result = new StringBuilder("op_batch_cs: records " + records + ", counts " + compact(counts) + ", vectors " + vectors + ", errors without vector " + errors);
int first = -1;
int last = -1;
String previous = null;
for (int i = 0; i < vectors; i++) {
int record = in.readInt();
String text = status(in);
if (text.equals(previous) && record == last + 1) {
last = record;
continue;
}
appendVectors(result, first, last, previous);
first = record;
last = record;
previous = text;
}
appendVectors(result, first, last, previous);
List<String> plain = new ArrayList<>();
for (int i = 0; i < errors; i++) {
plain.add(String.valueOf(in.readInt()));
}
if (!plain.isEmpty()) {
result.append("; without vector: records ").append(compact(plain));
}
return result.toString();
}
static void appendVectors(StringBuilder result, int first, int last, String text) {
if (text != null) {
result.append(first == last ? "; record " + first : "; records " + first + "-" + last).append(": ").append(text);
}
}
static String compact(List<String> values) {
List<String> result = new ArrayList<>();
for (int i = 0; i < values.size(); ) {
int j = i;
while (j < values.size() && values.get(j).equals(values.get(i))) {
j++;
}
result.add(j - i > 2 ? values.get(i) + " x" + (j - i) : String.join(", ", values.subList(i, j)));
i = j;
}
return "[" + String.join(", ", result) + "]";
}
static String status(XdrInputStream in) throws IOException {
FbExceptionBuilder builder = new FbExceptionBuilder();
List<String> codes = new ArrayList<>();
List<String> items = new ArrayList<>();
for (int arg = in.readInt(); arg != 0; arg = in.readInt()) { // isc_arg_end
switch (arg) {
case 1 -> {
int code = in.readInt();
if (code != 0) {
builder.exception(code);
codes.add(code + " " + name(code));
items.add("gds " + code);
}
}
case 18 -> { // isc_arg_warning
int code = in.readInt();
if (code != 0) {
builder.warning(code);
codes.add("warning " + code + " " + name(code));
items.add("warning " + code);
}
}
case 4 -> { // isc_arg_number
int value = in.readInt();
builder.messageParameter(value);
items.add("number " + value);
}
case 19 -> { // isc_arg_sql_state
String value = new String(in.readBuffer(), StandardCharsets.UTF_8);
builder.sqlState(value);
items.add("sqlstate " + value);
}
default -> {
String value = new String(in.readBuffer(), StandardCharsets.UTF_8);
builder.messageParameter(value);
items.add("string '" + value.replace("\n", "\\n") + "'");
}
}
}
if (codes.isEmpty()) {
return "success";
}
String message = String.valueOf(builder.toFlatSQLException().getMessage());
int end = message.indexOf(" [SQLState:");
return "error " + String.join(", ", codes) + " '" + (end < 0 ? message : message.substring(0, end)).replace("\n", " ") + "' " + items;
}
static String error(Exception ex) {
if (ex instanceof EOFException || ex instanceof SocketException || ex.getCause() instanceof EOFException || ex.getCause() instanceof SocketException) {
return "connection closed";
}
if (ex instanceof SQLException sql) {
String message = String.valueOf(sql.getMessage()).split("\n")[0];
int end = message.indexOf(" [SQLState:");
String name = name(sql.getErrorCode());
return "error " + sql.getErrorCode() + (name.isEmpty() ? "" : " " + name) + " '" + (end < 0 ? message : message.substring(0, end)) + "'";
}
return "error " + ex;
}
static String name(int code) {
if (code == 0) {
return "";
}
for (Class<?> each : new Class<?>[]{ISCConstants.class, JaybirdErrorCodes.class}) {
for (Field field : each.getFields()) {
try {
if (field.getType() == int.class && field.getInt(null) == code && (field.getName().startsWith("isc_") || field.getName().startsWith("jb_"))) {
return field.getName();
}
} catch (IllegalAccessException ignored) {
}
}
}
return "(name unknown)";
}
static byte[] xdr(Object... values) {
ByteArrayOutputStream result = new ByteArrayOutputStream();
for (Object each : values) {
byte[] bytes;
if (each instanceof Integer value) {
bytes = new byte[]{(byte) (value >>> 24), (byte) (value >>> 16), (byte) (value >>> 8), (byte) (int) value};
} else if (each instanceof Long value) {
bytes = new byte[8];
for (int i = 0; i < 8; i++) {
bytes[i] = (byte) (value >>> (56 - 8 * i));
}
} else if (each instanceof String value) {
byte[] text = value.getBytes(StandardCharsets.UTF_8);
result.writeBytes(xdr(text.length));
bytes = text;
} else {
bytes = (byte[]) each;
}
result.writeBytes(bytes);
result.writeBytes(new byte[(4 - bytes.length % 4) % 4]);
}
return result.toByteArray();
}
static void batchCreate(XdrOutputStream out, int h, String blr, int messageLength, String pb) throws IOException {
out.writeInt(99); // op_batch_create
out.writeInt(h);
out.writeBuffer(HexFormat.of().parseHex(blr));
out.writeInt(messageLength);
out.writeBuffer(HexFormat.of().parseHex(pb));
}
static void batchMsg(XdrOutputStream out, int h, int count, byte[] messages) throws IOException {
out.writeInt(100); // op_batch_msg
out.writeInt(h);
out.writeInt(count);
out.write(messages, 0, messages.length);
}
static void batchExec(XdrOutputStream out, int h, int transaction) throws IOException {
out.writeInt(101); // op_batch_exec
out.writeInt(h);
out.writeInt(transaction);
}
static void handleOp(XdrOutputStream out, int op, int h) throws IOException {
out.writeInt(op);
out.writeInt(h);
}
static void sync(XdrOutputStream out) throws IOException {
out.writeInt(110); // op_batch_sync
}
static void readDirectly(String sql) {
Properties props = new Properties();
props.setProperty("user", System.getProperty("server.user", "<server user>"));
props.setProperty("password", System.getProperty("server.password", "<server password>"));
props.setProperty("encoding", "UTF8");
props.setProperty("wireCrypt", "DISABLED");
props.setProperty("maxInlineBlobSize", "0");
List<String> rows = new ArrayList<>();
try (Connection connection = DriverManager.getConnection(SERVER_URL, props);
Statement statement = connection.createStatement();
ResultSet rs = statement.executeQuery(sql)) {
int columns = rs.getMetaData().getColumnCount();
while (rs.next()) {
List<String> row = new ArrayList<>();
for (int i = 1; i <= columns; i++) {
Object value = rs.getObject(i);
row.add(value == null ? "NULL" : value instanceof byte[] bytes ? HexFormat.of().formatHex(bytes) : value instanceof String text ? "'" + text + "'" : String.valueOf(value));
}
rows.add(String.join(" ", row));
}
System.out.println("rows read directly from server -> " + (rows.isEmpty() ? "none" : String.join("; ", rows)));
} catch (SQLException ex) {
System.out.println("rows read directly from server -> " + error(ex));
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org