You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "ShenFeng312 (via GitHub)" <gi...@apache.org> on 2025/08/20 12:10:26 UTC
[PR] binary support [shardingsphere]
ShenFeng312 opened a new pull request, #36372:
URL: https://github.com/apache/shardingsphere/pull/36372
Fixes #ISSUSE_ID.
Changes proposed in this pull request:
-
---
Before committing this PR, I'm sure that I have checked the following options:
- [ ] My code follows the [code of conduct](https://shardingsphere.apache.org/community/en/involved/conduct/code/) of this project.
- [ ] I have self-reviewed the commit code.
- [ ] I have (or in comment I request) added corresponding labels for the pull request.
- [ ] I have passed maven check locally : `./mvnw clean install -B -T1C -Dmaven.javadoc.skip -Dmaven.jacoco.skip -e`.
- [ ] I have made corresponding changes to the documentation.
- [ ] I have added corresponding unit tests for my changes.
- [ ] I have updated the Release Notes of the current development version. For more details, see [Update Release Note](https://shardingsphere.apache.org/community/en/involved/contribute/contributor/)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "RaigorJiang (via GitHub)" <gi...@apache.org>.
RaigorJiang commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3575032058
> But please note, your implementation in #32848 is incorrect. PG's `text[]` does not represent just a one-dimensional `text[]`; it can be multi-dimensional.
@ShenFeng312 Okay, I will test it based on the information you provided. Thank you for your prompt feedback.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "RaigorJiang (via GitHub)" <gi...@apache.org>.
RaigorJiang commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3243558703
Hi @ShenFeng312 , thank you for your great work. Your thoroughness is evident in this pull request.
However, the code contains many irregularities, and some of the logic is complex and difficult to read, so it cannot be merged at this time.
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on code in PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#discussion_r2328956489
##########
database/protocol/dialect/postgresql/src/main/java/org/postgresql/jdbc/ShardingSpherePgArrayUtils.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.postgresql.jdbc;
+
+import lombok.SneakyThrows;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.PostgreSQLArrayParameterDecoder;
+import org.apache.shardingsphere.infra.exception.generic.UnsupportedSQLOperationException;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.encoder.ArrayEncoding;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.encoder.ArrayEncoder;
+import org.postgresql.core.BaseConnection;
+import org.postgresql.core.Oid;
+
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.Charset;
+import java.sql.SQLException;
+
+public class ShardingSpherePgArrayUtils {
+
+ /**
+ * encode pgArray.
+ * @param array PgArrayObject
+ * @param charset charset
+ * @return binary bytes array
+ * @throws UnsupportedSQLOperationException if array is not PgArray
+ */
+ @SneakyThrows(SQLException.class)
+ public static byte[] getBinaryBytes(final Object array, final Charset charset) {
+ PgArray pgArray;
+ if (array instanceof PgArray) {
+ pgArray = (PgArray) array;
+ } else {
+ throw new UnsupportedSQLOperationException("can not encode" + array.getClass());
+ }
+ byte[] bytes = pgArray.toBytes();
+
+ if (bytes != null) {
+ return bytes;
+ }
+ BaseConnection connection = pgArray.connection;
+
+ assert connection != null;
+ int pgType = connection.getTypeInfo().getPGArrayType(pgArray.getBaseTypeName());
+
+ Object realArray;
+ if (pgType == Oid.NUMERIC_ARRAY) {
+ realArray = new PostgreSQLArrayParameterDecoder().decodeNumberArray(pgArray.fieldString);
+ } else {
+ realArray = pgArray.getArray();
+ }
+
+ final ArrayEncoder arraySupport = ArrayEncoding.getArrayEncoder(realArray);
+ ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+ arraySupport.toBinaryRepresentation(realArray, pgType, byteArrayOutputStream, charset);
+ return byteArrayOutputStream.toByteArray();
+
Review Comment:
fixed
##########
database/protocol/dialect/postgresql/src/main/java/org/postgresql/jdbc/ShardingSpherePgArrayUtils.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.postgresql.jdbc;
Review Comment:
fixed
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLArrayBinaryProtocolValue.java:
##########
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol;
+
+import org.apache.shardingsphere.database.protocol.postgresql.payload.PostgreSQLPacketPayload;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.decoder.PgBinaryObj;
+import org.postgresql.core.Oid;
+import org.postgresql.jdbc.ShardingSpherePgArrayUtils;
+
+import java.nio.ByteBuffer;
+import java.util.HashMap;
+import java.util.Map;
+
+public final class PostgreSQLArrayBinaryProtocolValue implements PostgreSQLBinaryProtocolValue {
+
+ public static final PostgreSQLArrayBinaryProtocolValue INSTANCE = new PostgreSQLArrayBinaryProtocolValue();
+
+ private static final Map<Integer, String> OID_TYPE_NAME = new HashMap<>();
+
+ static {
+ OID_TYPE_NAME.put(Oid.BOOL, "bool[]");
+ OID_TYPE_NAME.put(Oid.BYTEA, "bytea[]");
+ // oidTypeName.put(Oid.CHAR_ARRAY, "char[]");
+ // oidTypeName.put(Oid.NAME_ARRAY, "name[]");
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "RaigorJiang (via GitHub)" <gi...@apache.org>.
RaigorJiang commented on code in PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#discussion_r2553969984
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLStringBinaryProtocolValue.java:
##########
@@ -26,25 +26,26 @@ public final class PostgreSQLStringBinaryProtocolValue implements PostgreSQLBina
@Override
public int getColumnLength(final PostgreSQLPacketPayload payload, final Object value) {
- if (value instanceof byte[]) {
- return ((byte[]) value).length;
- }
- return value.toString().getBytes(payload.getCharset()).length;
+ return -1;
Review Comment:
Why is ColumnLength a fixed value of -1? This is incorrect.
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/util/codec/decoder/ArrayDecoder.java:
##########
@@ -0,0 +1,55 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.decoder;
+
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+
+/**
+ * postgresql array decoder.
Review Comment:
Specialized terms and sentences should all be capitalized.
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/util/codec/decoder/BooleanArrayDecoder.java:
##########
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.decoder;
+
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+
+/**
+ * BooleanArrayDecoder.
Review Comment:
Words in the comments should be separated, not camelCase.
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLTimeStampBinaryProtocolValue.java:
##########
@@ -17,30 +17,34 @@
package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.PostgreSQLBinaryTimestampUtils;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.decoder.PgBinaryObj;
import org.apache.shardingsphere.database.protocol.postgresql.payload.PostgreSQLPacketPayload;
-import org.apache.shardingsphere.infra.exception.generic.UnsupportedSQLOperationException;
+
+import java.sql.Timestamp;
/**
- * Binary protocol value for boolean array for PostgreSQL.
+ * Binary protocol value for time for PostgreSQL.
*/
-public final class PostgreSQLBoolArrayBinaryProtocolValue implements PostgreSQLBinaryProtocolValue {
-
- private static final PostgreSQLArrayParameterDecoder ARRAY_PARAMETER_DECODER = new PostgreSQLArrayParameterDecoder();
+public final class PostgreSQLTimeStampBinaryProtocolValue implements PostgreSQLBinaryProtocolValue {
@Override
public int getColumnLength(final PostgreSQLPacketPayload payload, final Object value) {
- throw new UnsupportedSQLOperationException("PostgreSQLBoolArrayBinaryProtocolValue.getColumnLength()");
+ return 8;
}
@Override
public Object read(final PostgreSQLPacketPayload payload, final int parameterValueLength) {
- byte[] bytes = new byte[parameterValueLength];
+ // 获取pg 时间戳
Review Comment:
Chinese comments are not accepted.
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLStringBinaryProtocolValue.java:
##########
@@ -26,25 +26,26 @@ public final class PostgreSQLStringBinaryProtocolValue implements PostgreSQLBina
@Override
public int getColumnLength(final PostgreSQLPacketPayload payload, final Object value) {
- if (value instanceof byte[]) {
- return ((byte[]) value).length;
- }
- return value.toString().getBytes(payload.getCharset()).length;
+ return -1;
}
@Override
public Object read(final PostgreSQLPacketPayload payload, final int parameterValueLength) {
byte[] result = new byte[parameterValueLength];
payload.getByteBuf().readBytes(result);
- return new String(result);
+ return new String(result, payload.getCharset());
}
@Override
public void write(final PostgreSQLPacketPayload payload, final Object value) {
if (value instanceof byte[]) {
+ payload.writeInt4(((byte[]) value).length);
+ // we may convert charset?
payload.writeBytes((byte[]) value);
} else {
- payload.writeStringEOF(value.toString());
+ byte[] bytes = value.toString().getBytes(payload.getCharset());
+ payload.writeInt4(bytes.length);
Review Comment:
PostgreSQL DataRowPacket has a unified write length operation (based on getColumnLength()), so individual value implementations do not need to call writeInt4.
This is clearly incorrect logic.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3574333076
> I will provide a PR for reference later. (like https://github.com/apache/shardingsphere/pull/32848)
Thank you. If you can resubmit and improve it, that would of course be the best. The original intention of submitting this PR was to demonstrate how to correctly solve the serialization and deserialization issues. Feel free to use the code in this PR and make modifications anywhere. I have no intention of insisting that you must use my code.@RaigorJiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "RaigorJiang (via GitHub)" <gi...@apache.org>.
RaigorJiang commented on code in PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#discussion_r2314755812
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLArrayParameterDecoder.java:
##########
@@ -17,174 +17,242 @@
package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol;
-import com.google.common.base.Preconditions;
-import org.apache.shardingsphere.infra.exception.ShardingSpherePreconditions;
-import org.apache.shardingsphere.infra.exception.generic.UnsupportedSQLOperationException;
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.postgresql.core.Parser;
+import org.postgresql.util.internal.Nullness;
-import java.nio.charset.StandardCharsets;
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.stream.Collectors;
+import java.lang.reflect.Array;
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.List;
/**
* PostgreSQL array parameter decoder.
*/
public final class PostgreSQLArrayParameterDecoder {
- private static final String[] EMPTY_STRING_ARRAY = new String[0];
-
/**
- * Decode int2 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * decode numberArray String to a array.
+ * @param parameterValue arrayString
+ * @return array
*/
- public short[] decodeInt2Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- short[] result = new short[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Short.parseShort(each);
+ public Object decodeNumberArray(final String parameterValue) {
+
+ PgDimensionsArrayList list = decodeFromString(parameterValue, ',');
+ int dims = list.dimensionsCount;
+ final int[] dimensionLengths = new int[dims];
+ dimensionLengths[0] = list.size();
+ List tmpList = (List) list.get(0);
+ for (int i = 1; i < dims; i++) {
+ dimensionLengths[i] = Nullness.castNonNull(tmpList, "first element of adjustedList is null").size();
+ if (i != dims - 1) {
+ tmpList = (List) tmpList.get(0);
+ }
}
- return result;
- }
-
- /**
- * Decode int4 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
- */
- public int[] decodeInt4Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- int[] result = new int[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Integer.parseInt(each);
+ Object[] array = (Object[]) Array.newInstance(Number.class, dimensionLengths);
+ if (array instanceof Number[]) {
+ parserNumber((Number[]) array, list);
+ } else {
+ storeStringValues(array, list, dimensionLengths, 0);
}
- return result;
+ return array;
}
/**
- * Decode int8 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * decode and copy to result.
+ * @param array result
+ * @param list source String array
+ * @param dimensionLengths dimensionLengths
+ * @param dim dim
*/
- public long[] decodeInt8Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- long[] result = new long[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Long.parseLong(each);
+ private static void storeStringValues(final Object[] array, final List list, final int[] dimensionLengths,
+ final int dim) {
+
+ for (int i = 0; i < dimensionLengths[dim]; i++) {
+ Object element = Nullness.castNonNull(list.get(i), "list.get(i)");
+ if (dim == dimensionLengths.length - 2) {
+ parserNumber((Number[]) array[i], (List) element);
+ } else {
+ storeStringValues((Object[]) array[i], (List) element, dimensionLengths, dim + 1);
+ }
}
- return result;
}
/**
- * Decode float4 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * parse a String list to Number[].
+ * @param target Number[]
+ * @param source String list
*/
- public float[] decodeFloat4Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- float[] result = new float[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Float.parseFloat(each);
+ private static void parserNumber(final Number[] target, final List source) {
+ for (int i = 0; i < target.length; i++) {
+ Object o = source.get(i);
+ if (o == null) {
+ continue;
+ }
+ target[i] = parseNumber(o.toString());
}
- return result;
}
/**
- * Decode float8 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * parse String to Number.
+ * @param each String
+ * @return BigDecimal or Double
*/
- public double[] decodeFloat8Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- double[] result = new double[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Double.parseDouble(each);
+ private static Number parseNumber(final String each) {
+ String value = each;
+ if (value.startsWith("\"") && value.endsWith("\"") && value.length() > 2) {
+ value = each.substring(1, value.length() - 1);
}
- return result;
- }
-
- /**
- * Decode bool array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
- */
- public boolean[] decodeBoolArray(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- boolean[] result = new boolean[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Boolean.parseBoolean(each);
+ if (Double.toString(Double.NaN).equals(value)) {
+ return Double.NaN;
}
- return result;
- }
-
- /**
- * Decode string array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
- */
- public String[] decodeStringArray(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- return parameterElements.toArray(EMPTY_STRING_ARRAY);
+ if (Double.toString(Double.POSITIVE_INFINITY).equals(value)) {
+ return Double.POSITIVE_INFINITY;
+ }
+ if (Double.toString(Double.NEGATIVE_INFINITY).equals(value)) {
+ return Double.POSITIVE_INFINITY;
+ }
+ return new BigDecimal(value);
}
/**
- * Decode parameter in text mode.
- *
- * @param value protocol parameter value
- * @return decoded parameter value elements
+ * decode String array.
+ * @param fieldString string result
+ * @param delim delim
+ * @return PgDimensionsArrayList
*/
- private Collection<String> decodeText(final String value) {
- Preconditions.checkArgument(value.length() >= 2, "value length less than 2");
- Preconditions.checkArgument('{' == value.charAt(0) && '}' == value.charAt(value.length() - 1), "value not start with '{' or not end with '}'");
- String[] elements = value.substring(1, value.length() - 1).split(",");
- return Arrays.stream(elements).map(each -> "NULL".equals(each) ? null : decodeElementText(each)).collect(Collectors.toList());
- }
-
- private static String decodeElementText(final String element) {
- String result = element;
- if ('"' == result.charAt(0) && '"' == result.charAt(result.length() - 1)) {
- result = result.substring(1, result.length() - 1);
+ public PgDimensionsArrayList decodeFromString(final String fieldString, final char delim) {
+
+ final PgDimensionsArrayList arrayList = new PgDimensionsArrayList();
+
+ if (fieldString == null) {
+ return arrayList;
}
- while (result.contains("\\\"")) {
- result = result.replace("\\\"", "\"");
+
+ final char[] chars = fieldString.toCharArray();
+ StringBuilder buffer = null;
+ boolean insideString = false;
+
+ // needed for checking if NULL value occurred
+ boolean wasInsideString = false;
+
+ // array dimension arrays
+ final List<PgDimensionsArrayList> dims = new ArrayList<>();
+
+ // currently processed array
+ PgDimensionsArrayList curArray = arrayList;
+
+ // Starting with 8.0 non-standard (beginning index
+ // isn't 1) bounds the dimensions are returned in the
+ // data formatted like so "[0:3]={0,1,2,3,4}".
+ // Older versions simply do not return the bounds.
+ //
+ // Right now we ignore these bounds, but we could
+ // consider allowing these index values to be used
+ // even though the JDBC spec says 1 is the first
+ // index. I'm not sure what a client would like
+ // to see, so we just retain the old behavior.
+ int startOffset = 0;
+
+ if (chars[0] == '[') {
+ while (chars[startOffset] != '=') {
+ startOffset++;
+ }
+ startOffset++;
}
- while (result.contains("\\\\")) {
- result = result.replace("\\\\", "\\");
+ int i = startOffset;
+ while (i < chars.length) {
+
+ // escape character that we need to skip
+ if (chars[i] == '\\') {
+ i++;
+ } else if (!insideString && chars[i] == '{') {
+ // subarray start
+ if (dims.isEmpty()) {
+ dims.add(arrayList);
+ } else {
+ PgDimensionsArrayList a = new PgDimensionsArrayList();
Review Comment:
Variable names like `a` and `p` are difficult to understand. When submitting code to open source projects, you need to pay attention to coding standards.
##########
database/protocol/dialect/postgresql/src/test/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLArrayBinaryProtocolValueTest.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol;
+
+//
+//
+//import io.netty.buffer.ByteBuf;
+//import io.netty.buffer.Unpooled;
+//import org.apache.shardingsphere.db.protocol.postgresql.payload.PostgreSQLPacketPayload;
+//import org.junit.jupiter.api.AfterAll;
+//import org.junit.jupiter.api.BeforeAll;
+//import org.junit.jupiter.api.Test;
+//import org.postgresql.Driver;
+//import org.postgresql.core.BaseConnection;
+//import org.postgresql.jdbc.PgArray;
+//import org.testcontainers.containers.PostgreSQLContainer;
+//import org.testcontainers.utility.DockerImageName;
+//
+//import java.nio.charset.StandardCharsets;
+//import java.sql.*;
+//
+//import java.util.Properties;
+//import java.util.TimeZone;
+//
+//import static org.apache.shardingsphere.db.protocol.postgresql.packet.command.query.extended.bind.protocol.PostgreSQLArrayBinaryProtocolValue.instance;
+//import static org.junit.jupiter.api.Assertions.*;
+//
+public class PostgreSQLArrayBinaryProtocolValueTest {
Review Comment:
This class is already fully commented, why submit to PR?
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLArrayBinaryProtocolValue.java:
##########
@@ -0,0 +1,83 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol;
+
+import org.apache.shardingsphere.database.protocol.postgresql.payload.PostgreSQLPacketPayload;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.decoder.PgBinaryObj;
+import org.postgresql.core.Oid;
+import org.postgresql.jdbc.ShardingSpherePgArrayUtils;
+
+import java.nio.ByteBuffer;
+import java.util.HashMap;
+import java.util.Map;
+
+public final class PostgreSQLArrayBinaryProtocolValue implements PostgreSQLBinaryProtocolValue {
+
+ public static final PostgreSQLArrayBinaryProtocolValue INSTANCE = new PostgreSQLArrayBinaryProtocolValue();
+
+ private static final Map<Integer, String> OID_TYPE_NAME = new HashMap<>();
+
+ static {
+ OID_TYPE_NAME.put(Oid.BOOL, "bool[]");
+ OID_TYPE_NAME.put(Oid.BYTEA, "bytea[]");
+ // oidTypeName.put(Oid.CHAR_ARRAY, "char[]");
+ // oidTypeName.put(Oid.NAME_ARRAY, "name[]");
Review Comment:
Why are these comments? Are they not supported yet? Should they be marked as TODO?
##########
database/protocol/dialect/postgresql/src/main/java/org/postgresql/jdbc/ShardingSpherePgArrayUtils.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.postgresql.jdbc;
Review Comment:
The package name of the class we implement should not be org.postgresql.jdbc, which will cause misunderstanding to users.
##########
database/protocol/dialect/postgresql/src/main/java/org/postgresql/jdbc/ShardingSpherePgArrayUtils.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.postgresql.jdbc;
+
+import lombok.SneakyThrows;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.PostgreSQLArrayParameterDecoder;
+import org.apache.shardingsphere.infra.exception.generic.UnsupportedSQLOperationException;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.encoder.ArrayEncoding;
+import org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.encoder.ArrayEncoder;
+import org.postgresql.core.BaseConnection;
+import org.postgresql.core.Oid;
+
+import java.io.ByteArrayOutputStream;
+import java.nio.charset.Charset;
+import java.sql.SQLException;
+
+public class ShardingSpherePgArrayUtils {
+
+ /**
+ * encode pgArray.
+ * @param array PgArrayObject
+ * @param charset charset
+ * @return binary bytes array
+ * @throws UnsupportedSQLOperationException if array is not PgArray
+ */
+ @SneakyThrows(SQLException.class)
+ public static byte[] getBinaryBytes(final Object array, final Charset charset) {
+ PgArray pgArray;
+ if (array instanceof PgArray) {
+ pgArray = (PgArray) array;
+ } else {
+ throw new UnsupportedSQLOperationException("can not encode" + array.getClass());
+ }
+ byte[] bytes = pgArray.toBytes();
+
+ if (bytes != null) {
+ return bytes;
+ }
+ BaseConnection connection = pgArray.connection;
+
+ assert connection != null;
+ int pgType = connection.getTypeInfo().getPGArrayType(pgArray.getBaseTypeName());
+
+ Object realArray;
+ if (pgType == Oid.NUMERIC_ARRAY) {
+ realArray = new PostgreSQLArrayParameterDecoder().decodeNumberArray(pgArray.fieldString);
+ } else {
+ realArray = pgArray.getArray();
+ }
+
+ final ArrayEncoder arraySupport = ArrayEncoding.getArrayEncoder(realArray);
+ ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
+ arraySupport.toBinaryRepresentation(realArray, pgType, byteArrayOutputStream, charset);
+ return byteArrayOutputStream.toByteArray();
+
Review Comment:
Useless blank lines do not meet the requirements of checkstyle
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3209156316
@terrymanu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3241012675
Hi @terrymanu @RaigorJiang, any updates on this? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3574351103
But please note, your implementation in #32848 is incorrect. PG's `text[]` does not represent just a one-dimensional `text[]`; it can be multi-dimensional.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3573974564
see https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/jdbc/ArrayEncoding.java and https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/jdbc/ArrayDecoding.java @RaigorJiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3580477211
> 3\. There are clearly incorrect parts of the business logic, such as `PostgreSQLStringBinaryProtocolValue#write`.
For performance considerations, I believe it's reasonable to return -1 when the length cannot be determined, and then obtain the value when writing. I have noted the reason for the change in the interface. If we convert the string into a byte array just to get its length, I think that’s a waste of performance. It's completely unnecessary, because when writing we already obtain its byte array anyway
@RaigorJiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on code in PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#discussion_r2314785340
##########
database/protocol/dialect/postgresql/src/main/java/org/postgresql/jdbc/ShardingSpherePgArrayUtils.java:
##########
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.postgresql.jdbc;
Review Comment:
In fact, the reason I am using this package name is that PgArray#connection is protected. If I were to access it from a different package, it would require an unnecessary extra reflection call.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3301805865
Hi! @RaigorJiang
Thank you very much for your kind words and encouragement, I really appreciate it. 🙏
I have carefully followed the ShardingSphere code style guide and tried my best to fix all the issues I could identify. Since the document contains many detailed rules and I might have misunderstood some points, it’s possible that I still missed a few places.
If you don’t mind, I’d really appreciate your help in pointing out any remaining style problems so that I can fully align with the project standards.
Thanks again for your support!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3209274905
@terrymanu @RaigorJiang
I ran the integration tests and unit tests locally, and everything seems fine. In this PR, I have removed some unnecessary test cases and incorrect implementations. However, I am not sure how to add my integration tests since they seem to be incompatible with the previous ones. Additionally, I couldn't find any testcontainers-related dependencies in both the `shardingsphere-protocol-postgresql` module and its parent module. Where should I add them? Please help me complete the PR. Below is my test code. There's also a part in `PostgreSQLArrayBinaryProtocolValueTest`, but due to dependency issues, I have temporarily commented it out.
```java
import org.apache.shardingsphere.db.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.decoder.PgBinaryObj;
import org.apache.shardingsphere.db.protocol.postgresql.packet.command.query.extended.bind.protocol.util.codec.encoder.*;
import org.postgresql.core.Oid;
import org.postgresql.core.QueryExecutor;
import org.postgresql.jdbc.PgConnection;
import org.postgresql.jdbc.PgStatement;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.math.BigDecimal;
import java.nio.charset.StandardCharsets;
import java.sql.*;
import java.util.Arrays;
import java.util.TimeZone;
public class ClientTest {
public static void main(String[] args) throws IOException {
String url = "jdbc:postgresql://localhost:3307/test";
String username = "root";
String password = "root";
Short[][] param1 = new Short[2][4];
param1[0][0] = 1;
param1[0][1] = 2;
param1[1][0] = 3;
Integer[][] param2 = new Integer[2][4];
param2[0][0] = 1;
param2[0][1] = 2;
param2[1][0] = 3;
Long[][] param3 = new Long[2][4];
param3[0][0] = 1L;
param3[0][1] = 2L;
param3[1][0] = 3L;
Float[][] param4 = new Float[2][4];
param4[0][0] = 1.1F;
param4[0][1] = 2.2F;
param4[1][0] = Float.NaN;
param4[1][1] = Float.POSITIVE_INFINITY;
param4[1][2] = Float.NEGATIVE_INFINITY;
Double[][] param5 = new Double[2][4];
param5[0][0] = 1.1;
param5[0][1] = 2.2;
param5[1][0] = Double.NaN;
param5[1][1] = Double.POSITIVE_INFINITY;
param5[1][2] = Double.NEGATIVE_INFINITY;
Number[][] param6 = new Number[2][4];
param6[0][0] = new BigDecimal("1.1");
param6[0][1] = new BigDecimal("2.2");
param6[1][0] = Double.NaN;
param6[1][1] = Double.POSITIVE_INFINITY;
param6[1][2] = Double.NEGATIVE_INFINITY;
Boolean[][] param7 = new Boolean[2][4];
param7[0][0] = true;
param7[0][1] = false;
String[][] param8 = new String[2][4];
param8[0][0] = "aaa";
param8[0][1] = "bbbb";
String[][] param9 = new String[2][4];
param9[0][0] = "aaa";
param9[0][1] = "bbbb";
byte[][][] param10 = new byte[2][4][];
param10[0][0] = new byte[10];
param10[0][1] = "aaa".getBytes(StandardCharsets.UTF_8);
String query = "SELECT\n" +
" ?::int2[] AS int2_array, -- smallint\n" +
" ?::int4[] AS int4_array, -- integer\n" +
" ?::int8[] AS int8_array, -- bigint\n" +
" ?::float4[] AS float4_array, -- real\n" +
" ?::float8[] AS float8_array, -- double precision\n" +
" ?::numeric[] AS numeric_array, -- numeric / decimal\n" +
" ?::bool[] AS bool_array, -- boolean\n" +
" ?::text[] AS text_array, -- text\n" +
" ?::varchar[] AS varchar_array, -- varchar\n" +
" ?::bytea[] AS bytea_array, -- bytea\n" +
" ?::date[] AS date_array, -- date\n" +
" ?::time[] AS time_array, -- time without time zone\n" +
" ?::timestamp[] AS timestamp_array, -- timestamp\n" +
" ?::bool AS boolv, -- bool\n" +
" ?::bytea AS byteav, -- bytea\n" +
" ?::date AS datev, -- date\n" +
" ?::float8 AS float8v, -- float8\n" +
" ?::float4 AS float4v, -- float4\n" +
" ?::int2 AS int2v, -- int2\n" +
" ?::int4 AS int4v, -- int4\n" +
" ?::int8 AS int8v, -- int8\n" +
" ?::numeric AS numericv, -- numeric\n" +
" ?::text AS textv, -- text\n" +
" ?::varchar AS varcharv, -- varchar\n" +
" ?::time AS timev, -- time\n" +
" ?::timestamp AS timestampv -- timestamp\n";
int[] arrayOids = new int[]{Oid.INT2_ARRAY, Oid.INT4_ARRAY, Oid.INT8_ARRAY, Oid.FLOAT4_ARRAY, Oid.FLOAT8_ARRAY, Oid.NUMERIC_ARRAY, Oid.BOOL_ARRAY, Oid.TEXT_ARRAY, Oid.VARCHAR_ARRAY, Oid.BYTEA_ARRAY};
String[] arrayOidNames = new String[]{"int2[]", "int4[]", "int8[]", "float4[]", "float8[]", "numeric[]", "bool[]", "text[]", "varchar[]", "bytea[]"};
byte[][] sendResult = new byte[26][];
Object[] params = new Object[]{param1, param2, param3, param4, param5, param6, param7, param8, param9, param10};
try (Connection connection = DriverManager.getConnection(url, username, password)) {
PgConnection pgConnection = (PgConnection) connection;
QueryExecutor queryExecutor = pgConnection.getQueryExecutor();
queryExecutor.addBinarySendOid(Oid.DATE);
queryExecutor.addBinarySendOid(Oid.DATE_ARRAY);
queryExecutor.addBinarySendOid(Oid.TIME);
queryExecutor.addBinarySendOid(Oid.TIME_ARRAY);
queryExecutor.addBinarySendOid(Oid.BOOL_ARRAY);
queryExecutor.addBinarySendOid(Oid.TIMESTAMP);
queryExecutor.addBinarySendOid(Oid.TIMESTAMP_ARRAY);
queryExecutor.addBinarySendOid(Oid.NUMERIC_ARRAY);
queryExecutor.addBinarySendOid(Oid.VARCHAR);
queryExecutor.addBinarySendOid(Oid.TEXT);
queryExecutor.addBinarySendOid(Oid.BOOL);
queryExecutor.addBinaryReceiveOid(Oid.DATE);
queryExecutor.addBinaryReceiveOid(Oid.DATE_ARRAY);
queryExecutor.addBinaryReceiveOid(Oid.TIME);
queryExecutor.addBinaryReceiveOid(Oid.TIME_ARRAY);
queryExecutor.addBinaryReceiveOid(Oid.BOOL_ARRAY);
queryExecutor.addBinaryReceiveOid(Oid.TIMESTAMP);
queryExecutor.addBinaryReceiveOid(Oid.TIMESTAMP_ARRAY);
queryExecutor.addBinaryReceiveOid(Oid.NUMERIC_ARRAY);
queryExecutor.addBinaryReceiveOid(Oid.VARCHAR);
queryExecutor.addBinaryReceiveOid(Oid.TEXT);
queryExecutor.addBinaryReceiveOid(Oid.BOOL);
PreparedStatement preparedStatement = connection.prepareStatement(query);
PgStatement pgStatement = (PgStatement) preparedStatement;
pgStatement.setPrepareThreshold(-1);
for (int i = 0; i < arrayOids.length; i++) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ArrayEncoding.getArrayEncoder(params[i]).toBinaryRepresentation((Object[]) params[i], arrayOids[i], baos);
byte[] byteArray = baos.toByteArray();
PgBinaryObj pgBinaryObj = new PgBinaryObj(byteArray);
pgBinaryObj.setType(arrayOidNames[i]);
sendResult[i] = byteArray;
preparedStatement.setObject(i + 1, pgBinaryObj);
}
Date[][] dates = new Date[2][4];
//1970-01-01
dates[0][0] = new Date(-TimeZone.getDefault().getRawOffset());
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ArrayEncoding.getArrayEncoder(dates).toBinaryRepresentation(dates, Oid.DATE_ARRAY, baos);
byte[] byteArray = baos.toByteArray();
PgBinaryObj dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("date[]");
sendResult[10] = byteArray;
preparedStatement.setObject(11, dateBinary);
baos.reset();
Time[][] times = new Time[2][4];
//time0
times[0][0] = new Time(-TimeZone.getDefault().getRawOffset());
ArrayEncoding.getArrayEncoder(times).toBinaryRepresentation(times, Oid.TIME_ARRAY, baos);
byteArray = baos.toByteArray();
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("time[]");
sendResult[11] = byteArray;
preparedStatement.setObject(12, dateBinary);
baos.reset();
Timestamp[][] timestamps = new Timestamp[2][4];
//timestamps0 1970-01-01
timestamps[0][0] = new Timestamp(-TimeZone.getDefault().getRawOffset());
ArrayEncoding.getArrayEncoder(timestamps).toBinaryRepresentation(timestamps, Oid.TIMESTAMP_ARRAY, baos);
byteArray = baos.toByteArray();
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("timestamp[]");
sendResult[12] = byteArray;
preparedStatement.setObject(13, dateBinary);
baos.reset();
BooleanArrayEncoder.INSTANCE.write(true,baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length);
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("bool");
sendResult[13] = byteArray;
preparedStatement.setObject(14,dateBinary);
byteArray = new byte[10];
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("bytea");
sendResult[14] = byteArray;
preparedStatement.setObject(15, dateBinary);
baos.reset();
DateArrayEncoder.INSTANCE.write(new Date(-TimeZone.getDefault().getRawOffset()),baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length);
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("date");
sendResult[15] = byteArray;
preparedStatement.setObject(16, dateBinary);
baos.reset();
Float8ArrayEncoder.INSTANCE.write(1.2,baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length);
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("float8");
sendResult[16] = byteArray;
preparedStatement.setObject(17, dateBinary);
baos.reset();
Float4ArrayEncoder.INSTANCE.write(1.2f,baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length);
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("float4");
sendResult[17] = byteArray;
preparedStatement.setObject(18, dateBinary);
baos.reset();
Int2ArrayEncoder.INSTANCE.write((short) 1,baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length);
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("int2");
sendResult[18] = byteArray;
preparedStatement.setObject(19, dateBinary);
baos.reset();
Int4ArrayEncoder.INSTANCE.write( 1,baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length);
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("int4");
sendResult[19] = byteArray;
preparedStatement.setObject(20, dateBinary);
baos.reset();
Int8ArrayEncoder.INSTANCE.write(1L,baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length);
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("int8");
sendResult[20] = byteArray;
preparedStatement.setObject(21, dateBinary);
baos.reset();
NumericArrayEncoder.INSTANCE.write(new BigDecimal("1.2"),baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length );
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("numeric");
sendResult[21] = byteArray;
preparedStatement.setObject(22, dateBinary);
baos.reset();
StringArrayEncoder.INSTANCE.write("testString",baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length );
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("text");
sendResult[22] = byteArray;
preparedStatement.setObject(23, dateBinary);
baos.reset();
StringArrayEncoder.INSTANCE.write("testVarchar",baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length );
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("varchar");
sendResult[23] = byteArray;
preparedStatement.setObject(24, dateBinary);
baos.reset();
TimeArrayEncoder.INSTANCE.write(new Time(-TimeZone.getDefault().getRawOffset()),baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length );
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("time");
sendResult[24] = byteArray;
preparedStatement.setObject(25, dateBinary);
baos.reset();
TimestampArrayEncoder.INSTANCE.write(new Timestamp(-TimeZone.getDefault().getRawOffset()),baos);
byteArray = baos.toByteArray();
byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length );
dateBinary = new PgBinaryObj(byteArray);
dateBinary.setType("timestamp");
sendResult[25] = byteArray;
preparedStatement.setObject(26, dateBinary);
//
// baos.reset();
// Int2ArrayEncoder.INSTANCE.write((short) 1,baos);
// byteArray = baos.toByteArray();
// byteArray = Arrays.copyOfRange(byteArray, 4, byteArray.length );
// dateBinary = new PgBinaryObj(byteArray);
// dateBinary.setType("int2");
// sendResult[18] = byteArray;
preparedStatement.execute();
ResultSet resultSet = preparedStatement.getResultSet();
resultSet.next();
Object value = resultSet.getObject(6);
// value.toString();
value = resultSet.getObject(11);
value = resultSet.getObject(12);
value = resultSet.getObject(13);
value = resultSet.getObject(14);
for (int col = 1; col <= resultSet.getMetaData().getColumnCount(); col++) {
value = resultSet.getBytes(col);
boolean equals = new String((byte[]) value, StandardCharsets.US_ASCII).equals(new String((byte[]) sendResult[col - 1], StandardCharsets.US_ASCII));
System.out.println("col:"+col+ equals);
}
} catch (SQLException e) {
e.printStackTrace();
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3426003236
ping @RaigorJiang @terrymanu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3212649171
@terrymanu @RaigorJiang Hello, all test cases passed except for one Spotless exception. Can we proceed with the code review? By the way, I'd also like to discuss how to add my integration tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3209284855
Additionally, this PR contains some unused Decoder classes. I will keep them if we need them, but if not, I will remove them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3573970554
> Hi @ShenFeng312
>
> I discussed this with several committers familiar with the Proxy protocol, and we reached the following consensus:
>
> 1. The current PR involves too many changes, including adjustments to data processing logic and abstraction levels, making review difficult;
> 2. The code has many non-standard aspects, including naming conventions and comments;
> 3. There are clearly incorrect parts of the business logic, such as `PostgreSQLStringBinaryProtocolValue#write`.
>
> Based on the above information, we speculate that some code in this PR was generated by a large model? Some code appears to have not been rigorously considered.
>
> Therefore, we suggest you withdraw this PR and submit it multiple times with small-scale adjustments, ensuring the correctness of the code logic and comments, as well as consistency in style.
I don't think you have carefully reviewed this PR. The serialization and deserialization logic for arrays in this PR comes from part of the code in pgjdbc. I have abstracted and optimized it to support both binary and text formats. As a result, it carries the coding style of pg-jdbc. @RaigorJiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "RaigorJiang (via GitHub)" <gi...@apache.org>.
RaigorJiang commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3574047019
> > Hi @ShenFeng312
> > I discussed this with several committers familiar with the Proxy protocol, and we reached the following consensus:
> >
> > 1. The current PR involves too many changes, including adjustments to data processing logic and abstraction levels, making review difficult;
> > 2. The code has many non-standard aspects, including naming conventions and comments;
> > 3. There are clearly incorrect parts of the business logic, such as `PostgreSQLStringBinaryProtocolValue#write`.
> >
> > Based on the above information, we speculate that some code in this PR was generated by a large model? Some code appears to have not been rigorously considered.
> > Therefore, we suggest you withdraw this PR and submit it multiple times with small-scale adjustments, ensuring the correctness of the code logic and comments, as well as consistency in style.
>
> I don't think you have carefully reviewed this PR. The serialization and deserialization logic for arrays in this PR comes from part of the code in pgjdbc. I have abstracted and optimized it to support both binary and text formats. As a result, it carries the coding style of pg-jdbc. @RaigorJiang
@ShenFeng312
In fact, copying `ArrayDecoding.java` from pgjdbc is a very bad practice because its code is difficult to read and maintain.
Furthermore, you didn't specify the source of the copy in the comments of class. So, when pgjdbc is updated or bugs are fixed, who will maintain the copied code in ShardingSphere?
I noticed that you mainly use `ArrayDecoding` to implement the functionality of `PostgreSQLStringArrayBinaryProtocolValue`. This can actually be accomplished without copying its code; I will provide a PR for reference later. (like #32848)
Finally, we cannot assume that because the code is copied, we can break ShardingSphere's coding standards; this is unsustainable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3574229769
I think what you're doing is meaningless. When deserializing a multi-dimensional array string into a multi-dimensional array, it feels more like a JSON deserialization tool. It is inherently difficult to maintain, which means there are many variables related to the algorithm. When each variable requires a meaningful name, it becomes very hard to manage. Putting these issues aside, you should also notice the different handling between single-level arrays and multi-level nested arrays. I admit that the original code from pg is poor and its abstraction is incorrect. I have corrected these issues. However, I didn't choose to rewrite the 'JSON deserialization code' because rewriting this code wouldn't likely result in much difference from the current version. It’s more like an algorithmic problem. Furthermore, I believe that the primary focus of the code here should be performance, not readability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on code in PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#discussion_r2328956669
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLArrayParameterDecoder.java:
##########
@@ -17,174 +17,242 @@
package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol;
-import com.google.common.base.Preconditions;
-import org.apache.shardingsphere.infra.exception.ShardingSpherePreconditions;
-import org.apache.shardingsphere.infra.exception.generic.UnsupportedSQLOperationException;
+import org.checkerframework.checker.nullness.qual.Nullable;
+import org.postgresql.core.Parser;
+import org.postgresql.util.internal.Nullness;
-import java.nio.charset.StandardCharsets;
-import java.util.Arrays;
-import java.util.Collection;
-import java.util.stream.Collectors;
+import java.lang.reflect.Array;
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.List;
/**
* PostgreSQL array parameter decoder.
*/
public final class PostgreSQLArrayParameterDecoder {
- private static final String[] EMPTY_STRING_ARRAY = new String[0];
-
/**
- * Decode int2 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * decode numberArray String to a array.
+ * @param parameterValue arrayString
+ * @return array
*/
- public short[] decodeInt2Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- short[] result = new short[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Short.parseShort(each);
+ public Object decodeNumberArray(final String parameterValue) {
+
+ PgDimensionsArrayList list = decodeFromString(parameterValue, ',');
+ int dims = list.dimensionsCount;
+ final int[] dimensionLengths = new int[dims];
+ dimensionLengths[0] = list.size();
+ List tmpList = (List) list.get(0);
+ for (int i = 1; i < dims; i++) {
+ dimensionLengths[i] = Nullness.castNonNull(tmpList, "first element of adjustedList is null").size();
+ if (i != dims - 1) {
+ tmpList = (List) tmpList.get(0);
+ }
}
- return result;
- }
-
- /**
- * Decode int4 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
- */
- public int[] decodeInt4Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- int[] result = new int[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Integer.parseInt(each);
+ Object[] array = (Object[]) Array.newInstance(Number.class, dimensionLengths);
+ if (array instanceof Number[]) {
+ parserNumber((Number[]) array, list);
+ } else {
+ storeStringValues(array, list, dimensionLengths, 0);
}
- return result;
+ return array;
}
/**
- * Decode int8 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * decode and copy to result.
+ * @param array result
+ * @param list source String array
+ * @param dimensionLengths dimensionLengths
+ * @param dim dim
*/
- public long[] decodeInt8Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- long[] result = new long[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Long.parseLong(each);
+ private static void storeStringValues(final Object[] array, final List list, final int[] dimensionLengths,
+ final int dim) {
+
+ for (int i = 0; i < dimensionLengths[dim]; i++) {
+ Object element = Nullness.castNonNull(list.get(i), "list.get(i)");
+ if (dim == dimensionLengths.length - 2) {
+ parserNumber((Number[]) array[i], (List) element);
+ } else {
+ storeStringValues((Object[]) array[i], (List) element, dimensionLengths, dim + 1);
+ }
}
- return result;
}
/**
- * Decode float4 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * parse a String list to Number[].
+ * @param target Number[]
+ * @param source String list
*/
- public float[] decodeFloat4Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- float[] result = new float[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Float.parseFloat(each);
+ private static void parserNumber(final Number[] target, final List source) {
+ for (int i = 0; i < target.length; i++) {
+ Object o = source.get(i);
+ if (o == null) {
+ continue;
+ }
+ target[i] = parseNumber(o.toString());
}
- return result;
}
/**
- * Decode float8 array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
+ * parse String to Number.
+ * @param each String
+ * @return BigDecimal or Double
*/
- public double[] decodeFloat8Array(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- double[] result = new double[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Double.parseDouble(each);
+ private static Number parseNumber(final String each) {
+ String value = each;
+ if (value.startsWith("\"") && value.endsWith("\"") && value.length() > 2) {
+ value = each.substring(1, value.length() - 1);
}
- return result;
- }
-
- /**
- * Decode bool array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
- */
- public boolean[] decodeBoolArray(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- boolean[] result = new boolean[parameterElements.size()];
- int index = 0;
- for (String each : parameterElements) {
- result[index++] = Boolean.parseBoolean(each);
+ if (Double.toString(Double.NaN).equals(value)) {
+ return Double.NaN;
}
- return result;
- }
-
- /**
- * Decode string array parameter.
- *
- * @param parameterBytes protocol parameter value
- * @param isBinary whether parameter value is binary or not
- * @return int array
- */
- public String[] decodeStringArray(final byte[] parameterBytes, final boolean isBinary) {
- ShardingSpherePreconditions.checkState(!isBinary, () -> new UnsupportedSQLOperationException("binary mode"));
- String parameterValue = new String(parameterBytes, StandardCharsets.UTF_8);
- Collection<String> parameterElements = decodeText(parameterValue);
- return parameterElements.toArray(EMPTY_STRING_ARRAY);
+ if (Double.toString(Double.POSITIVE_INFINITY).equals(value)) {
+ return Double.POSITIVE_INFINITY;
+ }
+ if (Double.toString(Double.NEGATIVE_INFINITY).equals(value)) {
+ return Double.POSITIVE_INFINITY;
+ }
+ return new BigDecimal(value);
}
/**
- * Decode parameter in text mode.
- *
- * @param value protocol parameter value
- * @return decoded parameter value elements
+ * decode String array.
+ * @param fieldString string result
+ * @param delim delim
+ * @return PgDimensionsArrayList
*/
- private Collection<String> decodeText(final String value) {
- Preconditions.checkArgument(value.length() >= 2, "value length less than 2");
- Preconditions.checkArgument('{' == value.charAt(0) && '}' == value.charAt(value.length() - 1), "value not start with '{' or not end with '}'");
- String[] elements = value.substring(1, value.length() - 1).split(",");
- return Arrays.stream(elements).map(each -> "NULL".equals(each) ? null : decodeElementText(each)).collect(Collectors.toList());
- }
-
- private static String decodeElementText(final String element) {
- String result = element;
- if ('"' == result.charAt(0) && '"' == result.charAt(result.length() - 1)) {
- result = result.substring(1, result.length() - 1);
+ public PgDimensionsArrayList decodeFromString(final String fieldString, final char delim) {
+
+ final PgDimensionsArrayList arrayList = new PgDimensionsArrayList();
+
+ if (fieldString == null) {
+ return arrayList;
}
- while (result.contains("\\\"")) {
- result = result.replace("\\\"", "\"");
+
+ final char[] chars = fieldString.toCharArray();
+ StringBuilder buffer = null;
+ boolean insideString = false;
+
+ // needed for checking if NULL value occurred
+ boolean wasInsideString = false;
+
+ // array dimension arrays
+ final List<PgDimensionsArrayList> dims = new ArrayList<>();
+
+ // currently processed array
+ PgDimensionsArrayList curArray = arrayList;
+
+ // Starting with 8.0 non-standard (beginning index
+ // isn't 1) bounds the dimensions are returned in the
+ // data formatted like so "[0:3]={0,1,2,3,4}".
+ // Older versions simply do not return the bounds.
+ //
+ // Right now we ignore these bounds, but we could
+ // consider allowing these index values to be used
+ // even though the JDBC spec says 1 is the first
+ // index. I'm not sure what a client would like
+ // to see, so we just retain the old behavior.
+ int startOffset = 0;
+
+ if (chars[0] == '[') {
+ while (chars[startOffset] != '=') {
+ startOffset++;
+ }
+ startOffset++;
}
- while (result.contains("\\\\")) {
- result = result.replace("\\\\", "\\");
+ int i = startOffset;
+ while (i < chars.length) {
+
+ // escape character that we need to skip
+ if (chars[i] == '\\') {
+ i++;
+ } else if (!insideString && chars[i] == '{') {
+ // subarray start
+ if (dims.isEmpty()) {
+ dims.add(arrayList);
+ } else {
+ PgDimensionsArrayList a = new PgDimensionsArrayList();
Review Comment:
fixed
##########
database/protocol/dialect/postgresql/src/test/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLArrayBinaryProtocolValueTest.java:
##########
@@ -0,0 +1,139 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.database.protocol.postgresql.packet.command.query.extended.bind.protocol;
+
+//
+//
+//import io.netty.buffer.ByteBuf;
+//import io.netty.buffer.Unpooled;
+//import org.apache.shardingsphere.db.protocol.postgresql.payload.PostgreSQLPacketPayload;
+//import org.junit.jupiter.api.AfterAll;
+//import org.junit.jupiter.api.BeforeAll;
+//import org.junit.jupiter.api.Test;
+//import org.postgresql.Driver;
+//import org.postgresql.core.BaseConnection;
+//import org.postgresql.jdbc.PgArray;
+//import org.testcontainers.containers.PostgreSQLContainer;
+//import org.testcontainers.utility.DockerImageName;
+//
+//import java.nio.charset.StandardCharsets;
+//import java.sql.*;
+//
+//import java.util.Properties;
+//import java.util.TimeZone;
+//
+//import static org.apache.shardingsphere.db.protocol.postgresql.packet.command.query.extended.bind.protocol.PostgreSQLArrayBinaryProtocolValue.instance;
+//import static org.junit.jupiter.api.Assertions.*;
+//
+public class PostgreSQLArrayBinaryProtocolValueTest {
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3274032901
PTAL @RaigorJiang @terrymanu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on code in PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#discussion_r2558700852
##########
database/protocol/dialect/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLStringBinaryProtocolValue.java:
##########
@@ -26,25 +26,26 @@ public final class PostgreSQLStringBinaryProtocolValue implements PostgreSQLBina
@Override
public int getColumnLength(final PostgreSQLPacketPayload payload, final Object value) {
- if (value instanceof byte[]) {
- return ((byte[]) value).length;
- }
- return value.toString().getBytes(payload.getCharset()).length;
+ return -1;
Review Comment:
For performance considerations, I believe it's reasonable to return -1 when the length cannot be determined, and then obtain the value when writing. I have noted the reason for the change in the interface.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3580173485
>
> I noticed that you mainly use `ArrayDecoding` to implement the functionality of `PostgreSQLStringArrayBinaryProtocolValue`. This can actually be accomplished without copying its code; I will provide a PR for reference later. (like #32848)
For performance considerations, I believe it's reasonable to return -1 when the length cannot be determined, and then obtain the value when writing. I have noted the reason for the change in the interface. If we convert the string into a byte array just to get its length, I think that’s a waste of performance. It's completely unnecessary, because when writing we already obtain its byte array anyway
@RaigorJiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3494432705
Hi @RaigorJiang @terrymanu I just wanted to check if there are any updates on this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] postgresql binary support [shardingsphere]
Posted by "ShenFeng312 (via GitHub)" <gi...@apache.org>.
ShenFeng312 commented on PR #36372:
URL: https://github.com/apache/shardingsphere/pull/36372#issuecomment-3209973670
[ERROR] /home/shenfeng/github/shardingsphere/database/protocol/type/postgresql/src/main/java/org/apache/shardingsphere/database/protocol/postgresql/packet/command/query/extended/bind/protocol/PostgreSQLArrayParameterDecoder.java:168:18: Control variable 'i' is modified. [ModifiedControlVariable] Are you serious?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org