You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@dubbo.apache.org by "mcck (via GitHub)" <gi...@apache.org> on 2024/10/30 07:35:21 UTC
[PR] 修复InputStream反序列化bug [dubbo-hessian-lite]
mcck opened a new pull request, #76:
URL: https://github.com/apache/dubbo-hessian-lite/pull/76
@AlbumenJ This code cannot completely solve the problem of deserializing InputStream：[Fix input stream unable to being deserialized #14837](https://github.com/apache/dubbo/pull/14837)
The code before modification is:
```java
public Object readObject(AbstractHessianInput in) throws IOException {
return in.readInputStream();
}
```
In `in.readInputStream()`, he only read `InputStream` of `tag`, and his data was not read. This will cause an error in the '_offset' when `org.apache.dubbo.rpc.protocol.dubbo.DecodeableRpcResult#decode(org.apache.dubbo.remoting.Channel, java.io.InputStream)` reads the map in the `handleAttachment(in);` method, throwing an exception;
So when deserializing InputStream, it is necessary to read its bytes and move _osffet to the correct position.
related issues：[方法返回InputStream时expected map/object at java.lang.String异常](https://github.com/apache/dubbo-hessian-lite/issues/75)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [PR] 修复InputStream反序列化bug [dubbo-hessian-lite]
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ commented on code in PR #76:
URL: https://github.com/apache/dubbo-hessian-lite/pull/76#discussion_r1823676548
##########
java-8-test/src/test/java/com/alibaba/com/caucho/hessian/io/HessianInputStreamDeserializerTest.java:
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.alibaba.com.caucho.hessian.io;
+
+import com.alibaba.com.caucho.hessian.io.base.SerializeTestBase;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.EnabledForJreRange;
+import org.junit.jupiter.api.condition.JRE;
+
+import java.io.*;
+import java.util.Arrays;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
+
+/**
+ * InputStreamDeserializer 测试
+ */
+public class HessianInputStreamDeserializerTest extends SerializeTestBase {
+
+ protected <T> T baseHessian2Serialize(T data) throws IOException {
+ ByteArrayOutputStream bout = new ByteArrayOutputStream();
+ Hessian2Output out = new Hessian2Output(bout);
+
+ out.writeObject(data);
+ out.flush();
+
+ ByteArrayInputStream bin = new ByteArrayInputStream(bout.toByteArray());
+ Hessian2Input input = new Hessian2Input(bin);
+ Object obj = input.readObject(InputStream.class);
+ return (T) obj;
+ }
+
+ byte[] returnBytes(int size){
+ byte[] bytes = new byte[size];
+ Arrays.fill(bytes, (byte) 'A');
+ bytes[0] = '&';
+ bytes[bytes.length-3] = '$';
+ bytes[bytes.length-2] = '%';
+ bytes[bytes.length-1] = '#';
+ return bytes;
+ }
+
+ /**
+ * 测试1024字节，返回 ByteArrayInputStream
+ */
+ @Test
+ @EnabledForJreRange(max = JRE.JAVA_11)
+ public void test1024Byte() throws IOException {
+ int size = 1024;
+ byte[] bytes = returnBytes(size);
+ InputStream source = new ByteArrayInputStream(bytes);
+ InputStream result = baseHessian2Serialize(source);
+
+ byte[] resultByte = new byte[size];
+ result.read(resultByte);
+
+ assertInstanceOf(ByteArrayInputStream.class, result, "类型错误");
+ assertEquals(bytes.length, resultByte.length);
+ assertEquals(new String(bytes), new String(resultByte));
+ }
+
+ /**
+ * 测试8192字节，返回 ByteArrayInputStream
+ */
+ @Test
+ @EnabledForJreRange(max = JRE.JAVA_11)
+ public void test8192Byte() throws IOException {
+ int size = 8192;
+ byte[] bytes = returnBytes(size);
+ InputStream source = new ByteArrayInputStream(bytes);
+ InputStream result = baseHessian2Serialize(source);
+
+ byte[] resultByte = new byte[size];
+ result.read(resultByte);
+
+ assertInstanceOf(ByteArrayInputStream.class, result, "类型错误");
+ assertEquals(bytes.length, resultByte.length);
+ assertEquals(new String(bytes), new String(resultByte));
+ }
+
+ /**
+ * 测试8193字节，返回 ByteArrayInputStream
+ * 比缓冲区多一个字节
+ */
+ @Test
+ @EnabledForJreRange(max = JRE.JAVA_11)
+ public void test8193Byte() throws IOException {
+ int size = 8193;
+ byte[] bytes = returnBytes(size);
+ InputStream source = new ByteArrayInputStream(bytes);
+ InputStream result = baseHessian2Serialize(source);
+
+ byte[] resultByte = new byte[size];
+ result.read(resultByte);
+
+ assertInstanceOf(FileInputStream.class, result, "类型错误");
+ assertEquals(bytes.length, resultByte.length);
+ assertEquals(new String(bytes), new String(resultByte));
+ }
+
+ /**
+ * 测试81920字节，返回 ByteArrayInputStream
+ * 比缓冲区多10倍字节
+ */
Review Comment:
Please comment in English
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [PR] Fix InputStream deserialization bug [dubbo-hessian-lite]
Posted by "AlbumenJ (via GitHub)" <gi...@apache.org>.
AlbumenJ commented on code in PR #76:
URL: https://github.com/apache/dubbo-hessian-lite/pull/76#discussion_r1974636073
##########
hessian-lite/src/main/java/com/alibaba/com/caucho/hessian/io/InputStreamDeserializer.java:
##########
@@ -48,20 +48,78 @@
package com.alibaba.com.caucho.hessian.io;
-import java.io.IOException;
+import java.io.*;
+import java.util.UUID;
/**
+ * InputStream Deserializer
* Serializing a stream object.
+ * @author HeYuJie
+ * @date 2024/10/31
*/
public class InputStreamDeserializer extends AbstractDeserializer {
public static final InputStreamDeserializer DESER
= new InputStreamDeserializer();
+ // TODO Allow external configuration
+ @SuppressWarnings("FieldCanBeLocal")
+ private final int bufferSize = Hessian2Output.SIZE;
+ // TODO Allow external configuration
+ private final File tmpDir;
+
public InputStreamDeserializer() {
+ tmpDir = new File(System.getProperty("java.io.tmpdir"), "dubbo-"+ System.currentTimeMillis());
+ if(!tmpDir.exists()){
+ //noinspection ResultOfMethodCallIgnored
+ tmpDir.mkdirs();
+ }
}
- public Object readObject(AbstractHessianInput in)
- throws IOException {
- return in.readInputStream();
+ public Object readObject(AbstractHessianInput in) throws IOException {
+
+ FileOutputStream out = null;
+ try {
+ @SuppressWarnings("resource")
+ InputStream input = in.readInputStream();
+ // Read twice the size of the buffer (16k)
+ byte[] bytes = new byte[bufferSize * 2];
+ File file = null;
+ while (true) { // Loop reading
+
+ int len = input.read(bytes, 0, bytes.length);
+
+ if (out == null) {
+
+ // If the length of InputStream is less than the buffer, creating a byte stream returns
+ if (len <= bufferSize) {
+ byte[] buff = new byte[len];
+ System.arraycopy(bytes, 0, buff, 0, len);
+ return new ByteArrayInputStream(buff);
+ }
+
+ // If the length of InputStream is greater than the buffer, create a temporary file and return it
+ String name = String.format("%d-%s.dubbo.tmp", System.currentTimeMillis(), UUID.randomUUID().toString().replace("-", ""));
+ file = new File(tmpDir, name);
+ // Close the stream in finally
+ //noinspection resource
+ out = new FileOutputStream(file);
Review Comment:
Using temp file in `serilalization` directly is not a good idea
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org
Re: [PR] 修复InputStream反序列化bug [dubbo-hessian-lite]
Posted by "mcck (via GitHub)" <gi...@apache.org>.
mcck commented on code in PR #76:
URL: https://github.com/apache/dubbo-hessian-lite/pull/76#discussion_r1823738320
##########
java-8-test/src/test/java/com/alibaba/com/caucho/hessian/io/HessianInputStreamDeserializerTest.java:
##########
@@ -0,0 +1,136 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.alibaba.com.caucho.hessian.io;
+
+import com.alibaba.com.caucho.hessian.io.base.SerializeTestBase;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.EnabledForJreRange;
+import org.junit.jupiter.api.condition.JRE;
+
+import java.io.*;
+import java.util.Arrays;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertInstanceOf;
+
+/**
+ * InputStreamDeserializer 测试
+ */
+public class HessianInputStreamDeserializerTest extends SerializeTestBase {
+
+ protected <T> T baseHessian2Serialize(T data) throws IOException {
+ ByteArrayOutputStream bout = new ByteArrayOutputStream();
+ Hessian2Output out = new Hessian2Output(bout);
+
+ out.writeObject(data);
+ out.flush();
+
+ ByteArrayInputStream bin = new ByteArrayInputStream(bout.toByteArray());
+ Hessian2Input input = new Hessian2Input(bin);
+ Object obj = input.readObject(InputStream.class);
+ return (T) obj;
+ }
+
+ byte[] returnBytes(int size){
+ byte[] bytes = new byte[size];
+ Arrays.fill(bytes, (byte) 'A');
+ bytes[0] = '&';
+ bytes[bytes.length-3] = '$';
+ bytes[bytes.length-2] = '%';
+ bytes[bytes.length-1] = '#';
+ return bytes;
+ }
+
+ /**
+ * 测试1024字节，返回 ByteArrayInputStream
+ */
+ @Test
+ @EnabledForJreRange(max = JRE.JAVA_11)
+ public void test1024Byte() throws IOException {
+ int size = 1024;
+ byte[] bytes = returnBytes(size);
+ InputStream source = new ByteArrayInputStream(bytes);
+ InputStream result = baseHessian2Serialize(source);
+
+ byte[] resultByte = new byte[size];
+ result.read(resultByte);
+
+ assertInstanceOf(ByteArrayInputStream.class, result, "类型错误");
+ assertEquals(bytes.length, resultByte.length);
+ assertEquals(new String(bytes), new String(resultByte));
+ }
+
+ /**
+ * 测试8192字节，返回 ByteArrayInputStream
+ */
+ @Test
+ @EnabledForJreRange(max = JRE.JAVA_11)
+ public void test8192Byte() throws IOException {
+ int size = 8192;
+ byte[] bytes = returnBytes(size);
+ InputStream source = new ByteArrayInputStream(bytes);
+ InputStream result = baseHessian2Serialize(source);
+
+ byte[] resultByte = new byte[size];
+ result.read(resultByte);
+
+ assertInstanceOf(ByteArrayInputStream.class, result, "类型错误");
+ assertEquals(bytes.length, resultByte.length);
+ assertEquals(new String(bytes), new String(resultByte));
+ }
+
+ /**
+ * 测试8193字节，返回 ByteArrayInputStream
+ * 比缓冲区多一个字节
+ */
+ @Test
+ @EnabledForJreRange(max = JRE.JAVA_11)
+ public void test8193Byte() throws IOException {
+ int size = 8193;
+ byte[] bytes = returnBytes(size);
+ InputStream source = new ByteArrayInputStream(bytes);
+ InputStream result = baseHessian2Serialize(source);
+
+ byte[] resultByte = new byte[size];
+ result.read(resultByte);
+
+ assertInstanceOf(FileInputStream.class, result, "类型错误");
+ assertEquals(bytes.length, resultByte.length);
+ assertEquals(new String(bytes), new String(resultByte));
+ }
+
+ /**
+ * 测试81920字节，返回 ByteArrayInputStream
+ * 比缓冲区多10倍字节
+ */
Review Comment:
Okay, it has been modified
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@dubbo.apache.org
For additional commands, e-mail: notifications-help@dubbo.apache.org