You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brpc.apache.org by "dashjay (via GitHub)" <gi...@apache.org> on 2024/07/03 04:13:56 UTC
[I] GRPC python client can not read big response from brpc server (brpc)
dashjay opened a new issue, #2676:
URL: https://github.com/apache/brpc/issues/2676
**Describe the bug (描述bug)**
When server response a big value(more than 4096K), client will get error or hanging.
**To Reproduce (复现方法)**
I edited the example echo client/server, client send 8M message to server and receive response result success.
https://github.com/dashjay/brpc/commit/923555a205288ebe1e0c251a4e83506e2f7a1bb4
Then when I generated a python grpc client and write a small case
```
from math import exp
import grpc
import echo_pb2
import echo_pb2_grpc
import random
import time
import string
keySize = 4096
size = keySize * 1024
v_large = ''.join(random.choices(string.ascii_lowercase, k=size))
bv_large = str.encode(v_large)
options = [
('grpc.max_receive_message_length', 83804160),
('grpc.max_send_message_length', 83804160),
]
channel = grpc.insecure_channel('0.0.0.0:8000', [])
client = echo_pb2_grpc.EchoServiceStub(channel)
req = echo_pb2.EchoRequest(message = bv_large)
def send():
resp = client.Echo(req, timeout=10)
print(len(resp.message))
if __name__ == '__main__':
while True:
try:
send()
except KeyboardInterrupt as e:
break
```
server run normally:
```
I0703 12:02:46.774291 5379 755914244353 server.cpp:60] Received request[log_id=0] from 127.0.0.1:53151 to 127.0.0.1:8000: length(4194304) (attached=)
I0703 12:02:46.787413 5379 755914244353 server.cpp:89] req: length(4194318) res: length(4194318)
```
but grpc python client:
```
bytedance@FY2MY2TWDK ~/D/c/b/e/echo_c++ (issue/server-no-buffer-for-grpc-client)> /Users/bytedance/.pyenv/versions/3.10.13/bin/python3 python_client.py
Traceback (most recent call last):
File "/codes/brpc/example/echo_c++/python_client.py", line 31, in <module>
send()
File "/codes/brpc/example/echo_c++/python_client.py", line 24, in send
resp = client.Echo(req, timeout=10)
File "/usr/lib/python3.10/site-packages/grpc/_channel.py", line 1181, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/usr/python3.10/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "Received RST_STREAM with error code 3"
debug_error_string = "UNKNOWN:Error received from peer {created_time:"2024-07-03T12:02:46.789109+08:00", grpc_status:13, grpc_message:"Received RST_STREAM with error code 3"}"
>
```
**Expected behavior (期望行为)**
Get the echo value.
I found on web and get this https://groups.google.com/g/grpc-io/c/3h-Q5Vjq5aU ,I suspect this has something to do with it
**Versions (各种版本)**
OS: Darwin FY2MY2TWDK 23.3.0 Darwin Kernel Version 23.3.0
Compiler: `Apple clang version 15.0.0 (clang-1500.3.9.4)`
brpc: `master`
protobuf: ` protobuf: stable 27.1 (bottled)`
**Additional context/screenshots (更多上下文/截图)**
BRPC server interact with BRPC client good
<img width="1670" alt="截屏2024-07-03 12 10 52" src="https://github.com/apache/brpc/assets/45532257/72a6ed42-c22b-4cb9-8b72-7c0c5ee5c790">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
Re: [I] GRPC python client can not read big response from BRPC server (brpc)
Posted by "dashjay (via GitHub)" <gi...@apache.org>.
dashjay commented on issue #2676:
URL: https://github.com/apache/brpc/issues/2676#issuecomment-2213364819
You can easily use command like this to access BRPC server with http proto
```
dd if=/dev/random of=/tmp/test bs=4M count=1
echo "{\"message\": \"$(cat /tmp/test | base64)\"}" > test.1
curl -H 'Content-Type: application/json' --data-binary "@./test.1" 0.0.0.0:8000/EchoService/Echo > /tmp/test.2
```
But you can not use grpc client to do this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
Re: [I] GRPC python client can not read big response from BRPC server (brpc)
Posted by "dashjay (via GitHub)" <gi...@apache.org>.
dashjay commented on issue #2676:
URL: https://github.com/apache/brpc/issues/2676#issuecomment-2355389575
> We encountered the same problem, how was it solved?
@wlleiiwang Hi, finally we solve this problem by invoking the HTTP interface directly, brpc server implement http interface automaticly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org
Re: [I] GRPC python client can not read big response from BRPC server (brpc)
Posted by "wlleiiwang (via GitHub)" <gi...@apache.org>.
wlleiiwang commented on issue #2676:
URL: https://github.com/apache/brpc/issues/2676#issuecomment-2330938072
We encountered the same problem, how was it solved?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@brpc.apache.org
For additional commands, e-mail: dev-help@brpc.apache.org