You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "powersj (via GitHub)" <gi...@apache.org> on 2024/06/13 15:46:15 UTC
[I] [GO] Flight query hides internal error messages [arrow]
powersj opened a new issue, #42145:
URL: https://github.com/apache/arrow/issues/42145
### Describe the bug, including details regarding any error messages, version, and platform.
## Background
We have found that there is a difference between libraires when running the same query. The Go library in this example seems to hide or prevent an internal error from showing up. While the Python library throws an exception with the error (which is what we expect).
Happy to provide the token I am using to aid in reproducing with the code used below.
## Expected result
An error message from the client:
```
External error: Panic: internal error: entered unreachable code. gRPC client debug context: UNKNOWN:Error received from peer ipv4:34.196.233.7:443 {created_time:"2024-06-13T09:01:48.010211097-06:00", grpc_status:13, grpc_message:"Join Error (panic)\ncaused by\nExternal error: Panic: internal error: entered unreachable code"}. Client context: OK
```
## Actual result
No rows or errors are returned making it seem like the query was correct, but no data was found.
## Go code example
```go
package main
import (
"context"
"crypto/tls"
"encoding/json"
"fmt"
"github.com/apache/arrow/go/v17/arrow/flight"
"github.com/apache/arrow/go/v17/arrow/ipc"
"github.com/apache/arrow/go/v17/arrow/memory"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/metadata"
)
var (
token = ""
host = "us-east-1-1.aws.cloud2.influxdata.com"
port = "443"
database = "test"
language = "sql"
query = `
SELECT array_to_string(array_agg(instance),',')
FROM (
SELECT DISTINCT instance
FROM repro
WHERE time >= now() - INTERVAL '1 hours'
)`
)
func main() {
ticketJSON, err := json.Marshal(map[string]interface{}{
"database": database,
"sql_query": query,
"query_type": language,
})
if err != nil {
panic(fmt.Errorf("unable to serialize ticket data to JSON: %w", err))
}
ticket := &flight.Ticket{Ticket: ticketJSON}
opts := []grpc.DialOption{
grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{})),
}
client, err := flight.NewClientWithMiddleware(fmt.Sprintf("%s:%s", host, port), nil, nil, opts...)
if err != nil {
panic(fmt.Errorf("unable to create flight client: %w", err))
}
ctx := metadata.AppendToOutgoingContext(context.Background(), "authorization", "Bearer "+token)
stream, err := client.DoGet(ctx, ticket)
if err != nil {
panic(fmt.Errorf("failed calling client: %w", err))
}
reader, err := flight.NewRecordReader(stream, ipc.WithAllocator(memory.DefaultAllocator))
if err != nil {
panic(fmt.Errorf("unable to get flight reader: %w", err))
}
for reader.Next() {
fmt.Println(reader.Record())
}
reader.Release()
fmt.Println("Query executed successfully!")
}
```
outputs:
```s
❯ go run .
Query executed successfully!
```
## Python code example
```py3
#!/usr/bin/env python3
import json
from pyarrow.flight import FlightClient, Ticket, FlightCallOptions
token=""
host="us-east-1-1.aws.cloud2.influxdata.com"
port=443
database="test"
language = "sql"
query = """
SELECT array_to_string(array_agg(instance),',')
FROM (
SELECT DISTINCT instance
FROM repro
WHERE time >= now() - INTERVAL '1 hours'
)
"""
flight_client = FlightClient(f"grpc+tls://{host}:{port}")
opts = {
"headers": [(b"authorization", f"Bearer {token}".encode('utf-8'))],
"timeout": 300
}
options = FlightCallOptions(**opts)
ticket_data = {
"database": database,
"sql_query": query,
"query_type": language
}
ticket = Ticket(json.dumps(ticket_data).encode('utf-8'))
flight_reader = flight_client.do_get(ticket, options)
flight_reader.read_all()
```
outputs:
```s
❯ ./query.py
Traceback (most recent call last):
File "/home/powersj/influxdb-client-tools/python/v3/./query.py", line 37, in <module>
flight_reader.read_all()
File "pyarrow/_flight.pyx", line 1075, in pyarrow._flight.FlightStreamReader.read_all
File "pyarrow/_flight.pyx", line 1078, in pyarrow._flight.FlightStreamReader.read_all
File "pyarrow/_flight.pyx", line 55, in pyarrow._flight.check_flight_status
pyarrow._flight.FlightInternalError: Flight returned internal error, with message: Join Error (panic)
caused by
External error: Panic: internal error: entered unreachable code. gRPC client debug context: UNKNOWN:Error received from peer ipv4:34.196.233.7:443 {created_time:"2024-06-13T09:25:46.298673099-06:00", grpc_status:13, grpc_message:"Join Error (panic)\ncaused by\nExternal error: Panic: internal error: entered unreachable code"}. Client context: OK
```
### Component(s)
Go
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [GO] Flight query hides internal error messages [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #42145:
URL: https://github.com/apache/arrow/issues/42145#issuecomment-2166925740
Ah the way it works IIRC is that you have to check it after `reader.Next()` returns false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [GO] Flight query hides internal error messages [arrow]
Posted by "powersj (via GitHub)" <gi...@apache.org>.
powersj commented on issue #42145:
URL: https://github.com/apache/arrow/issues/42145#issuecomment-2166073084
> I can't quite check it right now but I believe `reader` has an `Err` that needs to be explicitly polled
You are correct. Reader does have a `Err`, however it appears to be nil as well. I have updated the code example to check that for non-nil.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Go] Flight query hides internal error messages [arrow]
Posted by "Yaswi (via GitHub)" <gi...@apache.org>.
Yaswi commented on issue #42145:
URL: https://github.com/apache/arrow/issues/42145#issuecomment-2599018093
@lidavidm, @powersj Is there any possibility that we can remove this ```gRPC client debug context ``` and the error after that coming up on our client? Its seems to be repetitive.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Go] Flight query hides internal error messages [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #42145:
URL: https://github.com/apache/arrow/issues/42145#issuecomment-2599346583
Can you please file a new issue with a reproducer/some context?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [GO] Flight query hides internal error messages [arrow]
Posted by "lidavidm (via GitHub)" <gi...@apache.org>.
lidavidm commented on issue #42145:
URL: https://github.com/apache/arrow/issues/42145#issuecomment-2166065894
I can't quite check it right now but I believe `reader` has an `Err` that needs to be explicitly polled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [GO] Flight query hides internal error messages [arrow]
Posted by "powersj (via GitHub)" <gi...@apache.org>.
powersj closed issue #42145: [GO] Flight query hides internal error messages
URL: https://github.com/apache/arrow/issues/42145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [GO] Flight query hides internal error messages [arrow]
Posted by "powersj (via GitHub)" <gi...@apache.org>.
powersj commented on issue #42145:
URL: https://github.com/apache/arrow/issues/42145#issuecomment-2166993332
> Ah the way it works IIRC is that you have to check it after `reader.Next()` returns false.
That was it! Thank you very much!
```sh
❯ go run .
panic: unable to read flight data: rpc error: code = Internal desc = Join Error (panic)
caused by
External error: Panic: internal error: entered unreachable code
goroutine 1 [running]:
main.main()
/home/powersj/influxdb-client-tools/go/query/main.go:68 +0x676
exit status 2
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org