You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "xborder (via GitHub)" <gi...@apache.org> on 2025/01/13 14:40:57 UTC
[I] [FLIGHT][GO] [arrow-adbc]
xborder opened a new issue, #2434:
URL: https://github.com/apache/arrow-adbc/issues/2434
### What would you like help with?
Hi,
I'm testing a connector that uses ADBC Go driver against a flight server. While I can obtain information about catalogs, schemas and tables, when it comes to the columns everything is empty.
I believe the issue is with the GO driver and compliance with FlightSQL but I'd like to get some validation on my interpretation.
FlightSQL specification states the following for `CommandGetTables` response:
```
....
The returned Arrow schema will be:
<
catalog_name: utf8,
db_schema_name: utf8,
table_name: utf8 not null,
table_type: utf8 not null,
[optional] table_schema: bytes not null (schema of the table as described in Schema.fbs::Schema,
it is serialized as an IPC message.)
>
Fields on table_schema may contain the following metadata:
- ARROW:FLIGHT:SQL:CATALOG_NAME - Table's catalog name
- ARROW:FLIGHT:SQL:DB_SCHEMA_NAME - Database schema name
- ARROW:FLIGHT:SQL:TABLE_NAME - Table name
- ARROW:FLIGHT:SQL:TYPE_NAME - The data source-specific name for the data type of the column.
- ARROW:FLIGHT:SQL:PRECISION - Column precision/size
- ARROW:FLIGHT:SQL:SCALE - Column scale/decimal digits if applicable
- ARROW:FLIGHT:SQL:IS_AUTO_INCREMENT - "1" indicates if the column is auto incremented, "0" otherwise.
- ARROW:FLIGHT:SQL:IS_CASE_SENSITIVE - "1" indicates if the column is case-sensitive, "0" otherwise.
- ARROW:FLIGHT:SQL:IS_READ_ONLY - "1" indicates if the column is read only, "0" otherwise.
- ARROW:FLIGHT:SQL:IS_SEARCHABLE - "1" indicates if the column is searchable via WHERE clause, "0" otherwise.
....
```
- I can see this information being returned to the driver so the server I'm testing against is respecting the specification.
- Regardless of the server returns, the result of GetObjects doesn't have type information about the existing columns
- While debugging the driver I think the issue is in how the column information is being processed (in `[shared_utils.go](https://github.com/apache/arrow-adbc/blob/c3149c0746d09877709e7625b7199e102d6d9c94/go/adbc/driver/internal/shared_utils.go#L378)`)
- If there's the columns contain metadata, there is no code looking for the metadata keys described in the specification
- JAVA ADBC seems to have a different [implementation](https://github.com/apache/arrow-adbc/blob/c3149c0746d09877709e7625b7199e102d6d9c94/java/driver/flight-sql/src/main/java/org/apache/arrow/adbc/driver/flightsql/GetObjectsMetadataReaders.java#L589C21-L589C22) where these metadata keys are used
Questions:
1. Does this mean that GO driver currently is not respecting the FlightSQL spec?
2. If this is the case what should it look for in the metadata? The `ARROW:FLIGHT:SQL:` keys described in the spec or the `xdbc_` keys? Should it look for `ARROW:FLIGHT:SQL:` and fallback to `xdbc_`?
3. Where can I get more information about these `xdbc_` metadata fields?
Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [FLIGHT][GO] CommandGetTables metadata being ignored [arrow-adbc]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade closed issue #2434: [FLIGHT][GO] CommandGetTables metadata being ignored
URL: https://github.com/apache/arrow-adbc/issues/2434
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [FLIGHT][GO] CommandGetTables metadata being ignored [arrow-adbc]
Posted by "davidhcoe (via GitHub)" <gi...@apache.org>.
davidhcoe commented on issue #2434:
URL: https://github.com/apache/arrow-adbc/issues/2434#issuecomment-2590082824
I think the genesis of this was as a utility function for non-FlightSQL databases.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [FLIGHT][GO] CommandGetTables metadata being ignored [arrow-adbc]
Posted by "xborder (via GitHub)" <gi...@apache.org>.
xborder commented on issue #2434:
URL: https://github.com/apache/arrow-adbc/issues/2434#issuecomment-2589443708
Cool, thank you for the confirmation. I'll try to fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [FLIGHT][GO] CommandGetTables metadata being ignored [arrow-adbc]
Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on issue #2434:
URL: https://github.com/apache/arrow-adbc/issues/2434#issuecomment-2587977818
Looks like you're absolutely correct, we're not properly propagating the metadata from flight sql.
> If this is the case what should it look for in the metadata? The ARROW:FLIGHT:SQL: keys described in the spec or the xdbc_ keys? Should it look for ARROW:FLIGHT:SQL: and fallback to xdbc_?
It should definitely be using the correct `ARROW:FLIGHT:SQL:*` keys and fallback to looking for the `XDBC_*` ones. I'll look into fixing this when I have the opportunity. That said, if you would like to fix it yourself, i'll happily review it!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org