You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by "grundprinzip (via GitHub)" <gi...@apache.org> on 2025/01/06 14:15:00 UTC
[I] Support Struct Conversion when reading Arrow data [spark-connect-go]
grundprinzip opened a new issue, #114:
URL: https://github.com/apache/spark-connect-go/issues/114
The following query fails:
```golang
_, err := spark.Sql(ctx, "SELECT struct(1, 'a') as col")
if err != nil {
log.Fatalf("Failed: %s", err)
}
```
with the error:
```
2025/01/06 15:13:51 Failed: unsupported arrow data type STRUCT
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [I] Support Struct Conversion when reading Arrow data [spark-connect-go]
Posted by "kronsbein (via GitHub)" <gi...@apache.org>.
kronsbein commented on issue #114:
URL: https://github.com/apache/spark-connect-go/issues/114#issuecomment-2578309298
Hi @grundprinzip, I’m new to Spark Connect Go, but want to start contributing. I’d like to take this up if it’s still available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [I] Support Struct Conversion when reading Arrow data [spark-connect-go]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip closed issue #114: Support Struct Conversion when reading Arrow data
URL: https://github.com/apache/spark-connect-go/issues/114
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [I] Support Struct Conversion when reading Arrow data [spark-connect-go]
Posted by "kronsbein (via GitHub)" <gi...@apache.org>.
kronsbein commented on issue #114:
URL: https://github.com/apache/spark-connect-go/issues/114#issuecomment-2579573771
Great, thank you, will do! I was wondering about how to implement this dynamically when the structs schema is unknown. Would it make sense to convert them into a `map[string]any`, using field names as keys and field values as values? Or would it be better to use reflection to create and populate a generic struct type dynamically at runtime?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [I] Support Struct Conversion when reading Arrow data [spark-connect-go]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip commented on issue #114:
URL: https://github.com/apache/spark-connect-go/issues/114#issuecomment-2578409855
Absolutely, please feel free to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org
Re: [I] Support Struct Conversion when reading Arrow data [spark-connect-go]
Posted by "grundprinzip (via GitHub)" <gi...@apache.org>.
grundprinzip commented on issue #114:
URL: https://github.com/apache/spark-connect-go/issues/114#issuecomment-2579583919
I think map[string]any is probably better because the receiver will have a hard time expecting the struct the right way.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org