You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@texera.apache.org by "Yicong-Huang (via GitHub)" <gi...@apache.org> on 2026/05/03 01:35:28 UTC
[I] ArrowUtils round-trip loses AttributeType.ANY (becomes STRING) [texera]
Yicong-Huang opened a new issue, #4762:
URL: https://github.com/apache/texera/issues/4762
### What happened?
`common/workflow-core/src/main/scala/org/apache/texera/amber/util/ArrowUtils.scala::fromAttributeType` maps `STRING`, `LARGE_BINARY`, and `ANY` all to `ArrowType.Utf8.INSTANCE`. `LARGE_BINARY` is recovered via field metadata (`texera_type=LARGE_BINARY`) by `toTexeraSchema`, but `ANY` carries no metadata, so a schema round-trip (`toTexeraSchema(fromTexeraSchema(schema))`) silently turns every `ANY` attribute into `STRING`. The cross-language schema bridge therefore loses the `ANY` distinction entirely.
### How to reproduce?
```scala
import org.apache.texera.amber.core.tuple.{Attribute, AttributeType, Schema}
import org.apache.texera.amber.util.ArrowUtils
val original = Schema(List(new Attribute("v", AttributeType.ANY)))
val recovered = ArrowUtils.toTexeraSchema(ArrowUtils.fromTexeraSchema(original))
// recovered.getAttributes.head.getType == AttributeType.STRING (information lost)
```
### Version
1.1.0-incubating (Pre-release/Master)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] ArrowUtils round-trip loses AttributeType.ANY (becomes STRING) [texera]
Posted by "kunwp1 (via GitHub)" <gi...@apache.org>.
kunwp1 closed issue #4762: ArrowUtils round-trip loses AttributeType.ANY (becomes STRING)
URL: https://github.com/apache/texera/issues/4762
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] ArrowUtils round-trip loses AttributeType.ANY (becomes STRING) [texera]
Posted by "Ma77Ball (via GitHub)" <gi...@apache.org>.
Ma77Ball commented on issue #4762:
URL: https://github.com/apache/texera/issues/4762#issuecomment-4405183183
/take
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@texera.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org