You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "atongpu (via GitHub)" <gi...@apache.org> on 2025/04/21 08:33:51 UTC
[I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
atongpu opened a new issue, #9382:
URL: https://github.com/apache/incubator-gluten/issues/9382
### Description
I encountered the issue described in the title while running the following SQL:
> set spark.gluten.enabled=true;
create TABLE test.test_gluten using parquet CLUSTERED BY (id) sorted by (name) INTO 1000 BUCKETS as select id, cast(id as string) as name from range(10000);
### Gluten version
Gluten-1.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf closed issue #9382: [VL] Unsupported native write: Non-compatible hive bucket write is not supported
URL: https://github.com/apache/incubator-gluten/issues/9382
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
Posted by "wangyum (via GitHub)" <gi...@apache.org>.
wangyum commented on issue #9382:
URL: https://github.com/apache/incubator-gluten/issues/9382#issuecomment-2819734898
It is parquet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
Posted by "atongpu (via GitHub)" <gi...@apache.org>.
atongpu commented on issue #9382:
URL: https://github.com/apache/incubator-gluten/issues/9382#issuecomment-2817940566

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #9382:
URL: https://github.com/apache/incubator-gluten/issues/9382#issuecomment-2819682669
Is it hive or iceberg? iceberg write isn't supported yet. it's still WIP
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
Posted by "JkSelf (via GitHub)" <gi...@apache.org>.
JkSelf commented on issue #9382:
URL: https://github.com/apache/incubator-gluten/issues/9382#issuecomment-2819865860
@atongpu @wangyum Velox does not support bucket with non-partitioned table. You can refer [here](https://github.com/apache/incubator-gluten/blob/main/backends-velox/src/main/scala/org/apache/gluten/backendsapi/velox/VeloxBackend.scala#L351-L359).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
Posted by "wangyum (via GitHub)" <gi...@apache.org>.
wangyum commented on issue #9382:
URL: https://github.com/apache/incubator-gluten/issues/9382#issuecomment-2818815850
@JkSelf Both are repartitioning, but writing to non-bucket table is much faster than writing to bucket table. We better support writing to bucket table:
```sql
create table db.gluten_bucket using parquet
using parquet
CLUSTERED BY (id)
INTO 5000 BUCKETS
as
SELECT * from db.table
distribute by id;
create table db.gluten_repartition using parquet
using parquet
as
SELECT /*+ REPARTITION(5000, id) */ * from db.table;
```
<img width="959" alt="Image" src="https://github.com/user-attachments/assets/48583bd8-b874-4614-a6f4-2611c808b34d" />
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] Unsupported native write: Non-compatible hive bucket write is not supported [incubator-gluten]
Posted by "wangyum (via GitHub)" <gi...@apache.org>.
wangyum commented on issue #9382:
URL: https://github.com/apache/incubator-gluten/issues/9382#issuecomment-2823799559
Thank you @JkSelf .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org