You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "aqluheng (via GitHub)" <gi...@apache.org> on 2024/11/04 12:51:42 UTC
[I] [VL] TableScan may report an error when using filecache. [incubator-gluten]
aqluheng opened a new issue, #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805
### Backend
VL (Velox)
### Bug description
When using Gluten-1.2.0, an error is reported after a filecache hit once filecache is set.
```shell
spark.gluten.sql.columnar.backend.velox.fileHandleCacheEnabled true
spark.gluten.sql.columnar.backend.velox.cacheEnabled true
spark.gluten.sql.columnar.backend.velox.memCacheSize 2500000000
spark.gluten.sql.columnar.backend.velox.ssdCacheSize 0
```
TPCDS32 q9 will report this error.
24/11/04 20:45:08 WARN TaskSetManager: Lost task 91.0 in stage 6.0 (TID 12659) (node3 executor 20): org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Error during calling Java code from native code: org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: No magic bytes found at end of the Parquet file
Retriable: False
Expression: strncmp(copy.data() + readSize - 4, "PAR1", 4) == 0
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]
Posted by "Zand100 (via GitHub)" <gi...@apache.org>.
Zand100 commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-2458644227
Hi, I would like to work on adding filecache to Gluten, to learn more about Gluten. Do you think that's feasible for someone new to the project? How would you recommend me to get started, please?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]
Posted by "beliefer (via GitHub)" <gi...@apache.org>.
beliefer commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-3509699432
@aqluheng I met the same issue. How did you fix it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-2455874996
Filecache actually doesn't work in Gluten. There is no UT to track it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]
Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-2459205515
@Zand100 I don't think it's a good start. The part is too complex. To fully enable file cache, we need much modification to Velox, even need to refactor the cache code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org