You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "aqluheng (via GitHub)" <gi...@apache.org> on 2024/11/04 12:51:42 UTC

[I] [VL] TableScan may report an error when using filecache. [incubator-gluten]

aqluheng opened a new issue, #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805

   ### Backend
   
   VL (Velox)
   
   ### Bug description
   
   When using Gluten-1.2.0, an error is reported after a filecache hit once filecache is set.
   ```shell
   spark.gluten.sql.columnar.backend.velox.fileHandleCacheEnabled  true
   spark.gluten.sql.columnar.backend.velox.cacheEnabled            true
   spark.gluten.sql.columnar.backend.velox.memCacheSize            2500000000
   spark.gluten.sql.columnar.backend.velox.ssdCacheSize            0
   ``` 
   TPCDS32 q9 will report this error.
   24/11/04 20:45:08 WARN TaskSetManager: Lost task 91.0 in stage 6.0 (TID 12659) (node3 executor 20): org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Error during calling Java code from native code: org.apache.gluten.exception.GlutenException: org.apache.gluten.exception.GlutenException: Exception: VeloxRuntimeError
   Error Source: RUNTIME
   Error Code: INVALID_STATE
   Reason: No magic bytes found at end of the Parquet file
   Retriable: False
   Expression: strncmp(copy.data() + readSize - 4, "PAR1", 4) == 0
   
   ### Spark version
   
   None
   
   ### Spark configurations
   
   _No response_
   
   ### System information
   
   _No response_
   
   ### Relevant logs
   
   _No response_


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]

Posted by "Zand100 (via GitHub)" <gi...@apache.org>.
Zand100 commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-2458644227

   Hi, I would like to work on adding filecache to Gluten, to learn more about Gluten. Do you think that's feasible for someone new to the project? How would you recommend me to get started, please? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]

Posted by "beliefer (via GitHub)" <gi...@apache.org>.
beliefer commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-3509699432

   @aqluheng I met the same issue. How did you fix it?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]

Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-2455874996

   Filecache actually doesn't work in Gluten. There is no UT to track it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org


Re: [I] [VL] TableScan may report an error when using filecache. [incubator-gluten]

Posted by "FelixYBW (via GitHub)" <gi...@apache.org>.
FelixYBW commented on issue #7805:
URL: https://github.com/apache/incubator-gluten/issues/7805#issuecomment-2459205515

   @Zand100 I don't think it's a good start. The part is too complex. To fully enable file cache, we need much modification to Velox, even need to refactor the cache code.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org