You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "wangyum (via GitHub)" <gi...@apache.org> on 2025/10/28 03:53:59 UTC
[PR] [MINOR][VL] Improve temp parquet directory naming in FunctionsValidateSuite [incubator-gluten]
wangyum opened a new pull request, #10955:
URL: https://github.com/apache/incubator-gluten/pull/10955
## What changes are proposed in this pull request?
https://github.com/apache/incubator-gluten/blob/a8f45879d13e193733859bc9174612634df99c58/backends-velox/src/test/scala/org/apache/gluten/functions/FunctionsValidateSuite.scala#L70-L76
Spark writes output to a path. Our internal Spark distribution adds extra validation logic that rejects writes targeting a file path. This change ensures we always provide a directory path, improving compatibility with both upstream and internal Spark builds.
```
Cause: org.apache.hadoop.fs.ParentNotDirectoryException: Parent path is not a directory: file:/tmp/8263767318181008243.parquet
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:691)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:672)
at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:788)
at org.apache.spark.sql.InsertOperationLockUtil$.tryLock(InsertOperationLockUtil.scala:111)
at org.apache.spark.sql.InsertOperationLockUtil$.pickInsertOperationLock(InsertOperationLockUtil.scala:49)
at org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelationCommand.detectConflict(InsertIntoHadoopFsRelationCommand.scala:389)
...
```
## How was this patch tested?
Existing test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [MINOR][VL] Improve temp parquet directory in FunctionsValidateSuite [incubator-gluten]
Posted by "wForget (via GitHub)" <gi...@apache.org>.
wForget merged PR #10955:
URL: https://github.com/apache/incubator-gluten/pull/10955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org