You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Eric Marnadi (Jira)" <ji...@apache.org> on 2024/11/28 01:02:00 UTC
[jira] [Updated] (SPARK-50442) Use Guava cache to store Avro encoders
[ https://issues.apache.org/jira/browse/SPARK-50442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Marnadi updated SPARK-50442:
---------------------------------
Description:
There are maven errors introduced by the guava dependency in `sql/core`, as we use the Guava cache to store the Avro encoders, outlined in this comment: [https://github.com/apache/spark/pull/48401#issuecomment-2504353098]
Change the NonFateSharingCache to not use the Guava cache
It looks like the PR CI doesn't run the suites with Maven, which caused this build error to go unnoticed on merge, but was caught in post-merge. To test this, we need to run the following commands:
{code:java}
build/mvn clean install -DskipTests -pl sql/core -am
build/mvn test -pl sql/core -DwildcardSuites=none -Dtest=test.org.apache.spark.sql.JavaDatasetSuite{code}
Re-introduce use of Guava cache without build errors
> Use Guava cache to store Avro encoders
> --------------------------------------
>
> Key: SPARK-50442
> URL: https://issues.apache.org/jira/browse/SPARK-50442
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Affects Versions: 4.0.0
> Reporter: Eric Marnadi
> Priority: Major
>
> There are maven errors introduced by the guava dependency in `sql/core`, as we use the Guava cache to store the Avro encoders, outlined in this comment: [https://github.com/apache/spark/pull/48401#issuecomment-2504353098]
> Change the NonFateSharingCache to not use the Guava cache
> It looks like the PR CI doesn't run the suites with Maven, which caused this build error to go unnoticed on merge, but was caught in post-merge. To test this, we need to run the following commands:
> {code:java}
> build/mvn clean install -DskipTests -pl sql/core -am
> build/mvn test -pl sql/core -DwildcardSuites=none -Dtest=test.org.apache.spark.sql.JavaDatasetSuite{code}
> Re-introduce use of Guava cache without build errors
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org