You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "rangareddy (via GitHub)" <gi...@apache.org> on 2025/05/13 09:46:34 UTC
[PR] [HUDI-9400] - Included Caffeine jar to the Hudi Hive Sync Bundle [hudi]
rangareddy opened a new pull request, #13296:
URL: https://github.com/apache/hudi/pull/13296
### Change Logs
Got the following exception while running the Hudi Hive Sync Bundle inside Docker image.
```
Exception in thread "main" java.lang.NoClassDefFoundError: com/github/benmanes/caffeine/cache/Caffeine
at org.apache.hudi.avro.AvroSchemaCache.<clinit>(AvroSchemaCache.java:36)
at org.apache.hudi.common.table.log.block.HoodieDataBlock.<init>(HoodieDataBlock.java:115)
at org.apache.hudi.common.table.log.block.HoodieHFileDataBlock.<init>(HoodieHFileDataBlock.java:79)
at org.apache.hudi.common.table.log.HoodieLogFileReader.readBlock(HoodieLogFileReader.java:202)
at org.apache.hudi.common.table.log.HoodieLogFileReader.next(HoodieLogFileReader.java:386)
at org.apache.hudi.common.table.log.HoodieLogFormatReader.next(HoodieLogFormatReader.java:102)
at org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner.scanInternalV1(AbstractHoodieLogRecordScanner.java:282)
at org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner.scanInternal(AbstractHoodieLogRecordScanner.java:252)
at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.performScan(HoodieMergedLogRecordScanner.java:207)
at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner.<init>(HoodieMergedLogRecordScanner.java:123)
at org.apache.hudi.common.table.log.HoodieMergedLogRecordScanner$Builder.build(HoodieMergedLogRecordScanner.java:490)
at org.apache.hudi.metadata.HoodieMetadataLogRecordReader$Builder.build(HoodieMetadataLogRecordReader.java:230)
at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:517)
at org.apache.hudi.metadata.HoodieBackedTableMetadata.openReaders(HoodieBackedTableMetadata.java:436)
at org.apache.hudi.metadata.HoodieBackedTableMetadata.getOrCreateReaders(HoodieBackedTableMetadata.java:421)
at org.apache.hudi.metadata.HoodieBackedTableMetadata.lookupKeysFromFileSlice(HoodieBackedTableMetadata.java:304)
at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordsByKeys(HoodieBackedTableMetadata.java:264)
at org.apache.hudi.metadata.HoodieBackedTableMetadata.getRecordByKey(HoodieBackedTableMetadata.java:155)
at org.apache.hudi.metadata.BaseTableMetadata.fetchAllPartitionPaths(BaseTableMetadata.java:316)
at org.apache.hudi.metadata.BaseTableMetadata.getAllPartitionPaths(BaseTableMetadata.java:124)
at org.apache.hudi.common.fs.FSUtils.getAllPartitionPaths(FSUtils.java:249)
at org.apache.hudi.sync.common.HoodieSyncClient.getAllPartitionPathsOnStorage(HoodieSyncClient.java:123)
at org.apache.hudi.hive.HiveSyncTool.syncAllPartitions(HiveSyncTool.java:469)
at org.apache.hudi.hive.HiveSyncTool.validateAndSyncPartitions(HiveSyncTool.java:321)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:261)
at org.apache.hudi.hive.HiveSyncTool.doSync(HiveSyncTool.java:189)
at org.apache.hudi.hive.HiveSyncTool.syncHoodieTable(HiveSyncTool.java:177)
at org.apache.hudi.hive.HiveSyncTool.main(HiveSyncTool.java:547)
Caused by: java.lang.ClassNotFoundException: com.github.benmanes.caffeine.cache.Caffeine
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 28 more
```
### Impact
Unable to perform the Hudi Hive Sync inside docker image.
Needs to include caffeine jar to the Hudi Hive Sync bundle.
```xml
<include>com.github.ben-manes.caffeine:caffeine</include>
```
**Output:**
```sh
% jar tvf packaging/hudi-hive-sync-bundle/target/hudi-hive-sync-bundle-1.1.0-SNAPSHOT.jar | grep 'caffeine' | head -5
0 Sun May 02 19:46:48 IST 2021 com/github/benmanes/caffeine/
2208 Sun May 02 19:46:48 IST 2021 com/github/benmanes/caffeine/SCQHeader$HeadAndTailRef.class
1040 Sun May 02 19:46:48 IST 2021 com/github/benmanes/caffeine/SCQHeader$HeadRef.class
2406 Sun May 02 19:46:48 IST 2021 com/github/benmanes/caffeine/SCQHeader$PadHead.class
2498 Sun May 02 19:46:48 IST 2021 com/github/benmanes/caffeine/SCQHeader$PadHeadAndTail.class
```
### Risk level (write none, low medium or high below)
Medium
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9400] - Included Caffeine jar to the Hudi Hive Sync Bundle [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13296:
URL: https://github.com/apache/hudi/pull/13296#issuecomment-2875790901
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10b656caf7e839bb5d39cfc03a35525f855666b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "e10b656caf7e839bb5d39cfc03a35525f855666b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e10b656caf7e839bb5d39cfc03a35525f855666b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9400] - Included Caffeine jar to the Hudi Hive Sync Bundle [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13296:
URL: https://github.com/apache/hudi/pull/13296#issuecomment-2875797473
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10b656caf7e839bb5d39cfc03a35525f855666b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5434",
"triggerID" : "e10b656caf7e839bb5d39cfc03a35525f855666b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e10b656caf7e839bb5d39cfc03a35525f855666b Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5434)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9400] - Included Caffeine jar to the Hudi Hive Sync Bundle [hudi]
Posted by "zhangyue19921010 (via GitHub)" <gi...@apache.org>.
zhangyue19921010 merged PR #13296:
URL: https://github.com/apache/hudi/pull/13296
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-9400] - Included Caffeine jar to the Hudi Hive Sync Bundle [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13296:
URL: https://github.com/apache/hudi/pull/13296#issuecomment-2876041099
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "e10b656caf7e839bb5d39cfc03a35525f855666b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5434",
"triggerID" : "e10b656caf7e839bb5d39cfc03a35525f855666b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* e10b656caf7e839bb5d39cfc03a35525f855666b Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=5434)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org