You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "the-other-tim-brown (via GitHub)" <gi...@apache.org> on 2025/04/15 17:40:39 UTC
[PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
the-other-tim-brown opened a new pull request, #13148:
URL: https://github.com/apache/hudi/pull/13148
### Change Logs
General refactoring to move away from the spark specific broadcast nomenclature for ReaderContext to a serializable factory that serves basically the same purpose. Main changes:
- Serializable factory for generating the ReaderContext
- EngineContext will generate the factory since these readers will be engine specific
- ReaderContext now contains the storage configuration that reads should use
- Clean up use of closeable on existing code in the path
### Impact
This is mainly cleanup and setup work for making the FileGroupReader the default throughout the codebase.
### Risk level (write none, low medium or high below)
None
### Documentation Update
_Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none"._
- _The config description must be updated if new configs are added or the default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the [instruction](https://hudi.apache.org/contribute/developer-setup#website) to make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807187444
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 23382bd415614a5f92b15175bf4524f0fa6be670 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870)
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2045889367
##########
hudi-common/src/main/java/org/apache/hudi/common/util/LocalAvroSchemaCache.java:
##########
@@ -63,10 +59,4 @@ public Integer cacheSchema(Schema schema) {
public Option<Schema> getSchema(Integer versionId) {
return Option.ofNullable(this.versionIdToSchema.get(versionId));
}
-
- @Override
- public void close() {
- this.schemaToVersionId.clear();
Review Comment:
seems not releated with the change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807151867
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85a3a4aa7d1fba2e551bc1f37795075ddccb487e Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869)
* 23382bd415614a5f92b15175bf4524f0fa6be670 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2045887871
##########
hudi-common/src/main/java/org/apache/hudi/common/engine/ReaderContextFactory.java:
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hudi.common.engine;
+
+import java.io.Serializable;
+
+public interface ReaderContextFactory<T> extends Serializable {
Review Comment:
Can we add some doc to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2808085441
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
}, {
"hash" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"triggerType" : "PUSH"
}, {
"hash" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878",
"triggerID" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* 40c41a5b4a638c6f2ee4ec787359169e9acae44e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871)
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
* 68d520b92b1afa559ef2fe0eca780842067cc105 UNKNOWN
* 21216b5e0e839bcb59326730558ba903cd43dbd1 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807115762
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85a3a4aa7d1fba2e551bc1f37795075ddccb487e Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807003854
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85a3a4aa7d1fba2e551bc1f37795075ddccb487e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2047110008
##########
hudi-common/src/main/java/org/apache/hudi/common/util/LocalAvroSchemaCache.java:
##########
@@ -63,10 +59,4 @@ public Integer cacheSchema(Schema schema) {
public Option<Schema> getSchema(Integer versionId) {
return Option.ofNullable(this.versionIdToSchema.get(versionId));
}
-
- @Override
- public void close() {
- this.schemaToVersionId.clear();
Review Comment:
If the cleanup should be part of a follow up PR, then I can remove the changes related to this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2806998555
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85a3a4aa7d1fba2e551bc1f37795075ddccb487e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807197908
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 23382bd415614a5f92b15175bf4524f0fa6be670 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870)
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* 40c41a5b4a638c6f2ee4ec787359169e9acae44e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2045889682
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/common/SparkReaderContextFactory.java:
##########
@@ -55,31 +53,16 @@
import scala.Tuple2;
import scala.collection.JavaConverters;
-/**
- * Broadcast variable management for Spark.
- */
-public class SparkBroadcastManager extends EngineBroadcastManager {
-
- private final transient HoodieEngineContext context;
- private final transient HoodieTableMetaClient metaClient;
+class SparkReaderContextFactory implements ReaderContextFactory<InternalRow> {
Review Comment:
Add doc for this class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807310334
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 23382bd415614a5f92b15175bf4524f0fa6be670 Azure: [CANCELED](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870)
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* 40c41a5b4a638c6f2ee4ec787359169e9acae44e Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807513734
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* 40c41a5b4a638c6f2ee4ec787359169e9acae44e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807997587
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* 40c41a5b4a638c6f2ee4ec787359169e9acae44e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871)
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2808281173
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
}, {
"hash" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"triggerType" : "PUSH"
}, {
"hash" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878",
"triggerID" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
* 68d520b92b1afa559ef2fe0eca780842067cc105 UNKNOWN
* 21216b5e0e839bcb59326730558ba903cd43dbd1 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2046128760
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -319,9 +319,6 @@ public void close() throws IOException {
if (recordBuffer != null) {
recordBuffer.close();
}
- if (readerContext != null) {
- readerContext.close();
Review Comment:
what's the problem here with the close?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2809839222
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
}, {
"hash" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"triggerType" : "PUSH"
}, {
"hash" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878",
"triggerID" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"triggerType" : "PUSH"
}, {
"hash" : "a52ce5f99a36143a9b3674132c1112583d6e7527",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a52ce5f99a36143a9b3674132c1112583d6e7527",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
* 68d520b92b1afa559ef2fe0eca780842067cc105 UNKNOWN
* 21216b5e0e839bcb59326730558ba903cd43dbd1 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878)
* a52ce5f99a36143a9b3674132c1112583d6e7527 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2810088453
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
}, {
"hash" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"triggerType" : "PUSH"
}, {
"hash" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878",
"triggerID" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"triggerType" : "PUSH"
}, {
"hash" : "a52ce5f99a36143a9b3674132c1112583d6e7527",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4900",
"triggerID" : "a52ce5f99a36143a9b3674132c1112583d6e7527",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
* 68d520b92b1afa559ef2fe0eca780842067cc105 UNKNOWN
* a52ce5f99a36143a9b3674132c1112583d6e7527 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4900)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2047995743
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -319,9 +319,6 @@ public void close() throws IOException {
if (recordBuffer != null) {
recordBuffer.close();
}
- if (readerContext != null) {
- readerContext.close();
Review Comment:
you mean the schema cache is small?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807126648
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85a3a4aa7d1fba2e551bc1f37795075ddccb487e Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869)
* 23382bd415614a5f92b15175bf4524f0fa6be670 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2807183548
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 85a3a4aa7d1fba2e551bc1f37795075ddccb487e Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869)
* 23382bd415614a5f92b15175bf4524f0fa6be670 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870)
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2808071695
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
}, {
"hash" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"triggerType" : "PUSH"
}, {
"hash" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* 40c41a5b4a638c6f2ee4ec787359169e9acae44e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871)
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
* 68d520b92b1afa559ef2fe0eca780842067cc105 UNKNOWN
* 21216b5e0e839bcb59326730558ba903cd43dbd1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2809862390
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
}, {
"hash" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"triggerType" : "PUSH"
}, {
"hash" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878",
"triggerID" : "21216b5e0e839bcb59326730558ba903cd43dbd1",
"triggerType" : "PUSH"
}, {
"hash" : "a52ce5f99a36143a9b3674132c1112583d6e7527",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4900",
"triggerID" : "a52ce5f99a36143a9b3674132c1112583d6e7527",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
* 68d520b92b1afa559ef2fe0eca780842067cc105 UNKNOWN
* 21216b5e0e839bcb59326730558ba903cd43dbd1 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4878)
* a52ce5f99a36143a9b3674132c1112583d6e7527 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4900)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2048002135
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -319,9 +319,6 @@ public void close() throws IOException {
if (recordBuffer != null) {
recordBuffer.close();
}
- if (readerContext != null) {
- readerContext.close();
Review Comment:
No, the schema cache does not actually have anything that requires closing. The cache object is only referenced from the context that creates it. Once that context is no longer referenced, the cache is eligible for GC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 merged PR #13148:
URL: https://github.com/apache/hudi/pull/13148
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2046880700
##########
hudi-common/src/main/java/org/apache/hudi/common/util/LocalAvroSchemaCache.java:
##########
@@ -63,10 +59,4 @@ public Integer cacheSchema(Schema schema) {
public Option<Schema> getSchema(Integer versionId) {
return Option.ofNullable(this.versionIdToSchema.get(versionId));
}
-
- @Override
- public void close() {
- this.schemaToVersionId.clear();
Review Comment:
It's part of the cleanup. Generally there seems to be over use of closeable in some new code. This cleanup will make it easier to be consistent with enforcing that all closeables are closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "the-other-tim-brown (via GitHub)" <gi...@apache.org>.
the-other-tim-brown commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2046877179
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -319,9 +319,6 @@ public void close() throws IOException {
if (recordBuffer != null) {
recordBuffer.close();
}
- if (readerContext != null) {
- readerContext.close();
Review Comment:
The reader context doesn't have anything to close so there is no need for this line currently and no need to make the class implement closeable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #13148:
URL: https://github.com/apache/hudi/pull/13148#discussion_r2045763183
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/compact/HoodieCompactor.java:
##########
@@ -297,14 +286,12 @@ public List<WriteStatus> compact(HoodieCompactionHandler compactionHandler,
* Execute a single compaction operation and report back status.
*/
public List<WriteStatus> compact(HoodieCompactionHandler compactionHandler,
- HoodieTableMetaClient metaClient,
- HoodieWriteConfig writeConfig,
- CompactionOperation operation,
- String instantTime,
- Option<EngineBroadcastManager> broadcastManagerOpt) throws IOException {
+ HoodieWriteConfig writeConfig,
Review Comment:
let's fix the indentation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [HUDI-8637] Refactor broadcast to a more generic reader context factory [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #13148:
URL: https://github.com/apache/hudi/pull/13148#issuecomment-2808051752
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4869",
"triggerID" : "85a3a4aa7d1fba2e551bc1f37795075ddccb487e",
"triggerType" : "PUSH"
}, {
"hash" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4870",
"triggerID" : "23382bd415614a5f92b15175bf4524f0fa6be670",
"triggerType" : "PUSH"
}, {
"hash" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "a25e7bdb3c383751972d8aa37b219a2ff233bcdc",
"triggerType" : "PUSH"
}, {
"hash" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871",
"triggerID" : "40c41a5b4a638c6f2ee4ec787359169e9acae44e",
"triggerType" : "PUSH"
}, {
"hash" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "cb08a635f469b222df4e3d117c9793cbe33da8a0",
"triggerType" : "PUSH"
}, {
"hash" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "68d520b92b1afa559ef2fe0eca780842067cc105",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* a25e7bdb3c383751972d8aa37b219a2ff233bcdc UNKNOWN
* 40c41a5b4a638c6f2ee4ec787359169e9acae44e Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=4871)
* cb08a635f469b222df4e3d117c9793cbe33da8a0 UNKNOWN
* 68d520b92b1afa559ef2fe0eca780842067cc105 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org