You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/12/07 01:13:26 UTC
[GitHub] [hudi] alexeykudinkin opened a new pull request, #7395: [MINOR] FIxing performance traps in Spark SQL `MERGE INTO` implementation
alexeykudinkin opened a new pull request, #7395:
URL: https://github.com/apache/hudi/pull/7395
### Change Logs
This PR cleans up Spark SQL Merge Into implementation fixing some of the performance traps:
- `SqlTypedRecord` relies on cache using Avro's `Schema`s as a key, therefore entailing that every lookup would entail `Schema.equals` which in turn has non-trivial overhead. Additional problem is that it's done for _every_ field, not just per-record which exacerbates the problem for wider tables.
Changelog
- Cleaned up `SqlTypedRecord`
- Cleaned up `ExpressionCodeGen` to instead handle just Catalyst's `InternalRow`
- Fixed caches w/ expression-evaluators to include target schema into the key to make sure there are no collisions in the cache in case multiple statements might be run w/in the same session
### Impact
Should considerably improve performance for those who use Merge Into
### Risk level (write none, low medium or high below)
Low
### Documentation Update
N/A
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on a diff in pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
yihua commented on code in PR #7395:
URL: https://github.com/apache/hudi/pull/7395#discussion_r1042806294
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala:
##########
@@ -242,11 +287,12 @@ class ExpressionPayload(record: GenericRecord,
*
* @return
*/
- private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): IndexedRecord = {
+ private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): GenericRecord = {
val leftSchema = sourceRecord.getSchema
val joinSchema = getMergedSchema(leftSchema, targetRecord.getSchema)
- val values = new ArrayBuffer[AnyRef]()
+ // TODO rebase onto JoinRecord
Review Comment:
still needed?
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala:
##########
@@ -289,65 +328,90 @@ object ExpressionPayload {
val PAYLOAD_RECORD_AVRO_SCHEMA = "hoodie.payload.record.schema"
/**
- * A cache for the serializedConditionAssignments to the compiled class after CodeGen.
- * The Map[IExpressionEvaluator, IExpressionEvaluator] is the map of the condition expression
- * to the assignments expression.
+ * To avoid compiling projections for Merge Into expressions for every record these
+ * are cached under a key of
+ * <ol>
+ * <li>Expression's (textual) representation</li>
+ * <li>Expected input-schema</li>
+ * </ol>
+ *
+ * NOTE: Schema is required b/c these cache is static and might be shared by multiple
+ * executed statements w/in a single Spark session
*/
- private val cache = Caffeine.newBuilder()
+ private val projectionsCache = Caffeine.newBuilder()
.maximumSize(1024)
- .build[String, Map[IExpressionEvaluator, IExpressionEvaluator]]()
+ .build[(String, Schema), Map[Projection, Projection]]()
private val schemaCache = Caffeine.newBuilder()
.maximumSize(16).build[String, Schema]()
+ private val mergedSchemaCache = Caffeine.newBuilder()
+ .maximumSize(16).build[(Schema, Schema), Schema]()
+
+ private val avroDeserializerCache = Caffeine.newBuilder()
+ .maximumSize(16).build[Schema, HoodieAvroDeserializer]()
+
+ private val avroSerializerCache = Caffeine.newBuilder()
+ .maximumSize(16).build[Schema, HoodieAvroSerializer]()
+
private def parseSchema(schemaStr: String): Schema = {
schemaCache.get(schemaStr,
new Function[String, Schema] {
override def apply(t: String): Schema = new Schema.Parser().parse(t)
})
}
+ private def getAvroDeserializerFor(schema: Schema) = {
+ avroDeserializerCache.get(schema, new Function[Schema, HoodieAvroDeserializer] {
+ override def apply(t: Schema): HoodieAvroDeserializer =
+ sparkAdapter.createAvroDeserializer(schema, convertAvroSchemaToStructType(schema))
+ })
+ }
+
+ private def getAvroSerializerFor(schema: Schema) = {
+ avroSerializerCache.get(schema, new Function[Schema, HoodieAvroSerializer] {
+ override def apply(t: Schema): HoodieAvroSerializer =
+ sparkAdapter.createAvroSerializer(convertAvroSchemaToStructType(schema), schema, isNullable(schema))
+ })
+ }
+
/**
- * Do the CodeGen for each condition and assignment expressions.We will cache it to reduce
+ * Do the CodeGen for each condition and assignment expressions.We will projectionsCache it to reduce
Review Comment:
nit: no need to fix this appearance of `cache`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope commented on a diff in pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
codope commented on code in PR #7395:
URL: https://github.com/apache/hudi/pull/7395#discussion_r1043166522
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala:
##########
@@ -242,11 +287,12 @@ class ExpressionPayload(record: GenericRecord,
*
* @return
*/
- private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): IndexedRecord = {
+ private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): GenericRecord = {
Review Comment:
Taking `IndexedRecord` as input and returning `GenericRecord` as output. Why so? Is it because if we later indeed have to deserialize the record then we do so by name?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [HUDI-5347][Stacked on 7370] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1341745074
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
}, {
"hash" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"triggerType" : "PUSH"
}, {
"hash" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50663b998e3395cd4e997f9efadd8cc0da3af5a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493)
* 17dd63164f5f134ea4a35d7ec95a3699a8b76d44 UNKNOWN
* dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [MINOR][Stacked on 7370] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1340289928
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50663b998e3395cd4e997f9efadd8cc0da3af5a4 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1341816323
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
}, {
"hash" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"triggerType" : "PUSH"
}, {
"hash" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13529",
"triggerID" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"triggerType" : "PUSH"
}, {
"hash" : "4372aa6594fbdfe29a89df7a347d589f2433dc84",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "4372aa6594fbdfe29a89df7a347d589f2433dc84",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50663b998e3395cd4e997f9efadd8cc0da3af5a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493)
* 17dd63164f5f134ea4a35d7ec95a3699a8b76d44 UNKNOWN
* dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13529)
* 4372aa6594fbdfe29a89df7a347d589f2433dc84 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7395:
URL: https://github.com/apache/hudi/pull/7395#discussion_r1042847867
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala:
##########
@@ -242,11 +287,12 @@ class ExpressionPayload(record: GenericRecord,
*
* @return
*/
- private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): IndexedRecord = {
+ private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): GenericRecord = {
val leftSchema = sourceRecord.getSchema
val joinSchema = getMergedSchema(leftSchema, targetRecord.getSchema)
- val values = new ArrayBuffer[AnyRef]()
+ // TODO rebase onto JoinRecord
Review Comment:
Yeah, leaving these for subsequent follow-up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7395:
URL: https://github.com/apache/hudi/pull/7395#discussion_r1043553515
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala:
##########
@@ -242,11 +287,12 @@ class ExpressionPayload(record: GenericRecord,
*
* @return
*/
- private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): IndexedRecord = {
+ private def joinRecord(sourceRecord: IndexedRecord, targetRecord: IndexedRecord): GenericRecord = {
Review Comment:
It's just interface misalignment. Didn't want to overload this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1341820648
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
}, {
"hash" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"triggerType" : "PUSH"
}, {
"hash" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"status" : "CANCELED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13529",
"triggerID" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"triggerType" : "PUSH"
}, {
"hash" : "4372aa6594fbdfe29a89df7a347d589f2433dc84",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13530",
"triggerID" : "4372aa6594fbdfe29a89df7a347d589f2433dc84",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 17dd63164f5f134ea4a35d7ec95a3699a8b76d44 UNKNOWN
* dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245 Azure: [CANCELED](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13529)
* 4372aa6594fbdfe29a89df7a347d589f2433dc84 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13530)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] alexeykudinkin commented on a diff in pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
alexeykudinkin commented on code in PR #7395:
URL: https://github.com/apache/hudi/pull/7395#discussion_r1042848019
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/payload/ExpressionPayload.scala:
##########
@@ -289,65 +328,90 @@ object ExpressionPayload {
val PAYLOAD_RECORD_AVRO_SCHEMA = "hoodie.payload.record.schema"
/**
- * A cache for the serializedConditionAssignments to the compiled class after CodeGen.
- * The Map[IExpressionEvaluator, IExpressionEvaluator] is the map of the condition expression
- * to the assignments expression.
+ * To avoid compiling projections for Merge Into expressions for every record these
+ * are cached under a key of
+ * <ol>
+ * <li>Expression's (textual) representation</li>
+ * <li>Expected input-schema</li>
+ * </ol>
+ *
+ * NOTE: Schema is required b/c these cache is static and might be shared by multiple
+ * executed statements w/in a single Spark session
*/
- private val cache = Caffeine.newBuilder()
+ private val projectionsCache = Caffeine.newBuilder()
.maximumSize(1024)
- .build[String, Map[IExpressionEvaluator, IExpressionEvaluator]]()
+ .build[(String, Schema), Map[Projection, Projection]]()
private val schemaCache = Caffeine.newBuilder()
.maximumSize(16).build[String, Schema]()
+ private val mergedSchemaCache = Caffeine.newBuilder()
+ .maximumSize(16).build[(Schema, Schema), Schema]()
+
+ private val avroDeserializerCache = Caffeine.newBuilder()
+ .maximumSize(16).build[Schema, HoodieAvroDeserializer]()
+
+ private val avroSerializerCache = Caffeine.newBuilder()
+ .maximumSize(16).build[Schema, HoodieAvroSerializer]()
+
private def parseSchema(schemaStr: String): Schema = {
schemaCache.get(schemaStr,
new Function[String, Schema] {
override def apply(t: String): Schema = new Schema.Parser().parse(t)
})
}
+ private def getAvroDeserializerFor(schema: Schema) = {
+ avroDeserializerCache.get(schema, new Function[Schema, HoodieAvroDeserializer] {
+ override def apply(t: Schema): HoodieAvroDeserializer =
+ sparkAdapter.createAvroDeserializer(schema, convertAvroSchemaToStructType(schema))
+ })
+ }
+
+ private def getAvroSerializerFor(schema: Schema) = {
+ avroSerializerCache.get(schema, new Function[Schema, HoodieAvroSerializer] {
+ override def apply(t: Schema): HoodieAvroSerializer =
+ sparkAdapter.createAvroSerializer(convertAvroSchemaToStructType(schema), schema, isNullable(schema))
+ })
+ }
+
/**
- * Do the CodeGen for each condition and assignment expressions.We will cache it to reduce
+ * Do the CodeGen for each condition and assignment expressions.We will projectionsCache it to reduce
Review Comment:
Will address in a follow-up (have another PR for Spark SQL)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1342304815
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
}, {
"hash" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"triggerType" : "PUSH"
}, {
"hash" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13529",
"triggerID" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"triggerType" : "PUSH"
}, {
"hash" : "4372aa6594fbdfe29a89df7a347d589f2433dc84",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13530",
"triggerID" : "4372aa6594fbdfe29a89df7a347d589f2433dc84",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 17dd63164f5f134ea4a35d7ec95a3699a8b76d44 UNKNOWN
* 4372aa6594fbdfe29a89df7a347d589f2433dc84 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13530)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [MINOR][Stacked on 7370] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1340285491
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50663b998e3395cd4e997f9efadd8cc0da3af5a4 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [HUDI-5347][Stacked on 7370] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1341753029
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
}, {
"hash" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"triggerType" : "PUSH"
}, {
"hash" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13529",
"triggerID" : "dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50663b998e3395cd4e997f9efadd8cc0da3af5a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493)
* 17dd63164f5f134ea4a35d7ec95a3699a8b76d44 UNKNOWN
* dd2f7e1d7d690fd0c2df2204d4f3b3d55c2e2245 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13529)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [HUDI-5347][Stacked on 7370] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1341736465
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
}, {
"hash" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "17dd63164f5f134ea4a35d7ec95a3699a8b76d44",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50663b998e3395cd4e997f9efadd8cc0da3af5a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493)
* 17dd63164f5f134ea4a35d7ec95a3699a8b76d44 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] codope merged pull request #7395: [HUDI-5347] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
codope merged PR #7395:
URL: https://github.com/apache/hudi/pull/7395
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #7395: [MINOR][Stacked on 7370] FIxing performance traps in Spark SQL `MERGE INTO` implementation
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #7395:
URL: https://github.com/apache/hudi/pull/7395#issuecomment-1340383714
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493",
"triggerID" : "50663b998e3395cd4e997f9efadd8cc0da3af5a4",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 50663b998e3395cd4e997f9efadd8cc0da3af5a4 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=13493)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org