You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/05/09 06:24:36 UTC
[GitHub] [hudi] jinxing64 opened a new pull request, #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
jinxing64 opened a new pull request, #5534:
URL: https://github.com/apache/hudi/pull/5534
## What is the purpose of the pull request
In current code, AlterHoodieTableDropPartitionCommand and TruncateHoodieTableCommand require all partition fields should be specified when droping / truncating partitions(s), otherwise complain with AnalyisException (HoodieSqlCommonUtils#normalizePartitionSpec).
But native Spark/Hive SQL have no such limitations – – partition matching is provided as a functionality helping user to manage partition(s) in an easy way. Say dropping partitions with a single SQL "alter table test drop partition (year='2020')", but rather to specify all the partitions from (year='2020', month='01', day='01') to (year='2020', month='12', day='31')
This PR propose to refine the partition matching logic when drop / truncate partition and remove the limitation mentioned above.
## Brief change log
- Refine HoodieSqlCommonUtils#normalizePartitionSpec -- Reuse Spark utilities for column verifying;
- HoodieSqlCommonUtils#getMatchingPartitions provides as a utility for partition matching;
- Corresponding change in AlterHoodieTableDropPartitionCommand and TruncateHoodieTableCommand
## Verify this pull request
- Added tests in TestAlterTableDropPartition and TestTruncateTable
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] XuQianJin-Stars commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r872949175
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTableDropPartition.scala:
##########
@@ -81,13 +81,13 @@ class TestAlterTableDropPartition extends HoodieSparkSqlTestBase {
}
Seq(false, true).foreach { urlencode =>
- test(s"Drop single-partition table' partitions, urlencode: $urlencode") {
+ test(s"Drop single-partition table's partitions, urlencode: $urlencode") {
withTempDir { tmp =>
val tableName = generateTableName
val tablePath = s"${tmp.getCanonicalPath}/$tableName"
import spark.implicits._
- val df = Seq((1, "z3", "v1", "2021/10/01"), (2, "l4", "v1", "2021/10/02"))
+ val df = Seq((1, "z3", "v1", "2021#10#01"), (2, "l4", "v1", "2021#10#02"))
Review Comment:
"#" While reducing errors, but increasing ut readability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1134228759
@XuQianJin-Stars
Mind to take another look :) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133582400
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5799064ab62362256a2c78d36d0c34596756a581 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808)
* 99ae88c935d7c47737e3ca2e2237000b12669944 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] XuQianJin-Stars commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
XuQianJin-Stars commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1123140593
> @xushiyan @XuQianJin-Stars Please take a look if it benefits hudi sql.
Sorry for the late reply, I will try my best to review this pr today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r879059238
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTableDropPartition.scala:
##########
@@ -80,111 +80,92 @@ class TestAlterTableDropPartition extends HoodieSparkSqlTestBase {
checkAnswer(s"show partitions $tableName")(Seq.empty: _*)
}
Review Comment:
For SQL testing, the valid config is (encodePartition=true, keyGen=SqlKeyGenerator). This PR simplify as below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r867670047
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTableDropPartition.scala:
##########
@@ -81,13 +81,13 @@ class TestAlterTableDropPartition extends HoodieSparkSqlTestBase {
}
Seq(false, true).foreach { urlencode =>
- test(s"Drop single-partition table' partitions, urlencode: $urlencode") {
+ test(s"Drop single-partition table's partitions, urlencode: $urlencode") {
withTempDir { tmp =>
val tableName = generateTableName
val tablePath = s"${tmp.getCanonicalPath}/$tableName"
import spark.implicits._
Review Comment:
It makes no sense to config urlencode=false when partition value contains slash -- '/' , it brings much obstacles when parsing partition values from path, say when executing 'show partitions'. I replace "/" with "#", which triggers character escaping but doesn't break partition path parsing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r878651499
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala:
##########
@@ -113,7 +113,10 @@ object HoodieWriterUtils {
def getOriginKeyGenerator(parameters: Map[String, String]): String = {
val kg = parameters.getOrElse(KEYGENERATOR_CLASS_NAME.key(), null)
if (classOf[SqlKeyGenerator].getCanonicalName == kg) {
- parameters.getOrElse(SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME, null)
+ parameters.get(SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME) match {
Review Comment:
Cover the case when originKeyGen is a SqlKeyGenerator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133816870
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
}, {
"hash" : "0024d0306e8671655004ff995c7754a99de6c59d",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0024d0306e8671655004ff995c7754a99de6c59d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 99b80b43a703e51e449e6a8670675a67a3554d3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813)
* 0024d0306e8671655004ff995c7754a99de6c59d UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1134068120
@hudi-bot
run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1120707522
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5deab6cb23cab172ce1e1940255e61ba3ab62e75 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1120882409
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5deab6cb23cab172ce1e1940255e61ba3ab62e75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133634907
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 99b80b43a703e51e449e6a8670675a67a3554d3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r879055918
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/catalyst/catalog/HoodieCatalogTable.scala:
##########
@@ -274,15 +274,15 @@ class HoodieCatalogTable(val spark: SparkSession, val table: CatalogTable) exten
}
} else {
extraConfig(HoodieTableConfig.HIVE_STYLE_PARTITIONING_ENABLE.key) = "true"
- extraConfig(HoodieTableConfig.URL_ENCODE_PARTITIONING.key) = HoodieTableConfig.URL_ENCODE_PARTITIONING.defaultValue()
+ extraConfig(HoodieTableConfig.URL_ENCODE_PARTITIONING.key) = "true"
Review Comment:
Current Hudi ProvidesHoodieConfig set default SQL configuration as (hivePartitioning=true, encodePartitionining=true, keyGenerator=SqlKeyGenerator). The change here adjusts configuration of HoodieCatalogTable be consistent with ProvidesConfig.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133588546
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5799064ab62362256a2c78d36d0c34596756a581 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808)
* 99ae88c935d7c47737e3ca2e2237000b12669944 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810)
* 956b4ff53aba2e91775ff1021c43d97d8a36f1e8 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133830072
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
}, {
"hash" : "0024d0306e8671655004ff995c7754a99de6c59d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829",
"triggerID" : "0024d0306e8671655004ff995c7754a99de6c59d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0024d0306e8671655004ff995c7754a99de6c59d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r879057203
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/ProvidesHoodieConfig.scala:
##########
@@ -124,7 +124,7 @@ trait ProvidesHoodieConfig extends Logging {
val preCombineField = hoodieCatalogTable.preCombineKey.getOrElse("")
val hiveStylePartitioningEnable = Option(tableConfig.getHiveStylePartitioningEnable).getOrElse("true")
Review Comment:
SQL functionality relies heavily on partition management and a encoded `partitionPath` is the key for support.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1141114566
@XuQianJin-Stars @xushiyan
Gentle ping to take another review ~ I updated the PR accordingly ~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] xushiyan commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
xushiyan commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1297013320
@jinxing64 are you planning to resolve conflicts and resume the work? thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r879053177
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala:
##########
@@ -113,7 +113,10 @@ object HoodieWriterUtils {
def getOriginKeyGenerator(parameters: Map[String, String]): String = {
val kg = parameters.getOrElse(KEYGENERATOR_CLASS_NAME.key(), null)
if (classOf[SqlKeyGenerator].getCanonicalName == kg) {
Review Comment:
Cover the case when origin is a SqlKeyGenerator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133599084
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 956b4ff53aba2e91775ff1021c43d97d8a36f1e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133626997
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 956b4ff53aba2e91775ff1021c43d97d8a36f1e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812)
* 99b80b43a703e51e449e6a8670675a67a3554d3e Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133622859
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 956b4ff53aba2e91775ff1021c43d97d8a36f1e8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812)
* 99b80b43a703e51e449e6a8670675a67a3554d3e UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1120705233
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5deab6cb23cab172ce1e1940255e61ba3ab62e75 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] yihua commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
yihua commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1244903086
cc @alexeykudinkin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133620024
@hudi-bot
run please.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133558223
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5799064ab62362256a2c78d36d0c34596756a581 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r878651499
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala:
##########
@@ -113,7 +113,10 @@ object HoodieWriterUtils {
def getOriginKeyGenerator(parameters: Map[String, String]): String = {
val kg = parameters.getOrElse(KEYGENERATOR_CLASS_NAME.key(), null)
if (classOf[SqlKeyGenerator].getCanonicalName == kg) {
- parameters.getOrElse(SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME, null)
+ parameters.get(SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME) match {
Review Comment:
Cover the case when originKeyGen is a SqlKeyGenerator.
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala:
##########
@@ -113,7 +113,10 @@ object HoodieWriterUtils {
def getOriginKeyGenerator(parameters: Map[String, String]): String = {
val kg = parameters.getOrElse(KEYGENERATOR_CLASS_NAME.key(), null)
if (classOf[SqlKeyGenerator].getCanonicalName == kg) {
- parameters.getOrElse(SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME, null)
+ parameters.get(SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME) match {
Review Comment:
Cover the case when originKeyGen is a SqlKeyGenerator.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133544458
WIP !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1134138326
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
}, {
"hash" : "0024d0306e8671655004ff995c7754a99de6c59d",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829",
"triggerID" : "0024d0306e8671655004ff995c7754a99de6c59d",
"triggerType" : "PUSH"
}, {
"hash" : "d834cab7033af3caa93e0baeb4975eab5258b31b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8839",
"triggerID" : "d834cab7033af3caa93e0baeb4975eab5258b31b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* d834cab7033af3caa93e0baeb4975eab5258b31b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8839)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r879058082
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/SqlKeyGenerator.scala:
##########
@@ -53,8 +55,17 @@ class SqlKeyGenerator(props: TypedProperties) extends ComplexKeyGenerator(props)
keyGenProps.remove(SqlKeyGenerator.ORIGIN_KEYGEN_CLASS_NAME)
val convertedKeyGenClassName = SqlKeyGenerator.getRealKeyGenClassName(props)
keyGenProps.put(HoodieWriteConfig.KEYGENERATOR_CLASS_NAME.key, convertedKeyGenClassName)
Review Comment:
SqlKeyGenerator as the default keyGenerator for Spark SQL. The change below checks if `encodePartition` config is turned on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121792309
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5deab6cb23cab172ce1e1940255e61ba3ab62e75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512)
* 697a8c68d64bd9144cb81c4c918e85d15b81a15b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121787903
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5deab6cb23cab172ce1e1940255e61ba3ab62e75 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512)
* 697a8c68d64bd9144cb81c4c918e85d15b81a15b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133860539
@hudi-bot
run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133860713
@hudi-bot
run azure
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133596685
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 99ae88c935d7c47737e3ca2e2237000b12669944 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810)
* 956b4ff53aba2e91775ff1021c43d97d8a36f1e8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133817159
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
}, {
"hash" : "0024d0306e8671655004ff995c7754a99de6c59d",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829",
"triggerID" : "0024d0306e8671655004ff995c7754a99de6c59d",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 99b80b43a703e51e449e6a8670675a67a3554d3e Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813)
* 0024d0306e8671655004ff995c7754a99de6c59d Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1120697816
wip
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121902158
@xushiyan @XuQianJin-Stars
Please take a look if it benefits hudi sql.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r873099587
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTableDropPartition.scala:
##########
@@ -81,13 +81,13 @@ class TestAlterTableDropPartition extends HoodieSparkSqlTestBase {
}
Seq(false, true).foreach { urlencode =>
- test(s"Drop single-partition table' partitions, urlencode: $urlencode") {
+ test(s"Drop single-partition table's partitions, urlencode: $urlencode") {
withTempDir { tmp =>
val tableName = generateTableName
val tablePath = s"${tmp.getCanonicalPath}/$tableName"
import spark.implicits._
- val df = Seq((1, "z3", "v1", "2021/10/01"), (2, "l4", "v1", "2021/10/02"))
+ val df = Seq((1, "z3", "v1", "2021#10#01"), (2, "l4", "v1", "2021#10#02"))
Review Comment:
Emm, I think I don't quite get what your mean ~ More specifically ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133793391
@hudi-bot
test please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133548109
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 697a8c68d64bd9144cb81c4c918e85d15b81a15b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536)
* 5799064ab62362256a2c78d36d0c34596756a581 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133547703
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 697a8c68d64bd9144cb81c4c918e85d15b81a15b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536)
* 5799064ab62362256a2c78d36d0c34596756a581 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133544457
WIP !
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1133579115
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 5799064ab62362256a2c78d36d0c34596756a581 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808)
* 99ae88c935d7c47737e3ca2e2237000b12669944 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1134074714
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
}, {
"hash" : "0024d0306e8671655004ff995c7754a99de6c59d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829",
"triggerID" : "0024d0306e8671655004ff995c7754a99de6c59d",
"triggerType" : "PUSH"
}, {
"hash" : "d834cab7033af3caa93e0baeb4975eab5258b31b",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "d834cab7033af3caa93e0baeb4975eab5258b31b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0024d0306e8671655004ff995c7754a99de6c59d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829)
* d834cab7033af3caa93e0baeb4975eab5258b31b UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1134076751
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
}, {
"hash" : "5799064ab62362256a2c78d36d0c34596756a581",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8808",
"triggerID" : "5799064ab62362256a2c78d36d0c34596756a581",
"triggerType" : "PUSH"
}, {
"hash" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8810",
"triggerID" : "99ae88c935d7c47737e3ca2e2237000b12669944",
"triggerType" : "PUSH"
}, {
"hash" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8812",
"triggerID" : "956b4ff53aba2e91775ff1021c43d97d8a36f1e8",
"triggerType" : "PUSH"
}, {
"hash" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8813",
"triggerID" : "99b80b43a703e51e449e6a8670675a67a3554d3e",
"triggerType" : "PUSH"
}, {
"hash" : "0024d0306e8671655004ff995c7754a99de6c59d",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829",
"triggerID" : "0024d0306e8671655004ff995c7754a99de6c59d",
"triggerType" : "PUSH"
}, {
"hash" : "d834cab7033af3caa93e0baeb4975eab5258b31b",
"status" : "PENDING",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8839",
"triggerID" : "d834cab7033af3caa93e0baeb4975eab5258b31b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0024d0306e8671655004ff995c7754a99de6c59d Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8829)
* d834cab7033af3caa93e0baeb4975eab5258b31b Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8839)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] jinxing64 commented on a diff in pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
jinxing64 commented on code in PR #5534:
URL: https://github.com/apache/hudi/pull/5534#discussion_r867670047
##########
hudi-spark-datasource/hudi-spark/src/test/scala/org/apache/spark/sql/hudi/TestAlterTableDropPartition.scala:
##########
@@ -81,13 +81,13 @@ class TestAlterTableDropPartition extends HoodieSparkSqlTestBase {
}
Seq(false, true).foreach { urlencode =>
- test(s"Drop single-partition table' partitions, urlencode: $urlencode") {
+ test(s"Drop single-partition table's partitions, urlencode: $urlencode") {
withTempDir { tmp =>
val tableName = generateTableName
val tablePath = s"${tmp.getCanonicalPath}/$tableName"
import spark.implicits._
Review Comment:
It makes no sense to config urlencode=false when partition value contains slash -- '/' , it brings much obstacles when parsing partition values from path -- running 'show partitions' fails easily in scenarios. I replace "/" with "#", which triggers character escaping but doesn't break partition path parsing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] hudi-bot commented on pull request #5534: [HUDI-4056] Refine partition matching in when drop / truncate partition.
Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5534:
URL: https://github.com/apache/hudi/pull/5534#issuecomment-1121856086
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"status" : "DELETED",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8512",
"triggerID" : "5deab6cb23cab172ce1e1940255e61ba3ab62e75",
"triggerType" : "PUSH"
}, {
"hash" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536",
"triggerID" : "697a8c68d64bd9144cb81c4c918e85d15b81a15b",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 697a8c68d64bd9144cb81c4c918e85d15b81a15b Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8536)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org