You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/12/31 03:47:14 UTC
[GitHub] [hudi] peng-xin opened a new issue #2397: [SUPPORT]
peng-xin opened a new issue #2397:
URL: https://github.com/apache/hudi/issues/2397
**_Tips before filing an issue_**
- Have you gone through our [FAQs](https://cwiki.apache.org/confluence/display/HUDI/FAQ)?
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apache.org.
- If you have triaged this as a bug, then file an [issue](https://issues.apache.org/jira/projects/HUDI/issues) directly.
**Describe the problem you faced**
when i have a empty partition(partition table),hudi create a partition `partition_name=default`,the partition field type is date.
that will cause an exception when sync the table to Apache Hive metastore.
`Failed in executing SQL ALTER TABLE `hudi`.`f_mid_order_details_ro` ADD IF NOT EXISTS PARTITION (`payment_date`='2019-10-07') LOCATION '/user/hive/warehouse/hudi/hudi/f_mid_order_details/payment_date=2019-10-07' PARTITION (`payment_date`='default') LOCATION '/user/hive/warehouse/hudi/hudi/f_mid_order_details/payment_date=default' `
this is the write option
`hoodie.datasource.hive_sync.table -> f_mid_order_details
hoodie.bloom.index.update.partition.path -> true
hoodie.bloom.index.filter.type -> DYNAMIC_V0
hoodie.datasource.write.keygenerator.class -> org.apache.hudi.keygen.SimpleKeyGenerator
hoodie.datasource.hive_sync.database -> hudi
hoodie.datasource.write.table.type -> MERGE_ON_READ
hoodie.datasource.write.partitionpath.field -> payment_date
hoodie.datasource.hive_sync.partition_fields -> payment_date
hoodie.datasource.hive_sync.partition_extractor_class -> org.apache.hudi.hive.MultiPartKeysValueExtractor
hoodie.datasource.write.recordkey.field -> key
hoodie.datasource.hive_sync.enable -> true
hoodie.upsert.shuffle.parallelism -> 100
hoodie.index.type -> GLOBAL_BLOOM
hoodie.datasource.hive_sync.jdbcurl -> jdbc:hive2://172.16.117.73:10000
hoodie.compact.inline -> true
hoodie.datasource.write.precombine.field -> ts
hoodie.table.name -> f_mid_order_details
hoodie.datasource.write.hive_style_partitioning -> true
hoodie.datasource.write.operation -> upsert`
A clear and concise description of the problem.
**To Reproduce**
Steps to reproduce the behavior:
1.
2.
3.
4.
**Expected behavior**
A clear and concise description of what you expected to happen.
**Environment Description**
* Hudi version :
hudi-0.6
* Spark version :
spark-2.4.4
* Hive version :
hive-2.3.4
* Hadoop version :
hadoop-2.7.3
* Storage (HDFS/S3/GCS..) :
hdfs
* Running on Docker? (yes/no) :
no
**Additional context**
Add any other context about the problem here.
**Stacktrace**
```Add the stacktrace of the error.```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #2397: [SUPPORT]
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #2397:
URL: https://github.com/apache/hudi/issues/2397#issuecomment-753716419
@peng-xin : Sorry for the delay. Somehow, I missed this GH issue. Looking at how Hive handles Null values, this behavior is consistent with Hive null handling. what is your partition path type ?One option here is to replace MultiPartKeysValueExtractor with your own implementation which handles the default value in partition-path differently based on your use case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #2397: [SUPPORT]
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #2397:
URL: https://github.com/apache/hudi/issues/2397#issuecomment-765212048
Please reopen if this is still an issue ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar closed issue #2397: [SUPPORT]
Posted by GitBox <gi...@apache.org>.
bvaradar closed issue #2397:
URL: https://github.com/apache/hudi/issues/2397
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org