You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/05/18 08:48:24 UTC

[GitHub] [hudi] gaogao110 opened a new pull request, #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

gaogao110 opened a new pull request, #5625:
URL: https://github.com/apache/hudi/pull/5625

   ## *Tips*
   - *Thank you very much for contributing to Apache Hudi.*
   - *Please review https://hudi.apache.org/contribute/how-to-contribute before opening a pull request.*
   
   ## What is the purpose of the pull request
   
   fix flink write hudi sync hive 
    if Hive table already exists throw an exception
   
   ## Brief change log
   
   
     - *Modify hudi-flink-bundle pom.xml*
   
   Add 
   `                <relocation>
                     <pattern>org.apache.hadoop.hive.serde2.</pattern>
                     <shadedPattern>${flink.bundle.shade.prefix}org.apache.hadoop.hive.serde2.</shadedPattern>
                   </relocation>`
   
   
   ## Verify this pull request
   
   *(Please pick either of the following options)*
   
   This pull request is a trivial rework / code cleanup without any test coverage.
   
   *(or)*
   
   This pull request is already covered by existing tests, such as *(please describe tests)*.
   
   (or)
   
   This change added tests and can be verified as follows:
   
   
   
   
   
   
   
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] gaogao110 commented on pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
gaogao110 commented on PR #5625:
URL: https://github.com/apache/hudi/pull/5625#issuecomment-1165038331

   I test from  0.11.1 version and master 
   No problem was found
   
   I think this issue has been fixed in 0.11.1.
   Because of hive dependencies in 0.10.1, 0.11 has been deleted
   
   I will use the new version and close this problem. Thank you for your reply


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5625:
URL: https://github.com/apache/hudi/pull/5625#issuecomment-1129958250

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "673d5d86837dd907d872f9134ffdcfd4eeb7b156",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8739",
       "triggerID" : "673d5d86837dd907d872f9134ffdcfd4eeb7b156",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 673d5d86837dd907d872f9134ffdcfd4eeb7b156 Azure: [SUCCESS](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8739) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on a diff in pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #5625:
URL: https://github.com/apache/hudi/pull/5625#discussion_r881590696


##########
packaging/hudi-flink-bundle/pom.xml:
##########
@@ -289,6 +289,10 @@
                   <shadedPattern>org.apache.hudi.org.apache.hadoop.metrics2.util.MetricSampleQuantiles
                   </shadedPattern>
                 </relocation>
+                <relocation>
+                  <pattern>org.apache.hadoop.hive.serde2.</pattern>
+                  <shadedPattern>${flink.bundle.shade.prefix}org.apache.hadoop.hive.serde2.</shadedPattern>
+                </relocation>

Review Comment:
   can you also help me understand: is this a problem only when hive table already exists? 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5625:
URL: https://github.com/apache/hudi/pull/5625#issuecomment-1129778762

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "673d5d86837dd907d872f9134ffdcfd4eeb7b156",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "673d5d86837dd907d872f9134ffdcfd4eeb7b156",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 673d5d86837dd907d872f9134ffdcfd4eeb7b156 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] nsivabalan commented on pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
nsivabalan commented on PR #5625:
URL: https://github.com/apache/hudi/pull/5625#issuecomment-1164753800

   @gaogao110 : can you follow up on Raymond's comment above. Please update here once the patch is ready to review again. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] gaogao110 commented on pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
gaogao110 commented on PR #5625:
URL: https://github.com/apache/hudi/pull/5625#issuecomment-1137186870

   Yes, this will only happen if the hive table already exists&nbsp; and flink-connector-sql-hive.jar also exists in flink lib
   
   
   I did the tests and version 0.10.1 will always exist&nbsp;
   
   
   This is a conflict between two JAR packages and I think it is best to fix it
   
   
   Because Hudi is often used in combination with Hive
   
   
   
   
   
   ------------------&nbsp;原始邮件&nbsp;------------------
   发件人:                                                                                                                        "apache/hudi"                                                                                    ***@***.***&gt;;
   发送时间:&nbsp;2022年5月25日(星期三) 晚上8:27
   ***@***.***&gt;;
   ***@***.******@***.***&gt;;
   主题:&nbsp;Re: [apache/hudi] [HUDI-4118] fix sync hive Hive table already exists throw an exception (PR #5625)
   
   
   
   
   
    
   @xushiyan commented on this pull request.
    
    
   In packaging/hudi-flink-bundle/pom.xml:
    &gt; +                <relocation&gt; +                  <pattern&gt;org.apache.hadoop.hive.serde2.</pattern&gt; +                  <shadedPattern&gt;${flink.bundle.shade.prefix}org.apache.hadoop.hive.serde2.</shadedPattern&gt; +                </relocation&gt;  
   can you also help me understand: is this a problem only when hive table already exists?
    
   —
   Reply to this email directly, view it on GitHub, or unsubscribe.
   You are receiving this because you were mentioned.Message ID: ***@***.***&gt;


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] hudi-bot commented on pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
hudi-bot commented on PR #5625:
URL: https://github.com/apache/hudi/pull/5625#issuecomment-1129782658

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "673d5d86837dd907d872f9134ffdcfd4eeb7b156",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8739",
       "triggerID" : "673d5d86837dd907d872f9134ffdcfd4eeb7b156",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 673d5d86837dd907d872f9134ffdcfd4eeb7b156 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=8739) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] xushiyan commented on a diff in pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
xushiyan commented on code in PR #5625:
URL: https://github.com/apache/hudi/pull/5625#discussion_r881588154


##########
packaging/hudi-flink-bundle/pom.xml:
##########
@@ -289,6 +289,10 @@
                   <shadedPattern>org.apache.hudi.org.apache.hadoop.metrics2.util.MetricSampleQuantiles
                   </shadedPattern>
                 </relocation>
+                <relocation>
+                  <pattern>org.apache.hadoop.hive.serde2.</pattern>
+                  <shadedPattern>${flink.bundle.shade.prefix}org.apache.hadoop.hive.serde2.</shadedPattern>
+                </relocation>

Review Comment:
   @gaogao110 from the ticket it reports as a problem in 0.10.1. Does this also fix master version ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [hudi] gaogao110 closed pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception

Posted by GitBox <gi...@apache.org>.
gaogao110 closed pull request #5625: [HUDI-4118] fix sync hive Hive table already exists throw an exception
URL: https://github.com/apache/hudi/pull/5625


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org