You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "empcl (via GitHub)" <gi...@apache.org> on 2023/12/02 16:54:46 UTC

[PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

empcl opened a new pull request, #10229:
URL: https://github.com/apache/hudi/pull/10229

   …ronizes to Hive
   
   ### Change Logs
   
   _Resolve type conversion exception when Spark writes to Hudi and synchronizes to Hive._
   
   ### Impact
   
   _no._
   
   ### Risk level (write none, low medium or high below)
   
   _no._
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10229:
URL: https://github.com/apache/hudi/pull/10229#issuecomment-1837308154

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283",
       "triggerID" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "triggerType" : "PUSH"
     }, {
       "hash" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "1837302544",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 56c80f47aba8b643ced31941d4e77b92f5a4adb8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "empcl (via GitHub)" <gi...@apache.org>.
empcl commented on PR #10229:
URL: https://github.com/apache/hudi/pull/10229#issuecomment-1837302544

   @hudi-bot run azure


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "empcl (via GitHub)" <gi...@apache.org>.
empcl closed pull request #10229: [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch…
URL: https://github.com/apache/hudi/pull/10229


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10229:
URL: https://github.com/apache/hudi/pull/10229#issuecomment-1837203468

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56c80f47aba8b643ced31941d4e77b92f5a4adb8 UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "danny0405 (via GitHub)" <gi...@apache.org>.
danny0405 commented on code in PR #10229:
URL: https://github.com/apache/hudi/pull/10229#discussion_r1413311630


##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/CreateHoodieTableCommand.scala:
##########
@@ -164,6 +161,20 @@ object CreateHoodieTableCommand {
     }
   }
 
+  private def convertFieldTypes(fieldTypes: StructType): StructType = {
+    val fields = new Array[StructField](fieldTypes.fields.length)
+    var index = 0

Review Comment:
   Should we just fix the hive sync schema instead? Modification in here affects beyond the Hive scope I guess.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10229:
URL: https://github.com/apache/hudi/pull/10229#issuecomment-1837309139

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283",
       "triggerID" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "triggerType" : "PUSH"
     }, {
       "hash" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283",
       "triggerID" : "1837302544",
       "triggerType" : "MANUAL"
     } ]
   }-->
   ## CI report:
   
   * 56c80f47aba8b643ced31941d4e77b92f5a4adb8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10229:
URL: https://github.com/apache/hudi/pull/10229#issuecomment-1837205177

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283",
       "triggerID" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56c80f47aba8b643ced31941d4e77b92f5a4adb8 Azure: [PENDING](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [HUDI-7167] Resolve type conversion exception when Spark writes to Hudi and synch… [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #10229:
URL: https://github.com/apache/hudi/pull/10229#issuecomment-1837241167

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "status" : "FAILURE",
       "url" : "https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283",
       "triggerID" : "56c80f47aba8b643ced31941d4e77b92f5a4adb8",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * 56c80f47aba8b643ced31941d4e77b92f5a4adb8 Azure: [FAILURE](https://dev.azure.com/apache-hudi-ci-org/785b6ef4-2f42-4a89-8f0e-5f0d7039a0cc/_build/results?buildId=21283) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org