You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/09/19 06:29:46 UTC

[GitHub] [dolphinscheduler] zeaper123 opened a new issue, #12042: [Bug] [task failed] task of Data_Quality failed

zeaper123 opened a new issue, #12042:
URL: https://github.com/apache/dolphinscheduler/issues/12042

   ### Search before asking
   
   - [X] I had searched in the [issues](https://github.com/apache/dolphinscheduler/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### What happened
   
   dolphinscheduler version: 3.0.0
   spark version: 2.4.6
   
   After I installed dolphinscheduler，I can submit the normal spark parogram to yarn without any errors，
   But task of Data_Quality moudle failed when I try to run 
   
   error.log:
   `Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.hive.shims.HadoopShims.getHadoopConfNames()Ljava/util/Map;
   		at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.<clinit>(OrcInputFormat.java:117)
   		at java.lang.Class.forName0(Native Method)
   		at java.lang.Class.forName(Class.java:348)
   		at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
   		at org.apache.spark.sql.hive.client.HiveClientImpl$.org$apache$spark$sql$hive$client$HiveClientImpl$$toInputFormat(HiveClientImpl.scala:912)
   		at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$toHiveTable$6.apply(HiveClientImpl.scala:948)
   		at org.apache.spark.sql.hive.client.HiveClientImpl$$anonfun$toHiveTable$6.apply(HiveClientImpl.scala:948)
   		at scala.Option.map(Option.scala:146)
   		at org.apache.spark.sql.hive.client.HiveClientImpl$.toHiveTable(HiveClientImpl.scala:948)`
   
   I have confirmed that there is org.apache.hadoop.hive.shims.HadoopShims in hive-shims-common.2.3.6.jar, but org.apache.hadoop.hive.shims.HadoopShims does not have getHadoopConfNames function.
   
   
   ### What you expected to happen
   
   run success
   
   ### How to reproduce
   
   waiting for solution
   
   ### Anything else
   
   _No response_
   
   ### Version
   
   3.0.0
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] zeaper123 closed issue #12042: [Bug] [task failed] task of Data_Quality failed

Posted by GitBox <gi...@apache.org>.
zeaper123 closed issue #12042: [Bug] [task failed]  task of Data_Quality failed
URL: https://github.com/apache/dolphinscheduler/issues/12042


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] zeaper123 commented on issue #12042: [Bug] [task failed] task of Data_Quality failed

Posted by GitBox <gi...@apache.org>.
zeaper123 commented on issue #12042:
URL: https://github.com/apache/dolphinscheduler/issues/12042#issuecomment-1252103307

   I have replaced spark2.4.6 with spark2.4.0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] github-actions[bot] commented on issue #12042: [Bug] [task failed] task of Data_Quality failed

Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #12042:
URL: https://github.com/apache/dolphinscheduler/issues/12042#issuecomment-1250623204

   Thank you for your feedback, we have received your issue, Please wait patiently for a reply.
   * In order for us to understand your request as soon as possible, please provide detailed information、version or pictures.
   * If you haven't received a reply for a long time, you can [join our slack](https://s.apache.org/dolphinscheduler-slack) and send your question to channel `#troubleshooting`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org