You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@kyuubi.apache.org by "lordk911 (via GitHub)" <gi...@apache.org> on 2023/10/13 08:41:55 UTC

[I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

lordk911 opened a new issue, #5414:
URL: https://github.com/apache/kyuubi/issues/5414

   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   
   
   ### Search before asking
   
   - [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Describe the bug
   
   spark: 3.3.3
   remote hive:    hdp3.1.0
   local hive:    hdp3.1.0
   spark-hive-connector: master
   complie commond: mvn clean install -pl extensions/spark/kyuubi-spark-connector-hive -DskipTests -Dspark.version=3.3.3
   
   from spark-shell I can run : spark.sql("select * from hive_catalog.crm_bi.XXX").show , but when I run spark.sql("select count(*) from hive_catalog.crm_bi.XXX").show I will got error: 
   
   `java.lang.NumberFormatException: For input string: ""
           at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
           at java.lang.Integer.parseInt(Integer.java:592)
           at java.lang.Integer.parseInt(Integer.java:615)
           at org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(ColumnProjectionUtils.java:186)
           at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.genIncludedColumns(OrcInputFormat.java:399)
           at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.createReaderFromFile(OrcInputFormat.java:322)
           at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$OrcRecordReader.<init>(OrcInputFormat.java:232)
           at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1848)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory$$anon$2.liftedTree1$1(HivePartitionReaderFactory.scala:130)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory$$anon$2.<init>(HivePartitionReaderFactory.scala:129)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory.createPartitionWritableReader(HivePartitionReaderFactory.scala:122)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory.buildReaderInternal(HivePartitionReaderFactory.scala:91)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory.$anonfun$createReader$1(HivePartitionReaderFactory.scala:75)
           at scala.collection.Iterator$$anon$10.next(Iterator.scala:461)
           at org.apache.kyuubi.spark.connector.hive.read.SparkFilePartitionReader.getNextReader(SparkFilePartitionReader.scala:99)
           at org.apache.kyuubi.spark.connector.hive.read.SparkFilePartitionReader.next(SparkFilePartitionReader.scala:46)
           at org.apache.spark.sql.execution.datasources.v2.PartitionIterator.hasNext(DataSourceRDD.scala:119)
           at org.apache.spark.sql.execution.datasources.v2.MetricsIterator.hasNext(DataSourceRDD.scala:156)
           at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1(DataSourceRDD.scala:63)
           at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.$anonfun$hasNext$1$adapted(DataSourceRDD.scala:63)
           at scala.Option.exists(Option.scala:376)
           at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
           at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.advanceToNextIter(DataSourceRDD.scala:97)
           at org.apache.spark.sql.execution.datasources.v2.DataSourceRDD$$anon$1.hasNext(DataSourceRDD.scala:63)
           at org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:37)
           at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
           at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.hashAgg_doAggregateWithoutKey_0$(Unknown Source)
           at org.apache.spark.sql.catalyst.expressions.GeneratedClass$GeneratedIteratorForCodegenStage1.processNext(Unknown Source)
           at org.apache.spark.sql.execution.BufferedRowIterator.hasNext(BufferedRowIterator.java:43)
           at org.apache.spark.sql.execution.WholeStageCodegenExec$$anon$1.hasNext(WholeStageCodegenExec.scala:760)
           at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:460)
           at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:140)
           at org.apache.spark.shuffle.ShuffleWriteProcessor.write(ShuffleWriteProcessor.scala:59)
           at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
           at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:52)
           at org.apache.spark.scheduler.Task.run(Task.scala:136)
           at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:548)
           at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1504)
           at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:551)
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
           at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
           at java.lang.Thread.run(Thread.java:748)`
   
   after I got this error I can't run spark.sql("select * from hive_catalog.crm_bi.XXX").show again , will got the same error
   
   ### Affects Version(s)
   
   master
   
   ### Kyuubi Server Log Output
   
   _No response_
   
   ### Kyuubi Engine Log Output
   
   _No response_
   
   ### Kyuubi Server Configurations
   
   _No response_
   
   ### Kyuubi Engine Configurations
   
   ```yaml
   spark.sql.catalog.hive_catalog     org.apache.kyuubi.spark.connector.hive.HiveTableCatalog
   spark.sql.catalog.hive_catalog.hive.metastore.uris     thrift://hdpdev246:9083,thrift://hdpdev248:9083
   
   I also tried to config metastore other information: 
   spark.sql.catalog.hive_catalog.spark.sql.hive.metastore.version     3.1.0
   spark.sql.catalog.hive_catalog.spark.sql.hive.metastore.jars     path
   spark.sql.catalog.hive_catalog.spark.sql.hive.metastore.jars.path     file:///data/soft/standalone-metastore/*.jar
   
   this not help
   ```
   
   
   ### Additional context
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix.
   - [ ] No. I cannot submit a PR at this time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 closed issue #5414: [Bug] spark-hive-connector read faild with count(*)
URL: https://github.com/apache/kyuubi/issues/5414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 closed issue #5414: [Bug] spark-hive-connector read faild with count(*)
URL: https://github.com/apache/kyuubi/issues/5414


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1761149264

   the querying table is not a partioned table


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1761266241

   > also cc @cxzl25, is it an ORC issue?
   
   I try to select count against parquet table, got same error :
   `ERROR TaskSetManager: Task 0 in stage 4.0 failed 1 times; aborting job
   org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 4.0 failed 1 times, most recent failure: Lost task 0.0 in stage 4.0 (TID 4) (hdpdev246.glodondg.com executor driver): java.lang.NumberFormatException: For input string: ""
           at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
           at java.lang.Integer.parseInt(Integer.java:592)
           at java.lang.Integer.parseInt(Integer.java:615)
           at org.apache.hadoop.hive.serde2.ColumnProjectionUtils.getReadColumnIDs(ColumnProjectionUtils.java:186)
           at org.apache.hadoop.hive.ql.io.parquet.read.DataWritableReadSupport.init(DataWritableReadSupport.java:377)
           at org.apache.hadoop.hive.ql.io.parquet.ParquetRecordReaderBase.getSplit(ParquetRecordReaderBase.java:84)
           at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:75)
           at org.apache.hadoop.hive.ql.io.parquet.read.ParquetRecordReaderWrapper.<init>(ParquetRecordReaderWrapper.java:60)
           at org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat.getRecordReader(MapredParquetInputFormat.java:75)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory$$anon$2.liftedTree1$1(HivePartitionReaderFactory.scala:130)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory$$anon$2.<init>(HivePartitionReaderFactory.scala:129)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory.createPartitionWritableReader(HivePartitionReaderFactory.scala:122)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory.buildReaderInternal(HivePartitionReaderFactory.scala:91)
           at org.apache.kyuubi.spark.connector.hive.read.HivePartitionReaderFactory.$anonfun$createReader$1(HivePartitionReaderFactory.scala:75)`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1763779086

   It should be due to environmental issues, but I couldn't find out where the problem was. The problem disappeared on its own. If there are any new findings, I will update again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "lordk911 (via GitHub)" <gi...@apache.org>.
lordk911 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1764154666

   Thank you everyone, very helpful extension. Then I just want to mention how my environment is working normally.
   I have modified my spark-default.conf and added the following content : 
   ```
   spark.sql.hive.metastore.jars /data/soft/spark3/standalone-metastore/*
   spark.sql.hive.metastore.version 3.1.2
   ```
   After normal operation, even if I comment out the above configuration and start a new Spark application, it can still work normally.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "Yikf (via GitHub)" <gi...@apache.org>.
Yikf commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1763947633

   Sorry for late reply, Thanks @lordk911 for reporting this issue, this is a BUG as @cxzl25 said. Will fix it on https://github.com/apache/kyuubi/pull/5424.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1761153857

   cc @zhaomin1423 @Yikf 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "cxzl25 (via GitHub)" <gi...@apache.org>.
cxzl25 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1763875083

   Thanks to @lordk911 for reporting this issue
   
   I think this is a bug, HiveReader.initializeHiveConf polluted the Hiveconf instance, causing `hive.io.file.readcolumn.ids` to add a lot of dirty values.
   
   Can you propose a PR to help fix it?
   
   https://github.com/apache/kyuubi/blob/4bb67bdf3bcabd135102bd85c8992e9b52ba9356/extensions/spark/kyuubi-spark-connector-hive/src/main/scala/org/apache/kyuubi/spark/connector/hive/read/HiveScan.scala#L66-L73
   
   https://github.com/apache/hive/blob/d9e96c96a09398d79d9f866b914d994403f4928a/serde/src/java/org/apache/hadoop/hive/serde2/ColumnProjectionUtils.java#L115-L124
   
   https://github.com/apache/spark/blob/c69117742b7c05fe58ca2b94cd713231dc8f7a3d/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/orc/OrcScan.scala#L64-L72
   
   <img width="1463" alt="image" src="https://github.com/apache/kyuubi/assets/3898450/c0adc063-0002-4a21-8cfa-89842c08db11">
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1761155379

   also cc @cxzl25, is it a ORC issue?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org


Re: [I] [Bug] spark-hive-connector read faild with count(*) [kyuubi]

Posted by "pan3793 (via GitHub)" <gi...@apache.org>.
pan3793 commented on issue #5414:
URL: https://github.com/apache/kyuubi/issues/5414#issuecomment-1763801729

   @lordk911 all right, thanks for testing this module


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@kyuubi.apache.org
For additional commands, e-mail: notifications-help@kyuubi.apache.org