[GitHub] [iceberg] wangxujin1221 commented on issue #3921: A BaseDataReader error always occurs when traversing a partitioned table

[GitBox <gi...@apache.org>]

wangxujin1221 commented on issue #3921:
URL: https://github.com/apache/iceberg/issues/3921#issuecomment-1017032910


   > 
   
   Hi @RussellSpitzer ， thanks for your reply. Actually when i use year or month as the partion rules, it worked and without error above. And i also tried with code like:
   ```
   dataset.foreachPartition(par ->{
                       par.forEachRemaining(row ->{
                           try{
                               OUTPUT_LIST.put(row);
                           }catch(InterruptedException e){
                               logger.error("iceberg records caching failed in LinkedBlockingQueue.",e);
                           }
                       });
                   });
   ```            
   But i still got this error. I don't know whether this error relatted to the total number of files of the hive table. Tables partioned by year or bucket only have 0.5k where table partioned by day has 73k. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org