[jira] [Created] (HBASE-26693) Throw exception if region has same start key when writePartitions use HFileOutputFormat2

["mengqi (Jira)" <ji...@apache.org>]

mengqi created HBASE-26693:
------------------------------

             Summary: Throw exception if region has same start key when writePartitions use HFileOutputFormat2
                 Key: HBASE-26693
                 URL: https://issues.apache.org/jira/browse/HBASE-26693
             Project: HBase
          Issue Type: Improvement
            Reporter: mengqi


Some exception info is not friendly in MapReduce task log such as:
{code:java}
2022-01-21 06:23:39,156 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.IllegalArgumentException: Can't read partitions file
	at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:116)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:719)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:797)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:344)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:169)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1865)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.io.IOException: Wrong number of partitions in keyset
	at org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:90)
	... 10 more
 {code}
when I write hfile use HFileOutputFormat2 in a MapReduce task.

I can not understand what happend to my job. And got some wrong suggestions such as: trucate table or add cancelDeleteOnExit code in my code. Those are useless and stupid. 

 

*The real reason is my table has same start key ,and I didn't find this problem in hbck web ui overlaps list immediately.* 

{*}So{*}， I thanks it's a good idea to check same start key in writePartitions(), we can print each same start key  and throw a same start key exception if my table regions has same start key.

It is more friendly exception info than 'Can't read partitions file'.

Here is my code  [https://github.com/Mengqi777/hbase/commit/d8dc093695d02cfb16ba303389de9f7cef778ee1|https://github.com/Mengqi777/hbase/commit/d8dc093695d02cfb16ba303389de9f7cef778ee1],]

can I create a pull request ?

 

 

 

 

 

 

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)