[jira] Commented: (HBASE-1923) Bulk incremental load into an existing table

["HBase Review Board (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-1923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12873912#action_12873912 ] 

HBase Review Board commented on HBASE-1923:
-------------------------------------------

Message from: "Todd Lipcon" <to...@cloudera.com>

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.hbase.org/r/87/
-----------------------------------------------------------

(Updated 2010-06-01 00:33:30.030753)


Review request for hbase, stack and Jonathan Gray.


Changes
-------

One more day of hacking on this, adds some docs, cleans up the command line parameters, fixes a couple bugs identified in cluster testing.

Calling this one final, I swear :)

I don't know how to build the forrest docs in trunk, but the new xml file does pass xmllint.


Summary
-------

Here's a first patch that implements bulk import into existing tables. This applies on top of HBASE-2586 and HBASE-2588 - I've pushed the series of the three to my github: http://github.com/toddlipcon/hbase/tree/hfof-review

I have some TODOs left that I want to take care of before this gets committed, but since it's a pretty large patch, I figured I'd get it out for review ASAP.

The stuff in the hadoopbackport package is essentially copypaste from Hadoop trunk, so you can ignore that in the review.


This addresses bug HBASE-1923.
    http://issues.apache.org/jira/browse/HBASE-1923


Diffs (updated)
-----

  pom.xml 0a009cf 
  src/docs/src/documentation/content/xdocs/bulk-loads.xml PRE-CREATION 
  src/docs/src/documentation/content/xdocs/site.xml 0d644f5 
  src/main/java/org/apache/hadoop/hbase/HRegionInfo.java 29b0cd6 
  src/main/java/org/apache/hadoop/hbase/io/ImmutableBytesWritable.java 0a9ec4b 
  src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java b912a85 
  src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 4cbe52a 
  src/main/java/org/apache/hadoop/hbase/mapreduce/Driver.java 3d40695 
  src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java 9c8e53e 
  src/main/java/org/apache/hadoop/hbase/mapreduce/ImportTsv.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/mapreduce/PutSortReducer.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/mapreduce/SimpleTotalOrderPartitioner.java af3d588 
  src/main/java/org/apache/hadoop/hbase/mapreduce/TableMapReduceUtil.java 07d7911 
  src/main/java/org/apache/hadoop/hbase/mapreduce/hadoopbackport/InputSampler.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/mapreduce/hadoopbackport/TotalOrderPartitioner.java PRE-CREATION 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java 287cd48 
  src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 78f3223 
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 7de766d 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 80bf09a 
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java c8941f1 
  src/main/java/org/apache/hadoop/hbase/util/Bytes.java a53dafe 
  src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java ed8709f 
  src/test/java/org/apache/hadoop/hbase/mapreduce/NMapInputFormat.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat.java d04ced2 
  src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java PRE-CREATION 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java f1566d3 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestHRegionInfo.java fcb22fb 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java 4595e6e 
  src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 2e4c7df 
  src/test/java/org/apache/hadoop/hbase/util/TestBytes.java c7361cb 

Diff: http://review.hbase.org/r/87/diff


Testing
-------

Primary unit/functional testing, a bit of pseudo-distributed testing. Plan on doing full system tests before commit as well.


Thanks,

Todd




> Bulk incremental load into an existing table
> --------------------------------------------
>
>                 Key: HBASE-1923
>                 URL: https://issues.apache.org/jira/browse/HBASE-1923
>             Project: HBase
>          Issue Type: New Feature
>          Components: client, mapred, regionserver, scripts
>    Affects Versions: 0.21.0
>            Reporter: anty.rao
>            Assignee: Todd Lipcon
>         Attachments: hbase-1923-prelim.txt, hbase-1923.txt
>
>
> hbase-48 is about bulk load of a new table,maybe it's more practicable to bulk load aganist a existing table.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.