You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hive.apache.org by Roshan Naik <ro...@hortonworks.com> on 2014/03/27 23:24:04 UTC

Review Request 19754: Defines a api for streaming data into Hive using ACID support.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19754/
-----------------------------------------------------------

Review request for hive.


Bugs: HIVE-5687
    https://issues.apache.org/jira/browse/HIVE-5687


Repository: hive-git


Description
-------

Defines an API for streaming data into Hive using ACID support.


Diffs
-----

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 209349b 
  packaging/pom.xml de9b002 
  packaging/src/main/assembly/src.xml bdaa47b 
  pom.xml 7343683 
  streaming/pom.xml PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/AbstractLazySimpleRecordWriter.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/ConnectionError.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/DelimitedInputWriter.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/HeartBeatFailure.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/HiveEndPoint.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/ImpersonationFailed.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidColumn.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidPartition.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidTable.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidTrasactionState.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/PartitionCreationFailed.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/QueryFailedException.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/RecordWriter.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/SerializationError.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/StreamingConnection.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/StreamingException.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/StreamingIOFailure.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/TransactionBatch.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/TransactionBatchUnAvailable.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/TransactionError.java PRE-CREATION 
  streaming/src/test/org/apache/hive/streaming/StreamingIntegrationTester.java PRE-CREATION 
  streaming/src/test/org/apache/hive/streaming/TestDelimitedInputWriter.java PRE-CREATION 
  streaming/src/test/org/apache/hive/streaming/TestStreaming.java PRE-CREATION 
  streaming/src/test/sit PRE-CREATION 

Diff: https://reviews.apache.org/r/19754/diff/


Testing
-------

Unit tests included. Also done manual testing by streaming data using flume.


Thanks,

Roshan Naik

Re: Review Request 19754: Defines a api for streaming data into Hive using ACID support.

Posted by Lefty Leverenz <le...@gmail.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19754/#review39817
-----------------------------------------------------------



hcatalog/streaming/pom.xml
<https://reviews.apache.org/r/19754/#comment72461>

    typo:  artifectId should be artifactId



hcatalog/streaming/pom.xml
<https://reviews.apache.org/r/19754/#comment72462>

    typo:  artifectId should be artifactId



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java
<https://reviews.apache.org/r/19754/#comment72463>

    suggestion for Txnid:  either spell out transaction ("transaction ID" -- preferable) or use capital I like the parameter (TxnId)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java
<https://reviews.apache.org/r/19754/#comment72464>

    Why does the parameter name have both-caps "ID" for maxTxnID while it's init-cap "Id" for minTxnId?  Are parameter names case-sensitive?
    
    Also a suggestion for Txnid in description:  either spell out transaction ("transaction ID" -- preferable) or use capital ID like the parameter (TxnID).



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java
<https://reviews.apache.org/r/19754/#comment72465>

    Same question as line 108 about minTxnId vs maxTxnID capitalization



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72520>

    Nit:  period at the end (next line too)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72466>

    Editorial nits:  Please capitalize "nulls" and end the second sentence with a period (next line) just for consistency with the first sentence.



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72467>

    Grammar nit:  Remove "s" from "indicates" because the subjects are plural.



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72468>

    Consistency nit:  Since other param descriptions are capitalized on the first word, please do the same here.
    
    Bonus points if you capitalize all the param descriptions in this patch, but I'm not going to comment on all of them.  You could argue for a rule that only capitalizes full sentences and proper nouns like Hive, in which case [pun alert] it's okay to leave "input" uncapitalized.  But I favor visual consistency over rule consistency, except when I'm inconsistent.
    
    Terminal periods aren't essential (given the typical style of javadocs) but they're recommended when a description has multiple sentences.  Hm, but that's inconsistent with my visual consistency preference.  Why am I wasting your time with this trivia?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72517>

    should "endpoint" be explained? (your call)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72469>

    Editorial nit:  "non existing" seems okay in this context, but "nonexistent" is the real word (your choice).
    
    Consistency nit again:  Since other exception descriptions are capitalized on the first word, please do the same here.



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72470>

    ditto line 57



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72471>

    ditto line 58



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72472>

    ditto line 59 (capitalization)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72474>

    ditto line 60



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72473>

    Hive nit:  please capitalize "hive"
    
    Editorial nits:  please capitalize "a" and perhaps spell out configuration in "conf object" unless conf is the proper term for the object



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72475>

    ditto line 65



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72516>

    ditto line 59



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72518>

    ditto line 60



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72519>

    ditto line 79



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72521>

    ditto line 59 (capitalization)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72522>

    ditto line 65



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72525>

    do you want to document the return?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72526>

    do you want to document tbl?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java
<https://reviews.apache.org/r/19754/#comment72527>

    add @param conf



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72528>

    Hive nit:  please capitalize hive (also, end point is a single word elsewhere)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72529>

    missing a method description



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72530>

    why is the comma here rather than on the last line? (just curious)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72531>

    want to document the return?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72542>

    proxyUser isn't a parameter of this method (does that matter?)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72534>

    IOException isn't in the throws list



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72532>

    want to document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72533>

    exception InvalidTable isn't documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72541>

    Nits:  first comma belongs on previous line, second comma doesn't need space before it



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72535>

    Hive nit:  please capitalize hive



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72536>

    want to document the return?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72543>

    proxyUser isn't a parameter of this method (does that matter?)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72539>

    IOException isn't in the throws list



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72537>

    want to document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72538>

    exception InvalidTable isn't documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72540>

    Nits:  first comma belongs on previous line, second comma doesn't need space before it



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72544>

    please change hdfs and hive to HDFS and Hive



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72545>

    add a period after "null"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72546>

    document the return?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72549>

    IOException isn't in the throws list



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72550>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72548>

    @param conf & @throws InvalidTable aren't documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72547>

    unnecessary space before a comma



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72551>

    no method description ... but this is private so doc not crucial



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72552>

    Nit:  end point is one word elsewhere



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72553>

    Description starts "of prody user" -- should be "UGI of proxy user" (and could add description that starts on line 134).



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72556>

    comma belongs on previous line



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72558>

    want to document the return?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72563>

    throws list has StreamingException, not StreamingIOFailure



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72561>

    proxyUser isn't a parameter of this method (does that matter?)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72560>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72559>

    comma belongs on previous line



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72564>

    no param descriptions, but this is a private method so javadoc not published



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72565>

    comma belongs on previous line



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72566>

    @throws ImpersonationFailed & InterruptedException not documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72567>

    document the return?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72569>

    capitalize "get" & change "tramsaction" to "transaction"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72568>

    document the return?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72570>

    insert "in" --> "remaining [in] this batch"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72572>

    throws list has StreamingException, not StreamingIOFailure



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72571>

    SerializationError is not in the throws list



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72573>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72574>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72577>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72578>

    throws list has StreamingException, not StreamingIOFailure



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72575>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72576>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72580>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72581>

    3 exceptions not documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72579>

    comma belongs on previous line



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72582>

    throws list has StreamingException, not StreamingIOFailure



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java
<https://reviews.apache.org/r/19754/#comment72583>

    2 exceptions not documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java
<https://reviews.apache.org/r/19754/#comment72588>

    Hive nit:  capitalize "hive" (also change "Writes" to "Write" for consistency -- I didn't comment on a few other instances of this because these nits are overwhelming and mostly unimportant)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java
<https://reviews.apache.org/r/19754/#comment72591>

    optional:  spell out transaction for Txn



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java
<https://reviews.apache.org/r/19754/#comment72593>

    @throws StreamingException not documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java
<https://reviews.apache.org/r/19754/#comment72594>

    ditto line 28 (exception doc)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java
<https://reviews.apache.org/r/19754/#comment72595>

    ditto line 28 (exception doc)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java
<https://reviews.apache.org/r/19754/#comment72596>

    ditto line 28 (exception doc) and two params not documented (also, different capitalization of id in minTxnId & maxTxnID)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java
<https://reviews.apache.org/r/19754/#comment72597>

    ditto line 28 (exception doc)



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java
<https://reviews.apache.org/r/19754/#comment72601>

    omit "is" at beginning of definition, then capitalize "a"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java
<https://reviews.apache.org/r/19754/#comment72602>

    document the return? -- yes, done in line 38 so this line isn't needed



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java
<https://reviews.apache.org/r/19754/#comment72604>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java
<https://reviews.apache.org/r/19754/#comment72608>

    InvalidPartition not in the throws list, but InterruptedException needs to be documented



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java
<https://reviews.apache.org/r/19754/#comment72606>

    document this exception?



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72609>

    no method description and no exception descriptions



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72610>

    no method description and no exception descriptions



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72611>

    Nit:  end point is a single word elsewhere



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72612>

    conf could be spelled out (configuration) in the description



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72614>

    skimpy definition but okay for private method



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72615>

    ditto line 88



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72616>

    ditto



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72617>

    ditto line 88



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java
<https://reviews.apache.org/r/19754/#comment72618>

    ditto line 88



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72619>

    spell out Txn



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72620>

    "in interrupted" (should be "is")



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72621>

    Nit:  capitalize "get"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72622>

    Nit:  capitalize "get"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72623>

    "in" --> "is"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72624>

    "in" --> "is"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72625>

    "remaining [in] this batch"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72626>

    "in" --> "is"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72627>

    "in" --> "is"



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72628>

    capitalize "hive" and spell out txn



hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java
<https://reviews.apache.org/r/19754/#comment72629>

    "in" --> "is"


- Lefty Leverenz


On April 8, 2014, 6:27 p.m., Roshan Naik wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/19754/
> -----------------------------------------------------------
> 
> (Updated April 8, 2014, 6:27 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-5687
>     https://issues.apache.org/jira/browse/HIVE-5687
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Defines an API for streaming data into Hive using ACID support.
> 
> 
> Diffs
> -----
> 
>   hcatalog/pom.xml 50ce296 
>   hcatalog/streaming/pom.xml PRE-CREATION 
>   hcatalog/streaming/src/docs/package.html PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/ConnectionError.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HeartBeatFailure.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/ImpersonationFailed.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidColumn.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidPartition.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidTable.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidTrasactionState.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/PartitionCreationFailed.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/QueryFailedException.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/SerializationError.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingException.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingIOFailure.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatchUnAvailable.java PRE-CREATION 
>   hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionError.java PRE-CREATION 
>   hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/StreamingIntegrationTester.java PRE-CREATION 
>   hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestDelimitedInputWriter.java PRE-CREATION 
>   hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java PRE-CREATION 
>   hcatalog/streaming/src/test/sit PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1bbe02e 
>   packaging/pom.xml de9b002 
>   packaging/src/main/assembly/src.xml bdaa47b 
> 
> Diff: https://reviews.apache.org/r/19754/diff/
> 
> 
> Testing
> -------
> 
> Unit tests included. Also done manual testing by streaming data using flume.
> 
> 
> Thanks,
> 
> Roshan Naik
> 
>

Re: Review Request 19754: Defines a api for streaming data into Hive using ACID support.

Posted by Roshan Naik <ro...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19754/
-----------------------------------------------------------

(Updated April 8, 2014, 6:27 p.m.)


Review request for hive.


Changes
-------

addressing review comments.
 - move to hcatalog
 - expose HiveConf to client API


Bugs: HIVE-5687
    https://issues.apache.org/jira/browse/HIVE-5687


Repository: hive-git


Description
-------

Defines an API for streaming data into Hive using ACID support.


Diffs (updated)
-----

  hcatalog/pom.xml 50ce296 
  hcatalog/streaming/pom.xml PRE-CREATION 
  hcatalog/streaming/src/docs/package.html PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/AbstractRecordWriter.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/ConnectionError.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/DelimitedInputWriter.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HeartBeatFailure.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/HiveEndPoint.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/ImpersonationFailed.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidColumn.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidPartition.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidTable.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/InvalidTrasactionState.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/PartitionCreationFailed.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/QueryFailedException.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/RecordWriter.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/SerializationError.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingConnection.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingException.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StreamingIOFailure.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/StrictJsonWriter.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatch.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionBatchUnAvailable.java PRE-CREATION 
  hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/TransactionError.java PRE-CREATION 
  hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/StreamingIntegrationTester.java PRE-CREATION 
  hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestDelimitedInputWriter.java PRE-CREATION 
  hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/TestStreaming.java PRE-CREATION 
  hcatalog/streaming/src/test/sit PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1bbe02e 
  packaging/pom.xml de9b002 
  packaging/src/main/assembly/src.xml bdaa47b 

Diff: https://reviews.apache.org/r/19754/diff/


Testing
-------

Unit tests included. Also done manual testing by streaming data using flume.


Thanks,

Roshan Naik

Re: Review Request 19754: Defines a api for streaming data into Hive using ACID support.

Posted by Roshan Naik <ro...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/19754/
-----------------------------------------------------------

(Updated April 1, 2014, 11:53 p.m.)


Review request for hive.


Changes
-------

updating patch


Bugs: HIVE-5687
    https://issues.apache.org/jira/browse/HIVE-5687


Repository: hive-git


Description
-------

Defines an API for streaming data into Hive using ACID support.


Diffs (updated)
-----

  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1bbe02e 
  packaging/pom.xml de9b002 
  packaging/src/main/assembly/src.xml bdaa47b 
  pom.xml 7343683 
  streaming/pom.xml PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/AbstractRecordWriter.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/ConnectionError.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/DelimitedInputWriter.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/HeartBeatFailure.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/HiveEndPoint.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/ImpersonationFailed.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidColumn.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidPartition.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidTable.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/InvalidTrasactionState.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/PartitionCreationFailed.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/QueryFailedException.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/RecordWriter.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/SerializationError.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/StreamingConnection.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/StreamingException.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/StreamingIOFailure.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/StrictJsonWriter.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/TransactionBatch.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/TransactionBatchUnAvailable.java PRE-CREATION 
  streaming/src/java/org/apache/hive/streaming/TransactionError.java PRE-CREATION 
  streaming/src/test/org/apache/hive/streaming/StreamingIntegrationTester.java PRE-CREATION 
  streaming/src/test/org/apache/hive/streaming/TestDelimitedInputWriter.java PRE-CREATION 
  streaming/src/test/org/apache/hive/streaming/TestStreaming.java PRE-CREATION 
  streaming/src/test/sit PRE-CREATION 

Diff: https://reviews.apache.org/r/19754/diff/


Testing
-------

Unit tests included. Also done manual testing by streaming data using flume.


Thanks,

Roshan Naik