You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Navis (Created) (JIRA)" <ji...@apache.org> on 2012/02/28 01:49:49 UTC
[jira] [Created] (HIVE-2828) make timestamp accessible in the hbase
KeyValue
make timestamp accessible in the hbase KeyValue
------------------------------------------------
Key: HIVE-2828
URL: https://issues.apache.org/jira/browse/HIVE-2828
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Reporter: Navis
Assignee: Navis
Priority: Trivial
Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
{code}
CREATE TABLE hbase_table (key int, value string, time timestamp)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase
KeyValue
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2828:
------------------------------
Attachment: HIVE-2828.D1989.3.patch
navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
Reviewers: JIRA
1. Rebased on trunk
2. Support string type for timestamp column
REVISION DETAIL
https://reviews.facebook.net/D1989
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
hbase-handler/src/test/queries/hbase_timestamp.q
hbase-handler/src/test/results/hbase_timestamp.q.out
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch, HIVE-2828.D1989.3.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2828) make timestamp accessible in the
hbase KeyValue
Posted by "Navis (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217894#comment-13217894 ]
Navis commented on HIVE-2828:
-----------------------------
No and no as far as I know, but might be implemented by modifying HiveHBaseTableInputFormat and HBaseStorageHandler.
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase
KeyValue
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2828:
------------------------------
Attachment: HIVE-2828.D1989.5.patch
navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
Reviewers: JIRA
Rebased on trunk
REVISION DETAIL
https://reviews.facebook.net/D1989
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
hbase-handler/src/test/queries/hbase_timestamp.q
hbase-handler/src/test/results/hbase_timestamp.q.out
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch, HIVE-2828.D1989.3.patch, HIVE-2828.D1989.4.patch, HIVE-2828.D1989.5.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase
KeyValue
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2828:
------------------------------
Attachment: HIVE-2828.D1989.1.patch
navis requested code review of "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
Reviewers: JIRA
DPAL-861 make timestamp accessible in the hbase KeyValue
Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
CREATE TABLE hbase_table (key int, value string, time timestamp)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D1989
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java
hbase-handler/src/test/queries/hbase_timestamp.q
hbase-handler/src/test/results/hbase_timestamp.q.out
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/4245/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2828) make timestamp accessible in the
hbase KeyValue
Posted by "Anoop Sam John (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217873#comment-13217873 ]
Anoop Sam John commented on HIVE-2828:
--------------------------------------
Does HBase integration in Hive allows retrieval of multiple versions ( diff time stamp ) of the same cell?
Also whether allows specifying the time range while select such that the cell value versions within this time range will be retrieved?
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase
KeyValue
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2828:
------------------------------
Attachment: HIVE-2828.D1989.4.patch
navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
Reviewers: JIRA
Rebased on trunk
REVISION DETAIL
https://reviews.facebook.net/D1989
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
hbase-handler/src/test/queries/hbase_timestamp.q
hbase-handler/src/test/results/hbase_timestamp.q.out
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch, HIVE-2828.D1989.3.patch, HIVE-2828.D1989.4.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase
KeyValue
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2828:
------------------------------
Attachment: HIVE-2828.D1989.2.patch
navis updated the revision "HIVE-2828 [jira] make timestamp accessible in the hbase KeyValue".
Reviewers: JIRA
Applied HIVE-2771
REVISION DETAIL
https://reviews.facebook.net/D1989
AFFECTED FILES
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseSerDe.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HBaseStorageHandler.java.rej
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java.rej
hbase-handler/src/java/org/apache/hadoop/hive/hbase/LazyHBaseRow.java
hbase-handler/src/test/org/apache/hadoop/hive/hbase/TestLazyHBaseObject.java
hbase-handler/src/test/queries/external_table_ppd.q
hbase-handler/src/test/queries/hbase_timestamp.q
hbase-handler/src/test/results/external_table_ppd.q.out
hbase-handler/src/test/results/hbase_timestamp.q.out
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2828) make timestamp accessible in the
hbase KeyValue
Posted by "Navis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423554#comment-13423554 ]
Navis commented on HIVE-2828:
-----------------------------
If what you meant was is it possible to 'group by timestamp', it's possible.
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch, HIVE-2828.D1989.3.patch, HIVE-2828.D1989.4.patch, HIVE-2828.D1989.5.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2828) make timestamp accessible in the hbase
KeyValue
Posted by "Navis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Navis updated HIVE-2828:
------------------------
Status: Patch Available (was: Open)
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch, HIVE-2828.D1989.3.patch, HIVE-2828.D1989.4.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2828) make timestamp accessible in the
hbase KeyValue
Posted by "Hans Uhlig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13423347#comment-13423347 ]
Hans Uhlig commented on HIVE-2828:
----------------------------------
Would this allow aggregation across timestamps as well?
> make timestamp accessible in the hbase KeyValue
> ------------------------------------------------
>
> Key: HIVE-2828
> URL: https://issues.apache.org/jira/browse/HIVE-2828
> Project: Hive
> Issue Type: Improvement
> Components: HBase Handler
> Reporter: Navis
> Assignee: Navis
> Priority: Trivial
> Attachments: HIVE-2828.D1989.1.patch, HIVE-2828.D1989.2.patch, HIVE-2828.D1989.3.patch, HIVE-2828.D1989.4.patch, HIVE-2828.D1989.5.patch
>
>
> Originated from HIVE-2781 and not accepted, but I think this could be helpful to someone.
> By using special column notation ':timestamp' in HBASE_COLUMNS_MAPPING, user might access timestamp value in hbase KeyValue.
> {code}
> CREATE TABLE hbase_table (key int, value string, time timestamp)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf:string,:timestamp")
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira