You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "praveen sripati (Created) (JIRA)" <ji...@apache.org> on 2012/03/28 03:33:22 UTC
[jira] [Created] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Create InputFormats/OutputFormats for HBase
-------------------------------------------
Key: HAMA-544
URL: https://issues.apache.org/jira/browse/HAMA-544
Project: Hama
Issue Type: Sub-task
Components: bsp
Reporter: praveen sripati
Assignee: praveen sripati
Priority: Minor
Fix For: 0.5.0, 0.6.0
Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253182#comment-13253182 ]
Thomas Jungblut commented on HAMA-544:
--------------------------------------
I think we should add this to our contrib package.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253298#comment-13253298 ]
Edward J. Yoon commented on HAMA-544:
-------------------------------------
We should add a Wikipage.
+1
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253220#comment-13253220 ]
Edward J. Yoon commented on HAMA-544:
-------------------------------------
Most of NoSQL solutions supports MR. Once proved value of BSP computing power, they will have to support Hama BSP in the future.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252273#comment-13252273 ]
Edward J. Yoon commented on HAMA-544:
-------------------------------------
By the way, shouldn't we contribute this to HBase?
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "praveen sripati (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253307#comment-13253307 ]
praveen sripati commented on HAMA-544:
--------------------------------------
Ed - Debarshi Basak mentioned he was working on it. So, I haven't spent time on it. Not sure how much progress he has done. If you want to implement it, please go ahead.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255250#comment-13255250 ]
Edward J. Yoon commented on HAMA-544:
-------------------------------------
Option 2 (add BSP input output formats and its example)'s advantages is to show a best practice for BSP computing model based new applications on Accumulo.
Option 3 also looks good to me but not great, considering user accessibility.
I know your concerns about option 2, but I'll bet that our basic BSP message-passing and input/output interfaces don't change.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "praveen sripati (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240163#comment-13240163 ]
praveen sripati commented on HAMA-544:
--------------------------------------
TableInputFormat from HBase can be ported.
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapred/TableInputFormat.html
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/TableInputFormat.html
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.5.0, 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "debarshi basak (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243197#comment-13243197 ]
debarshi basak commented on HAMA-544:
-------------------------------------
Hi praveen, i was working on integrating exposing hama apis for hbase...i was wondering what is the procedure for creating a patch?Is there any documents that i can follow?
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.5.0, 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253088#comment-13253088 ]
Edward J. Yoon commented on HAMA-544:
-------------------------------------
What concerns me is about compatibility issue of our all releases (Like HDFS[1]). So, I'm planning to add this to HBase project[2].
1. http://wiki.apache.org/hama/CompatibilityTable
2. http://markmail.org/thread/odckj37jpoqke6sy
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon updated HAMA-544:
--------------------------------
Affects Version/s: (was: 0.5.0)
Fix Version/s: (was: 0.6.0)
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp core
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253270#comment-13253270 ]
Edward J. Yoon commented on HAMA-544:
-------------------------------------
Praveen,
Are you going to implement HBase table input output formats?
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Edward J. Yoon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253202#comment-13253202 ]
Edward J. Yoon commented on HAMA-544:
-------------------------------------
Think about various input formats e.g., cassandra, accumulo, hbase, ..., etc.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "praveen sripati (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13252328#comment-13252328 ]
praveen sripati commented on HAMA-544:
--------------------------------------
I am OK with either of the option, but till Hama API becomes stable we might consider keeping it in Hama only. Either way we have to add the Hama dependency to HBase or HBase dependency to Hama.
Ed, I think your feedback will apply for other source/sinks (GraphDB etc) also.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Thomas Jungblut (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thomas Jungblut updated HAMA-544:
---------------------------------
Affects Version/s: 0.5.0
Fix Version/s: (was: 0.5.0)
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Keith Turner (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255003#comment-13255003 ]
Keith Turner commented on HAMA-544:
-----------------------------------
I am an Accumulo developer and I was looking at ACCUMULO-532. I was thinking about the problem of where this code should be put. It seems like there are three options.
# Put hama input/output formats in Hama
# Put hama input/output formats in cassandra, accumulo, hbase, ..., etc
# Put hama input/output formats on something like github
I am not sure what the best option is, we are trying to figure that out. Below are some thoughts about this issue.
Option 3 may not be a blessed apache way of doing business.
One thing nice about option 3, is that if accumulo-hama has a serious bug that it can release immediately w/o waiting. For options 1 and 2, either hama or accumulo must release to fix a serious hama-accumulo bug. Options 3 may also make it easier to use a newer version of hama w/ an older version of accumulo. If accumulo ships with hama, and you have an older version of accumulo it probably depends on an older version of hama. This may not be an issue if the hama API is really stable.
With option 3, Accumulo could include a hama jar in contrib w/ a link to github.
>From the perspective of Accumulo, we have to work this same issue out for lots of projects. For example accumulo could ship with connectors for pig, hive, gora, hama, etc. This increases the # of dependencies that accumulo has that users may not need. Currently the accumulo pig adapter is on github.
Apache Gora seems to be doing option 1. They have gora-core, gora-accumulo, gora-hbase, etc. Each one of these are maven sub projects of Gora. One nice thing about this for the gora case is that all of the gora stores can share test code. For example gora-accumlo extends a test class from gora-core for testing.
I suppose option 1 is bad because hama is a subsystem like map reduce. For example gora and pig depend on map reduce, and its probably ok to make them depend on hbase or accumulo. However, you would not want map reduce or hama to depend on a certain version of accumulo or hbase. If accumulo-1.3 jars were in the map reduce system lib dir, I do not think user accumulo-1.4 jars can override those. I suspect the same is true for hama, which is why option 1 is bad?
I wrote the gora-accumulo backend and then I wrote goraci (https://github.com/keith-turner/goraci). To make goraci easy to run I wrote a slightly complex script and pom. Maybe I was being a bit OCD, but when I ran goraci against accumulo I only wanted the jars that were needed on the classpath. For example, I did not want hbase jars when running goraci against accumulo and visa versa. So what steps would the user have to go through to have hama read from accumulo w/ options 2 vs 3? Seems like the main diff is add one extra jar to the classpath? Is there any other burden? Its nice to make things as easy as possible for the user.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13253277#comment-13253277 ]
Thomas Jungblut commented on HAMA-544:
--------------------------------------
You are right, but we must keep track who has which implementations to notify them when something changed. We should add a Wikipage.
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Affects Versions: 0.5.0
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HAMA-544) Create InputFormats/OutputFormats for
HBase
Posted by "Thomas Jungblut (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13243212#comment-13243212 ]
Thomas Jungblut commented on HAMA-544:
--------------------------------------
Have a look here: http://wiki.apache.org/hama/HowToContribute
> Create InputFormats/OutputFormats for HBase
> -------------------------------------------
>
> Key: HAMA-544
> URL: https://issues.apache.org/jira/browse/HAMA-544
> Project: Hama
> Issue Type: Sub-task
> Components: bsp
> Reporter: praveen sripati
> Assignee: praveen sripati
> Priority: Minor
> Fix For: 0.5.0, 0.6.0
>
>
> Create InputFormats/OutputFormats for HBase
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira