You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "edward yoon (JIRA)" <ji...@apache.org> on 2007/07/13 13:35:04 UTC
[jira] Created: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
[HbaseShell] Relational Algrebra Operators
------------------------------------------
Key: HADOOP-1608
URL: https://issues.apache.org/jira/browse/HADOOP-1608
Project: Hadoop
Issue Type: Improvement
Components: contrib/hbase
Affects Versions: 0.13.0
Environment: All environments
Reporter: edward yoon
Priority: Minor
Fix For: 0.14.0
Development of relational algebra operators has begun.
* Projection
* Selection
* Product
* Rename
* Group
* Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
RE: Relational Algrebra Operators on hbase shell.
Posted by edward yoon <we...@udanax.org>.
Hi, community.
There was a misunderstanding in my last writing, please ignore it. Thanks.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org> From: webmaster@udanax.org> To: hadoop-dev@lucene.apache.org> Subject: Relational Algrebra Operators on hbase shell.> Date: Mon, 5 Nov 2007 23:14:01 +0000> > > > One last thing, hbase shell (relational) operators running mapreduce jobs begins to impinge on PIG territory. We should be careful and avoid overlap/duplicating work. Would it make sense doing further operators as PIG user defined functions? (I suppose we'll be able to tell better after PIG-6, the hbase load/store, is done. Smile).> > I don't think so.> It's just a relational algebra basics.> No these works, no SQL-like HQL, RDF, Matrix works.> What is an overlapping character?> I need the exact explain.> > Also, If Pig has time to discuss matrix operations or linear systems, > Can i said "No, it's a hbase shell territory." ??> > I feel stuffy becuase i'm just going round in circles discussing the plans of shell operator on wiki. > (almost for the past two months) > > I bet my name on shell works and Other members,too.> At least, I have rights to make it.> > Any advice are welcome.> > Edward J. Yoon,> Thanks.> ------------------------------> B. Regards,> Edward yoon @ NHN, corp.> Home : http://www.udanax.org> > > > Date: Mon, 5 Nov 2007 14:04:51 -0800> > From: jira@apache.org> > To: hadoop-dev@lucene.apache.org> > Subject: [jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra Operators> >> >> > [ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540287 ]> >> > stack commented on HADOOP-1608:> > -------------------------------> >> > Edward:> >> > I do not understand what you mean by the following in your 02/Nov/07 06:09 PM comment: "I'll implement relational algebra operators to the tentative language and HQL by Sub-Tasks List sequence."> >> > In org.apache.hadoop.hbase.shell.algebra, test for output table presence and creation if missing is duplicated code in Selection, Projection, DuplicateTable, etc. I tried to move this duplicated code back up into the RelationalOperation class as a utility but noticed then if table already exists, we don't call initJob because we return early (See end of the getConf in DuplicateTable for example). Is running one of these operators a second time, after table has been created, a problem? Have you tried it?> >> > I did not have a mapreduce cluster running and so the last line hung for ever (You should add to help need for a mapreduce cluster).> > {code}> > Hbase> aaa = table('y');> > Hbase> aaa;> > Syntax error : Type 'help;' for usage.Message : Encountered ";" at line 1, column 5.> > Hbase> show aaa;> > Missing parameters. Please check 'Show' syntax> > Hbase> bbb = group aaa by ('x'); Hbase> save bbb into table ('a');> > {code}> >> > Note, it would be nice if you entered a variable name only if you got a description of the variable content or perhaps doing 'show VARIABLE_NAME' output some kind of description.> >> > I then started up a cluster and did the simplest of operations:> > {code}> > Hbase> aaa = table('x');> > Hbase> save aaa into table(aaaaaa);> > 07/11/05 21:26:57 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).Job job_200711052125_0001 is still running........Job failed.> > {code}> >> > Job failed because of 'Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapred.TableSplit'. How do you run on a cluster Edward? Do you copy the hbase jar into the hadoop lib dir all over the cluster? Does HADOOP-1622 help here?> >> > On job failure, should the table be removed? Currently, I have a new table 'aaaaaa' with nothing in it.> >> > The output of Group, Selection, etc. is saved to an 'output' table?> >> > It would be good if we could get into the help listing somewhere the operators and types of conditions allowed on selection (>, > > One last thing, hbase shell (relational) operators running mapreduce jobs begins to impinge on PIG territory. We should be careful and avoid overlap/duplicating work. Would it make sense doing further operators as PIG user defined functions? (I suppose we'll be able to tell better after PIG-6, the hbase load/store, is done. Smile).> >> > Otherwise, patch looks good Edward.> >> >> >> >> >> >> [Hbase Shell] Relational Algrebra Operators> >> -------------------------------------------> >>> >> Key: HADOOP-1608> >> URL: https://issues.apache.org/jira/browse/HADOOP-1608> >> Project: Hadoop> >> Issue Type: Improvement> >> Components: contrib/hbase> >> Affects Versions: 0.14.1> >> Environment: All environments> >> Reporter: Edward Yoon> >> Priority: Minor> >> Fix For: 0.16.0> >>> >> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt> >>> >>> >> Development of relational algebra operators has begun.> >> * Projection> >> ** selects a subset of the columnfamilies of a relation> >> ** Result = π ~column_list~ (Relation)> >> {code}> >> Hbase> Result = Relation.Projection('year','legnth');> >> Hbase> save Result into table('result');> >> {code}> >> * Selection> >> ** See : HADOOP-2003 issue's description> >> {code}> >> Hbase> Result = Relation.Selection(length> 100 and studioName = 'Fox');> >> Hbase> save Result into table('result');> >> {code}> >> * Group> >> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.> >> * θ Join> >> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition> >> {code}> >> Hbase> R1 = table('movieLog_table');> >> Hbase> R2 = table('personInfo_table');> >> Hbase> Result = R1.join(R1.producer: = R2.ROW) and R2;> >> or Result = R1.join(R1.actor:hero = R2.Row) and R2;> >> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;> >> {code}> >> > --> > This message is automatically generated by JIRA.> > -> > You can reply to this email to add a comment to the issue online.> >> > _________________________________________________________________> Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare!> http://onecare.live.com/standard/en-us/purchase/trial.aspx?s_cid=wl_hotmailnews
_________________________________________________________________
Peek-a-boo FREE Tricks & Treats for You!
http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us
Relational Algrebra Operators on hbase shell.
Posted by edward yoon <we...@udanax.org>.
> One last thing, hbase shell (relational) operators running mapreduce jobs begins to impinge on PIG territory. We should be careful and avoid overlap/duplicating work. Would it make sense doing further operators as PIG user defined functions? (I suppose we'll be able to tell better after PIG-6, the hbase load/store, is done. Smile).
I don't think so.
It's just a relational algebra basics.
No these works, no SQL-like HQL, RDF, Matrix works.
What is an overlapping character?
I need the exact explain.
Also, If Pig has time to discuss matrix operations or linear systems,
Can i said "No, it's a hbase shell territory." ??
I feel stuffy becuase i'm just going round in circles discussing the plans of shell operator on wiki.
(almost for the past two months)
I bet my name on shell works and Other members,too.
At least, I have rights to make it.
Any advice are welcome.
Edward J. Yoon,
Thanks.
------------------------------
B. Regards,
Edward yoon @ NHN, corp.
Home : http://www.udanax.org
> Date: Mon, 5 Nov 2007 14:04:51 -0800
> From: jira@apache.org
> To: hadoop-dev@lucene.apache.org
> Subject: [jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra Operators
>
>
> [ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540287 ]
>
> stack commented on HADOOP-1608:
> -------------------------------
>
> Edward:
>
> I do not understand what you mean by the following in your 02/Nov/07 06:09 PM comment: "I'll implement relational algebra operators to the tentative language and HQL by Sub-Tasks List sequence."
>
> In org.apache.hadoop.hbase.shell.algebra, test for output table presence and creation if missing is duplicated code in Selection, Projection, DuplicateTable, etc. I tried to move this duplicated code back up into the RelationalOperation class as a utility but noticed then if table already exists, we don't call initJob because we return early (See end of the getConf in DuplicateTable for example). Is running one of these operators a second time, after table has been created, a problem? Have you tried it?
>
> I did not have a mapreduce cluster running and so the last line hung for ever (You should add to help need for a mapreduce cluster).
> {code}
> Hbase> aaa = table('y');
> Hbase> aaa;
> Syntax error : Type 'help;' for usage.Message : Encountered ";" at line 1, column 5.
> Hbase> show aaa;
> Missing parameters. Please check 'Show' syntax
> Hbase> bbb = group aaa by ('x'); Hbase> save bbb into table ('a');
> {code}
>
> Note, it would be nice if you entered a variable name only if you got a description of the variable content or perhaps doing 'show VARIABLE_NAME' output some kind of description.
>
> I then started up a cluster and did the simplest of operations:
> {code}
> Hbase> aaa = table('x');
> Hbase> save aaa into table(aaaaaa);
> 07/11/05 21:26:57 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).Job job_200711052125_0001 is still running........Job failed.
> {code}
>
> Job failed because of 'Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapred.TableSplit'. How do you run on a cluster Edward? Do you copy the hbase jar into the hadoop lib dir all over the cluster? Does HADOOP-1622 help here?
>
> On job failure, should the table be removed? Currently, I have a new table 'aaaaaa' with nothing in it.
>
> The output of Group, Selection, etc. is saved to an 'output' table?
>
> It would be good if we could get into the help listing somewhere the operators and types of conditions allowed on selection (>,
> One last thing, hbase shell (relational) operators running mapreduce jobs begins to impinge on PIG territory. We should be careful and avoid overlap/duplicating work. Would it make sense doing further operators as PIG user defined functions? (I suppose we'll be able to tell better after PIG-6, the hbase load/store, is done. Smile).
>
> Otherwise, patch looks good Edward.
>
>
>
>
>
>> [Hbase Shell] Relational Algrebra Operators
>> -------------------------------------------
>>
>> Key: HADOOP-1608
>> URL: https://issues.apache.org/jira/browse/HADOOP-1608
>> Project: Hadoop
>> Issue Type: Improvement
>> Components: contrib/hbase
>> Affects Versions: 0.14.1
>> Environment: All environments
>> Reporter: Edward Yoon
>> Priority: Minor
>> Fix For: 0.16.0
>>
>> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>>
>>
>> Development of relational algebra operators has begun.
>> * Projection
>> ** selects a subset of the columnfamilies of a relation
>> ** Result = π ~column_list~ (Relation)
>> {code}
>> Hbase> Result = Relation.Projection('year','legnth');
>> Hbase> save Result into table('result');
>> {code}
>> * Selection
>> ** See : HADOOP-2003 issue's description
>> {code}
>> Hbase> Result = Relation.Selection(length> 100 and studioName = 'Fox');
>> Hbase> save Result into table('result');
>> {code}
>> * Group
>> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
>> * θ Join
>> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
>> {code}
>> Hbase> R1 = table('movieLog_table');
>> Hbase> R2 = table('personInfo_table');
>> Hbase> Result = R1.join(R1.producer: = R2.ROW) and R2;
>> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
>> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
>> {code}
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
_________________________________________________________________
Boo! Scare away worms, viruses and so much more! Try Windows Live OneCare!
http://onecare.live.com/standard/en-us/purchase/trial.aspx?s_cid=wl_hotmailnews
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540309 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
Thanks, stack.
I'll fix them.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
TestBooleanExpression failed
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v01.txt
I had checked you comments, fixed it.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516263 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
I am always concerned about implementing SQL-like syntax, familiar to many users.
But..... There are many reasons why I chose to implement the syntax.
* Consideration for all commands such as data relational, aggregation, and processing management.
* Differences between RDBMS and Multi Dimensional Model.
I think it is important to make users more aware of HBase's data model characteristics.
That way, teh should be able to use HBase at its full potential.
I have introduced multi dimensional model(Hadoop + Hbase + MapReduce) as a sort of sistributed computing platform to application developers, using SQL-like ysntax examples.
In the end, it has create more confusion among them, as most of them tried to relate it to traditional RDMBS.
Also besides relational operation, I will add aggregation and anlaysis functions,
and the will be considerate of all the grammars to make the whole thing more consistent.
I have a plan to shore up wiki pages and will finish implementing the planned features.
I hope to get more advice and tips from many of you people and make it up to 0.0.2 version.
Thank you.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546531 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
1. SELECT * FROM y WHERE x = 'g'
2. SELECT x FROM y WHERE x = 'g'
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Assignee: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517423 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
I think relational algebra can be used for preparing an operation table to control the configuration of the logic matrix.
So, i think it was not bad.
If you have any good name idea, feel free to let me know.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516180 ]
stack commented on HADOOP-1608:
-------------------------------
Hey Edward:
This looks like a sweet addition.
A couple of comments.
{code}
HBase > meta = select --META--;
Syntax error : Type 'help' for usage: Encountered "meta" at line 1, column 2.
HBase > A = select --META--;
Syntax error : Type 'help' for usage: Encountered "select" at line 1, column 6.
HBase > meta = table('--META--');
Syntax error : Type 'help' for usage: Encountered "meta" at line 1, column 2.
HBase > A = table('--META--');
HBase > a = table('--META--');
HBase > abc = table('--META--');
Syntax error : Type 'help' for usage: Encountered "abc" at line 1, column 2.
HBase > a
--> ;
Syntax error : Type 'help' for usage: Encountered ";" at line 1, column 4.
{code}
Looks like variable names can only be of one character. That seems like an artificial limitation.
Why can I not have select results stored to a variable?
Entering a variable name, I was expecting something to happen. In python, the interpreter echos the value (but in bash it complains 'command not found').
I shouldn't get the below
{code}
HBase > a = table('--META--');
HBase > store a to table('xyz');
Exception in thread "main" java.lang.NullPointerException
at org.apache.hadoop.hbase.shell.batch.Projection.getColumns(Projection.java:89)
at org.apache.hadoop.hbase.shell.batch.Projection.getConf(Projection.java:49)
at org.apache.hadoop.hbase.shell.StoreCommand.execute(StoreCommand.java:38)
at org.apache.hadoop.hbase.Shell.main(Shell.java:63)
{code}
This is a big addition yet no unit tests. What would you think of launching a little minihbasecluster and then against it running your shell commands (Have your junit tests subclass HBaseTestCase)
A one line class javadoc comment on what the class is for would not go amiss.
In the likes of BinOperationFilter (BinaryOperationFilter would be a better name), why not make the filter data member private? Same for the maps.
In the same class, you split on ' OR '. Is this split being done on the product of your javacc parse or is it directly operating on user input? If the latter, what if tabs or new-lines delimit the OR?
More comments to follow.... I'll post these for now (My connection is spotty)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1608:
---------------------------------
Fix Version/s: (was: 0.14.0)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545331 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12370172/patch_v20.txt
against trunk revision r598113.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs -1. The patch appears to introduce new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1156/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1156/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1156/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1156/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Affects Version/s: (was: 0.14.3)
0.14.1
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: shell_v03.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase > save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
or Result = R1.join(R1.actor:hero = R2.Row) and R2;
or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
{code}
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase > save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
{code}
Update Description.
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
or Result = R1.join(R1.actor:hero = R2.Row) and R2;
or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
{code}
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Affects Version/s: (was: 0.13.1)
Status: Open (was: Patch Available)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
I miss some code.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
And i submit this patch.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v03.txt
I had successfully tested my patch on my computer.
Re-trying.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538279 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
Thanks Vuk.
I'll do it.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516189 ]
Jeff Hammerbacher commented on HADOOP-1608:
-------------------------------------------
Hey stack,
I actually like Edward's diction and implementation. The semantic gap between relational operators and standard SQL implementations is wide enough that implementing the relational operators themselves is the correct approach. A SQL-like query language can easily be written that translates queries into a tree of relational operators, as you point out (javacc being like yacc for java, i assume--new to java). That's a different problem though.
Apologies if I am restating information you already know. Keeping the raw relational operators separate from the query language that leverages those operators seems like an important design goal though and now seemed like as good a time as any to jump into the discussion.
Jeff
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v07.txt
* added 'GROUP' operation. more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
{code}
Hbase > group R by (column-list);
{code}
TODO :
* self join
* join of two relations R1 and R2
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12536618 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
And, I would like to hear your opinions about shell sub-packages for mapred/filter classes and hbase mapred/filter packages.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
I would like to submit this patch to svn trunk.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532847 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367187/patch_v02.txt
against trunk revision r582443.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/899/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/899/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/899/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/899/console
This message is automatically generated.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12543448 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I'd like to submit to svn tree.
If you are not sure about committing this patch, let me know.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516184 ]
stack commented on HADOOP-1608:
-------------------------------
Isn't 'projection' what is more commonly known in db-speak as a 'view'? How hard would it be to implement 'projection' -- or view -- in a more sql-like manner (see mysql page for example: http://dev.mysql.com/doc/refman/5.0/en/create-view.html)
'selection' is too close in operation to 'select'. Why not name them the same (and have 'select' pick up the 'selection' capabilities)?
'store' seems to be done in (non-standard) SQL as a suffix 'into' on select statements.
In fact, how hard would it be to add all of these new capabilities in a more sql-like fashion? Below I redo your examples:
{code}
HBase >SELECT 'year', 'length' FROM 'movieLog_table' SORT BY 'year' INTO table 'temp_table';
...
OR
HBase> CREATE VIEW year_length AS SELECT 'year', 'length' FROM 'movieLog_table' SORT BY 'year';
HBase> SELECT year_length INTO TABLE 'temp_table';
...
HBase > SELECT 'year', 'length' FROM 'movieLog_table' SORT BY 'year' WHERE ('length' > 100) INTO table 'temp_table2';
...
HBase > SELECT 'year', 'length', 'studioName' FROM 'movieLog_table' WHERE 'length' > 100 SORT BY 'studioName' INTO FILE '/tmp/x.txt';
{code}
(There's a javacc pl/sql parser over in the javacc grammers page)
In this wiki page, http://wiki.apache.org/lucene-hadoop/Hbase/ShellPlans, when you say 'relational operator', in what sense do you mean? (database? computer science?). Can you add examples of how you would express matrix add, delete,and transpositions in hbase shell? It would help figure if going too far down the sql path would take you away from being able to add support for these operations (though it seems like your intent is to add traditional rdbms sql-like operations for hbase, is that right?)
Thanks Edward
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12543254 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12369641/patch_v18.txt
against trunk revision r595563.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1108/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1108/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1108/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1108/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [Hbase Shell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534493 ]
udanax edited comment on HADOOP-1608 at 10/13/07 3:07 AM:
---------------------------------------------------------------
>> The hadoop trunk is about to be branched in preparation for the hadoop 0.15.0 release. Its to happen any day now. Meantime, we are trying to hold off committing big new features.
Yeah, I finally understood perfectly. (sorry on my clumsy english and versioning)
Thanks, St.Ack
>> When you say in 'network mode', do you mean distributed?
Yes. When running on 4 node cluster, I recieve "User classes may not be found" error message.
So, I used setJar() method.
was (Author: udanax):
>> When you say in 'network mode', do you mean distributed?
Yes.
When running on 4 node cluster, I recieve "User classes may not be found" error message.
So, I used setJar() method.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
I'll more work on this issue while we waited for 0.15
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Attachment: shell_v02.patch
This patch
* adds (project, select, sort, substitute, store) command.
{code}
Examples.
HBase > A = table('movieLog_table');
HBase > B = A.projection('year','length');
HBase > [C = sort B by ('year');]
HBase > store B to table('temp_table');
...
HBase > A = table('movieLog_table');
HBase > B = A.selection(length > 100);
HBase > store B to table('temp_table2');
...
HBase > A = table('movieLog_table');
HBase > B = A.projection('year','length','studioName');
HBase > C = B.selection(length > 100);
HBase > D = sort C by ('studioName');
HBase > store D to table('temp_table3');
--Output Table Creating.
--Job 'job_200707200959_0003' is submitted
--Job job_200707200959_0003 is still running.
--Job job_200707200959_0003 is still running.
--Job job_200707200959_0003 is still running.
Job Successful.(47.86 sec)
HBase > select temp_table3;
+------+----------------------+----------------------+----------------------+
| No. | Row | Column | Cell |
+------+----------------------+----------------------+----------------------+
| 1 | Disney | length: | 124 |
+------+----------------------+----------------------+----------------------+
| 2 | Disney | studioName: | Disney |
+------+----------------------+----------------------+----------------------+
| 3 | Disney | year: | 1991 |
+------+----------------------+----------------------+----------------------+
| 4 | Fox | length: | 124 |
+------+----------------------+----------------------+----------------------+
| 5 | Fox | studioName: | Fox |
+------+----------------------+----------------------+----------------------+
| 6 | Fox | year: | 1977 |
+------+----------------------+----------------------+----------------------+
{code}
but, I should make logical operators(AND, OR, NOT) in selection command syntax.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Attachments: shell_r_operators_v01.patch, shell_v02.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v12.txt
* added arbitrary boolean expression in join_condition
{code}
Hbase> a = table(movieLog_table);
Hbase> b = table('personInfo_table');
Hbase> c = a.join(a.producer: = b.row and a.length > 100 or a.inColor = 'false') and b;
Hbase> save c into table('result');
{code}
* not yet implemented : ( a.producer: = b.row or a.length > 100 ) or ( a.producer: = b.name: )
** I think it needs map/reduce processing twice.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534392 ]
stack commented on HADOOP-1608:
-------------------------------
Edward: The hadoop trunk is about to be branched in preparation for the hadoop 0.15.0 release. Its to happen any day now. Meantime, we are trying to hold off committing big new features.
When you say in 'network mode', do you mean distributed?
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544716 ]
stack commented on HADOOP-1608:
-------------------------------
Here's a few comments on latest Edward:
+ Exceptions are suppressed in maps and configurations. They are just printed on stderr, which is better than nothing, but jobs should fail. Create a Runtime exception from the checked exception so you can throw it where exceptions are not allowed. When you have a ParseException but you're only allowed throw ParseException, and the map task should fail, create an IOE out of the PE and throw that.
+ Also:
[23:06] <st^ack> edward: When I do SAVE A INTO table('table_name');
[23:06] <st^ack> what column does it go into?
+ The GROUP help says this: "Group rows by value of an attribute and apply aggregate function independently to each group of rows"... but I do not see an 'aggregate function' in the example given. Am I missing something?
+ The SAVE command wants to run a mapreduce job to copy into result table. Should say it runs a MR job in help.
I took the patch for a run... things seem to basically work. Good stuff.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533227 ]
stack commented on HADOOP-1608:
-------------------------------
Ignore comment above where I say it does not add new operators to the shell -- I see now that it does.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516587 ]
edward yoon edited comment on HADOOP-1608 at 7/30/07 6:25 PM:
--------------------------------------------------------------
Ok, Stack.
Here's One of My Proposal Ideas.
I will more think about injection of general MDX queries.
and seperate them numeric tool and process control shell.
see : http://publib.boulder.ibm.com/infocenter/ablxhelp/v8r4m0/index.jsp?topic=/com.ibm.db2.abx.cub.doc/abx-c-cube-supportedmdxsyntax.html
Note, we must think about difference between traditional DBMS and Hbase.
{code}
udanax $> bin/shell
Hbase > ~~~ MDX Style queries with multi dimensional tables ~~~ (manipulating tables.)
Hbase > _
Hbase > NumShell;
Numshell > ~~~ Matlab or Octave Style queries with mapped matrix in tables~~~ (manipulating tables..)
Numshell > exit;
Hbase > ..
Hbase > exit;
udanax $> exit
{code}
Feel free to disagree with me Stack.
was:
Ok, Stack.
Here's One of My Proposal Ideas.
I will more think about injection of general MDX queries.
and seperate them numeric tool and process control shell.
see : http://publib.boulder.ibm.com/infocenter/ablxhelp/v8r4m0/index.jsp?topic=/com.ibm.db2.abx.cub.doc/abx-c-cube-supportedmdxsyntax.html
Note, we must think about difference between traditional DBMS and Hbase.
{code}
udanax $> bin/shell
Hbase > ~~~ MDX Style queries with multi dimensional tables ~~~ (manipulating tables.)
Hbase > _
Hbase > NumShell;
Numshell > ~~~ Matlab or Octave Style with mapped matrix in tables~~~ (manipulating tables..)
Numshell > exit;
Hbase > ..
Hbase > exit;
udanax $> exit
{code}
Feel free to disagree with me Stack.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: altools_v01.patch
skeleton code for altools sub-shell.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: altools_v01.patch, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516194 ]
stack commented on HADOOP-1608:
-------------------------------
Thanks for weighing in Jeff (and thanks for the educating pointer).
Currently, the shell is SQL-like (some data manipulation, some data definition). IMO, exposing relational algebra operators in the shell will make for a mix of operator types and will tend to confuse because folks are used to relational algebra's cousin SQL manipulating "databases". But as you suggest, an SQL shell for hbase mayhaps should be done as a different issue and would leverage Edward and crew's work done here?
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516306 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
If you have constructive ideas, please advise me.
Thank you.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534217 ]
stack commented on HADOOP-1608:
-------------------------------
Edward: OK if we wait on the 0.15.0 branching before we apply this patch (Should happen soon)?
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12542691 ]
stack commented on HADOOP-1608:
-------------------------------
Edward, can you say which of my comments from 05/Nov/07 02:04 PM above have been addressed in v17 of this patch? (That this patch does not work on clusters is being addressed outside of this issue). Thanks.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534493 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
>> When you say in 'network mode', do you mean distributed?
Yes.
When running on 4 node cluster, I recieve "User classes may not be found" error message.
So, I used setJar() method.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v06.txt
Sorry.spell error in test code.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532835 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367182/patch_v01.txt
against trunk revision r582443.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests -1. The patch failed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/898/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/898/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/898/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/898/console
This message is automatically generated.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538711 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12368656/patch_v14.txt
against trunk revision r589879.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1025/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1025/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1025/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1025/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch.txt
Regenerate the patch.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: altools_v01.patch, patch.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517309 ]
stack commented on HADOOP-1608:
-------------------------------
I like your suggestion of a sub-shell as a means of demarcating basic operators from non-sql relational algebra operators (Do you want to add your proposal to the wiki?)
One question: Is 'numeric' the right term to cover all the subshell adds? 'Numeric' covers linear algebra and matrix operators fine but do you think it encompasses relational algebra?
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539494 ]
stack commented on HADOOP-1608:
-------------------------------
Edward:
I'm having some difficulty following the model Edward particularly when there is a dearth of javadoc (You should at a minumum provide documentation of interfaces).
For example, the Operation interface has a getRelationalOperation method; RelationalOperation implements Operation. I would think the getRelationalOperation belongs in RelationalOperation or calling since RelationalOperation implements Operation, calling getOperation when the implemenation is a RelationalOperation, then you'd get back a RelationalOperation?
Do these mapreduce jobs work in a cluster of multiple machines?
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516587 ]
edward yoon edited comment on HADOOP-1608 at 7/30/07 6:10 PM:
--------------------------------------------------------------
Ok, Stack.
Here's One of My Proposal Ideas.
I will more think about injection of general MDX queries.
and seperate them numeric tool and process control shell.
see : http://publib.boulder.ibm.com/infocenter/ablxhelp/v8r4m0/index.jsp?topic=/com.ibm.db2.abx.cub.doc/abx-c-cube-supportedmdxsyntax.html
Note, we must think about difference between traditional DBMS and Hbase.
{code}
udanax $> bin/shell
Hbase > ~~~ MDX Style queries with multi dimensional tables ~~~ (manipulating tables.)
Hbase > _
Hbase > NumShell;
Numshell > ~~~ Matlab or Octave Style with mapped matrix in tables~~~ (manipulating tables..)
Numshell > exit;
Hbase > ..
Hbase > exit;
udanax $> exit
{code}
Feel free to disagree with me Stack.
was:
Ok, Stack.
Here's One of My Proposal Ideas.
I will more think about injection of general MDX queries.
and seperate with numeric tool and process control shell.
see : http://publib.boulder.ibm.com/infocenter/ablxhelp/v8r4m0/index.jsp?topic=/com.ibm.db2.abx.cub.doc/abx-c-cube-supportedmdxsyntax.html
and, we must think about difference between traditional DBMS and Hbase.
{code}
udanax $> bin/shell
Hbase > ~~~ MDX Style queries with multi dimensional table ~~~ (manipulating tables.)
Hbase > _
Hbase > NumShell;
Numshell > ~~~ Matlab or Octave Style with mapped matrix ~~~ (manipulating tables..)
Numshell > exit;
Hbase > ..
Hbase > exit;
udanax $> exit
{code}
Feel free to disagree with me Stack.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: shell_v04.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545358 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12370174/patch_v21.txt
against trunk revision r598113.
@author +1. The patch does not contain any @author tags.
javadoc -1. The javadoc tool appears to have generated messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs -1. The patch appears to introduce new Findbugs warnings.
core tests +1. The patch passed core unit tests.
findbugs -1. The patch appears to introduce new Findbugs warnings.
core tests -1. The patch failed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1157/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1157/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1157/artifact/trunk/build/test/checkstyle-errors.html
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1157/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1157/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1157/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516128 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
-1, new javadoc warnings
The javadoc tool appears to have generated warning messages when testing the latest attachment http://issues.apache.org/jira/secure/attachment/12362656/shell_v04.patch against trunk revision r560014.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/481/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/481/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.1
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516303 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
In addition,
After seeing Multi-Dimensional Model Style Data Storage(BigTable Clone)built on top of Hadoop,
I thought features such as Relational Algebra, Linear Algebra, Matrix operation, and etc. would be important.
{code}
edward yoon - [17/May/07 04:12 AM ] // Hadoop-1375
I have just started writing code for linear algebraic computation on Hadoop+Hbase based parallel machines.
I think It will makes the hadoop an even better platform for scientific and advanced analytics programming.
if i should be moving my classes to a different location, let me know how to do.
{code}
Therefore, I would like to develop relation and matrix operator commands first
and have them become consistent.
If HBase has online-transaction processing like HBase filter implemented,
I may make Select command more similar to SQL.
Also, I am willing to rearrange all the syntax if it is necessary.
However, in my opinion, Batch-type operations such as relational and matrix operators
would be better served with the current syntax.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v20.txt
- changed { SAVE command, GROUP command } help string
- When ParseException occurs, throwing the RuntimeException to job failure
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase> save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* Sort
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase> save Result into table('result');
{code}
* Product
* Rename
* Group
* Sort
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: shell_v07.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12537540 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
My codes are very clumsy.
So, I'll redesign classes with other good engineers and architecture designers in my company.
Also, I'll upload the UML document to the shell home wiki.
Please wait it.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532879 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367195/patch_v03.txt
against trunk revision r582443.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/900/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/900/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/900/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/900/console
This message is automatically generated.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: patch.txt)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12535385 ]
stack commented on HADOOP-1608:
-------------------------------
Comments on version 11 of this patch:
+ In BooleanExpression, you are doing a split using the strings " OR " and " AND ". Does this imply OR or AND can never appear at end of a line? (Or does the parser convert newlines to spaces?). Why not make UNION and INTERSECTION final static strings?
+ Comparator has a @SuppressWarnings("deprecation"). Do you need this? Same in IdentityFilterMap. Can you update your code so it uses non-deprecated API?
+ Cloning has one 'n' only so TableClonning should be CloningTable (Maybe better named as CloneTable?). In the getConf, you have an if/else. The else is not needed since you do a return out of the if block. Also in TableJoin and Projection (Is Projection a query? Or an expression? Same for Selection. Should there be a relational operators package rather than query and expression? Whats difference between a Command and an Expression? Why is Substitute a Command and Projection a query?).
Good stuff Edward.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516896 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
-1, build or testing failed
2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12362948/shell_v07.patch against trunk revision r561603.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/497/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/497/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531789 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
If it's okay with you, i would like to submit this patch.
and, If you are not sure about committing this patch, let me know.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: patch.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [Hbase Shell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539759 ]
udanax edited comment on HADOOP-1608 at 11/2/07 6:09 PM:
--------------------------------------------------------------
I'll implement relational algebra operators to the tentative language and HQL by Sub-Tasks List sequence.
Check Please.
1. Sort Join Implementation
2. The boolean expression can be added to the where clause of select statement
3. Aggregate Functions -----------------------------------------------------------------------------------> {Matrix issues}
4. Aggregate Functions in select statement
5. JOINs in select statement + [Hbase Shell] Cell-value index option using lucene.
was (Author: udanax):
I'll implement relational algebra operators to the tentative language and HQL by Sub-Tasks List sequence.
Check Please.
{code}
1. Sort Join Implementation
2. The boolean expression can be added to the where clause of select statement
3. Aggregate Functions -----------------------------------------------------------------------------------> {Matrix issues}
4. Aggregate Functions in select statement
5. JOINs in select statement + [Hbase Shell] Cell-value index option using lucene.
{code)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Vuk Ercegovac (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12538151 ]
Vuk Ercegovac commented on HADOOP-1608:
---------------------------------------
Hi Edward,
Is there a reason that GroupingFilterMap and IdentityFilterMap do not inherit from GroupingTableMap and IdentityTableMap instead of directly from TableMap? There seems to be some redundant code in the current patch. Alternatively, it may be better simply to enhance GroupingTableMap and IdentityTableMap with filtering.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517172 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
My suggestion is to develop HBase Shell in SQL-style, and develop numeric tools as a sub shell as described below.
If you are okay with it, I want the development to start.
{code}
HBase > help;
Type 'help <command>;' to see command-specific usage.
CLEAR Clear the screen.
CREATE Create a table.
DELETE Delete cell or row in table.
DESCRIBE Describe a table's columnfamilies.
DROP Drop columnfamilie(s) from a table or drop table(s).
EXIT Exit shell
INSERT Insert row into table.
SELECT Select values from a table.
SHOW List all tables or files.
NUMERIC HBase Numeric Analyzer Tool.
HBase > numeric;
Hbase Numeric Shell, 0.0.1 version
Type 'help;' for Hbase Numeric Analyzer Tool usage.
HBase Numeric > help;
Type 'help <command>;' to see command-specific usage.
STORE Store results to specified table(or file).
SUBSTITUTE Substitute expression to [A~Z].
TABLE Load a table.
Relational Algebra Operators.
PROJECTION Make a new relation set {cf_name1, ... , cf_name2}.
SELECTION Make a new relation set of specified Rows.
SORT Make a new sorted set, ordered according to columnfamilies.
HBase Numeric > exit;
HBase > exit;
nhn@yoon0105 ~/workspace/hadoop
$
{code}
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Fix Version/s: (was: 0.16.0)
0.15.1
Status: Patch Available (was: Open)
re-try
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.15.1
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v18.txt
- modified the show command. ( SHOW TABLES[or substitution variable name]; )
- modified the selection help string ( selection condition (>, <, AND, OR, etc.) )
- clean up the duplicated code
- moved the check code to saveCommand.java
- if job fail, delete the created output table
{code}
Hbase> a = table('movieLog_table');
Hbase> show a;
a = table(movieLog_table);
Hbase> b = a.selection(length > 100);
Hbase> show b;
b = a.selection(length: > 100);
Hbase>
{code}
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533234 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
Thanks for your advice.
The next patch code by your advice will improve.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546432 ]
stack commented on HADOOP-1608:
-------------------------------
Edward:
I committed the patch (Sorry it took so long). it basically works but there are some issues. Please file new JIRAs to fix.
'show table;' doesn't work ('table' is keyword in shell looks like).
I think you should peg the shell version same as rest of hadoop/hbase. Lets figure a way to get the version string into the shell preamble.
The below selection is not working properly. I would expect the result table to have nothing but one row with a cell value of 'g' -- not all content of source table.
{code}
hql > select * from y;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| x | x: | g |
+-------------------------+-------------------------+-------------------------+
| x | y: | e |
+-------------------------+-------------------------+-------------------------+
| x | z: | f |
+-------------------------+-------------------------+-------------------------+
3 row(s) in set (0.04 sec)
hql > show a;
a = table(y);
hql > b = a.selection(x = 'g');
hql > save b into table('e');
07/11/28 12:51:21 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
07/11/28 12:51:31 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
07/11/28 12:51:31 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
07/11/28 12:51:32 INFO mapred.MapTask: numReduceTasks: 1
07/11/28 12:51:32 INFO mapred.LocalJobRunner:
07/11/28 12:51:32 INFO mapred.TaskRunner: Task 'map_0000' done.
07/11/28 12:51:32 INFO mapred.LocalJobRunner: reduce > reduce
07/11/28 12:51:32 INFO mapred.TaskRunner: Task 'reduce_8stpva' done.
Successfully complete.
hql > select * from g;
'g' Table not found
hql > select * from e;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| x | x: | g |
+-------------------------+-------------------------+-------------------------+
| x | y: | e |
+-------------------------+-------------------------+-------------------------+
| x | z: | f |
+-------------------------+-------------------------+-------------------------+
3 row(s) in set (0.03 sec)
{code}
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Assignee: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
** ex. Result = Relation.Projection('year','legnth');
* Selection
** See : HADOOP-2003
* Product
* Rename
* Group
* Sort
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the attributes of a relation
** Result = π ~column_list~ (Relation)
* Selection
** See : HADOOP-2003
* Product
* Rename
* Group
* Sort
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546528 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
Selection is the row-based relational modeling operation.
So, i don't think it is a problem.
Your exepcted results are :
{code}
a = y.projection(x:);
b = a.selection(x = 'g');
{code}
And, If you find a bug, please let me know.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Assignee: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Attachment: shell_v06.patch
Fix it.
{code}
A = table('movieLog_table');
store A to table('temp_table');
{code}
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: added_test_code.txt)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v02.txt
Re-trying
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534505 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367680/patch_v11.txt
against trunk revision r584336.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/942/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/942/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/942/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/942/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
Build Locally
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Attachment: shell_v04.patch
Add some comments in code,
and if you agree, i would like to submit this patch.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516263 ]
edward yoon edited comment on HADOOP-1608 at 7/29/07 6:52 PM:
--------------------------------------------------------------
I am always concerned about implementing SQL-like syntax, familiar to many users.
But..... There are many reasons why I chose to implement the syntax.
* Consideration for all commands such as data relational, aggregation, and processing management.
* Differences between RDBMS and Multi Dimensional Model.
I think it is important to make users more aware of HBase's data model characteristics.
That way, they should be able to use HBase at its full potential.
I have introduced multi dimensional model(Hadoop + Hbase + MapReduce) as a sort of sistributed computing platform to application developers, using SQL-like ysntax examples.
In the end, it has create more confusion among them, as most of them tried to relate it to traditional RDMBS.
Also besides relational operation, I will add aggregation and anlaysis functions,
and the will be considerate of all the grammars to make the whole thing more consistent.
I have a plan to shore up wiki pages and will finish implementing the planned features.
I hope to get more advice and tips from many of you people and make it up to 0.0.2 version.
Thank you.
was:
I am always concerned about implementing SQL-like syntax, familiar to many users.
But..... There are many reasons why I chose to implement the syntax.
* Consideration for all commands such as data relational, aggregation, and processing management.
* Differences between RDBMS and Multi Dimensional Model.
I think it is important to make users more aware of HBase's data model characteristics.
That way, teh should be able to use HBase at its full potential.
I have introduced multi dimensional model(Hadoop + Hbase + MapReduce) as a sort of sistributed computing platform to application developers, using SQL-like ysntax examples.
In the end, it has create more confusion among them, as most of them tried to relate it to traditional RDMBS.
Also besides relational operation, I will add aggregation and anlaysis functions,
and the will be considerate of all the grammars to make the whole thing more consistent.
I have a plan to shore up wiki pages and will finish implementing the planned features.
I hope to get more advice and tips from many of you people and make it up to 0.0.2 version.
Thank you.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532483 ]
stack commented on HADOOP-1608:
-------------------------------
Regards submission of mapreduce jobs by the shell, do we need HADOOP-1622 committed? Let me know and I'll try and have a go at finishing it up. What of my comments above on 01/Aug/07 01:46 PM and on 28/Jul/07 12:38 PM have been addressed in this patch? Also, Edward, this large patch has no unit tests. Let me bet that if you add basic tests of the new functionality that in the end, you'll save both you and I a bunch of time as we do less than the tens of me-test/you-upload-new-patch that we did on HADOOP-1720 and HADOOP-1375.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: patch.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545418 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12370181/patch_v22.txt
against trunk revision r598152.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1159/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1159/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1159/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1159/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HADOOP-1608:
--------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed. Resolving. Thanks for the patch Edward.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Assignee: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: shell_v06.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v19.txt
add javacc element to build.xml
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v09.txt
Sorry, fix spell error.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540287 ]
stack commented on HADOOP-1608:
-------------------------------
Edward:
I do not understand what you mean by the following in your 02/Nov/07 06:09 PM comment: "I'll implement relational algebra operators to the tentative language and HQL by Sub-Tasks List sequence."
In org.apache.hadoop.hbase.shell.algebra, test for output table presence and creation if missing is duplicated code in Selection, Projection, DuplicateTable, etc. I tried to move this duplicated code back up into the RelationalOperation class as a utility but noticed then if table already exists, we don't call initJob because we return early (See end of the getConf in DuplicateTable for example). Is running one of these operators a second time, after table has been created, a problem? Have you tried it?
I did not have a mapreduce cluster running and so the last line hung for ever (You should add to help need for a mapreduce cluster).
{code}
Hbase> aaa = table('y');
Hbase> aaa;
Syntax error : Type 'help;' for usage.Message : Encountered ";" at line 1, column 5.
Hbase> show aaa;
Missing parameters. Please check 'Show' syntax
Hbase> bbb = group aaa by ('x'); Hbase> save bbb into table ('a');
{code}
Note, it would be nice if you entered a variable name only if you got a description of the variable content or perhaps doing 'show VARIABLE_NAME' output some kind of description.
I then started up a cluster and did the simplest of operations:
{code}
Hbase> aaa = table('x');
Hbase> save aaa into table(aaaaaa);
07/11/05 21:26:57 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).Job job_200711052125_0001 is still running........Job failed.
{code}
Job failed because of 'Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.mapred.TableSplit'. How do you run on a cluster Edward? Do you copy the hbase jar into the hadoop lib dir all over the cluster? Does HADOOP-1622 help here?
On job failure, should the table be removed? Currently, I have a new table 'aaaaaa' with nothing in it.
The output of Group, Selection, etc. is saved to an 'output' table?
It would be good if we could get into the help listing somewhere the operators and types of conditions allowed on selection (>, <, AND, OR, etc.).
One last thing, hbase shell (relational) operators running mapreduce jobs begins to impinge on PIG territory. We should be careful and avoid overlap/duplicating work. Would it make sense doing further operators as PIG user defined functions? (I suppose we'll be able to tell better after PIG-6, the hbase load/store, is done. Smile).
Otherwise, patch looks good Edward.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12513147 ]
stack commented on HADOOP-1608:
-------------------------------
Thinking more on my above comment, it won't get you far. It may work for all running in single context but will fail when you go to distribute your jar across multiple hosts.
Either copy the hadoop-hbase.jar to $HADOOP_HOME/lib everywhere on your cluster or build the hadoop-hbase.jar into your job jar under its lib directory.
Regards the latter, job jars submitted to hadoop are undone by the RunJar class out on each tasktracker node. Before invoking the jar's Main-Class from its MANIFEST.MF (or reading from the command-line if this is not set), it sets up the CLASSPATH so all jars found in the jars' lib directory, etc., are included. See the RunJar source for how this is done if you want to learn more (For an example, see nutch).
I took a look at bin/hadoop to add contribs to the classpath. We could make a patch to add 'build/contrib/*/classes'. This would work for development environment but for binary distributions, the distribution target could add contrib jars to the lib dir or into a hadoop-contrib sub-directory that got added to the classpath when bin/hadoop runs.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516265 ]
edward yoon edited comment on HADOOP-1608 at 7/29/07 7:13 PM:
--------------------------------------------------------------
to upload v06 patch.
was:
to upload v05 patch.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.15.1
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
to upload v05 patch.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Ko, Young-Su (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534742 ]
Ko, Young-Su commented on HADOOP-1608:
--------------------------------------
I like Edward's implementation. +1.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon reopened HADOOP-1608:
---------------------------------
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase > save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
{code}
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase> save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
description update.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539282 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
If you are not sure about committing this patch, let me know.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533643 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
>> InsertOneTableToAnother looks like it has general utility. Does it belong in the hbase mapred package? (Should it be named InsertOneTableIntoAnother?). Same for IdentityFilterMap?
I want them to be moved once they are surely classified as general classes.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: added_test_code.txt
Added test code.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: added_test_code.txt, patch.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Fix Version/s: 0.16.0
Status: Patch Available (was: Reopened)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544733 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
>> Exceptions are suppressed in maps and configurations. They are just printed on stderr, which is better than nothing, but jobs should fail. Create a Runtime exception from the checked exception so you can throw it where exceptions are not allowed. When you have a ParseException but you're only allowed throw ParseException, and the map task should fail, create an IOE out of the PE and throw that.
OK.
>> The GROUP help says this: "Group rows by value of an attribute and apply aggregate function independently to each group of rows"... but I do not see an 'aggregate function' in the example given. Am I missing something?
More details about 'GROUP' operation will be handled in HADOOP-1658 issue.
If you want, i'll drop it from the hadoop-1608.
But, i'd like to submit some classes for group to svn tree.
>> The SAVE command wants to run a mapreduce job to copy into result table. Should say it runs a MR job in help.
OK.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: altools_v01.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: patch.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [Hbase Shell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534263 ]
udanax edited comment on HADOOP-1608 at 10/12/07 3:30 AM:
---------------------------------------------------------------
I don't understand your 11/Oct/07 10:06 PM comment.
Could you repeat that?
----
I canceled the "submit patch".
When running in network mode, I recieve "User classes may not be found" error message.
Do you have any ideas other than setJar() method?
was (Author: udanax):
I don't understand your 11/Oct/07 10:06 PM comment.
Could you repeat that?
----
I canceled the "submit patch".
When running in network mode, I recieved "User classes my not be found" error message.
Do you have any ideas other than setJar() method?
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon resolved HADOOP-1608.
---------------------------------
Resolution: Fixed
I think this issue should first finalize syntax definition draft.
So, i will reopen after finalization.
http://wiki.apache.org/lucene-hadoop/Hbase/ShellPlans
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533640 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
PS.
I wanted to see how Hudson system reacts to my test code with my submissions.
I have gotten used to the system, and I will be more careful next time.
Thanks.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: shell_r_operators_v01.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [Hbase Shell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12543256 ]
udanax edited comment on HADOOP-1608 at 11/17/07 2:01 AM:
---------------------------------------------------------------
add a javacc element to build.xml
was (Author: udanax):
add javacc element to build.xml
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v17.txt
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
submit.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: shell_v02.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v10.txt
Fixed it using just setJar() method.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the attributes of a relation
** Result = π ~column_list~ (Relation)
* Selection
** See : HADOOP-2003
* Product
* Rename
* Group
* Sort
was:
Development of relational algebra operators has begun.
* Projection
* Selection
* Product
* Rename
* Group
* Sort
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, patch_v03.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the attributes of a relation
> ** Result = π ~column_list~ (Relation)
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12547400 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I think 'show table;' command didn't exist.
But, I agree that registered token can makes a little problem.
So, I think shell should clarify the quotations on grammar.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Assignee: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533215 ]
stack commented on HADOOP-1608:
-------------------------------
Hey Edward. Tests look great and thanks for picking up the new pattern where commands take a Writer in the constructor. Here's a couple of comments on the patch:
+ In TestBooleanExpression, you declare EXPRESSION but don't use it.
+ In TestBooleanTermFilter, you have a @SuppressWarnings on the hCluster yet you use it later. In this class, I see a setup but no corresponding takeDown shutting down the dfs and hbase mini clusters. It subclasses MultiRegionTable but doesn't seem to make use of any of the parent methods. Does it? If not, subclass HBaseTestCase or HBaseClusterTestCase instead.
+ Is the comparer class's name misspelled? Should it be Comparator?
+ InsertOneTableToAnother looks like it has general utility. Does it belong in the hbase mapred package? (Should it be named InsertOneTableIntoAnother?). Same for IdentityFilterMap?
+ The query package Selection class uses System.out logging. Add a logger?
+ Is it true that this patch does not add the new items to the shell?
+ Here are some old questions: What about variable names of more than one character? Entering a variable name, is its value emitted?
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
- edited some comments
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512848 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
Hi, All.
I want to use hbase.mapred on shell.
But, when I try to submit a job to the JobTracker, I get this exception error message.
{code}
2007-07-14 20:37:04,377 INFO org.apache.hadoop.mapred.TaskInProgress:
Error from task_200707142013_0004_r_000000_2:
java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
org.apache.hadoop.hbase.mapred.IdentityTableReduce
{code}
How can i fix it without another changes ?
Please give me some advice.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12545329 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I'd like to submit to svn tree.
If you are not sure about committing this patch, let me know.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516587 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
Ok, Stack.
Here's One of My Proposal Ideas.
I will injection of MDX queries into the hbase shell.
and seperate with numeric tool and process control shell.
see : http://publib.boulder.ibm.com/infocenter/ablxhelp/v8r4m0/index.jsp?topic=/com.ibm.db2.abx.cub.doc/abx-c-cube-supportedmdxsyntax.html
and, we must think about difference between traditional DBMS and Hbase.
{code}
udanax $> bin/shell
Hbase > ~~~ MDX Style queries with multi dimensional table ~~~ (manipulating tables.)
Hbase > _
Hbase > NumShell;
Numshell > ~~~ Matlab or Octave Style with mapped matrix ~~~ (manipulating tables..)
Numshell > exit;
Hbase > ..
Hbase > exit;
udanax $> exit
{code}
Feel free to disagree with me Stack.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516189 ]
Jeff Hammerbacher edited comment on HADOOP-1608 at 7/28/07 4:21 PM:
--------------------------------------------------------------------
Hey stack,
I actually like Edward's diction and implementation. The semantic gap between relational operators and standard SQL implementations is wide enough that implementing the relational operators themselves is the correct approach. A SQL-like query language can easily be written that translates queries into a tree of relational operators, as you point out (javacc being like yacc for java, i assume--new to java). That's a different problem though.
Apologies if I am restating information you already know. Keeping the raw relational operators separate from the query language that leverages those operators seems like an important design goal though and now seemed like as good a time as any to jump into the discussion.
Anyways, I am assuming Edward's use of "relational operator" is meant to reference standard set operators with their restrictions when applied to a relational algebra: http://en.wikipedia.org/wiki/Relational_algebra.
Feel free to disagree with me Edward.
Jeff
was:
Hey stack,
I actually like Edward's diction and implementation. The semantic gap between relational operators and standard SQL implementations is wide enough that implementing the relational operators themselves is the correct approach. A SQL-like query language can easily be written that translates queries into a tree of relational operators, as you point out (javacc being like yacc for java, i assume--new to java). That's a different problem though.
Apologies if I am restating information you already know. Keeping the raw relational operators separate from the query language that leverages those operators seems like an important design goal though and now seemed like as good a time as any to jump into the discussion.
Jeff
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [Hbase Shell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12536618 ]
udanax edited comment on HADOOP-1608 at 10/22/07 3:53 AM:
---------------------------------------------------------------
And, I would like to hear your opinions about shell sub-packages for mapred/filter classes and hbase mapred/filter packages.
May I touch hbase mapred/filter packages?
was (Author: udanax):
And, I would like to hear your opinions about shell sub-packages for mapred/filter classes and hbase mapred/filter packages.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532832 ]
udanax edited comment on HADOOP-1608 at 10/5/07 11:51 PM:
---------------------------------------------------------------
I had checked your comments, fixed it.
was (Author: udanax):
I had checked you comments, fixed it.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533269 ]
udanax edited comment on HADOOP-1608 at 10/8/07 10:14 PM:
---------------------------------------------------------------
>> + Here are some old questions: What about variable names of more than one character? Entering a variable name, is its value emitted?
I remove limitation.
{code}
Hbase> Relation = table('movieLog_table');
Hbase> Result = Relation.selection(length > 100);
Hbase> save Result into table('result');
Job job_local_1 is still running..
Successfully complete.
Hbase> select * from result;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | length: | 142 |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | producer: | Blair Peters |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | studioName: | Disney |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | year: | 1991 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Star Wars | length: | 124 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | producer: | George Lucas |
+-------------------------+-------------------------+-------------------------+
| Star Wars | studioName: | Fox |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user | 5 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | year: | 1977 |
+-------------------------+-------------------------+-------------------------+
13 row(s) in set (0.04 sec)
{code}
was (Author: udanax):
I remove limitation.
{code}
Hbase> Relation = table('movieLog_table');
Hbase> Result = Relation.selection(length > 100);
Hbase> save Result into table('result');
Job job_local_1 is still running..
Successfully complete.
Hbase> select * from result;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | length: | 142 |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | producer: | Blair Peters |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | studioName: | Disney |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | year: | 1991 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Star Wars | length: | 124 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | producer: | George Lucas |
+-------------------------+-------------------------+-------------------------+
| Star Wars | studioName: | Fox |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user | 5 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | year: | 1977 |
+-------------------------+-------------------------+-------------------------+
13 row(s) in set (0.04 sec)
{code}
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12543282 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12369713/patch_v19.txt
against trunk revision r595563.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1113/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1113/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1113/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1113/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12534263 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I don't understand your 11/Oct/07 10:06 PM comment.
Could you repeat that?
----
I canceled the "submit patch".
When running in network mode, I recieved "User classes my not be found" error message.
Do you have any ideas other than setJar() method?
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v14.txt
added expression parser.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v21.txt
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546678 ]
Hudson commented on HADOOP-1608:
--------------------------------
Integrated in Hadoop-Nightly #317 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/317/])
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Assignee: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540380 ]
stack commented on HADOOP-1608:
-------------------------------
Here's some history Vuk: HADOOP-1648.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Affects Version/s: 0.14.1
Status: Patch Available (was: Open)
I had successfully tested my patch on my computer.
I submit this patch.
If you are not sure about committing this patch, let me know.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-1608:
-------------------------------------
Assignee: Edward Yoon
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Assignee: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v05.txt
added 'IN, NOT IN' difference set operation.
{code}
A = table('movieLog_table');
B = A.selection(length > 100 and studioName not in ('Fox','YoungGu Art');
{code}
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: (was: shell_v05.patch)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> ** ex. Result = Relation.Projection('year','legnth');
> * Selection
> ** See : HADOOP-2003
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v16.txt
edited show command.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
re-submit
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533634 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12367430/patch_v09.txt
against trunk revision r583328.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/914/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/914/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/914/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/914/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v22.txt
{code}
[javadoc] /root/workspace/hadoop/src/contrib/hbase/src/java/org/apache/hadoop/ hbase/shell/algebra/OperationEvaluator.java:74: warning - @return tag has no arg uments.
{code}
Fixed it.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539932 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
If you are not sure about committing this patch, let me know.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539643 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12368861/patch_v15.txt
against trunk revision r591226.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1054/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1054/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1054/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1054/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase> save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase> save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* Sort
Summary: [Hbase Shell] Relational Algrebra Operators (was: [HbaseShell] Relational Algrebra Operators)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539495 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
>> I'm having some difficulty following the model Edward particularly when there is a dearth of javadoc (You should at a minumum provide documentation of interfaces).
Okay, i can adding more comments.
>> Do these mapreduce jobs work in a cluster of multiple machines?
Okay
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514591 ]
edward yoon commented on HADOOP-1608:
-------------------------------------
>> We could make a patch to add 'build/contrib/*/classes'
I agree with you Micheal Stack.
because i don't think i should make a new JobTracker stater all over again.
..
Then can this stack handle this issue?
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v15.txt
added some comments.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540323 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I now realize hadoop-1662 is required.
So i will wait.
Thanks.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Attachment: shell_v03.patch
* Fixed Filter Operation
selection(length > 100 and studioName = 'Fox');
selection(studioName = 'Fox' or studioName = 'Paramount');
selection(length > 100 and studioName = 'Fox' or studioName = 'Paramount');
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Attachment: shell_r_operators_v01.patch
Abstract and Interface Classes.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12513102 ]
stack commented on HADOOP-1608:
-------------------------------
Try this Edward:
{code}
% ${HBASE_HOME}/bin/hbase org.apache.hadoop.mapred.JobClient YOUR_JOB_JAR
{code}
The above does as '${HADOOP_HOME}/bin/hadoop jar YOUR_JOB_JAR' does only it has hadoop-hbase.jar on the CLASSPATH
Perhaps we should add to the hbase script support for a 'jar' argument?
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.0
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516303 ]
edward yoon edited comment on HADOOP-1608 at 7/30/07 3:25 AM:
--------------------------------------------------------------
In addition,
After seeing Multi-Dimensional Model Style Data Storage(BigTable Clone)built on top of Hadoop,
I thought features such as Relational Algebra, Linear Algebra, Matrix operation, and etc. would be important.
{code}
edward yoon - [17/May/07 04:12 AM ] // Hadoop-1375
I have just started writing code for linear algebraic computation on Hadoop+Hbase based parallel machines.
I think It will makes the hadoop an even better platform for scientific and advanced analytics programming.
if i should be moving my classes to a different location, let me know how to do.
{code}
Therefore, I would like to develop relation and matrix operator commands first
and have them become consistent.
If HBase has online-transaction processing like HBase filter implemented,
I may make Select command more similar to SQL.
Also, I am willing to rearrange all the syntax if it is necessary.
However, in my opinion, Batch-type analysis processing operations such as relational and matrix operators
would be better served with the current syntax.
was:
In addition,
After seeing Multi-Dimensional Model Style Data Storage(BigTable Clone)built on top of Hadoop,
I thought features such as Relational Algebra, Linear Algebra, Matrix operation, and etc. would be important.
{code}
edward yoon - [17/May/07 04:12 AM ] // Hadoop-1375
I have just started writing code for linear algebraic computation on Hadoop+Hbase based parallel machines.
I think It will makes the hadoop an even better platform for scientific and advanced analytics programming.
if i should be moving my classes to a different location, let me know how to do.
{code}
Therefore, I would like to develop relation and matrix operator commands first
and have them become consistent.
If HBase has online-transaction processing like HBase filter implemented,
I may make Select command more similar to SQL.
Also, I am willing to rearrange all the syntax if it is necessary.
However, in my opinion, Batch-type operations such as relational and matrix operators
would be better served with the current syntax.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516263 ]
edward yoon edited comment on HADOOP-1608 at 7/30/07 6:10 AM:
--------------------------------------------------------------
I am always concerned about implementing SQL-like syntax, familiar to many users.
But..... There are many reasons why I chose to implement the syntax.
* Consideration for all commands such as data relational, aggregation, and analysis processing management.
* Differences between RDBMS and Multi Dimensional Model.
I think it is important to make users more aware of HBase's data model characteristics.
That way, they should be able to use HBase at its full potential.
I have introduced multi dimensional model based data handling examples on distributed computing platform(Hadoop+Mapreduce) to application developers, using SQL-like ysntax examples.
In the end, it has create more confusion among them, as most of them tried to relate it to traditional RDMBS.
Also besides relational operation, I will add aggregation and anlaysis functions,
and the will be considerate of all the grammars to make the whole thing more consistent.
I have a plan to shore up wiki pages and will finish implementing the planned features.
I hope to get more advice and tips from many of you people and make it up to 0.0.2 version.
Thank you.
was:
I am always concerned about implementing SQL-like syntax, familiar to many users.
But..... There are many reasons why I chose to implement the syntax.
* Consideration for all commands such as data relational, aggregation, and processing management.
* Differences between RDBMS and Multi Dimensional Model.
I think it is important to make users more aware of HBase's data model characteristics.
That way, they should be able to use HBase at its full potential.
I have introduced multi dimensional model(Hadoop + Hbase + MapReduce) as a sort of sistributed computing platform to application developers, using SQL-like ysntax examples.
In the end, it has create more confusion among them, as most of them tried to relate it to traditional RDMBS.
Also besides relational operation, I will add aggregation and anlaysis functions,
and the will be considerate of all the grammars to make the whole thing more consistent.
I have a plan to shore up wiki pages and will finish implementing the planned features.
I hope to get more advice and tips from many of you people and make it up to 0.0.2 version.
Thank you.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [Hbase Shell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12544733 ]
udanax edited comment on HADOOP-1608 at 11/22/07 2:50 AM:
---------------------------------------------------------------
>> Exceptions are suppressed in maps and configurations. They are just printed on stderr, which is better than nothing, but jobs should fail. Create a Runtime exception from the checked exception so you can throw it where exceptions are not allowed. When you have a ParseException but you're only allowed throw ParseException, and the map task should fail, create an IOE out of the PE and throw that.
OK.
>> The GROUP help says this: "Group rows by value of an attribute and apply aggregate function independently to each group of rows"... but I do not see an 'aggregate function' in the example given. Am I missing something?
More details about 'GROUP' operation will be handled in HADOOP-1658 issue.
If you want, i'll drop it from the hadoop-1608.
But, i'd like to submit some classes for group to svn tree.
So, i'll change only "group help" content.
>> The SAVE command wants to run a mapreduce job to copy into result table. Should say it runs a MR job in help.
OK.
was (Author: udanax):
>> Exceptions are suppressed in maps and configurations. They are just printed on stderr, which is better than nothing, but jobs should fail. Create a Runtime exception from the checked exception so you can throw it where exceptions are not allowed. When you have a ParseException but you're only allowed throw ParseException, and the map task should fail, create an IOE out of the PE and throw that.
OK.
>> The GROUP help says this: "Group rows by value of an attribute and apply aggregate function independently to each group of rows"... but I do not see an 'aggregate function' in the example given. Am I missing something?
More details about 'GROUP' operation will be handled in HADOOP-1658 issue.
If you want, i'll drop it from the hadoop-1608.
But, i'd like to submit some classes for group to svn tree.
>> The SAVE command wants to run a mapreduce job to copy into result table. Should say it runs a MR job in help.
OK.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12517069 ]
stack commented on HADOOP-1608:
-------------------------------
The basic SQL operators are as I see it foundational Lets move the SQL-like data manipulatilon and data creation syntax closer to 'standard' SQL. It'll make it easier on folks getting started with HBase. This we can do as a separate issue. Someone else (perhaps I) can work on this.
Regards a mix of SQL and other operator types -- relational and linear algebra types -- all in a single shell, this can work as long each of the operators is coherent and it is clear to which domain each belongs. The danger to avoid is that the shell becomes cluttered with the user unable to figure which operators belong together.
I do not follow your proposal above? Are you suggesting that the shell prompt change as we move between domains (from 'basic' to 'mdx' to 'matlab'?).
Regards your last patch, v07, here are some comments Edward:
+ In general, the patch needs much more javadoc and commentary. What does the StoreCommand that is different from BasicCommand. It runs a mapreduce job to what end? What does the CompoundRestrictionMap do? Why a CommandPool? Also, most of your data members are default access. Private would be better (unless you have reason for them being otherwise accessible; e.g. access is needed unit testing. See 'Coding' section from http://wiki.apache.org/lucene-hadoop/CodeReviewChecklist where it says ' class & member access - as restricted as it can be (subject to testing requirements)').
+ TestRelationMap should be in the org.apache.hadoop.hbase.shell.batch.relation sub-package. The setup method from this class seems to be replicating much from MiniHBaseCluster. Take a look at TestScanner2. When test runs, root and meta tables have been added. In other words, you do not need to them explicitly yourself as you do in the setup. Then all your setup would have to do is startup the mini MR cluster (Should we add a MRTestCase and MRClusterTestCase in hbase tests?).
+ JobAnalysis javadoc and commentary is stunted. Is it to generate a configuration for a mapreduce job (After digging, this looks to be the case)? Why is the null key special from CommandPool iteration in constructor (Is this iteration safe? Can't CommandPool be updated concurrently?)?
+ CommandPool should have a private constructor so it can't be instantiated (This class as a Singleton might be a little more orthodox). Javadoc on why it exists would help.
+ The LOG class is wrong in RestrictionMap. Why suppress 'deprecation' in initJob? I'd suggest exposing the warning so it will get fixed. How about using the new filters that have been added to hbase to do your in-map filtering? At least for your examples in BinaryOperationFilter, it looks like they'd work? BinaryOperationFilter would parse the passsed strings and generate appropriate set of new filters. Why create the BinaryOperationFilter everytime the map is called?
+ ReturnMsg needs a toString or a getMessage method. Should type be named resultCode or errCode instead?.
+ Where do you discuss difference between 'basic', 'oltp', 'relation' and 'matrix'?
+ Seems odd to me that the LinearAlgebra can return a RelationalAlgebra (and vice-versa). Would having RelationAlgebra and LinearAlgebra marker interfaces rather than have JobConfGenerator have get*Algebra be cleaner? Why is CopyTable a RelationalAlgebra job (I understand why Projection is)?
+ Which of my previous review comments have your implemented (or put aside because you thought better)? What about variable names of more than one character? Entering a variable name, is its value emitted? Has the NPE been fixed? What about the issue I raise regards splitting on ' OR '? This latter for sure has not been addressed (Thanks for adding some unit tests)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v11.txt
Rebuilding code indentation
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Vuk Ercegovac (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540325 ]
Vuk Ercegovac commented on HADOOP-1608:
---------------------------------------
Hi stack,
The Class not found exception for TableSplit is due to HBase's jar not being found in the classpath
of the MapTask's JVM. There are several work arounds: 1) as stated, copy the hbase jar to hadoop/lib to
force its inclusion, 2) add hbase jar into the HADOOP_CLASSPATH which is picked up by the various hadoop
scripts, or 3) use the multiple jar file work around. I prefer option 2) but both 1) and 2) worked for me.
I have not tried option 3). Do you know of a preferred way to deal with such issues?
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Fix Version/s: (was: 0.15.1)
0.16.0
Status: Patch Available (was: Open)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533105 ]
udanax edited comment on HADOOP-1608 at 10/8/07 4:50 AM:
--------------------------------------------------------------
added 'IN, NOT IN' difference set operation.
{code}
A = table('movieLog_table');
B = A.selection(length > 100 and studioName not in ('Fox','YoungGu Art'));
{code}
was (Author: udanax):
added 'IN, NOT IN' difference set operation.
{code}
A = table('movieLog_table');
B = A.selection(length > 100 and studioName not in ('Fox','YoungGu Art');
{code}
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533972 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I would like to submit this patch to svn trunk.
If you are not sure about committing this patch, let me know.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Attachment: shell_v05.patch
Fixed javadoc warning.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.1
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546227 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
How long should i wait?
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt, patch_v18.txt, patch_v19.txt, patch_v20.txt, patch_v21.txt, patch_v22.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase> save Result into table('result');
{code}
* Product
* Rename
* Group
* Sort
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
** ex. Result = Relation.Projection('year','legnth');
* Selection
** See : HADOOP-2003
* Product
* Rename
* Group
* Sort
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase > save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
or Result = R1.join(R1.actor:hero = R2.Row) and R2;
or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
{code}
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase > save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
or Result = R1.join(R1.actor:hero = R2.Row) and R2;
or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
{code}
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v13.txt
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516153 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12362720/shell_v05.patch applied and successfully tested against trunk revision r560014.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/482/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/482/console
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12516587 ]
edward yoon edited comment on HADOOP-1608 at 7/30/07 5:53 PM:
--------------------------------------------------------------
Ok, Stack.
Here's One of My Proposal Ideas.
I will more think about injection of general MDX queries.
and seperate with numeric tool and process control shell.
see : http://publib.boulder.ibm.com/infocenter/ablxhelp/v8r4m0/index.jsp?topic=/com.ibm.db2.abx.cub.doc/abx-c-cube-supportedmdxsyntax.html
and, we must think about difference between traditional DBMS and Hbase.
{code}
udanax $> bin/shell
Hbase > ~~~ MDX Style queries with multi dimensional table ~~~ (manipulating tables.)
Hbase > _
Hbase > NumShell;
Numshell > ~~~ Matlab or Octave Style with mapped matrix ~~~ (manipulating tables..)
Numshell > exit;
Hbase > ..
Hbase > exit;
udanax $> exit
{code}
Feel free to disagree with me Stack.
was:
Ok, Stack.
Here's One of My Proposal Ideas.
I will injection of MDX queries into the hbase shell.
and seperate with numeric tool and process control shell.
see : http://publib.boulder.ibm.com/infocenter/ablxhelp/v8r4m0/index.jsp?topic=/com.ibm.db2.abx.cub.doc/abx-c-cube-supportedmdxsyntax.html
and, we must think about difference between traditional DBMS and Hbase.
{code}
udanax $> bin/shell
Hbase > ~~~ MDX Style queries with multi dimensional table ~~~ (manipulating tables.)
Hbase > _
Hbase > NumShell;
Numshell > ~~~ Matlab or Octave Style with mapped matrix ~~~ (manipulating tables..)
Numshell > exit;
Hbase > ..
Hbase > exit;
udanax $> exit
{code}
Feel free to disagree with me Stack.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Description:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase > save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
{code}
was:
Development of relational algebra operators has begun.
* Projection
** selects a subset of the columnfamilies of a relation
** Result = π ~column_list~ (Relation)
{code}
Hbase > Result = Relation.Projection('year','legnth');
Hbase > save Result into table('result');
{code}
* Selection
** See : HADOOP-2003 issue's description
{code}
Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
Hbase > save Result into table('result');
{code}
* Group
** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
* θ Join
** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
{code}
Hbase > R1 = table('movieLog_table');
Hbase > R2 = table('personInfo_table');
Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
{code}
update description.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2; or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v04.txt
Trivial test code update.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Affects Version/s: (was: 0.14.1)
0.14.3
Status: Patch Available (was: Open)
I had successfully tested my patch on my computer.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.3
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533269 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I remove limitation.
{code}
Hbase> Relation = table('movieLog_table');
Hbase> Result = Relation.selection(length > 100);
Hbase> save Result into table('result');
Job job_local_1 is still running..
Successfully complete.
Hbase> select * from result;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | length: | 142 |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | producer: | Blair Peters |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | studioName: | Disney |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Mighty Ducks | year: | 1991 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Star Wars | length: | 124 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | producer: | George Lucas |
+-------------------------+-------------------------+-------------------------+
| Star Wars | studioName: | Fox |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user | 5 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | year: | 1977 |
+-------------------------+-------------------------+-------------------------+
13 row(s) in set (0.04 sec)
{code}
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533028 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I'll implement relational algebra operators to the tentative language and HQL by Sub-Tasks List sequence.
If someone can start these work with me, add a comment to each issue.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase> Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase> save Result into table('result');
> {code}
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539533 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
let's go.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer');
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Status: Open (was: Patch Available)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Status: Patch Available (was: Open)
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532849 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I got a successful message when i tested my computer.
What's wrong?
{code}
[junit] Running org.apache.hadoop.hbase.shell.TestConsoleTable
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.016 sec
[junit] Running org.apache.hadoop.hbase.shell.TestHBaseShell
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 59.595 sec
[junit] Running org.apache.hadoop.hbase.shell.TestSubstitutionVariables
[junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 0.031 sec
[junit] Running org.apache.hadoop.hbase.shell.expression.TestBooleanExpressi
on
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.031 sec
[junit] Running org.apache.hadoop.hbase.shell.query.TestFilterMapReduce
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 21.812 sec
[junit] Running org.apache.hadoop.hbase.util.TestKeying
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.032 sec
[junit] Running org.onelab.test.TestFilter
[junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 0.046 sec
BUILD SUCCESSFUL
Total time: 27 minutes 42 seconds
{code}
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: added_test_code.txt, patch.txt, patch_v01.txt, patch_v02.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Fix Version/s: 0.14.0
Affects Version/s: (was: 0.13.0)
0.13.1
Status: Patch Available (was: Open)
Builds locally. Trying against hudson
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.13.1
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12541607 ]
Hadoop QA commented on HADOOP-1608:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12369317/patch_v17.txt
against trunk revision r593748.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1090/testReport/
Findbugs warnings: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1090/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1090/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1090/console
This message is automatically generated.
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt, patch_v16.txt, patch_v17.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [HbaseShell] Relational Algrebra
Operators
Posted by "edward yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edward yoon updated HADOOP-1608:
--------------------------------
Attachment: shell_v07.patch
added TestMapReduce.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: edward yoon
> Priority: Minor
> Fix For: 0.14.0
>
> Attachments: shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539759 ]
Edward Yoon commented on HADOOP-1608:
-------------------------------------
I'll implement relational algebra operators to the tentative language and HQL by Sub-Tasks List sequence.
Check Please.
{code}
1. Sort Join Implementation
2. The boolean expression can be added to the where clause of select statement
3. Aggregate Functions -----------------------------------------------------------------------------------> {Matrix issues}
4. Aggregate Functions in select statement
5. JOINs in select statement + [Hbase Shell] Cell-value index option using lucene.
{code)
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt, patch_v10.txt, patch_v11.txt, patch_v12.txt, patch_v13.txt, patch_v14.txt, patch_v15.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B ~1~,B ~2~,...,B ~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer: = R2.ROW) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row) and R2;
> or Result = R1.join(R1.actor:hero = R2.Row and R1.studioName = 'Fox' and R2.occupation = 'singer') and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [Hbase Shell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12533598 ]
udanax edited comment on HADOOP-1608 at 10/10/07 1:12 AM:
---------------------------------------------------------------
* added 'JOIN' operation. more details will be handled in HADOOP-2021 issue.
{code}
Hbase > a = table('movieLog_table');
Hbase > b = table('personInfo_table');
Hbase > c = a.Join(a.producer: = b.ROW) and b;
Hbase > save c into table('joinedTable');
Hbase > d = a.Join(a.actor:hero = b.ROW) and b;
Hbase > save d into table('heroActors_movieInfo');
Hbase > select * from joinedTable;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| Star Wars | birth: | May 14, 1944 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Star Wars | length: | 124 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | producer: | George Lucas |
+-------------------------+-------------------------+-------------------------+
| Star Wars | studioName: | Fox |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user | 5 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | year: | 1977 |
+-------------------------+-------------------------+-------------------------+
{code}
was (Author: udanax):
* added 'JOIN' operation. more details will be handled in HADOOP-2013 issue.
{code}
Hbase > a = table('movieLog_table');
Hbase > b = table('personInfo_table');
Hbase > c = a.Join(a.producer: = b.ROW) and b;
Hbase > save c into table('joinedTable');
Hbase > d = a.Join(a.actor:hero = b.ROW) and b;
Hbase > save d into table('heroActors_movieInfo');
Hbase > select * from joinedTable;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| Star Wars | birth: | May 14, 1944 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Star Wars | length: | 124 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | producer: | George Lucas |
+-------------------------+-------------------------+-------------------------+
| Star Wars | studioName: | Fox |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user | 5 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | year: | 1977 |
+-------------------------+-------------------------+-------------------------+
{code}
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Affects Versions: 0.14.1
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt, patch_v09.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-1608) [HbaseShell] Relational
Algrebra Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531789 ]
udanax edited comment on HADOOP-1608 at 10/3/07 12:10 AM:
---------------------------------------------------------------
I used only mapreduce job for relational operations. (I think scanner should be used for specific situation, too.)
I'll implement operations of relational,matrix,topological algebra features using mapreduce job.
And, Shell will provides a tentative language for those operations with the current syntax.
After all done, I think we can discuss the next steps to improve query language, optimizing the query execution process.
----
If it's okay with you, i would like to submit this patch.
and, If you are not sure about committing this patch, let me know.
was (Author: udanax):
If it's okay with you, i would like to submit this patch.
and, If you are not sure about committing this patch, let me know.
> [HbaseShell] Relational Algrebra Operators
> ------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Attachments: patch.txt, shell_r_operators_v01.patch, shell_v02.patch, shell_v03.patch, shell_v04.patch, shell_v05.patch, shell_v06.patch, shell_v07.patch
>
>
> Development of relational algebra operators has begun.
> * Projection
> * Selection
> * Product
> * Rename
> * Group
> * Sort
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1608) [Hbase Shell] Relational Algrebra
Operators
Posted by "Edward Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward Yoon updated HADOOP-1608:
--------------------------------
Attachment: patch_v08.txt
* added 'JOIN' operation. more details will be handled in HADOOP-2013 issue.
{code}
Hbase > a = table('movieLog_table');
Hbase > b = table('personInfo_table');
Hbase > c = a.Join(a.producer: = b.ROW) and b;
Hbase > save c into table('joinedTable');
Hbase > d = a.Join(a.actor:hero = b.ROW) and b;
Hbase > save d into table('heroActors_movieInfo');
Hbase > select * from joinedTable;
+-------------------------+-------------------------+-------------------------+
| Row | Column | Cell |
+-------------------------+-------------------------+-------------------------+
| Star Wars | birth: | May 14, 1944 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | inColor: | true |
+-------------------------+-------------------------+-------------------------+
| Star Wars | length: | 124 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | producer: | George Lucas |
+-------------------------+-------------------------+-------------------------+
| Star Wars | studioName: | Fox |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user | 5 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | vote:user 0 | 2 |
+-------------------------+-------------------------+-------------------------+
| Star Wars | year: | 1977 |
+-------------------------+-------------------------+-------------------------+
{code}
> [Hbase Shell] Relational Algrebra Operators
> -------------------------------------------
>
> Key: HADOOP-1608
> URL: https://issues.apache.org/jira/browse/HADOOP-1608
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Environment: All environments
> Reporter: Edward Yoon
> Priority: Minor
> Fix For: 0.16.0
>
> Attachments: patch_v01.txt, patch_v02.txt, patch_v03.txt, patch_v04.txt, patch_v05.txt, patch_v06.txt, patch_v07.txt, patch_v08.txt
>
>
> Development of relational algebra operators has begun.
> * Projection
> ** selects a subset of the columnfamilies of a relation
> ** Result = π ~column_list~ (Relation)
> {code}
> Hbase > Result = Relation.Projection('year','legnth');
> Hbase > save Result into table('result');
> {code}
> * Selection
> ** See : HADOOP-2003 issue's description
> {code}
> Hbase > Result = Relation.Selection(length > 100 and studioName = 'Fox');
> Hbase > save Result into table('result');
> {code}
> * Group
> ** more details about 'GROUP' operation will be handled in HADOOP-1658 issue.
> * θ Join
> ** The join of two relations R1(A ~1~,A ~2~,...,A ~n~) and R2(B~1~,B~2~,...,B~m~) is a relation with degree k=n+m and attributes (A ~1~,A ~2~,...,A ~n~, B ~1~,B ~2~,...,B ~m~) that satisfy the join condition
> {code}
> Hbase > R1 = table('movieLog_table');
> Hbase > R2 = table('personInfo_table');
> Hbase > Result = R1.join(R1.producer = R2.ROW) and R2;
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.