You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@mrql.apache.org by Marcos Freitas <ma...@idiro.com> on 2015/01/16 15:03:23 UTC

Error on union script

Hello,

I'm trying to do a union, followed by an select, if I run only the union it
works, but when I add the line with the 'a2 = SELECT' I keep getting an
error.
Here's the script I'm running:

ds =
source(line,'/tmp/file1','|',type(<city:string,country:string,pop:string>));
ds2 = source(line,'/tmp/file2
','|',type(<city:string,country:string,pop:string>));

a1 = (SELECT <city:d.city, country:d.country, pop:d.pop> FROM d in ds)
        union
        (SELECT <city:d2.city, country:d2.country, pop:d2.pop> FROM d2 in
ds2);

a2 = SELECT (a.city, a.country, a.pop) from a in a1;

dump '/tmp/test_union' from a2;


And here's the error I get:

Apache MRQL version 0.9.2 (compiled distributed Hadoop MapReduce mode with
2 reducers)
Query type: !bag(( string, string, string ))
*** Type error (line: 6, position: 75)
*** Undefined variable: x_8
Variable Types:
 x_10: ( < city: string, country: string, pop: string >, < city: string,
country: string, pop: string > )
 x_13: ( < city: string, country: string, pop: string >, < city: string,
country: string, pop: string > )
 x_12: < city: string, country: string, pop: string >
 x_11: < city: string, country: string, pop: string >
 x_5: < city: string, country: string, pop: string >
 x_4: < city: string, country: string, pop: string >
 x_5: < city: string, country: string, pop: string >
 x_4: < city: string, country: string, pop: string >
 x_3: < city: string, country: string, pop: string >
 x_5: < city: string, country: string, pop: string >
 x_4: < city: string, country: string, pop: string >
Run time: 0.049 secs

Can someone help me with that, as I didn't find many resources about Union
on MRQL.

Thanks,
Marcos Freitas


-- 
Marcos Freitas

Idiro Technologies
Clarendon House,
34-37 Clarendon St,
Dublin 2
Ireland

E: marcos.freitas@idiro.com

Re: Error on union script

Posted by Marcos Freitas <ma...@idiro.com>.

Hi Leonidas,

Thanks for your help. I've made the change that you suggested, but now I
get this exception:

Apache MRQL version 0.9.2 (compiled distributed Hadoop MapReduce mode with
2 reducers)
Query type: !bag(< city: string, country: string, pop: string >)
Physical plan:
Merge:
   left: cMap:
            input: Source (binary): "/user/marcos/tmp/mrql_union_bin"
   right: cMap:
             input: Source (binary): "/user/marcos/tmp/mrql_union_bin"
15/01/16 16:33:08 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
15/01/16 16:33:36 WARN mapred.JobClient: Use GenericOptionsParser for
parsing the arguments. Applications should implement Tool for the same.
Exception in thread "main" java.lang.Error: wrong MRData:
org.apache.mrql.MR_dataset@56f449b8
    at org.apache.mrql.Interpreter.reify(Interpreter.java:133)
    at
org.apache.mrql.Interpreter.new_distributed_binding(Interpreter.java:80)
    at org.apache.mrql.TopLevel.distributed_assign(TopLevel.java:126)
    at org.apache.mrql.TopLevel.evaluate_top_level(TopLevel.java:267)
    at org.apache.mrql.Translator.top_level(Translator.java:443)
    at
org.apache.mrql.CUP$MRQLParser$actions.CUP$MRQLParser$do_action(MRQLParser.java:4492)
    at org.apache.mrql.MRQLParser.do_action(MRQLParser.java:1977)
    at java_cup.runtime.lr_parser.parse(lr_parser.java:584)
    at org.apache.mrql.Main.main(Main.java:177)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)


Thanks,
Marcos Freitas


On 16 January 2015 at 13:59, Leonidas Fegaras <fe...@cse.uta.edu> wrote:

>  Hi Marcos,
> Yes, this is a bug. I will file a bug report on JIRA.
> In the meantime, you may use:
> store a1 := ...
> instead of:
> a1 = ...
> Thank you for reporting the error.
> Best
> Leonidas
>
>
> On 01/16/2015 08:03 AM, Marcos Freitas wrote:
>
>  Hello,
>
> I'm trying to do a union, followed by an select, if I run only the union
> it works, but when I add the line with the 'a2 = SELECT' I keep getting an
> error.
>  Here's the script I'm running:
>
>  ds =
> source(line,'/tmp/file1','|',type(<city:string,country:string,pop:string>));
> ds2 = source(line,'/tmp/file2
> ','|',type(<city:string,country:string,pop:string>));
>
> a1 = (SELECT <city:d.city, country:d.country, pop:d.pop> FROM d in ds)
>         union
>         (SELECT <city:d2.city, country:d2.country, pop:d2.pop> FROM d2 in
> ds2);
>
> a2 = SELECT (a.city, a.country, a.pop) from a in a1;
>
> dump '/tmp/test_union' from a2;
>
>
>  And here's the error I get:
>
>  Apache MRQL version 0.9.2 (compiled distributed Hadoop MapReduce mode
> with 2 reducers)
> Query type: !bag(( string, string, string ))
> *** Type error (line: 6, position: 75)
> *** Undefined variable: x_8
> Variable Types:
>  x_10: ( < city: string, country: string, pop: string >, < city: string,
> country: string, pop: string > )
>  x_13: ( < city: string, country: string, pop: string >, < city: string,
> country: string, pop: string > )
>  x_12: < city: string, country: string, pop: string >
>  x_11: < city: string, country: string, pop: string >
>  x_5: < city: string, country: string, pop: string >
>  x_4: < city: string, country: string, pop: string >
>  x_5: < city: string, country: string, pop: string >
>  x_4: < city: string, country: string, pop: string >
>  x_3: < city: string, country: string, pop: string >
>  x_5: < city: string, country: string, pop: string >
>  x_4: < city: string, country: string, pop: string >
> Run time: 0.049 secs
>
>  Can someone help me with that, as I didn't find many resources about
> Union on MRQL.
>
>  Thanks,
> Marcos Freitas
>
>
> --
>  Marcos Freitas
>
> Idiro Technologies
> Clarendon House,
> 34-37 Clarendon St,
> Dublin 2
> Ireland
>
> E: marcos.freitas@idiro.com
>
>
>


-- 
Marcos Freitas

Idiro Technologies
Clarendon House,
34-37 Clarendon St,
Dublin 2
Ireland

E: marcos.freitas@idiro.com

Re: Error on union script

Posted by Leonidas Fegaras <fe...@cse.uta.edu>.

Hi Marcos,
Yes, this is a bug. I will file a bug report on JIRA.
In the meantime, you may use:
store a1 := ...
instead of:
a1 = ...
Thank you for reporting the error.
Best
Leonidas

On 01/16/2015 08:03 AM, Marcos Freitas wrote:
> Hello,
>
> I'm trying to do a union, followed by an select, if I run only the 
> union it works, but when I add the line with the 'a2 = SELECT' I keep 
> getting an error.
> Here's the script I'm running:
>
> ds = 
> source(line,'/tmp/file1','|',type(<city:string,country:string,pop:string>));
> ds2 = source(line,'/tmp/file2 
> ','|',type(<city:string,country:string,pop:string>));
>
> a1 = (SELECT <city:d.city, country:d.country, pop:d.pop> FROM d in ds)
>         union
>         (SELECT <city:d2.city, country:d2.country, pop:d2.pop> FROM d2 
> in ds2);
>
> a2 = SELECT (a.city, a.country, a.pop) from a in a1;
>
> dump '/tmp/test_union' from a2;
>
>
> And here's the error I get:
>
> Apache MRQL version 0.9.2 (compiled distributed Hadoop MapReduce mode 
> with 2 reducers)
> Query type: !bag(( string, string, string ))
> *** Type error (line: 6, position: 75)
> *** Undefined variable: x_8
> Variable Types:
>  x_10: ( < city: string, country: string, pop: string >, < city: 
> string, country: string, pop: string > )
>  x_13: ( < city: string, country: string, pop: string >, < city: 
> string, country: string, pop: string > )
>  x_12: < city: string, country: string, pop: string >
>  x_11: < city: string, country: string, pop: string >
>  x_5: < city: string, country: string, pop: string >
>  x_4: < city: string, country: string, pop: string >
>  x_5: < city: string, country: string, pop: string >
>  x_4: < city: string, country: string, pop: string >
>  x_3: < city: string, country: string, pop: string >
>  x_5: < city: string, country: string, pop: string >
>  x_4: < city: string, country: string, pop: string >
> Run time: 0.049 secs
>
> Can someone help me with that, as I didn't find many resources about 
> Union on MRQL.
>
> Thanks,
> Marcos Freitas
>
>
> -- 
> Marcos Freitas
>
> Idiro Technologies
> Clarendon House,
> 34-37 Clarendon St,
> Dublin 2
> Ireland
>
> E: marcos.freitas@idiro.com <ma...@idiro.com>