You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Ruslan Salyakhov (JIRA)" <ji...@apache.org> on 2010/03/09 13:40:27 UTC
[jira] Created: (HIVE-1217) The MAPJOIN hint rises an error with
ORDER BY and LIMIT clauses
The MAPJOIN hint rises an error with ORDER BY and LIMIT clauses
---------------------------------------------------------------
Key: HIVE-1217
URL: https://issues.apache.org/jira/browse/HIVE-1217
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.5.0, 0.4.1
Reporter: Ruslan Salyakhov
Priority: Minor
the simple test-case:
hive> CREATE TABLE t1 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
hive> CREATE TABLE t2 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
hive> CREATE TABLE t3 (id INT, t1_id INT, t2_id INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
$ for x in `seq 0 9`; do echo -e "$x\tt1_$x"; done > t1_data
$ for x in `seq 0 9`; do echo -e "$x\tt2_$x"; done > t2_data
$ for x in `seq 0 99`; do echo -e "$x\t$(($RANDOM%10))\t$(($RANDOM%10))"; done > t3_data
$ hadoop fs -mkdir /test-hive/data/
$ hadoop fs -copyFromLocal t?_data /test-hive/data/
hive> load data inpath '/test-hive/data/t1_data' into table t1;
hive> load data inpath '/test-hive/data/t2_data' into table t2;
hive> load data inpath '/test-hive/data/t3_data' into table t3;
hive> select t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
works fine
hive> select /*+ MAPJOIN(t2,t3) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) limit 10;
works fine
hive> select /*+ MAPJOIN(t2,t3) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
taskdetails, Errors:
{code}
java.lang.RuntimeException
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:218)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:347)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)
... 4 more
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1217) The MAPJOIN hint rises an error with
ORDER BY and LIMIT clauses
Posted by "Ruslan Salyakhov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ruslan Salyakhov updated HIVE-1217:
-----------------------------------
Description:
the simple test-case:
hive> CREATE TABLE t1 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
hive> CREATE TABLE t2 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
hive> CREATE TABLE t3 (id INT, t1_id INT, t2_id INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
$ for x in `seq 0 9`; do echo -e "$x\tt1_$x"; done > t1_data
$ for x in `seq 0 9`; do echo -e "$x\tt2_$x"; done > t2_data
$ for x in `seq 0 99`; do echo -e "$x\t$(($RANDOM%10))\t$(($RANDOM%10))"; done > t3_data
$ hadoop fs -mkdir /test-hive/data/
$ hadoop fs -copyFromLocal t?_data /test-hive/data/
hive> load data inpath '/test-hive/data/t1_data' into table t1;
hive> load data inpath '/test-hive/data/t2_data' into table t2;
hive> load data inpath '/test-hive/data/t3_data' into table t3;
hive> select t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
works fine
hive> select /*+ MAPJOIN(t1,t2) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) limit 10;
works fine
hive> select /*+ MAPJOIN(t1,t2) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
taskdetails, Errors:
{code}
java.lang.RuntimeException
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:218)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:347)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)
... 4 more
{code}
was:
the simple test-case:
hive> CREATE TABLE t1 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
hive> CREATE TABLE t2 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
hive> CREATE TABLE t3 (id INT, t1_id INT, t2_id INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
$ for x in `seq 0 9`; do echo -e "$x\tt1_$x"; done > t1_data
$ for x in `seq 0 9`; do echo -e "$x\tt2_$x"; done > t2_data
$ for x in `seq 0 99`; do echo -e "$x\t$(($RANDOM%10))\t$(($RANDOM%10))"; done > t3_data
$ hadoop fs -mkdir /test-hive/data/
$ hadoop fs -copyFromLocal t?_data /test-hive/data/
hive> load data inpath '/test-hive/data/t1_data' into table t1;
hive> load data inpath '/test-hive/data/t2_data' into table t2;
hive> load data inpath '/test-hive/data/t3_data' into table t3;
hive> select t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
works fine
hive> select /*+ MAPJOIN(t2,t3) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) limit 10;
works fine
hive> select /*+ MAPJOIN(t2,t3) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
taskdetails, Errors:
{code}
java.lang.RuntimeException
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:218)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:347)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)
... 4 more
{code}
Release Note:
Description was updated:
use MAPJOIN(t1,t2) instead of MAPJOIN(t2,t3)
but the error still remains
> The MAPJOIN hint rises an error with ORDER BY and LIMIT clauses
> ---------------------------------------------------------------
>
> Key: HIVE-1217
> URL: https://issues.apache.org/jira/browse/HIVE-1217
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Affects Versions: 0.4.1, 0.5.0
> Reporter: Ruslan Salyakhov
> Priority: Minor
>
> the simple test-case:
> hive> CREATE TABLE t1 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
> hive> CREATE TABLE t2 (id INT, name STRING) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
> hive> CREATE TABLE t3 (id INT, t1_id INT, t2_id INT) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE;
> $ for x in `seq 0 9`; do echo -e "$x\tt1_$x"; done > t1_data
> $ for x in `seq 0 9`; do echo -e "$x\tt2_$x"; done > t2_data
> $ for x in `seq 0 99`; do echo -e "$x\t$(($RANDOM%10))\t$(($RANDOM%10))"; done > t3_data
> $ hadoop fs -mkdir /test-hive/data/
> $ hadoop fs -copyFromLocal t?_data /test-hive/data/
> hive> load data inpath '/test-hive/data/t1_data' into table t1;
> hive> load data inpath '/test-hive/data/t2_data' into table t2;
> hive> load data inpath '/test-hive/data/t3_data' into table t3;
> hive> select t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
> works fine
> hive> select /*+ MAPJOIN(t1,t2) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) limit 10;
> works fine
> hive> select /*+ MAPJOIN(t1,t2) */ t3.id, t1.name, t2.name from t3 join t1 on (t3.t1_id = t1.id) join t2 on (t3.t2_id = t2.id) order by t3.id limit 10;
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.ExecDriver
> taskdetails, Errors:
> {code}
> java.lang.RuntimeException
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:188)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:218)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
> at org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:81)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:386)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:598)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:347)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:171)
> ... 4 more
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.