You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Divya Gehlot <di...@gmail.com> on 2015/07/10 11:38:37 UTC
Java UDF Error: ERROR 1066: Unable to open iterator for alias
Hi
My input data format is
(message,NIL,2015-07-01,22:58:53.66,E,machine.com.name,12,0xd6,String,String
,0,0.0,key=value&key=123456789&key=value&key=US&key=COMPANY&key=MESSAGE&key=123456789&key=String&key=String&Key=String&Key=String)
I have written Java UDF as below to parse last string of input data
package com.pig.udf;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;
import org.apache.pig.EvalFunc;
import org.apache.pig.data.Tuple;
public class PigUDF extends EvalFunc<Map> {
@Override
public Map<String, String> exec(Tuple input) throws IOException {
// If tuple is null, has fewer than 3 values, or has an even number of
// values
if (input == null || input.size() < 3 || (input.size() % 2 == 0)) {
throw new IOException("Incorrect number of values.");
}
String source = (String) input.get(0);
System.out.println("input Source"+source);
String delim = (input.size() > 1) ? (String) input.get(1) : "&";
int length = (input.size() > 2) ? (Integer) input.get(2) : 0;
if (source == null || delim == null) {
return null;
}
String[] splits = source.split(delim, length);
System.out.println("Splits"+ splits);
ArrayList<String> arrayList = new ArrayList<String>(
Arrays.asList(splits));
Map<String, String> map = new HashMap<String, String>();
for (String keyValue : arrayList) {
int end = keyValue.indexOf('=');
if (end != -1) {
map.put(keyValue.substring(0, end), keyValue.substring(end + 1));
}
}
System.out.println("map"+map);
return map;
}
}
When I am running the my Pig Script with above Java UDF I am getting below
error
Pig Stack Trace
---------------
ERROR 1066: Unable to open iterator for alias C
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable
to open iterator for alias C
at org.apache.pig.PigServer.openIterator(PigServer.java:892)
at
org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:198)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:173)
at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
at org.apache.pig.Main.run(Main.java:607)
at org.apache.pig.Main.main(Main.java:156)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: Job terminated with anomalous status
FAILED
at org.apache.pig.PigServer.openIterator(PigServer.java:884)
... 13 more
Application Log
-------------------------------------------------------------------
Application application_1436453941326_0020 failed 2 times due to AM
Container for appattempt_1436453941326_0020_000002 exited with exitCode: 1
For more detailed output, check application tracking page:
http://quickstart.cloudera:8088/proxy/application_1436453941326_0020/Then,
click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1436453941326_0020_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at
org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
My Script is running fine without Java UDF function and giving me outfile
too.
The issue arises when I include Java UDF in my Pig Script.
There is no java version mismatch between my Java UDF and machine running
Pig
Any pointers will be appreciated
Re: Java UDF Error: ERROR 1066: Unable to open iterator for alias
Posted by "Jianfeng (Jeff) Zhang" <jz...@hortonworks.com>.
> Application application_1436453941326_0020 failed 2 times due to AM
> Container for appattempt_1436453941326_0020_000002 exited with exitCode:
>1
Could you check the yarn app log ?
Best Regard,
Jeff Zhang
On 7/10/15, 5:38 PM, "Divya Gehlot" <di...@gmail.com> wrote:
>Hi
>
>My input data format is
>(message,NIL,2015-07-01,22:58:53.66,E,machine.com.name,12,0xd6,String,Stri
>ng
>,0,0.0,key=value&key=123456789&key=value&key=US&key=COMPANY&key=MESSAGE&ke
>y=123456789&key=String&key=String&Key=String&Key=String)
>I have written Java UDF as below to parse last string of input data
>
>
> package com.pig.udf;
>
> import java.io.IOException;
> import java.util.ArrayList;
> import java.util.Arrays;
> import java.util.HashMap;
> import java.util.Map;
>
> import org.apache.pig.EvalFunc;
> import org.apache.pig.data.Tuple;
>
> public class PigUDF extends EvalFunc<Map> {
>
>
> @Override
> public Map<String, String> exec(Tuple input) throws IOException {
> // If tuple is null, has fewer than 3 values, or has an even number of
> // values
> if (input == null || input.size() < 3 || (input.size() % 2 == 0)) {
> throw new IOException("Incorrect number of values.");
> }
>
> String source = (String) input.get(0);
> System.out.println("input Source"+source);
> String delim = (input.size() > 1) ? (String) input.get(1) : "&";
> int length = (input.size() > 2) ? (Integer) input.get(2) : 0;
> if (source == null || delim == null) {
> return null;
> }
>
> String[] splits = source.split(delim, length);
> System.out.println("Splits"+ splits);
> ArrayList<String> arrayList = new ArrayList<String>(
> Arrays.asList(splits));
> Map<String, String> map = new HashMap<String, String>();
> for (String keyValue : arrayList) {
> int end = keyValue.indexOf('=');
> if (end != -1) {
> map.put(keyValue.substring(0, end), keyValue.substring(end + 1));
> }
>
> }
> System.out.println("map"+map);
>
> return map;
>
> }
>
> }
>
>
>When I am running the my Pig Script with above Java UDF I am getting below
>error
>
> Pig Stack Trace
> ---------------
> ERROR 1066: Unable to open iterator for alias C
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable
>to open iterator for alias C
> at org.apache.pig.PigServer.openIterator(PigServer.java:892)
> at
>org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774)
> at
>org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParse
>r.java:372)
> at
>org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
>98)
> at
>org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:1
>73)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:607)
> at org.apache.pig.Main.main(Main.java:156)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
>57)
> at
>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
>pl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException: Job terminated with anomalous status
>FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:884)
> ... 13 more
>
>
>
> Application Log
> -------------------------------------------------------------------
> Application application_1436453941326_0020 failed 2 times due to AM
>Container for appattempt_1436453941326_0020_000002 exited with exitCode: 1
> For more detailed output, check application tracking page:
>http://quickstart.cloudera:8088/proxy/application_1436453941326_0020/Then,
>click on links to logs of each attempt.
> Diagnostics: Exception from container-launch.
> Container id: container_1436453941326_0020_02_000001
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
> at org.apache.hadoop.util.Shell.run(Shell.java:455)
> at
>org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
> at
>org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchC
>ontainer(DefaultContainerExecutor.java:211)
> at
>org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai
>nerLaunch.call(ContainerLaunch.java:302)
> at
>org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.Contai
>nerLaunch.call(ContainerLaunch.java:82)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1145)
> at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:615)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1
> Failing this attempt. Failing the application.
>
>
>My Script is running fine without Java UDF function and giving me outfile
>too.
>The issue arises when I include Java UDF in my Pig Script.
>There is no java version mismatch between my Java UDF and machine running
>Pig
>Any pointers will be appreciated