Streaming_python, any help is appreciated.

[Edmund Day <ed...@yahoo.com>]

Hi,

When i run the following:

REGISTER /home/hduser/Documents/ccc/Research/phd/code/ECentre/scripts/bags.py USING streaming_python  AS bp;

raw = LOAD 'hdfs:///user/hduser/smsCorpus_en_2012.04.30_all.xml'   AS (line:chararray);


b = foreach raw generate bp.enumerate_bag(line);


I get 

Failed to parse: Pig script failed to parse: 
<file /home/hduser/Documents/ccc/Research/phd/code/ECentre/scripts/nltk.pig, line 13, column 25> Failed to generate logical plan. Nested exception: org.apache.pig.backend.executionengine.ExecException: ERROR 1070: Could not resolve bp.enumerate_bag using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

bags.py:

#!/usr/bin/env python 
def enumerate_bag(input):
    output = []
    for rank, item in enumerate(input):
        output.append(tuple([rank] + list(item)))
    return output


Can anyone tell me why? TIA!!!!


My version is:
Apache Pig version 0.12.2-SNAPSHOT (r: unknown) 
compiled Apr 29 2014, 13:40:45

Re: Streaming_python, any help is appreciated.

["edmundday@yahoo.com" <ed...@yahoo.com>]

The file just contains the pig lines in my original message. The 1500 error seems to be because pig is not registering my python file. I've got it to work using jython instead now, thanks anyway.

Sent from Yahoo Mail on Android

Re: Streaming_python, any help is appreciated.

[Mark Salter <ma...@talktalk.net>]

On 29/04/2014 14:47, Edmund Day wrote:
> <file /home/hduser/Documents/ccc/Research/phd/code/ECentre/scripts/nltk.pig, line 13, column 25>

What is in *this* file?

> ERROR 1070

Have you looked up this error value?

-- 
Mark