issues with Apache Pig: embedded

[Sameer Tilak <ss...@live.com>]

Hi everyone,
I want to read each line in file test.txt -- it will be something like "'Markov    chain Monte Carlo (MCMC) methods are an important algorithmic'"  -- just individual sentences and then I will do some text processing on each sentence in my udf. I get back a score for each of these sentences.  I am including only the relevant snippet of my python script.  Any help with this will be great!


// Script snippet:params ={'infile':  '/data/data.in', 'outfile': '/results/scores/', 'sentence': '0' };
f=open('./test.txt')
for currentline in f:    print 'Current line is' + currentline;     params["sentence"] = currentline;    params["outfile"] = '/results/scores/' + 'algo.out' + str(i);   bound = P.bind(params);      i = i + 1;    if result.isSuccessful() :        print 'Pig job succeeded'    else :        raise 'Pig job failed'
Does not work when the myudf.myfunc is parameterized -- $sentence.P = Pig.compile("A = LOAD '/data/data.in' AS (line: chararray); SCORES = FOREACH A GENERATE myudf.myfunc($sentence, line); STORE SCORES into '$outfile';")\
I get the following error:Current line is Markov    chain Monte Carlo (MCMC) methods are an   important algorithmic
2014-06-19 17:40:50,371 [main] INFO  org.apache.pig.scripting.BoundScript - Query to run:A = LOAD '/data/data.in' AS (line: chararray); SCORES = FOREACH A GENERATE  myudf.myfunc(Markov, line); STORE SCORES into '/results/scores/algo.out0';
2014-06-19 17:40:51,100 [main] ERROR org.apache.pig.Main - ERROR 1025:<line 1, column 110> Invalid field projection. Projected field [Markov] does not exist in schema: line:chararray.Details at logfile: /apps/software/pig-scripts/pig_1403224834638.log

Works fine when myudf.myfunc 1st parameter is hardcoded:
P = Pig.compile("A = LOAD '/data/data.in' AS (line: chararray); SCORES = FOREACH A GENERATE myudf.myfunc('Markov    chain Monte Carlo (MCMC) methods are an important algorithmic', line); STORE SCORES into '$outfile';");