PIG: python udf streaming error

["Amit Sharma (VAS)" <Am...@mtsindia.in>]

Hi,

while calling python udf from pig(local). i am getting following error (attempt logs) & in the console getting the error mentioned below. Not able to trace where i am doing wrong as same code was working earlier. Using RHEL(6.4) on 64 bit machine with 2.7.2 hadoop & 0.15 version pig & python 3.5
Traceback (most recent call last):
File "/tmp/controller2772959444531928936.py", line 356, in <module>
sys.argv[5], sys.argv[6], sys.argv[7], sys.argv[8])
File "/tmp/controller2772959444531928936.py", line 88, in main
input_str = self.get_next_input()
File "/tmp/controller2772959444531928936.py", line 164, in get_next_input
while input_str.endswith(END_RECORD_DELIM) == False:
TypeError: endswith first arg must be bytes or a tuple of bytes, not str
Following is the error at console:
java.lang.Exception: org.apache.pig.impl.streaming.StreamingUDFException: LINE : at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) Caused by: org.apache.pig.impl.streaming.StreamingUDFException: LINE : at org.apache.pig.impl.builtin.StreamingUDF$ProcessErrorThread.run(StreamingUDF.java:503)

Exception in thread "Thread-35" java.lang.NullPointerException at org.apache.pig.impl.builtin.StreamingUDF$ProcessOutputThread.run(StreamingUDF.java:468)


Following is the python code:
@outputSchema('output_field_name:chararray')
def readfileinlist(filename):
    with open(filename) as inputfile:
            lines = inputfile.read().splitlines()
    return lines

@outputSchema('output_field_name:boolean')
def intlgtinlist(srcgt,destgt,intgtllist):
    if srcgt.startswith(tuple(intgtllist)) or destgt.startswith(tuple(intgtllist)):
            return True
    else:
            return False

@outputSchema('output_field_name:boolean')
def checkintlgtincdrs(aparty,srcgt,destgt):
    intgtllist = []
    try:
            if( (len(srcgt) > 0 or len(destgt) > 0) and (srcgt or destgt) and aparty.isdigit()):
                    if os.path.isfile(INTERNATIONALGTPATH) and os.access(INTERNATIONALGTPATH, os.R_OK) and os.stat(INTERNATIONALGTPATH).st_size > 0:

                            #FUNCTION FOR READING THE FILE IN ARRAY/TUPLE
                            intgtllist = readfileinlist(INTERNATIONALGTPATH)

                            #CHECK FOR THE INPUT(ARG0) IN ARRAY/TUPLE
                            if intlgtinlist(srcgt,destgt,intgtllist):
                                    return True
                            else:
                                    return False
                    else:
                            return False
            else:
                    return False
    except OSError or IndexError:
            pass

    return True


Following is the pig script

 record = LOAD '/inreport/cdrs/ZTE_20160301*' USING PigStorage('|','-tagFile');

 REGISTER 'udf_smsiuc.py' using streaming_python as smsiucudfs;

 internationalcdrsfilter = FILTER record by smsiucudfs.checkintlgtincdrs($1,$26,$27);

\

Best regards

Amit Sharma


________________________________

This E-Mail may contain Confidential and/or legally privileged Information and is meant for the intended recipient(s) only. If you have received this e-mail in error and are not the intended recipient/s, Kindly notify the sender and then delete this e-mail immediately from your system. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail, its contents or its attachment/s other than by its intended recipient/s is strictly prohibited and may be unlawful.

Internet Communications cannot be guaranteed to be secure or error-free as information could be delayed, intercepted, corrupted, lost, or contain viruses. Sistema Shyam Teleservices Limited does not accept any liability for any errors, omissions, viruses or computer problems experienced by any recipient as a result of this e-mail.