You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Sushant Sawant <su...@karkinos.in> on 2022/07/07 07:16:32 UTC
Execute script code Jython2.7, code update not reflecting on server
Hi all,
Use case is to read email and extract body and other meta data and save it
in mongo.
msg.get_payload()
when I execute above line m getting, "2022=\n 15:23" in response. Quotable
printable.
msg.get_payload(decode=True)
when I execute above line m getting, "2022 15:23" in response. Quotable
printable is removed.This is as expected and works locally, when I pass eml
file. I am using the later one, "decode=True" on server but it is not
decoding quotable printable.
quopri.decodestring(body)
Then I tried above, but still it is not decoding as expected.Here is entire
the script m using in local, running on Jython 2.7.2
import email
import quopri
msg = email.message_from_file(open("some_eml.eml"))
body = ""
if msg.is_multipart():
for part in msg.walk():
ctype = part.get_content_type()
cdispo = str(part.get('Content-Disposition'))
if (ctype == 'text/plain' or ctype == 'text/html') and
'attachment' not in cdispo:
body = part.get_content() # decode
print(body)
break
else:
body_byte = msg.get_payload()
print(repr(body_byte))
body = body_byte.decode("utf-8", 'ignore')
print(repr(body))
utf = quopri.decodestring(body)
text = utf.decode('utf-8', errors='replace')
print(repr(text))
print(text)
One observation is, it is behaving as old script. Have restarted cluster.
Also did this, but didn't helped.nifi.flowcontroller.autoResumeState=false
nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
nifi.queue.swap.threshold=20000
nifi.swap.in.period=5 sec
nifi.swap.in.threads=1
nifi.swap.out.period=5 sec
nifi.swap.out.threads=4Any help appreciated.Also created question here,
https://community.cloudera.com/t5/Support-Questions/ExecuteScript-python-2-7-not-working-as-expected-on-server/td-p/346941
Re: Execute script code Jython2.7, code update not reflecting on server
Posted by Sushant Sawant <su...@karkinos.in>.
Yes very sure, have installed jython from jar and I can see log intellij
console(when I run script), it prints the entire jython executable path
following by my file path and then the actual output.
Thanks,
Sushant
On Sat, Jul 9, 2022, 7:33 PM Mike Thomsen <mi...@gmail.com> wrote:
> > However the same version of jython on windows
>
> Are you sure you're running **J**ython on Windows and not Python? The
> behavior you're describing sounds like the differences between Jython
> and CPython 3.
>
> On Fri, Jul 8, 2022 at 6:17 AM Sushant Sawant
> <su...@karkinos.in> wrote:
> >
> > Hi all,
> >
> > After further debugging more, here are the observation,
> >
> >
> >
> > No matter what encoding I do, reading string is giving a unicode string
> in jython nifi.
> > M unable to parse quotable printable characters since it requires a byte
> string as object.
> >
> >
> >
> > String is read from email body, is u'some email body' and not b'some
> email body'.
> >
> >
> > However the same version of jython on windows gives b'some email body'
> as a proper str object.
> >
> > Any help guys m stuck since 3 days in this.
> >
> >
> > On Thu, Jul 7, 2022 at 12:46 PM Sushant Sawant <
> sushant.sawant@karkinos.in> wrote:
> >>
> >> Hi all,
> >>
> >> Use case is to read email and extract body and other meta data and save
> it in mongo.
> >>
> >> msg.get_payload()
> >>
> >> when I execute above line m getting, "2022=\n 15:23" in response.
> Quotable printable.
> >>
> >> msg.get_payload(decode=True)
> >>
> >> when I execute above line m getting, "2022 15:23" in response. Quotable
> printable is removed.This is as expected and works locally, when I pass eml
> file. I am using the later one, "decode=True" on server but it is not
> decoding quotable printable.
> >>
> >> quopri.decodestring(body)
> >>
> >> Then I tried above, but still it is not decoding as expected.Here is
> entire the script m using in local, running on Jython 2.7.2
> >>
> >> import email
> >> import quopri
> >> msg = email.message_from_file(open("some_eml.eml"))
> >> body = ""
> >> if msg.is_multipart():
> >> for part in msg.walk():
> >> ctype = part.get_content_type()
> >> cdispo = str(part.get('Content-Disposition'))
> >> if (ctype == 'text/plain' or ctype == 'text/html') and
> 'attachment' not in cdispo:
> >> body = part.get_content() # decode
> >> print(body)
> >> break
> >> else:
> >> body_byte = msg.get_payload()
> >> print(repr(body_byte))
> >> body = body_byte.decode("utf-8", 'ignore')
> >> print(repr(body))
> >> utf = quopri.decodestring(body)
> >> text = utf.decode('utf-8', errors='replace')
> >> print(repr(text))
> >> print(text)
> >>
> >> One observation is, it is behaving as old script. Have restarted
> cluster.
> >> Also did this, but didn't
> helped.nifi.flowcontroller.autoResumeState=false
> >>
> nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
> >> nifi.queue.swap.threshold=20000
> >> nifi.swap.in.period=5 sec
> >> nifi.swap.in.threads=1
> >> nifi.swap.out.period=5 sec
> >> nifi.swap.out.threads=4Any help appreciated.Also created question here,
> https://community.cloudera.com/t5/Support-Questions/ExecuteScript-python-2-7-not-working-as-expected-on-server/td-p/346941
> >>
>
Re: Execute script code Jython2.7, code update not reflecting on server
Posted by Mike Thomsen <mi...@gmail.com>.
> However the same version of jython on windows
Are you sure you're running **J**ython on Windows and not Python? The
behavior you're describing sounds like the differences between Jython
and CPython 3.
On Fri, Jul 8, 2022 at 6:17 AM Sushant Sawant
<su...@karkinos.in> wrote:
>
> Hi all,
>
> After further debugging more, here are the observation,
>
>
>
> No matter what encoding I do, reading string is giving a unicode string in jython nifi.
> M unable to parse quotable printable characters since it requires a byte string as object.
>
>
>
> String is read from email body, is u'some email body' and not b'some email body'.
>
>
> However the same version of jython on windows gives b'some email body' as a proper str object.
>
> Any help guys m stuck since 3 days in this.
>
>
> On Thu, Jul 7, 2022 at 12:46 PM Sushant Sawant <su...@karkinos.in> wrote:
>>
>> Hi all,
>>
>> Use case is to read email and extract body and other meta data and save it in mongo.
>>
>> msg.get_payload()
>>
>> when I execute above line m getting, "2022=\n 15:23" in response. Quotable printable.
>>
>> msg.get_payload(decode=True)
>>
>> when I execute above line m getting, "2022 15:23" in response. Quotable printable is removed.This is as expected and works locally, when I pass eml file. I am using the later one, "decode=True" on server but it is not decoding quotable printable.
>>
>> quopri.decodestring(body)
>>
>> Then I tried above, but still it is not decoding as expected.Here is entire the script m using in local, running on Jython 2.7.2
>>
>> import email
>> import quopri
>> msg = email.message_from_file(open("some_eml.eml"))
>> body = ""
>> if msg.is_multipart():
>> for part in msg.walk():
>> ctype = part.get_content_type()
>> cdispo = str(part.get('Content-Disposition'))
>> if (ctype == 'text/plain' or ctype == 'text/html') and 'attachment' not in cdispo:
>> body = part.get_content() # decode
>> print(body)
>> break
>> else:
>> body_byte = msg.get_payload()
>> print(repr(body_byte))
>> body = body_byte.decode("utf-8", 'ignore')
>> print(repr(body))
>> utf = quopri.decodestring(body)
>> text = utf.decode('utf-8', errors='replace')
>> print(repr(text))
>> print(text)
>>
>> One observation is, it is behaving as old script. Have restarted cluster.
>> Also did this, but didn't helped.nifi.flowcontroller.autoResumeState=false
>> nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
>> nifi.queue.swap.threshold=20000
>> nifi.swap.in.period=5 sec
>> nifi.swap.in.threads=1
>> nifi.swap.out.period=5 sec
>> nifi.swap.out.threads=4Any help appreciated.Also created question here,https://community.cloudera.com/t5/Support-Questions/ExecuteScript-python-2-7-not-working-as-expected-on-server/td-p/346941
>>
Re: Execute script code Jython2.7, code update not reflecting on server
Posted by Sushant Sawant <su...@karkinos.in>.
Hi all,
After further debugging more, here are the observation,
No matter what encoding I do, reading string is giving a unicode string in
jython nifi.
M unable to parse quotable printable characters since it requires a byte
string as object.
String is read from email body, is u'some email body' and not b'some email
body'.
However the same version of jython on windows gives b'some email body' as a
proper str object.
Any help guys m stuck since 3 days in this.
On Thu, Jul 7, 2022 at 12:46 PM Sushant Sawant <su...@karkinos.in>
wrote:
> Hi all,
>
> Use case is to read email and extract body and other meta data and save it
> in mongo.
>
> msg.get_payload()
>
> when I execute above line m getting, "2022=\n 15:23" in response. Quotable
> printable.
>
> msg.get_payload(decode=True)
>
> when I execute above line m getting, "2022 15:23" in response. Quotable
> printable is removed.This is as expected and works locally, when I pass
> eml file. I am using the later one, "decode=True" on server but it is not
> decoding quotable printable.
>
> quopri.decodestring(body)
>
> Then I tried above, but still it is not decoding as expected.Here is
> entire the script m using in local, running on Jython 2.7.2
>
> import email
> import quopri
> msg = email.message_from_file(open("some_eml.eml"))
> body = ""
> if msg.is_multipart():
> for part in msg.walk():
> ctype = part.get_content_type()
> cdispo = str(part.get('Content-Disposition'))
> if (ctype == 'text/plain' or ctype == 'text/html') and 'attachment' not in cdispo:
> body = part.get_content() # decode
> print(body)
> break
> else:
> body_byte = msg.get_payload()
> print(repr(body_byte))
> body = body_byte.decode("utf-8", 'ignore')
> print(repr(body))
> utf = quopri.decodestring(body)
> text = utf.decode('utf-8', errors='replace')
> print(repr(text))
> print(text)
>
> One observation is, it is behaving as old script. Have restarted cluster.
> Also did this, but didn't helped.nifi.flowcontroller.autoResumeState=false
>
> nifi.swap.manager.implementation=org.apache.nifi.controller.FileSystemSwapManager
> nifi.queue.swap.threshold=20000
> nifi.swap.in.period=5 sec
> nifi.swap.in.threads=1
> nifi.swap.out.period=5 sec
> nifi.swap.out.threads=4Any help appreciated.Also created question here,
> https://community.cloudera.com/t5/Support-Questions/ExecuteScript-python-2-7-not-working-as-expected-on-server/td-p/346941
>
>