You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Andre <an...@fucs.org> on 2016/08/09 12:25:43 UTC

ExtractTNEFAttachments: (was Re: ListenSMTP processor)

All,

PR817[1] introduces an winmail.dat extractor.

Following people's feedback, I created a separate processor to handle the
TNEF attachments.

This means the typical deployment will look like:

(ListenSMTP || GetPOP3) --> ExtractEmailAttachments --> RouteOnAttribute
[filename=winmail.dat] --> ExtractTNEFAttachments


Since I could not generate a TNEF (where are the winmail.dats when you need
them?!?!) I ended up using the TNEFs available on POI's upstream test
units. winmail.dat donations to improve the test unit coverage are
welcome...

Please test, once you confirm this is working I will be happy to create a
processor to extract and parse TNEF body and mapi Attributes as well.

Cheers

[1]https://github.com/apache/nifi/pull/817/commits

On Mon, Jul 25, 2016 at 1:57 AM, Toivo Adams <to...@gmail.com> wrote:

> I support Oleg opinion.
> Do one thing and do it well.
>
> Thanks
> Toivo
>
>
>
> --
> View this message in context: http://apache-nifi-developer-
> list.39713.n7.nabble.com/ListenSMTP-processor-tp10510p12891.html
> Sent from the Apache NiFi Developer List mailing list archive at
> Nabble.com.
>

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

Posted by Joe Witt <jo...@gmail.com>.
Definitely best course of action is to use our own originally created
test data.  This can at times be very difficult but perhaps what Bryan
just pointed out helps.

Alternatively, we can of course include test artifacts in our source
repository but we must simply account for them in license and notice
and they must of course be valid ASLv2 source dependencies.  Those are
things which come from the category-a list seen here:
http://www.apache.org/legal/resolved.html#category-a

In short, this is totally doable we just must be really good stewards
of the L&N process.

On Tue, Aug 9, 2016 at 9:39 AM, Bryan Rosander <br...@gmail.com> wrote:
> Hi Andre,
>
> I found a superuser answer that seems like it might be helpful in forcing
> Outlook to use TNEF.
>
> http://superuser.com/questions/613014/how-do-i-force-outlook-to-send-an-email-message-to-have-a-winmail-dat-attachment#answer-638244
>
> Hope that helps.
>
> Thanks,
> Bryan
>
> On Tue, Aug 9, 2016 at 9:31 AM, Andre <an...@fucs.org> wrote:
>
>> Hi Joe,
>>
>> I am aware of it, reason I called it out openly so if someone can try to
>> assist.
>>
>> In the past, I have used created content within the junit or crafted it
>> within my lab, but in the case of TNEF I could not find a way of creating
>> the files (POI does not have this ability and outlook seems convinced to
>> create proper HTML attachments).
>>
>> As consequence I reached to the files stored in here:
>>
>> https://svn.apache.org/repos/asf/poi/tags/REL_3_14_BETA1/test-data/hmef/
>>
>> I also checked their NOTICE or LICENSE files but no references, nor are the
>> the samples mentioned within their maven profiles. Not ideal but so far the
>> best I could find so far. Worse thing comes, we restrict the test units to
>> an invalid content, at least we will know it fails safely. :-)
>>
>> Hope this helps to clarify.
>>
>> On Tue, Aug 9, 2016 at 11:19 PM, Joe Witt <jo...@gmail.com> wrote:
>>
>> > Andre
>> >
>> > We cannot copy source material even for testing unless we fully and
>> > properly account for licensing and notice concerns.
>> >
>> > Thanks
>> > Joe
>> >
>> > On Aug 9, 2016 8:25 AM, "Andre" <an...@fucs.org> wrote:
>> >
>> > > All,
>> > >
>> > > PR817[1] introduces an winmail.dat extractor.
>> > >
>> > > Following people's feedback, I created a separate processor to handle
>> the
>> > > TNEF attachments.
>> > >
>> > > This means the typical deployment will look like:
>> > >
>> > > (ListenSMTP || GetPOP3) --> ExtractEmailAttachments -->
>> RouteOnAttribute
>> > > [filename=winmail.dat] --> ExtractTNEFAttachments
>> > >
>> > >
>> > > Since I could not generate a TNEF (where are the winmail.dats when you
>> > need
>> > > them?!?!) I ended up using the TNEFs available on POI's upstream test
>> > > units. winmail.dat donations to improve the test unit coverage are
>> > > welcome...
>> > >
>> > > Please test, once you confirm this is working I will be happy to
>> create a
>> > > processor to extract and parse TNEF body and mapi Attributes as well.
>> > >
>> > > Cheers
>> > >
>> > > [1]https://github.com/apache/nifi/pull/817/commits
>> > >
>> > > On Mon, Jul 25, 2016 at 1:57 AM, Toivo Adams <to...@gmail.com>
>> > > wrote:
>> > >
>> > > > I support Oleg opinion.
>> > > > Do one thing and do it well.
>> > > >
>> > > > Thanks
>> > > > Toivo
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > View this message in context: http://apache-nifi-developer-
>> > > > list.39713.n7.nabble.com/ListenSMTP-processor-tp10510p12891.html
>> > > > Sent from the Apache NiFi Developer List mailing list archive at
>> > > > Nabble.com.
>> > > >
>> > >
>> >
>>

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

Posted by Andre <an...@fucs.org>.
Bryan,

Thanks for the message.

I've seen that link previously. Challenge is Exchange admins have the
ability to force the conversion of email into standard mime and suspect my
email server does that.

In any case, all sorted, I used an unused outlook license I had at home
together with mailtrap.io and generated a number of TNEFs. Will be happy to
replace the samples now.


Having said that... general peer reviews are still welcome. ;-)



Cheers

On Tue, Aug 9, 2016 at 11:39 PM, Bryan Rosander <br...@gmail.com>
wrote:

> Hi Andre,
>
> I found a superuser answer that seems like it might be helpful in forcing
> Outlook to use TNEF.
>
> http://superuser.com/questions/613014/how-do-i-force-outlook-to-send-an-
> email-message-to-have-a-winmail-dat-attachment#answer-638244
>
> Hope that helps.
>
> Thanks,
> Bryan
>
> On Tue, Aug 9, 2016 at 9:31 AM, Andre <an...@fucs.org> wrote:
>
> > Hi Joe,
> >
> > I am aware of it, reason I called it out openly so if someone can try to
> > assist.
> >
> > In the past, I have used created content within the junit or crafted it
> > within my lab, but in the case of TNEF I could not find a way of creating
> > the files (POI does not have this ability and outlook seems convinced to
> > create proper HTML attachments).
> >
> > As consequence I reached to the files stored in here:
> >
> > https://svn.apache.org/repos/asf/poi/tags/REL_3_14_BETA1/test-data/hmef/
> >
> > I also checked their NOTICE or LICENSE files but no references, nor are
> the
> > the samples mentioned within their maven profiles. Not ideal but so far
> the
> > best I could find so far. Worse thing comes, we restrict the test units
> to
> > an invalid content, at least we will know it fails safely. :-)
> >
> > Hope this helps to clarify.
> >
> > On Tue, Aug 9, 2016 at 11:19 PM, Joe Witt <jo...@gmail.com> wrote:
> >
> > > Andre
> > >
> > > We cannot copy source material even for testing unless we fully and
> > > properly account for licensing and notice concerns.
> > >
> > > Thanks
> > > Joe
> > >
> > > On Aug 9, 2016 8:25 AM, "Andre" <an...@fucs.org> wrote:
> > >
> > > > All,
> > > >
> > > > PR817[1] introduces an winmail.dat extractor.
> > > >
> > > > Following people's feedback, I created a separate processor to handle
> > the
> > > > TNEF attachments.
> > > >
> > > > This means the typical deployment will look like:
> > > >
> > > > (ListenSMTP || GetPOP3) --> ExtractEmailAttachments -->
> > RouteOnAttribute
> > > > [filename=winmail.dat] --> ExtractTNEFAttachments
> > > >
> > > >
> > > > Since I could not generate a TNEF (where are the winmail.dats when
> you
> > > need
> > > > them?!?!) I ended up using the TNEFs available on POI's upstream test
> > > > units. winmail.dat donations to improve the test unit coverage are
> > > > welcome...
> > > >
> > > > Please test, once you confirm this is working I will be happy to
> > create a
> > > > processor to extract and parse TNEF body and mapi Attributes as well.
> > > >
> > > > Cheers
> > > >
> > > > [1]https://github.com/apache/nifi/pull/817/commits
> > > >
> > > > On Mon, Jul 25, 2016 at 1:57 AM, Toivo Adams <to...@gmail.com>
> > > > wrote:
> > > >
> > > > > I support Oleg opinion.
> > > > > Do one thing and do it well.
> > > > >
> > > > > Thanks
> > > > > Toivo
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > View this message in context: http://apache-nifi-developer-
> > > > > list.39713.n7.nabble.com/ListenSMTP-processor-tp10510p12891.html
> > > > > Sent from the Apache NiFi Developer List mailing list archive at
> > > > > Nabble.com.
> > > > >
> > > >
> > >
> >
>

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

Posted by Bryan Rosander <br...@gmail.com>.
Hi Andre,

I found a superuser answer that seems like it might be helpful in forcing
Outlook to use TNEF.

http://superuser.com/questions/613014/how-do-i-force-outlook-to-send-an-email-message-to-have-a-winmail-dat-attachment#answer-638244

Hope that helps.

Thanks,
Bryan

On Tue, Aug 9, 2016 at 9:31 AM, Andre <an...@fucs.org> wrote:

> Hi Joe,
>
> I am aware of it, reason I called it out openly so if someone can try to
> assist.
>
> In the past, I have used created content within the junit or crafted it
> within my lab, but in the case of TNEF I could not find a way of creating
> the files (POI does not have this ability and outlook seems convinced to
> create proper HTML attachments).
>
> As consequence I reached to the files stored in here:
>
> https://svn.apache.org/repos/asf/poi/tags/REL_3_14_BETA1/test-data/hmef/
>
> I also checked their NOTICE or LICENSE files but no references, nor are the
> the samples mentioned within their maven profiles. Not ideal but so far the
> best I could find so far. Worse thing comes, we restrict the test units to
> an invalid content, at least we will know it fails safely. :-)
>
> Hope this helps to clarify.
>
> On Tue, Aug 9, 2016 at 11:19 PM, Joe Witt <jo...@gmail.com> wrote:
>
> > Andre
> >
> > We cannot copy source material even for testing unless we fully and
> > properly account for licensing and notice concerns.
> >
> > Thanks
> > Joe
> >
> > On Aug 9, 2016 8:25 AM, "Andre" <an...@fucs.org> wrote:
> >
> > > All,
> > >
> > > PR817[1] introduces an winmail.dat extractor.
> > >
> > > Following people's feedback, I created a separate processor to handle
> the
> > > TNEF attachments.
> > >
> > > This means the typical deployment will look like:
> > >
> > > (ListenSMTP || GetPOP3) --> ExtractEmailAttachments -->
> RouteOnAttribute
> > > [filename=winmail.dat] --> ExtractTNEFAttachments
> > >
> > >
> > > Since I could not generate a TNEF (where are the winmail.dats when you
> > need
> > > them?!?!) I ended up using the TNEFs available on POI's upstream test
> > > units. winmail.dat donations to improve the test unit coverage are
> > > welcome...
> > >
> > > Please test, once you confirm this is working I will be happy to
> create a
> > > processor to extract and parse TNEF body and mapi Attributes as well.
> > >
> > > Cheers
> > >
> > > [1]https://github.com/apache/nifi/pull/817/commits
> > >
> > > On Mon, Jul 25, 2016 at 1:57 AM, Toivo Adams <to...@gmail.com>
> > > wrote:
> > >
> > > > I support Oleg opinion.
> > > > Do one thing and do it well.
> > > >
> > > > Thanks
> > > > Toivo
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context: http://apache-nifi-developer-
> > > > list.39713.n7.nabble.com/ListenSMTP-processor-tp10510p12891.html
> > > > Sent from the Apache NiFi Developer List mailing list archive at
> > > > Nabble.com.
> > > >
> > >
> >
>

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

Posted by Andre <an...@fucs.org>.
Hi Joe,

I am aware of it, reason I called it out openly so if someone can try to
assist.

In the past, I have used created content within the junit or crafted it
within my lab, but in the case of TNEF I could not find a way of creating
the files (POI does not have this ability and outlook seems convinced to
create proper HTML attachments).

As consequence I reached to the files stored in here:

https://svn.apache.org/repos/asf/poi/tags/REL_3_14_BETA1/test-data/hmef/

I also checked their NOTICE or LICENSE files but no references, nor are the
the samples mentioned within their maven profiles. Not ideal but so far the
best I could find so far. Worse thing comes, we restrict the test units to
an invalid content, at least we will know it fails safely. :-)

Hope this helps to clarify.

On Tue, Aug 9, 2016 at 11:19 PM, Joe Witt <jo...@gmail.com> wrote:

> Andre
>
> We cannot copy source material even for testing unless we fully and
> properly account for licensing and notice concerns.
>
> Thanks
> Joe
>
> On Aug 9, 2016 8:25 AM, "Andre" <an...@fucs.org> wrote:
>
> > All,
> >
> > PR817[1] introduces an winmail.dat extractor.
> >
> > Following people's feedback, I created a separate processor to handle the
> > TNEF attachments.
> >
> > This means the typical deployment will look like:
> >
> > (ListenSMTP || GetPOP3) --> ExtractEmailAttachments --> RouteOnAttribute
> > [filename=winmail.dat] --> ExtractTNEFAttachments
> >
> >
> > Since I could not generate a TNEF (where are the winmail.dats when you
> need
> > them?!?!) I ended up using the TNEFs available on POI's upstream test
> > units. winmail.dat donations to improve the test unit coverage are
> > welcome...
> >
> > Please test, once you confirm this is working I will be happy to create a
> > processor to extract and parse TNEF body and mapi Attributes as well.
> >
> > Cheers
> >
> > [1]https://github.com/apache/nifi/pull/817/commits
> >
> > On Mon, Jul 25, 2016 at 1:57 AM, Toivo Adams <to...@gmail.com>
> > wrote:
> >
> > > I support Oleg opinion.
> > > Do one thing and do it well.
> > >
> > > Thanks
> > > Toivo
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-nifi-developer-
> > > list.39713.n7.nabble.com/ListenSMTP-processor-tp10510p12891.html
> > > Sent from the Apache NiFi Developer List mailing list archive at
> > > Nabble.com.
> > >
> >
>

Re: ExtractTNEFAttachments: (was Re: ListenSMTP processor)

Posted by Joe Witt <jo...@gmail.com>.
Andre

We cannot copy source material even for testing unless we fully and
properly account for licensing and notice concerns.

Thanks
Joe

On Aug 9, 2016 8:25 AM, "Andre" <an...@fucs.org> wrote:

> All,
>
> PR817[1] introduces an winmail.dat extractor.
>
> Following people's feedback, I created a separate processor to handle the
> TNEF attachments.
>
> This means the typical deployment will look like:
>
> (ListenSMTP || GetPOP3) --> ExtractEmailAttachments --> RouteOnAttribute
> [filename=winmail.dat] --> ExtractTNEFAttachments
>
>
> Since I could not generate a TNEF (where are the winmail.dats when you need
> them?!?!) I ended up using the TNEFs available on POI's upstream test
> units. winmail.dat donations to improve the test unit coverage are
> welcome...
>
> Please test, once you confirm this is working I will be happy to create a
> processor to extract and parse TNEF body and mapi Attributes as well.
>
> Cheers
>
> [1]https://github.com/apache/nifi/pull/817/commits
>
> On Mon, Jul 25, 2016 at 1:57 AM, Toivo Adams <to...@gmail.com>
> wrote:
>
> > I support Oleg opinion.
> > Do one thing and do it well.
> >
> > Thanks
> > Toivo
> >
> >
> >
> > --
> > View this message in context: http://apache-nifi-developer-
> > list.39713.n7.nabble.com/ListenSMTP-processor-tp10510p12891.html
> > Sent from the Apache NiFi Developer List mailing list archive at
> > Nabble.com.
> >
>