You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@isis.apache.org by Dileepa Jayakody <di...@gmail.com> on 2014/03/14 10:49:39 UTC

[GSOC] Building a "real-life" app in a suitable domain using Isis framework

Hi All,

I'm Dileepa Jayakody a MSc research student from University of Moratuwa,
Sri Lanka. I'm doing my research project on the topic : reputation
assessment in emails.
My project goal is to introduce reputation data as a attribute to emails,
which could be used for various applications such as spam-filtering,
priority inboxes, social-networking, etc.

I'm planning to adopt a prototype model to develop my application. And I
find Apache Isis a great framework to implement such applications mainly
focusing on my domain model. I'm interested in the GSoC idea : build a
"real-life" app in some suitable domain, along with a semi-academic
write-up of their learning [1] and wish to seek your opinion on whether
implementing my project using Apache Isis can be considered a GSoC project.
I'm willing to write a paper on the project implementation, highlighting
the features, usage details of the framework.

As suggested by Dan I took a look at the thesis on  "Naked Objects",
chapter 7 on the implementation comparison of CarServ (conventional vs Isis
usage). In this GSoC project idea, do you  think the student must do 2
developments; one in conventional way and another using Isis? In that case
it might be difficult for a research project such as mine.

WDYT?

Thanks,
Dileepa

[1] https://issues.apache.org/jira/browse/ISIS-736

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Hi Dan and all,

I have completed my proposal and have submitted it to google-melange system
:
http://www.google-melange.com/gsoc/proposal/review/student/google/gsoc2014/dileepaj/5662278724616192

I appreciate any comments to improve my proposal and hope to get selected
for Apache Isis this time :)

Thanks,
Dileepa


On Fri, Mar 21, 2014 at 2:21 PM, Dileepa Jayakody <dileepajayakody@gmail.com
> wrote:

> I have submitted a draft proposal to the google-melange system.
> I still have to complete the implementation and timeline parts of it.
> Will ping the dev list once I have completed them so you can give more
> feedback.
>
> Thanks,
> Dileepa
>
>
> On Fri, Mar 21, 2014 at 1:12 PM, Dileepa Jayakody <
> dileepajayakody@gmail.com> wrote:
>
>> Yes Dan,
>>
>> I'm in the process of writing it. Will submit it asap before the deadline.
>>
>> Thanks,
>> Dileepa
>>
>>
>> On Fri, Mar 21, 2014 at 1:00 PM, Dan Haywood <
>> dan@haywood-associates.co.uk> wrote:
>>
>>> Dileepa,
>>> just to remind you that the deadline for proposals is imminent, so submit
>>> your proposal on google melange very  soon.  Google won't allow any late
>>> submissions.
>>> Thx
>>> Dan
>>>
>>>
>>>
>>> On 20 March 2014 16:48, Dan Haywood <da...@haywood-associates.co.uk>
>>> wrote:
>>>
>>> >
>>> > On 20 March 2014 10:15, Dileepa Jayakody <dileepajayakody@gmail.com
>>> >wrote:
>>> >
>>> >>
>>> >> In the ReputationBox, another main component is the
>>> reputation-analysis
>>> >> component.
>>> >> [snip]
>>> >>
>>> >> The best opensource option to implement a recommendation engine I have
>>> >> seen
>>> >> so far is Apache Mahout.
>>> >>
>>> >
>>> > Had a quick look over the Mahout website [1] and read a blog or two
>>> [2];
>>> > does look like a good fit.
>>> >
>>> >
>>> >
>>> >>
>>> >> Any suggestions, pointers on how to integrate a Mahout
>>> machine-learning
>>> >> process to do the email analysis within the application will be
>>> greatly
>>> >> appreciated.
>>> >>
>>> >>
>>> > Seems that there are two phases to this... the initial bulk extract and
>>> > learning exercise, and then on-the-fly scoring.
>>> >
>>> > I suppose that the initial bulk extract and learning could be done as a
>>> > background task in Isis... but I'd probably spike it outside of Isis
>>> in the
>>> > first place.
>>> >
>>> > The on-the-fly scoring looks easy enough; a synchronous call from the
>>> RB
>>> > gmail client to the RB server hosted by Isis and its REST API.
>>> >
>>> > I think overall this might be more an a Mahout- than an Isis project;
>>> but
>>> > it'd be nice to combine all of these (plus Shiro for the auth stuff, of
>>> > course).
>>> >
>>> > Other opinions on this topic welcome...
>>> >
>>> > HTH
>>> > Dan
>>> >
>>> >
>>> >
>>> >> Thanks,
>>> >> Dileepa
>>> >>
>>> >>
>>> >>
>>> > [1] https://mahout.apache.org/
>>> > [2]
>>> >
>>> http://emmaespina.wordpress.com/2011/04/26/ham-spam-and-elephants-or-how-to-build-a-spam-filter-server-with-mahout/
>>> >
>>> >
>>> >
>>> >
>>>
>>
>>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

I have submitted a draft proposal to the google-melange system.
I still have to complete the implementation and timeline parts of it.
Will ping the dev list once I have completed them so you can give more
feedback.

Thanks,
Dileepa


On Fri, Mar 21, 2014 at 1:12 PM, Dileepa Jayakody <dileepajayakody@gmail.com
> wrote:

> Yes Dan,
>
> I'm in the process of writing it. Will submit it asap before the deadline.
>
> Thanks,
> Dileepa
>
>
> On Fri, Mar 21, 2014 at 1:00 PM, Dan Haywood <dan@haywood-associates.co.uk
> > wrote:
>
>> Dileepa,
>> just to remind you that the deadline for proposals is imminent, so submit
>> your proposal on google melange very  soon.  Google won't allow any late
>> submissions.
>> Thx
>> Dan
>>
>>
>>
>> On 20 March 2014 16:48, Dan Haywood <da...@haywood-associates.co.uk> wrote:
>>
>> >
>> > On 20 March 2014 10:15, Dileepa Jayakody <dileepajayakody@gmail.com
>> >wrote:
>> >
>> >>
>> >> In the ReputationBox, another main component is the reputation-analysis
>> >> component.
>> >> [snip]
>> >>
>> >> The best opensource option to implement a recommendation engine I have
>> >> seen
>> >> so far is Apache Mahout.
>> >>
>> >
>> > Had a quick look over the Mahout website [1] and read a blog or two [2];
>> > does look like a good fit.
>> >
>> >
>> >
>> >>
>> >> Any suggestions, pointers on how to integrate a Mahout machine-learning
>> >> process to do the email analysis within the application will be greatly
>> >> appreciated.
>> >>
>> >>
>> > Seems that there are two phases to this... the initial bulk extract and
>> > learning exercise, and then on-the-fly scoring.
>> >
>> > I suppose that the initial bulk extract and learning could be done as a
>> > background task in Isis... but I'd probably spike it outside of Isis in
>> the
>> > first place.
>> >
>> > The on-the-fly scoring looks easy enough; a synchronous call from the RB
>> > gmail client to the RB server hosted by Isis and its REST API.
>> >
>> > I think overall this might be more an a Mahout- than an Isis project;
>> but
>> > it'd be nice to combine all of these (plus Shiro for the auth stuff, of
>> > course).
>> >
>> > Other opinions on this topic welcome...
>> >
>> > HTH
>> > Dan
>> >
>> >
>> >
>> >> Thanks,
>> >> Dileepa
>> >>
>> >>
>> >>
>> > [1] https://mahout.apache.org/
>> > [2]
>> >
>> http://emmaespina.wordpress.com/2011/04/26/ham-spam-and-elephants-or-how-to-build-a-spam-filter-server-with-mahout/
>> >
>> >
>> >
>> >
>>
>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Yes Dan,

I'm in the process of writing it. Will submit it asap before the deadline.

Thanks,
Dileepa


On Fri, Mar 21, 2014 at 1:00 PM, Dan Haywood
<da...@haywood-associates.co.uk>wrote:

> Dileepa,
> just to remind you that the deadline for proposals is imminent, so submit
> your proposal on google melange very  soon.  Google won't allow any late
> submissions.
> Thx
> Dan
>
>
>
> On 20 March 2014 16:48, Dan Haywood <da...@haywood-associates.co.uk> wrote:
>
> >
> > On 20 March 2014 10:15, Dileepa Jayakody <dileepajayakody@gmail.com
> >wrote:
> >
> >>
> >> In the ReputationBox, another main component is the reputation-analysis
> >> component.
> >> [snip]
> >>
> >> The best opensource option to implement a recommendation engine I have
> >> seen
> >> so far is Apache Mahout.
> >>
> >
> > Had a quick look over the Mahout website [1] and read a blog or two [2];
> > does look like a good fit.
> >
> >
> >
> >>
> >> Any suggestions, pointers on how to integrate a Mahout machine-learning
> >> process to do the email analysis within the application will be greatly
> >> appreciated.
> >>
> >>
> > Seems that there are two phases to this... the initial bulk extract and
> > learning exercise, and then on-the-fly scoring.
> >
> > I suppose that the initial bulk extract and learning could be done as a
> > background task in Isis... but I'd probably spike it outside of Isis in
> the
> > first place.
> >
> > The on-the-fly scoring looks easy enough; a synchronous call from the RB
> > gmail client to the RB server hosted by Isis and its REST API.
> >
> > I think overall this might be more an a Mahout- than an Isis project; but
> > it'd be nice to combine all of these (plus Shiro for the auth stuff, of
> > course).
> >
> > Other opinions on this topic welcome...
> >
> > HTH
> > Dan
> >
> >
> >
> >> Thanks,
> >> Dileepa
> >>
> >>
> >>
> > [1] https://mahout.apache.org/
> > [2]
> >
> http://emmaespina.wordpress.com/2011/04/26/ham-spam-and-elephants-or-how-to-build-a-spam-filter-server-with-mahout/
> >
> >
> >
> >
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

Dileepa,
just to remind you that the deadline for proposals is imminent, so submit
your proposal on google melange very  soon.  Google won't allow any late
submissions.
Thx
Dan



On 20 March 2014 16:48, Dan Haywood <da...@haywood-associates.co.uk> wrote:

>
> On 20 March 2014 10:15, Dileepa Jayakody <di...@gmail.com>wrote:
>
>>
>> In the ReputationBox, another main component is the reputation-analysis
>> component.
>> [snip]
>>
>> The best opensource option to implement a recommendation engine I have
>> seen
>> so far is Apache Mahout.
>>
>
> Had a quick look over the Mahout website [1] and read a blog or two [2];
> does look like a good fit.
>
>
>
>>
>> Any suggestions, pointers on how to integrate a Mahout machine-learning
>> process to do the email analysis within the application will be greatly
>> appreciated.
>>
>>
> Seems that there are two phases to this... the initial bulk extract and
> learning exercise, and then on-the-fly scoring.
>
> I suppose that the initial bulk extract and learning could be done as a
> background task in Isis... but I'd probably spike it outside of Isis in the
> first place.
>
> The on-the-fly scoring looks easy enough; a synchronous call from the RB
> gmail client to the RB server hosted by Isis and its REST API.
>
> I think overall this might be more an a Mahout- than an Isis project; but
> it'd be nice to combine all of these (plus Shiro for the auth stuff, of
> course).
>
> Other opinions on this topic welcome...
>
> HTH
> Dan
>
>
>
>> Thanks,
>> Dileepa
>>
>>
>>
> [1] https://mahout.apache.org/
> [2]
> http://emmaespina.wordpress.com/2011/04/26/ham-spam-and-elephants-or-how-to-build-a-spam-filter-server-with-mahout/
>
>
>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

On 20 March 2014 10:15, Dileepa Jayakody <di...@gmail.com> wrote:

>
> In the ReputationBox, another main component is the reputation-analysis
> component.
> [snip]
> The best opensource option to implement a recommendation engine I have seen
> so far is Apache Mahout.
>

Had a quick look over the Mahout website [1] and read a blog or two [2];
does look like a good fit.

>
> Any suggestions, pointers on how to integrate a Mahout machine-learning
> process to do the email analysis within the application will be greatly
> appreciated.
>
>
Seems that there are two phases to this... the initial bulk extract and
learning exercise, and then on-the-fly scoring.

I suppose that the initial bulk extract and learning could be done as a
background task in Isis... but I'd probably spike it outside of Isis in the
first place.

The on-the-fly scoring looks easy enough; a synchronous call from the RB
gmail client to the RB server hosted by Isis and its REST API.

I think overall this might be more an a Mahout- than an Isis project; but
it'd be nice to combine all of these (plus Shiro for the auth stuff, of
course).

Other opinions on this topic welcome...

HTH
Dan

> Thanks,
> Dileepa
>
>
>
[1] https://mahout.apache.org/
[2]
http://emmaespina.wordpress.com/2011/04/26/ham-spam-and-elephants-or-how-to-build-a-spam-filter-server-with-mahout/

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Hi Dan,

Thanks again for your insight.

Based on our discussions, so far I have an understanding on how to
authorize the RB webapp using OAuth2 to connect to email server to retrieve
email and implement the ReputationStore using JDO spec (using Isis
persistence support).

Also a wicket based web application will be developed as the RB demo
application.

In the ReputationBox, another main component is the reputation-analysis
component.
This requires to process emails and predict the reputation of new emails.
i.e it will be a recommendation engine for emails. Drools is one option for
a rules engine but it doesn't do any machine learning or prediction AFAIK.
The best opensource option to implement a recommendation engine I have seen
so far is Apache Mahout.

Any suggestions, pointers on how to integrate a Mahout machine-learning
process to do the email analysis within the application will be greatly
appreciated.

Thanks,
Dileepa

On Thu, Mar 20, 2014 at 2:37 PM, Dan Haywood
<da...@haywood-associates.co.uk>wrote:

> On 20 March 2014 06:07, Dileepa Jayakody <di...@gmail.com>
> wrote:
>
> > Hi Dan and All,
> >
> > I was reading Isis JDO documentation to get an idea of how to implement
> the
> > ReputationStore to persist email reputation data. I'm interested in
> > learning more about deploying Isis JDO data to Google App Engine [1]. I
> see
> > that it explains about some limitations when deploying Isis to GAE and
> > would like to seek your feedback on implementing my application as
> > AppEngine compatible to deploy.
> >
> >
> Maurizio (cc'ed) has definitely deployed Isis to GAE, this page was written
> after a bit of feedback from him.  And he wrote the DHTMLX viewer and has a
> demo app running on GAE [2]
>
> So it's doable.  I just haven't done it yet myself.
>
> I think it'd be best not to get side-tracked by this, and focus on your
> core deliverables.  Later, we can do the work to GAE-ize the app.
>
>
>
> > Thanks,
> > Dileepa
> >
> >
> > [1]
> >
> >
> http://isis.apache.org/components/objectstores/jdo/deploying-on-the-google-app-engine.html
> >
> > [2] http://isis-viewer-dhtmlx.appspot.com/
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

On 20 March 2014 06:07, Dileepa Jayakody <di...@gmail.com> wrote:

> Hi Dan and All,
>
> I was reading Isis JDO documentation to get an idea of how to implement the
> ReputationStore to persist email reputation data. I'm interested in
> learning more about deploying Isis JDO data to Google App Engine [1]. I see
> that it explains about some limitations when deploying Isis to GAE and
> would like to seek your feedback on implementing my application as
> AppEngine compatible to deploy.
>
>
Maurizio (cc'ed) has definitely deployed Isis to GAE, this page was written
after a bit of feedback from him.  And he wrote the DHTMLX viewer and has a
demo app running on GAE [2]

So it's doable.  I just haven't done it yet myself.

I think it'd be best not to get side-tracked by this, and focus on your
core deliverables.  Later, we can do the work to GAE-ize the app.



> Thanks,
> Dileepa
>
>
> [1]
>
> http://isis.apache.org/components/objectstores/jdo/deploying-on-the-google-app-engine.html
>
> [2] http://isis-viewer-dhtmlx.appspot.com/

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Hi Dan and All,

I was reading Isis JDO documentation to get an idea of how to implement the
ReputationStore to persist email reputation data. I'm interested in
learning more about deploying Isis JDO data to Google App Engine [1]. I see
that it explains about some limitations when deploying Isis to GAE and
would like to seek your feedback on implementing my application as
AppEngine compatible to deploy.

Thanks,
Dileepa


[1]
http://isis.apache.org/components/objectstores/jdo/deploying-on-the-google-app-engine.html


On Thu, Mar 20, 2014 at 10:46 AM, Dileepa Jayakody <
dileepajayakody@gmail.com> wrote:

> Hi Dan,
>
>
> On Wed, Mar 19, 2014 at 2:32 PM, Dileepa Jayakody <
> dileepajayakody@gmail.com> wrote:
>
>> Thanks Dan, I will take a look at it. I'm currently drafting a proposal,
>> will send to the list asap for comments.
>>
>>
>> On Wed, Mar 19, 2014 at 12:57 PM, Dan Haywood <
>> dan@haywood-associates.co.uk> wrote:
>>
>>> Dileepa,
>>> You might want to take a look at the comment I just added to ISIS-743
>>> [1],
>>> answering a question raised about the proposed removal of a feature.  I
>>> used context.io to illustrate alternatives; might be helpful for you to
>>> grok too.
>>> Cheers
>>> Dan
>>>
>>>
>>> [1] https://issues.apache.org/jira/browse/ISIS-743
>>>
>>>
>>> On 18 March 2014 20:55, Dan Haywood <da...@haywood-associates.co.uk>
>>> wrote:
>>>
>>> > Hi Dileepa
>>> > OK, think I follow that ok.  Looks like you're doing some decent
>>> research
>>> > here and have a good handle on how to build this app.
>>> >
>>> > I notice that context.io is commercial, so we'll have to take care
>>> about
>>> > licensing.  If the final design ends up being a polling architecture,
>>> then
>>> > that will probably be some sort of domain service.  I think there
>>> should be
>>> > a (Java) interface to define the contract, with a fake implementation
>>> (also
>>> > useful for testing) as part of the Isis codebase, but then the "real"
>>> > context.io based impl can live externally up on github, as a
>>> third-party
>>> > contribution.
>>> >
>>>
>>
> Do you think using a commercial API such as context.io is not such good
> idea? If so I can look at implementing the email data handling part of RB
> as a separate domain service.
> For user authentication to email server (gmail) we can use Scribe [1] or
> OAuth2 extension developed for Apache Shiro [2] to perform the OAuth2
> authorization. And have a domain service implemented to perform email
> operations.
>
> I think these can be contributed as  separate domain services (integrate
> OAuth2 and handle email data) in Apache Isis.
> So the project will have several deliverables along the way.
>
> 1. OAuth2 client support in Apache Isis
> 2. Email data analysis domain service
> 3. ReputationBox as a demo application
>
> WDYT?
>
> Thanks,
> Dileepa
>
> [1] https://github.com/fernandezpablo85/scribe-java
> [2] https://github.com/bujiio/buji-pac4j
>
>  > If there are any other commercial dependencies, we would also need to
>>> have
>>> > these abstracted away similarly.
>>> >
>>> > Dan
>>> >
>>> >
>>> >
>>> >
>>> > On 18 March 2014 19:48, Dileepa Jayakody <dileepajayakody@gmail.com
>>> >wrote:
>>> >
>>> >> Hi Dan,
>>> >>
>>> >> Thanks for your valuable input.
>>> >> Please see my comments inline.
>>> >>
>>> >> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood
>>> >> <da...@haywood-associates.co.uk>wrote:
>>> >>
>>> >> > On 14 March 2014 11:09, Dileepa Jayakody <dileepajayakody@gmail.com
>>> >
>>> >> > wrote:
>>> >> >
>>> >> > >
>>> >> > > My current idea is that it will follow below flow of operations.
>>> >> > >
>>> >> > >  1. User authorizes ReputationBox to connect to his mailbox to
>>> read
>>> >> email
>>> >> > > (probably using OAuth2)
>>> >> > >
>>> >> >
>>> >> > I presume that this would be actioned from within ReputationBox
>>> (RB),
>>> >> > ultimately by invoking an action on some domain object or service?
>>> >> >
>>> >> >
>>> >> > At present the only thing a domain action can do is to return a URL;
>>> >> this
>>> >> > is then opened up (by the browser, via Ajax) in a separate tab).
>>> >> >
>>> >> > It isn't possible to set up cookies etc on this action, and I
>>> imagine
>>> >> that
>>> >> > they would be needed in order to do the oauth interaction dance.
>>> >> >
>>> >> > AFAIK, cookies are not required for OAuth2 handshake. It simply
>>> requires
>>> >> to redirect the browser to the authorization page provided by the
>>> OAuth2
>>> >> service provider. RB here will be the OAuth2 consumer, requesting the
>>> >> access token from the OAuth2 service provider (Google) to access the
>>> >> user's
>>> >> gmail access. (I'm planning to use Gmail as the email service provider
>>> >> with
>>> >> OAuth2 support.)
>>> >>
>>> >> Further to perform all the email data related requests, I'm planning
>>> to
>>> >> use
>>> >> context.io API:  http://context.io/.
>>> >> It is a REST email API to integrate email data into applications.
>>> >> Context.io internally supports OAuth2 authorization to connect
>>> mailboxes
>>> >> with applications.
>>> >>
>>> >> As a first iteration, I think it'd make most sense to implement this
>>> >> > outside of the Isis wicket viewer (ie in a custom servlet).  This
>>> custom
>>> >> > servlet could use the IsisSessionTemplate class to then interact
>>> with
>>> >> Isis,
>>> >> > and shove the relevant credentials into an appropriate domain
>>> object.
>>> >> >
>>> >> >
>>> >>
>>> >> >
>>> >> > >  2. ReputationBox performs an initial reputation-analysis process
>>> to
>>> >> > > build a reputation-index over the past emails imported as a batch.
>>> >> (This
>>> >> > > initial reputation-index will be used as the training-data to
>>> analyse
>>> >> new
>>> >> > > incoming emails)
>>> >> > >
>>> >> >
>>> >> > Once the oauth credentials have been obtained and are held within
>>> the
>>> >> > Isis-managed domain, then this looks like it should probably be done
>>> >> > asynchronously.
>>> >> >
>>> >> > Yes, the initial reputation analysis process will be done
>>> asynchronously
>>> >> and the backgroundService + Quartz scheduler suggestion you have
>>> given is
>>> >> most suitable to implement this.
>>> >>
>>> >>
>>> >> > You could the backgroundService + a Quartz scheduler to pick up a
>>> >> request
>>> >> > to perform the initial reputation analysis process, and have it
>>> store
>>> >> its
>>> >> > results in appropriate domain objects.
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > >  3. New emails are polled/ pushed to ReputationBox server and
>>> >> > > reputation-analysis is performed real-time to asses the
>>> reputation.
>>> >> > >
>>> >> >
>>> >> > I prefer the idea of pushing emails to RB, but if that's the case,
>>> then
>>> >> why
>>> >> > not use it everywhere (including the initial export?)  That'd
>>> remove the
>>> >> > need for the oauth/custom servlet stuff in (1) above.
>>> >> >
>>> >> > Pushing emails to the RB requires some push-based mechanism from
>>> >> email-server side. I'm not sure if that is possible with any email
>>> server
>>> >> to push emails to a third party application/server. Please let me
>>> know if
>>> >> that's possible, in which case above OAuth2 requirement to access
>>> gmail
>>> >> from RB application will be removed as you have suggested. :)
>>> >>
>>> >> Context.io uses a pull based mechanism to periodically retrieve email
>>> >> data.
>>> >>
>>> >> >
>>> >> >
>>> >> > >  4. Email reputation data is stored as a special header in the
>>> email
>>> >> > > itself OR stored in a special IMAP directory in the user's mailbox
>>> >> (need
>>> >> > to
>>> >> > > decide on the reputation data storage mechanism)
>>> >> > >
>>> >> >
>>> >> > This also sounds like the email server should call out to the RB
>>> server.
>>> >>
>>> >>  It'd be interesting to see how tools such as SpamAssassin [1] do
>>> this.
>>> >> >
>>> >>
>>> >> Thanks for the pointer, I will check that.
>>> >> Reputation-storage can also be done at RB server, since Isis support
>>> >> persistence storage. Each user will have a reputation-account and
>>> they can
>>> >> be persisted in ISIS server. The other option is to store the
>>> >> reputation-data in a special IMAP directory in the user's mailbox
>>> itself.
>>> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > > 5. ReputationBox client is either a web-app OR a plugin to an
>>> existing
>>> >> > > web-mail client (eg :gmail) that represents the reputation data
>>> of the
>>> >> > new
>>> >> > > emails (based on the reputation data in the email the client
>>> could be
>>> >> > > implemented as a priority-inbox, spam-filter, email categorizer
>>> etc)
>>> >> > >
>>> >> >
>>> >> >
>>> >> > The bit that's not clear to me is whether this is pull or push for
>>> both
>>> >> the
>>> >> > initial analysis and the ongoing attachment of headers.
>>> >>
>>> >>
>>> >> I'm thinking RB server will adopt a pull based model in general using
>>> >> Context.io. Context.io also have a feature called webhooks which
>>>  provides
>>> >> rule-based notifications pushed to the app through HTTP POST
>>> requests. I
>>> >> will check the possibility of push-based implementation for RB from
>>> >> context.io
>>> >>
>>> >> From your diagram
>>> >> > on the ISIS-736 ticket it looks a bit like the RB client is
>>> actually a
>>> >> > gmail plugin, and then the RB server is probably the domain
>>> surfaced by
>>> >> > Isis' Restful Objects viewer.
>>> >> >
>>> >> > The Wicket viewer would then be an "administrative" console to also
>>> >> > browse/amend reputation data.
>>> >> >
>>> >> > I'm thinking of developing the intial version of the RB client as a
>>> >> webapp
>>> >> to browse the reputation-data with emails. That way ISIS will also
>>> have
>>> >> another DEMO web-application with email data.
>>> >>
>>> >> The final production level client will be a gmail plugin.
>>> >>
>>> >>
>>> >> > Overall... think this is doable; just not sure if the oauth2
>>> >> integration is
>>> >> > actually required.
>>> >> >
>>> >> > Thanks and more suggestions are welcome. I will draft a proposal and
>>> >> send
>>> >> to this mail thread for your review.
>>> >>
>>> >> Regards,
>>> >> Dileepa
>>> >>
>>> >>
>>> >> > Dan
>>> >> >
>>> >> >
>>> >> > [1] http://spamassassin.apache.org/
>>> >> > [2]
>>> >> >
>>> >> >
>>> >>
>>> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > >
>>> >> > > Please see the high-level architecture diagram attached here to
>>> get a
>>> >> > > better idea of my system architecture. I can also send a SRS
>>> draft if
>>> >> you
>>> >> > > are interested.
>>> >> > > The entities I have in my mind are : email-sender, email-message,
>>> >> > > reputation-profile.
>>> >> > >
>>> >> > > Suggestions and ideas on how I can utilize Isis framework and it's
>>> >> tools
>>> >> > > for my application are most welcome.
>>> >> > >
>>> >> > > Thanks,
>>> >> > > Dileepa
>>> >> > >
>>> >> > >
>>> >> > >> In particular, I'm interested to know what the entities are, and
>>> I'm
>>> >> > also
>>> >> > >> interested in about the integrations between the app and the
>>> users'
>>> >> meil
>>> >> > >> client.  For example: how does the app get hold of these emails
>>> to
>>> >> > assess
>>> >> > >> reputation; is it a batch import, real-time, something else; and
>>> how
>>> >> > does
>>> >> > >> the Isis app then add in reputation scores later (does it
>>> interact
>>> >> with
>>> >> > >> the
>>> >> > >> email server, perhaps).
>>> >> > >>
>>> >> > >> Thx
>>> >> > >> Dan
>>> >> > >>
>>> >> > >>
>>> >> > >>
>>> >> > >>
>>> >> > >> On 14 March 2014 09:49, Dileepa Jayakody <
>>> dileepajayakody@gmail.com>
>>> >> > >> wrote:
>>> >> > >>
>>> >> > >> > Hi All,
>>> >> > >> >
>>> >> > >> > I'm Dileepa Jayakody a MSc research student from University of
>>> >> > Moratuwa,
>>> >> > >> > Sri Lanka. I'm doing my research project on the topic :
>>> reputation
>>> >> > >> > assessment in emails.
>>> >> > >> > My project goal is to introduce reputation data as a attribute
>>> to
>>> >> > >> emails,
>>> >> > >> > which could be used for various applications such as
>>> >> spam-filtering,
>>> >> > >> > priority inboxes, social-networking, etc.
>>> >> > >> >
>>> >> > >> > I'm planning to adopt a prototype model to develop my
>>> application.
>>> >> > And I
>>> >> > >> > find Apache Isis a great framework to implement such
>>> applications
>>> >> > mainly
>>> >> > >> > focusing on my domain model. I'm interested in the GSoC idea :
>>> >> build a
>>> >> > >> > "real-life" app in some suitable domain, along with a
>>> semi-academic
>>> >> > >> > write-up of their learning [1] and wish to seek your opinion on
>>> >> > whether
>>> >> > >> > implementing my project using Apache Isis can be considered a
>>> GSoC
>>> >> > >> project.
>>> >> > >> > I'm willing to write a paper on the project implementation,
>>> >> > highlighting
>>> >> > >> > the features, usage details of the framework.
>>> >> > >> >
>>> >> > >> > As suggested by Dan I took a look at the thesis on  "Naked
>>> >> Objects",
>>> >> > >> > chapter 7 on the implementation comparison of CarServ
>>> >> (conventional vs
>>> >> > >> Isis
>>> >> > >> > usage). In this GSoC project idea, do you  think the student
>>> must
>>> >> do 2
>>> >> > >> > developments; one in conventional way and another using Isis?
>>> In
>>> >> that
>>> >> > >> case
>>> >> > >> > it might be difficult for a research project such as mine.
>>> >> > >> >
>>> >> > >> > WDYT?
>>> >> > >> >
>>> >> > >> > Thanks,
>>> >> > >> > Dileepa
>>> >> > >> >
>>> >> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736
>>> >> > >> >
>>> >> > >>
>>> >> > >
>>> >> > >
>>> >> >
>>> >>
>>> >
>>> >
>>>
>>
>>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

On 20 March 2014 05:16, Dileepa Jayakody <di...@gmail.com> wrote:

> >
> Do you think using a commercial API such as context.io is not such good
> idea?

If you can find an open source equivalent (or roll your own), then that's
obviously preferable.  But if that's not feasible (or if you just run out
of time), then I think it's okay to integrate to context.io or similar, so
long as that code remains separate from Isis itself.

> If so I can look at implementing the email data handling part of RB
> as a separate domain service.
>

You should define a domain service anyway (EmailService or similar).  Then
have EmailServiceUsingContextIo or
EmailServiceUsingSomeCoolOpenSourceProject as the implementations.

Most of the services in the Isis applib are implemented this way, see for
example PublishingService [1], with a PublishingServiceJdo implementation
[2]

> For user authentication to email server (gmail) we can use Scribe [1] or
> OAuth2 extension developed for Apache Shiro [2] to perform the OAuth2
> authorization.

Both look good.  Scribe has MIT license, buji-pac4j has ASF v2.  Valid
licenses for ASF are listed in [5]

> And have a domain service implemented to perform email
> operations.
>
> I think these can be contributed as  separate domain services (integrate
> OAuth2 and handle email data) in Apache Isis.
>

I'm still a bit unclear as to how the orchestration with these external
services will work, because they would seem to sit "in front of" an Isis
application, rather than within it.  But I'm not saying you're wrong, just
that I don't yet have it clear in my own head.

> So the project will have several deliverables along the way.
>
> 1. OAuth2 client support in Apache Isis
> 2. Email data analysis domain service
> 3. ReputationBox as a demo application
>
> WDYT?
>
>
Would be very happy if that is what gets delivered.

Cheers
Dan

> Thanks,
> Dileepa
>
> [1] https://github.com/fernandezpablo85/scribe-java
> [2] https://github.com/bujiio/buji-pac4j
>
>
[3] http://s.apache.org/b30
[4] http://s.apache.org/6H2
[5] https://www.apache.org/legal/3party.html

>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Hi Dan,


On Wed, Mar 19, 2014 at 2:32 PM, Dileepa Jayakody <dileepajayakody@gmail.com
> wrote:

> Thanks Dan, I will take a look at it. I'm currently drafting a proposal,
> will send to the list asap for comments.
>
>
> On Wed, Mar 19, 2014 at 12:57 PM, Dan Haywood <
> dan@haywood-associates.co.uk> wrote:
>
>> Dileepa,
>> You might want to take a look at the comment I just added to ISIS-743 [1],
>> answering a question raised about the proposed removal of a feature.  I
>> used context.io to illustrate alternatives; might be helpful for you to
>> grok too.
>> Cheers
>> Dan
>>
>>
>> [1] https://issues.apache.org/jira/browse/ISIS-743
>>
>>
>> On 18 March 2014 20:55, Dan Haywood <da...@haywood-associates.co.uk> wrote:
>>
>> > Hi Dileepa
>> > OK, think I follow that ok.  Looks like you're doing some decent
>> research
>> > here and have a good handle on how to build this app.
>> >
>> > I notice that context.io is commercial, so we'll have to take care
>> about
>> > licensing.  If the final design ends up being a polling architecture,
>> then
>> > that will probably be some sort of domain service.  I think there
>> should be
>> > a (Java) interface to define the contract, with a fake implementation
>> (also
>> > useful for testing) as part of the Isis codebase, but then the "real"
>> > context.io based impl can live externally up on github, as a
>> third-party
>> > contribution.
>> >
>>
>
Do you think using a commercial API such as context.io is not such good
idea? If so I can look at implementing the email data handling part of RB
as a separate domain service.
For user authentication to email server (gmail) we can use Scribe [1] or
OAuth2 extension developed for Apache Shiro [2] to perform the OAuth2
authorization. And have a domain service implemented to perform email
operations.

I think these can be contributed as  separate domain services (integrate
OAuth2 and handle email data) in Apache Isis.
So the project will have several deliverables along the way.

1. OAuth2 client support in Apache Isis
2. Email data analysis domain service
3. ReputationBox as a demo application

WDYT?

Thanks,
Dileepa

[1] https://github.com/fernandezpablo85/scribe-java
[2] https://github.com/bujiio/buji-pac4j

> If there are any other commercial dependencies, we would also need to have
>> > these abstracted away similarly.
>> >
>> > Dan
>> >
>> >
>> >
>> >
>> > On 18 March 2014 19:48, Dileepa Jayakody <dileepajayakody@gmail.com
>> >wrote:
>> >
>> >> Hi Dan,
>> >>
>> >> Thanks for your valuable input.
>> >> Please see my comments inline.
>> >>
>> >> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood
>> >> <da...@haywood-associates.co.uk>wrote:
>> >>
>> >> > On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com>
>> >> > wrote:
>> >> >
>> >> > >
>> >> > > My current idea is that it will follow below flow of operations.
>> >> > >
>> >> > >  1. User authorizes ReputationBox to connect to his mailbox to read
>> >> email
>> >> > > (probably using OAuth2)
>> >> > >
>> >> >
>> >> > I presume that this would be actioned from within ReputationBox (RB),
>> >> > ultimately by invoking an action on some domain object or service?
>> >> >
>> >> >
>> >> > At present the only thing a domain action can do is to return a URL;
>> >> this
>> >> > is then opened up (by the browser, via Ajax) in a separate tab).
>> >> >
>> >> > It isn't possible to set up cookies etc on this action, and I imagine
>> >> that
>> >> > they would be needed in order to do the oauth interaction dance.
>> >> >
>> >> > AFAIK, cookies are not required for OAuth2 handshake. It simply
>> requires
>> >> to redirect the browser to the authorization page provided by the
>> OAuth2
>> >> service provider. RB here will be the OAuth2 consumer, requesting the
>> >> access token from the OAuth2 service provider (Google) to access the
>> >> user's
>> >> gmail access. (I'm planning to use Gmail as the email service provider
>> >> with
>> >> OAuth2 support.)
>> >>
>> >> Further to perform all the email data related requests, I'm planning to
>> >> use
>> >> context.io API:  http://context.io/.
>> >> It is a REST email API to integrate email data into applications.
>> >> Context.io internally supports OAuth2 authorization to connect
>> mailboxes
>> >> with applications.
>> >>
>> >> As a first iteration, I think it'd make most sense to implement this
>> >> > outside of the Isis wicket viewer (ie in a custom servlet).  This
>> custom
>> >> > servlet could use the IsisSessionTemplate class to then interact with
>> >> Isis,
>> >> > and shove the relevant credentials into an appropriate domain object.
>> >> >
>> >> >
>> >>
>> >> >
>> >> > >  2. ReputationBox performs an initial reputation-analysis process
>> to
>> >> > > build a reputation-index over the past emails imported as a batch.
>> >> (This
>> >> > > initial reputation-index will be used as the training-data to
>> analyse
>> >> new
>> >> > > incoming emails)
>> >> > >
>> >> >
>> >> > Once the oauth credentials have been obtained and are held within the
>> >> > Isis-managed domain, then this looks like it should probably be done
>> >> > asynchronously.
>> >> >
>> >> > Yes, the initial reputation analysis process will be done
>> asynchronously
>> >> and the backgroundService + Quartz scheduler suggestion you have given
>> is
>> >> most suitable to implement this.
>> >>
>> >>
>> >> > You could the backgroundService + a Quartz scheduler to pick up a
>> >> request
>> >> > to perform the initial reputation analysis process, and have it store
>> >> its
>> >> > results in appropriate domain objects.
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > >  3. New emails are polled/ pushed to ReputationBox server and
>> >> > > reputation-analysis is performed real-time to asses the reputation.
>> >> > >
>> >> >
>> >> > I prefer the idea of pushing emails to RB, but if that's the case,
>> then
>> >> why
>> >> > not use it everywhere (including the initial export?)  That'd remove
>> the
>> >> > need for the oauth/custom servlet stuff in (1) above.
>> >> >
>> >> > Pushing emails to the RB requires some push-based mechanism from
>> >> email-server side. I'm not sure if that is possible with any email
>> server
>> >> to push emails to a third party application/server. Please let me know
>> if
>> >> that's possible, in which case above OAuth2 requirement to access gmail
>> >> from RB application will be removed as you have suggested. :)
>> >>
>> >> Context.io uses a pull based mechanism to periodically retrieve email
>> >> data.
>> >>
>> >> >
>> >> >
>> >> > >  4. Email reputation data is stored as a special header in the
>> email
>> >> > > itself OR stored in a special IMAP directory in the user's mailbox
>> >> (need
>> >> > to
>> >> > > decide on the reputation data storage mechanism)
>> >> > >
>> >> >
>> >> > This also sounds like the email server should call out to the RB
>> server.
>> >>
>> >>  It'd be interesting to see how tools such as SpamAssassin [1] do this.
>> >> >
>> >>
>> >> Thanks for the pointer, I will check that.
>> >> Reputation-storage can also be done at RB server, since Isis support
>> >> persistence storage. Each user will have a reputation-account and they
>> can
>> >> be persisted in ISIS server. The other option is to store the
>> >> reputation-data in a special IMAP directory in the user's mailbox
>> itself.
>> >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > > 5. ReputationBox client is either a web-app OR a plugin to an
>> existing
>> >> > > web-mail client (eg :gmail) that represents the reputation data of
>> the
>> >> > new
>> >> > > emails (based on the reputation data in the email the client could
>> be
>> >> > > implemented as a priority-inbox, spam-filter, email categorizer
>> etc)
>> >> > >
>> >> >
>> >> >
>> >> > The bit that's not clear to me is whether this is pull or push for
>> both
>> >> the
>> >> > initial analysis and the ongoing attachment of headers.
>> >>
>> >>
>> >> I'm thinking RB server will adopt a pull based model in general using
>> >> Context.io. Context.io also have a feature called webhooks which
>>  provides
>> >> rule-based notifications pushed to the app through HTTP POST requests.
>> I
>> >> will check the possibility of push-based implementation for RB from
>> >> context.io
>> >>
>> >> From your diagram
>> >> > on the ISIS-736 ticket it looks a bit like the RB client is actually
>> a
>> >> > gmail plugin, and then the RB server is probably the domain surfaced
>> by
>> >> > Isis' Restful Objects viewer.
>> >> >
>> >> > The Wicket viewer would then be an "administrative" console to also
>> >> > browse/amend reputation data.
>> >> >
>> >> > I'm thinking of developing the intial version of the RB client as a
>> >> webapp
>> >> to browse the reputation-data with emails. That way ISIS will also have
>> >> another DEMO web-application with email data.
>> >>
>> >> The final production level client will be a gmail plugin.
>> >>
>> >>
>> >> > Overall... think this is doable; just not sure if the oauth2
>> >> integration is
>> >> > actually required.
>> >> >
>> >> > Thanks and more suggestions are welcome. I will draft a proposal and
>> >> send
>> >> to this mail thread for your review.
>> >>
>> >> Regards,
>> >> Dileepa
>> >>
>> >>
>> >> > Dan
>> >> >
>> >> >
>> >> > [1] http://spamassassin.apache.org/
>> >> > [2]
>> >> >
>> >> >
>> >>
>> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > >
>> >> > > Please see the high-level architecture diagram attached here to
>> get a
>> >> > > better idea of my system architecture. I can also send a SRS draft
>> if
>> >> you
>> >> > > are interested.
>> >> > > The entities I have in my mind are : email-sender, email-message,
>> >> > > reputation-profile.
>> >> > >
>> >> > > Suggestions and ideas on how I can utilize Isis framework and it's
>> >> tools
>> >> > > for my application are most welcome.
>> >> > >
>> >> > > Thanks,
>> >> > > Dileepa
>> >> > >
>> >> > >
>> >> > >> In particular, I'm interested to know what the entities are, and
>> I'm
>> >> > also
>> >> > >> interested in about the integrations between the app and the
>> users'
>> >> meil
>> >> > >> client.  For example: how does the app get hold of these emails to
>> >> > assess
>> >> > >> reputation; is it a batch import, real-time, something else; and
>> how
>> >> > does
>> >> > >> the Isis app then add in reputation scores later (does it interact
>> >> with
>> >> > >> the
>> >> > >> email server, perhaps).
>> >> > >>
>> >> > >> Thx
>> >> > >> Dan
>> >> > >>
>> >> > >>
>> >> > >>
>> >> > >>
>> >> > >> On 14 March 2014 09:49, Dileepa Jayakody <
>> dileepajayakody@gmail.com>
>> >> > >> wrote:
>> >> > >>
>> >> > >> > Hi All,
>> >> > >> >
>> >> > >> > I'm Dileepa Jayakody a MSc research student from University of
>> >> > Moratuwa,
>> >> > >> > Sri Lanka. I'm doing my research project on the topic :
>> reputation
>> >> > >> > assessment in emails.
>> >> > >> > My project goal is to introduce reputation data as a attribute
>> to
>> >> > >> emails,
>> >> > >> > which could be used for various applications such as
>> >> spam-filtering,
>> >> > >> > priority inboxes, social-networking, etc.
>> >> > >> >
>> >> > >> > I'm planning to adopt a prototype model to develop my
>> application.
>> >> > And I
>> >> > >> > find Apache Isis a great framework to implement such
>> applications
>> >> > mainly
>> >> > >> > focusing on my domain model. I'm interested in the GSoC idea :
>> >> build a
>> >> > >> > "real-life" app in some suitable domain, along with a
>> semi-academic
>> >> > >> > write-up of their learning [1] and wish to seek your opinion on
>> >> > whether
>> >> > >> > implementing my project using Apache Isis can be considered a
>> GSoC
>> >> > >> project.
>> >> > >> > I'm willing to write a paper on the project implementation,
>> >> > highlighting
>> >> > >> > the features, usage details of the framework.
>> >> > >> >
>> >> > >> > As suggested by Dan I took a look at the thesis on  "Naked
>> >> Objects",
>> >> > >> > chapter 7 on the implementation comparison of CarServ
>> >> (conventional vs
>> >> > >> Isis
>> >> > >> > usage). In this GSoC project idea, do you  think the student
>> must
>> >> do 2
>> >> > >> > developments; one in conventional way and another using Isis? In
>> >> that
>> >> > >> case
>> >> > >> > it might be difficult for a research project such as mine.
>> >> > >> >
>> >> > >> > WDYT?
>> >> > >> >
>> >> > >> > Thanks,
>> >> > >> > Dileepa
>> >> > >> >
>> >> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736
>> >> > >> >
>> >> > >>
>> >> > >
>> >> > >
>> >> >
>> >>
>> >
>> >
>>
>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Thanks Dan, I will take a look at it. I'm currently drafting a proposal,
will send to the list asap for comments.


On Wed, Mar 19, 2014 at 12:57 PM, Dan Haywood
<da...@haywood-associates.co.uk>wrote:

> Dileepa,
> You might want to take a look at the comment I just added to ISIS-743 [1],
> answering a question raised about the proposed removal of a feature.  I
> used context.io to illustrate alternatives; might be helpful for you to
> grok too.
> Cheers
> Dan
>
>
> [1] https://issues.apache.org/jira/browse/ISIS-743
>
>
> On 18 March 2014 20:55, Dan Haywood <da...@haywood-associates.co.uk> wrote:
>
> > Hi Dileepa
> > OK, think I follow that ok.  Looks like you're doing some decent research
> > here and have a good handle on how to build this app.
> >
> > I notice that context.io is commercial, so we'll have to take care about
> > licensing.  If the final design ends up being a polling architecture,
> then
> > that will probably be some sort of domain service.  I think there should
> be
> > a (Java) interface to define the contract, with a fake implementation
> (also
> > useful for testing) as part of the Isis codebase, but then the "real"
> > context.io based impl can live externally up on github, as a third-party
> > contribution.
> >
> > If there are any other commercial dependencies, we would also need to
> have
> > these abstracted away similarly.
> >
> > Dan
> >
> >
> >
> >
> > On 18 March 2014 19:48, Dileepa Jayakody <dileepajayakody@gmail.com
> >wrote:
> >
> >> Hi Dan,
> >>
> >> Thanks for your valuable input.
> >> Please see my comments inline.
> >>
> >> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood
> >> <da...@haywood-associates.co.uk>wrote:
> >>
> >> > On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com>
> >> > wrote:
> >> >
> >> > >
> >> > > My current idea is that it will follow below flow of operations.
> >> > >
> >> > >  1. User authorizes ReputationBox to connect to his mailbox to read
> >> email
> >> > > (probably using OAuth2)
> >> > >
> >> >
> >> > I presume that this would be actioned from within ReputationBox (RB),
> >> > ultimately by invoking an action on some domain object or service?
> >> >
> >> >
> >> > At present the only thing a domain action can do is to return a URL;
> >> this
> >> > is then opened up (by the browser, via Ajax) in a separate tab).
> >> >
> >> > It isn't possible to set up cookies etc on this action, and I imagine
> >> that
> >> > they would be needed in order to do the oauth interaction dance.
> >> >
> >> > AFAIK, cookies are not required for OAuth2 handshake. It simply
> requires
> >> to redirect the browser to the authorization page provided by the OAuth2
> >> service provider. RB here will be the OAuth2 consumer, requesting the
> >> access token from the OAuth2 service provider (Google) to access the
> >> user's
> >> gmail access. (I'm planning to use Gmail as the email service provider
> >> with
> >> OAuth2 support.)
> >>
> >> Further to perform all the email data related requests, I'm planning to
> >> use
> >> context.io API:  http://context.io/.
> >> It is a REST email API to integrate email data into applications.
> >> Context.io internally supports OAuth2 authorization to connect mailboxes
> >> with applications.
> >>
> >> As a first iteration, I think it'd make most sense to implement this
> >> > outside of the Isis wicket viewer (ie in a custom servlet).  This
> custom
> >> > servlet could use the IsisSessionTemplate class to then interact with
> >> Isis,
> >> > and shove the relevant credentials into an appropriate domain object.
> >> >
> >> >
> >>
> >> >
> >> > >  2. ReputationBox performs an initial reputation-analysis process to
> >> > > build a reputation-index over the past emails imported as a batch.
> >> (This
> >> > > initial reputation-index will be used as the training-data to
> analyse
> >> new
> >> > > incoming emails)
> >> > >
> >> >
> >> > Once the oauth credentials have been obtained and are held within the
> >> > Isis-managed domain, then this looks like it should probably be done
> >> > asynchronously.
> >> >
> >> > Yes, the initial reputation analysis process will be done
> asynchronously
> >> and the backgroundService + Quartz scheduler suggestion you have given
> is
> >> most suitable to implement this.
> >>
> >>
> >> > You could the backgroundService + a Quartz scheduler to pick up a
> >> request
> >> > to perform the initial reputation analysis process, and have it store
> >> its
> >> > results in appropriate domain objects.
> >> >
> >> >
> >> >
> >> >
> >> > >  3. New emails are polled/ pushed to ReputationBox server and
> >> > > reputation-analysis is performed real-time to asses the reputation.
> >> > >
> >> >
> >> > I prefer the idea of pushing emails to RB, but if that's the case,
> then
> >> why
> >> > not use it everywhere (including the initial export?)  That'd remove
> the
> >> > need for the oauth/custom servlet stuff in (1) above.
> >> >
> >> > Pushing emails to the RB requires some push-based mechanism from
> >> email-server side. I'm not sure if that is possible with any email
> server
> >> to push emails to a third party application/server. Please let me know
> if
> >> that's possible, in which case above OAuth2 requirement to access gmail
> >> from RB application will be removed as you have suggested. :)
> >>
> >> Context.io uses a pull based mechanism to periodically retrieve email
> >> data.
> >>
> >> >
> >> >
> >> > >  4. Email reputation data is stored as a special header in the email
> >> > > itself OR stored in a special IMAP directory in the user's mailbox
> >> (need
> >> > to
> >> > > decide on the reputation data storage mechanism)
> >> > >
> >> >
> >> > This also sounds like the email server should call out to the RB
> server.
> >>
> >>  It'd be interesting to see how tools such as SpamAssassin [1] do this.
> >> >
> >>
> >> Thanks for the pointer, I will check that.
> >> Reputation-storage can also be done at RB server, since Isis support
> >> persistence storage. Each user will have a reputation-account and they
> can
> >> be persisted in ISIS server. The other option is to store the
> >> reputation-data in a special IMAP directory in the user's mailbox
> itself.
> >>
> >> >
> >> >
> >> >
> >> >
> >> > > 5. ReputationBox client is either a web-app OR a plugin to an
> existing
> >> > > web-mail client (eg :gmail) that represents the reputation data of
> the
> >> > new
> >> > > emails (based on the reputation data in the email the client could
> be
> >> > > implemented as a priority-inbox, spam-filter, email categorizer etc)
> >> > >
> >> >
> >> >
> >> > The bit that's not clear to me is whether this is pull or push for
> both
> >> the
> >> > initial analysis and the ongoing attachment of headers.
> >>
> >>
> >> I'm thinking RB server will adopt a pull based model in general using
> >> Context.io. Context.io also have a feature called webhooks which
>  provides
> >> rule-based notifications pushed to the app through HTTP POST requests. I
> >> will check the possibility of push-based implementation for RB from
> >> context.io
> >>
> >> From your diagram
> >> > on the ISIS-736 ticket it looks a bit like the RB client is actually a
> >> > gmail plugin, and then the RB server is probably the domain surfaced
> by
> >> > Isis' Restful Objects viewer.
> >> >
> >> > The Wicket viewer would then be an "administrative" console to also
> >> > browse/amend reputation data.
> >> >
> >> > I'm thinking of developing the intial version of the RB client as a
> >> webapp
> >> to browse the reputation-data with emails. That way ISIS will also have
> >> another DEMO web-application with email data.
> >>
> >> The final production level client will be a gmail plugin.
> >>
> >>
> >> > Overall... think this is doable; just not sure if the oauth2
> >> integration is
> >> > actually required.
> >> >
> >> > Thanks and more suggestions are welcome. I will draft a proposal and
> >> send
> >> to this mail thread for your review.
> >>
> >> Regards,
> >> Dileepa
> >>
> >>
> >> > Dan
> >> >
> >> >
> >> > [1] http://spamassassin.apache.org/
> >> > [2]
> >> >
> >> >
> >>
> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png
> >> >
> >> >
> >> >
> >> >
> >> > >
> >> > > Please see the high-level architecture diagram attached here to get
> a
> >> > > better idea of my system architecture. I can also send a SRS draft
> if
> >> you
> >> > > are interested.
> >> > > The entities I have in my mind are : email-sender, email-message,
> >> > > reputation-profile.
> >> > >
> >> > > Suggestions and ideas on how I can utilize Isis framework and it's
> >> tools
> >> > > for my application are most welcome.
> >> > >
> >> > > Thanks,
> >> > > Dileepa
> >> > >
> >> > >
> >> > >> In particular, I'm interested to know what the entities are, and
> I'm
> >> > also
> >> > >> interested in about the integrations between the app and the users'
> >> meil
> >> > >> client.  For example: how does the app get hold of these emails to
> >> > assess
> >> > >> reputation; is it a batch import, real-time, something else; and
> how
> >> > does
> >> > >> the Isis app then add in reputation scores later (does it interact
> >> with
> >> > >> the
> >> > >> email server, perhaps).
> >> > >>
> >> > >> Thx
> >> > >> Dan
> >> > >>
> >> > >>
> >> > >>
> >> > >>
> >> > >> On 14 March 2014 09:49, Dileepa Jayakody <
> dileepajayakody@gmail.com>
> >> > >> wrote:
> >> > >>
> >> > >> > Hi All,
> >> > >> >
> >> > >> > I'm Dileepa Jayakody a MSc research student from University of
> >> > Moratuwa,
> >> > >> > Sri Lanka. I'm doing my research project on the topic :
> reputation
> >> > >> > assessment in emails.
> >> > >> > My project goal is to introduce reputation data as a attribute to
> >> > >> emails,
> >> > >> > which could be used for various applications such as
> >> spam-filtering,
> >> > >> > priority inboxes, social-networking, etc.
> >> > >> >
> >> > >> > I'm planning to adopt a prototype model to develop my
> application.
> >> > And I
> >> > >> > find Apache Isis a great framework to implement such applications
> >> > mainly
> >> > >> > focusing on my domain model. I'm interested in the GSoC idea :
> >> build a
> >> > >> > "real-life" app in some suitable domain, along with a
> semi-academic
> >> > >> > write-up of their learning [1] and wish to seek your opinion on
> >> > whether
> >> > >> > implementing my project using Apache Isis can be considered a
> GSoC
> >> > >> project.
> >> > >> > I'm willing to write a paper on the project implementation,
> >> > highlighting
> >> > >> > the features, usage details of the framework.
> >> > >> >
> >> > >> > As suggested by Dan I took a look at the thesis on  "Naked
> >> Objects",
> >> > >> > chapter 7 on the implementation comparison of CarServ
> >> (conventional vs
> >> > >> Isis
> >> > >> > usage). In this GSoC project idea, do you  think the student must
> >> do 2
> >> > >> > developments; one in conventional way and another using Isis? In
> >> that
> >> > >> case
> >> > >> > it might be difficult for a research project such as mine.
> >> > >> >
> >> > >> > WDYT?
> >> > >> >
> >> > >> > Thanks,
> >> > >> > Dileepa
> >> > >> >
> >> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736
> >> > >> >
> >> > >>
> >> > >
> >> > >
> >> >
> >>
> >
> >
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

Dileepa,
You might want to take a look at the comment I just added to ISIS-743 [1],
answering a question raised about the proposed removal of a feature.  I
used context.io to illustrate alternatives; might be helpful for you to
grok too.
Cheers
Dan


[1] https://issues.apache.org/jira/browse/ISIS-743


On 18 March 2014 20:55, Dan Haywood <da...@haywood-associates.co.uk> wrote:

> Hi Dileepa
> OK, think I follow that ok.  Looks like you're doing some decent research
> here and have a good handle on how to build this app.
>
> I notice that context.io is commercial, so we'll have to take care about
> licensing.  If the final design ends up being a polling architecture, then
> that will probably be some sort of domain service.  I think there should be
> a (Java) interface to define the contract, with a fake implementation (also
> useful for testing) as part of the Isis codebase, but then the "real"
> context.io based impl can live externally up on github, as a third-party
> contribution.
>
> If there are any other commercial dependencies, we would also need to have
> these abstracted away similarly.
>
> Dan
>
>
>
>
> On 18 March 2014 19:48, Dileepa Jayakody <di...@gmail.com>wrote:
>
>> Hi Dan,
>>
>> Thanks for your valuable input.
>> Please see my comments inline.
>>
>> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood
>> <da...@haywood-associates.co.uk>wrote:
>>
>> > On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com>
>> > wrote:
>> >
>> > >
>> > > My current idea is that it will follow below flow of operations.
>> > >
>> > >  1. User authorizes ReputationBox to connect to his mailbox to read
>> email
>> > > (probably using OAuth2)
>> > >
>> >
>> > I presume that this would be actioned from within ReputationBox (RB),
>> > ultimately by invoking an action on some domain object or service?
>> >
>> >
>> > At present the only thing a domain action can do is to return a URL;
>> this
>> > is then opened up (by the browser, via Ajax) in a separate tab).
>> >
>> > It isn't possible to set up cookies etc on this action, and I imagine
>> that
>> > they would be needed in order to do the oauth interaction dance.
>> >
>> > AFAIK, cookies are not required for OAuth2 handshake. It simply requires
>> to redirect the browser to the authorization page provided by the OAuth2
>> service provider. RB here will be the OAuth2 consumer, requesting the
>> access token from the OAuth2 service provider (Google) to access the
>> user's
>> gmail access. (I'm planning to use Gmail as the email service provider
>> with
>> OAuth2 support.)
>>
>> Further to perform all the email data related requests, I'm planning to
>> use
>> context.io API:  http://context.io/.
>> It is a REST email API to integrate email data into applications.
>> Context.io internally supports OAuth2 authorization to connect mailboxes
>> with applications.
>>
>> As a first iteration, I think it'd make most sense to implement this
>> > outside of the Isis wicket viewer (ie in a custom servlet).  This custom
>> > servlet could use the IsisSessionTemplate class to then interact with
>> Isis,
>> > and shove the relevant credentials into an appropriate domain object.
>> >
>> >
>>
>> >
>> > >  2. ReputationBox performs an initial reputation-analysis process to
>> > > build a reputation-index over the past emails imported as a batch.
>> (This
>> > > initial reputation-index will be used as the training-data to analyse
>> new
>> > > incoming emails)
>> > >
>> >
>> > Once the oauth credentials have been obtained and are held within the
>> > Isis-managed domain, then this looks like it should probably be done
>> > asynchronously.
>> >
>> > Yes, the initial reputation analysis process will be done asynchronously
>> and the backgroundService + Quartz scheduler suggestion you have given is
>> most suitable to implement this.
>>
>>
>> > You could the backgroundService + a Quartz scheduler to pick up a
>> request
>> > to perform the initial reputation analysis process, and have it store
>> its
>> > results in appropriate domain objects.
>> >
>> >
>> >
>> >
>> > >  3. New emails are polled/ pushed to ReputationBox server and
>> > > reputation-analysis is performed real-time to asses the reputation.
>> > >
>> >
>> > I prefer the idea of pushing emails to RB, but if that's the case, then
>> why
>> > not use it everywhere (including the initial export?)  That'd remove the
>> > need for the oauth/custom servlet stuff in (1) above.
>> >
>> > Pushing emails to the RB requires some push-based mechanism from
>> email-server side. I'm not sure if that is possible with any email server
>> to push emails to a third party application/server. Please let me know if
>> that's possible, in which case above OAuth2 requirement to access gmail
>> from RB application will be removed as you have suggested. :)
>>
>> Context.io uses a pull based mechanism to periodically retrieve email
>> data.
>>
>> >
>> >
>> > >  4. Email reputation data is stored as a special header in the email
>> > > itself OR stored in a special IMAP directory in the user's mailbox
>> (need
>> > to
>> > > decide on the reputation data storage mechanism)
>> > >
>> >
>> > This also sounds like the email server should call out to the RB server.
>>
>>  It'd be interesting to see how tools such as SpamAssassin [1] do this.
>> >
>>
>> Thanks for the pointer, I will check that.
>> Reputation-storage can also be done at RB server, since Isis support
>> persistence storage. Each user will have a reputation-account and they can
>> be persisted in ISIS server. The other option is to store the
>> reputation-data in a special IMAP directory in the user's mailbox itself.
>>
>> >
>> >
>> >
>> >
>> > > 5. ReputationBox client is either a web-app OR a plugin to an existing
>> > > web-mail client (eg :gmail) that represents the reputation data of the
>> > new
>> > > emails (based on the reputation data in the email the client could be
>> > > implemented as a priority-inbox, spam-filter, email categorizer etc)
>> > >
>> >
>> >
>> > The bit that's not clear to me is whether this is pull or push for both
>> the
>> > initial analysis and the ongoing attachment of headers.
>>
>>
>> I'm thinking RB server will adopt a pull based model in general using
>> Context.io. Context.io also have a feature called webhooks which  provides
>> rule-based notifications pushed to the app through HTTP POST requests. I
>> will check the possibility of push-based implementation for RB from
>> context.io
>>
>> From your diagram
>> > on the ISIS-736 ticket it looks a bit like the RB client is actually a
>> > gmail plugin, and then the RB server is probably the domain surfaced by
>> > Isis' Restful Objects viewer.
>> >
>> > The Wicket viewer would then be an "administrative" console to also
>> > browse/amend reputation data.
>> >
>> > I'm thinking of developing the intial version of the RB client as a
>> webapp
>> to browse the reputation-data with emails. That way ISIS will also have
>> another DEMO web-application with email data.
>>
>> The final production level client will be a gmail plugin.
>>
>>
>> > Overall... think this is doable; just not sure if the oauth2
>> integration is
>> > actually required.
>> >
>> > Thanks and more suggestions are welcome. I will draft a proposal and
>> send
>> to this mail thread for your review.
>>
>> Regards,
>> Dileepa
>>
>>
>> > Dan
>> >
>> >
>> > [1] http://spamassassin.apache.org/
>> > [2]
>> >
>> >
>> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png
>> >
>> >
>> >
>> >
>> > >
>> > > Please see the high-level architecture diagram attached here to get a
>> > > better idea of my system architecture. I can also send a SRS draft if
>> you
>> > > are interested.
>> > > The entities I have in my mind are : email-sender, email-message,
>> > > reputation-profile.
>> > >
>> > > Suggestions and ideas on how I can utilize Isis framework and it's
>> tools
>> > > for my application are most welcome.
>> > >
>> > > Thanks,
>> > > Dileepa
>> > >
>> > >
>> > >> In particular, I'm interested to know what the entities are, and I'm
>> > also
>> > >> interested in about the integrations between the app and the users'
>> meil
>> > >> client.  For example: how does the app get hold of these emails to
>> > assess
>> > >> reputation; is it a batch import, real-time, something else; and how
>> > does
>> > >> the Isis app then add in reputation scores later (does it interact
>> with
>> > >> the
>> > >> email server, perhaps).
>> > >>
>> > >> Thx
>> > >> Dan
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> On 14 March 2014 09:49, Dileepa Jayakody <di...@gmail.com>
>> > >> wrote:
>> > >>
>> > >> > Hi All,
>> > >> >
>> > >> > I'm Dileepa Jayakody a MSc research student from University of
>> > Moratuwa,
>> > >> > Sri Lanka. I'm doing my research project on the topic : reputation
>> > >> > assessment in emails.
>> > >> > My project goal is to introduce reputation data as a attribute to
>> > >> emails,
>> > >> > which could be used for various applications such as
>> spam-filtering,
>> > >> > priority inboxes, social-networking, etc.
>> > >> >
>> > >> > I'm planning to adopt a prototype model to develop my application.
>> > And I
>> > >> > find Apache Isis a great framework to implement such applications
>> > mainly
>> > >> > focusing on my domain model. I'm interested in the GSoC idea :
>> build a
>> > >> > "real-life" app in some suitable domain, along with a semi-academic
>> > >> > write-up of their learning [1] and wish to seek your opinion on
>> > whether
>> > >> > implementing my project using Apache Isis can be considered a GSoC
>> > >> project.
>> > >> > I'm willing to write a paper on the project implementation,
>> > highlighting
>> > >> > the features, usage details of the framework.
>> > >> >
>> > >> > As suggested by Dan I took a look at the thesis on  "Naked
>> Objects",
>> > >> > chapter 7 on the implementation comparison of CarServ
>> (conventional vs
>> > >> Isis
>> > >> > usage). In this GSoC project idea, do you  think the student must
>> do 2
>> > >> > developments; one in conventional way and another using Isis? In
>> that
>> > >> case
>> > >> > it might be difficult for a research project such as mine.
>> > >> >
>> > >> > WDYT?
>> > >> >
>> > >> > Thanks,
>> > >> > Dileepa
>> > >> >
>> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

Hi Dileepa
OK, think I follow that ok.  Looks like you're doing some decent research
here and have a good handle on how to build this app.

I notice that context.io is commercial, so we'll have to take care about
licensing.  If the final design ends up being a polling architecture, then
that will probably be some sort of domain service.  I think there should be
a (Java) interface to define the contract, with a fake implementation (also
useful for testing) as part of the Isis codebase, but then the "real"
context.io based impl can live externally up on github, as a third-party
contribution.

If there are any other commercial dependencies, we would also need to have
these abstracted away similarly.

Dan




On 18 March 2014 19:48, Dileepa Jayakody <di...@gmail.com> wrote:

> Hi Dan,
>
> Thanks for your valuable input.
> Please see my comments inline.
>
> On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood
> <da...@haywood-associates.co.uk>wrote:
>
> > On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com>
> > wrote:
> >
> > >
> > > My current idea is that it will follow below flow of operations.
> > >
> > >  1. User authorizes ReputationBox to connect to his mailbox to read
> email
> > > (probably using OAuth2)
> > >
> >
> > I presume that this would be actioned from within ReputationBox (RB),
> > ultimately by invoking an action on some domain object or service?
> >
> >
> > At present the only thing a domain action can do is to return a URL; this
> > is then opened up (by the browser, via Ajax) in a separate tab).
> >
> > It isn't possible to set up cookies etc on this action, and I imagine
> that
> > they would be needed in order to do the oauth interaction dance.
> >
> > AFAIK, cookies are not required for OAuth2 handshake. It simply requires
> to redirect the browser to the authorization page provided by the OAuth2
> service provider. RB here will be the OAuth2 consumer, requesting the
> access token from the OAuth2 service provider (Google) to access the user's
> gmail access. (I'm planning to use Gmail as the email service provider with
> OAuth2 support.)
>
> Further to perform all the email data related requests, I'm planning to use
> context.io API:  http://context.io/.
> It is a REST email API to integrate email data into applications.
> Context.io internally supports OAuth2 authorization to connect mailboxes
> with applications.
>
> As a first iteration, I think it'd make most sense to implement this
> > outside of the Isis wicket viewer (ie in a custom servlet).  This custom
> > servlet could use the IsisSessionTemplate class to then interact with
> Isis,
> > and shove the relevant credentials into an appropriate domain object.
> >
> >
>
> >
> > >  2. ReputationBox performs an initial reputation-analysis process to
> > > build a reputation-index over the past emails imported as a batch.
> (This
> > > initial reputation-index will be used as the training-data to analyse
> new
> > > incoming emails)
> > >
> >
> > Once the oauth credentials have been obtained and are held within the
> > Isis-managed domain, then this looks like it should probably be done
> > asynchronously.
> >
> > Yes, the initial reputation analysis process will be done asynchronously
> and the backgroundService + Quartz scheduler suggestion you have given is
> most suitable to implement this.
>
>
> > You could the backgroundService + a Quartz scheduler to pick up a request
> > to perform the initial reputation analysis process, and have it store its
> > results in appropriate domain objects.
> >
> >
> >
> >
> > >  3. New emails are polled/ pushed to ReputationBox server and
> > > reputation-analysis is performed real-time to asses the reputation.
> > >
> >
> > I prefer the idea of pushing emails to RB, but if that's the case, then
> why
> > not use it everywhere (including the initial export?)  That'd remove the
> > need for the oauth/custom servlet stuff in (1) above.
> >
> > Pushing emails to the RB requires some push-based mechanism from
> email-server side. I'm not sure if that is possible with any email server
> to push emails to a third party application/server. Please let me know if
> that's possible, in which case above OAuth2 requirement to access gmail
> from RB application will be removed as you have suggested. :)
>
> Context.io uses a pull based mechanism to periodically retrieve email data.
>
> >
> >
> > >  4. Email reputation data is stored as a special header in the email
> > > itself OR stored in a special IMAP directory in the user's mailbox
> (need
> > to
> > > decide on the reputation data storage mechanism)
> > >
> >
> > This also sounds like the email server should call out to the RB server.
>
>  It'd be interesting to see how tools such as SpamAssassin [1] do this.
> >
>
> Thanks for the pointer, I will check that.
> Reputation-storage can also be done at RB server, since Isis support
> persistence storage. Each user will have a reputation-account and they can
> be persisted in ISIS server. The other option is to store the
> reputation-data in a special IMAP directory in the user's mailbox itself.
>
> >
> >
> >
> >
> > > 5. ReputationBox client is either a web-app OR a plugin to an existing
> > > web-mail client (eg :gmail) that represents the reputation data of the
> > new
> > > emails (based on the reputation data in the email the client could be
> > > implemented as a priority-inbox, spam-filter, email categorizer etc)
> > >
> >
> >
> > The bit that's not clear to me is whether this is pull or push for both
> the
> > initial analysis and the ongoing attachment of headers.
>
>
> I'm thinking RB server will adopt a pull based model in general using
> Context.io. Context.io also have a feature called webhooks which  provides
> rule-based notifications pushed to the app through HTTP POST requests. I
> will check the possibility of push-based implementation for RB from
> context.io
>
> From your diagram
> > on the ISIS-736 ticket it looks a bit like the RB client is actually a
> > gmail plugin, and then the RB server is probably the domain surfaced by
> > Isis' Restful Objects viewer.
> >
> > The Wicket viewer would then be an "administrative" console to also
> > browse/amend reputation data.
> >
> > I'm thinking of developing the intial version of the RB client as a
> webapp
> to browse the reputation-data with emails. That way ISIS will also have
> another DEMO web-application with email data.
>
> The final production level client will be a gmail plugin.
>
>
> > Overall... think this is doable; just not sure if the oauth2 integration
> is
> > actually required.
> >
> > Thanks and more suggestions are welcome. I will draft a proposal and send
> to this mail thread for your review.
>
> Regards,
> Dileepa
>
>
> > Dan
> >
> >
> > [1] http://spamassassin.apache.org/
> > [2]
> >
> >
> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png
> >
> >
> >
> >
> > >
> > > Please see the high-level architecture diagram attached here to get a
> > > better idea of my system architecture. I can also send a SRS draft if
> you
> > > are interested.
> > > The entities I have in my mind are : email-sender, email-message,
> > > reputation-profile.
> > >
> > > Suggestions and ideas on how I can utilize Isis framework and it's
> tools
> > > for my application are most welcome.
> > >
> > > Thanks,
> > > Dileepa
> > >
> > >
> > >> In particular, I'm interested to know what the entities are, and I'm
> > also
> > >> interested in about the integrations between the app and the users'
> meil
> > >> client.  For example: how does the app get hold of these emails to
> > assess
> > >> reputation; is it a batch import, real-time, something else; and how
> > does
> > >> the Isis app then add in reputation scores later (does it interact
> with
> > >> the
> > >> email server, perhaps).
> > >>
> > >> Thx
> > >> Dan
> > >>
> > >>
> > >>
> > >>
> > >> On 14 March 2014 09:49, Dileepa Jayakody <di...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi All,
> > >> >
> > >> > I'm Dileepa Jayakody a MSc research student from University of
> > Moratuwa,
> > >> > Sri Lanka. I'm doing my research project on the topic : reputation
> > >> > assessment in emails.
> > >> > My project goal is to introduce reputation data as a attribute to
> > >> emails,
> > >> > which could be used for various applications such as spam-filtering,
> > >> > priority inboxes, social-networking, etc.
> > >> >
> > >> > I'm planning to adopt a prototype model to develop my application.
> > And I
> > >> > find Apache Isis a great framework to implement such applications
> > mainly
> > >> > focusing on my domain model. I'm interested in the GSoC idea :
> build a
> > >> > "real-life" app in some suitable domain, along with a semi-academic
> > >> > write-up of their learning [1] and wish to seek your opinion on
> > whether
> > >> > implementing my project using Apache Isis can be considered a GSoC
> > >> project.
> > >> > I'm willing to write a paper on the project implementation,
> > highlighting
> > >> > the features, usage details of the framework.
> > >> >
> > >> > As suggested by Dan I took a look at the thesis on  "Naked Objects",
> > >> > chapter 7 on the implementation comparison of CarServ (conventional
> vs
> > >> Isis
> > >> > usage). In this GSoC project idea, do you  think the student must
> do 2
> > >> > developments; one in conventional way and another using Isis? In
> that
> > >> case
> > >> > it might be difficult for a research project such as mine.
> > >> >
> > >> > WDYT?
> > >> >
> > >> > Thanks,
> > >> > Dileepa
> > >> >
> > >> > [1] https://issues.apache.org/jira/browse/ISIS-736
> > >> >
> > >>
> > >
> > >
> >
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Hi Dan,

Thanks for your valuable input.
Please see my comments inline.

On Tue, Mar 18, 2014 at 2:03 PM, Dan Haywood
<da...@haywood-associates.co.uk>wrote:

> On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com>
> wrote:
>
> >
> > My current idea is that it will follow below flow of operations.
> >
> >  1. User authorizes ReputationBox to connect to his mailbox to read email
> > (probably using OAuth2)
> >
>
> I presume that this would be actioned from within ReputationBox (RB),
> ultimately by invoking an action on some domain object or service?
>
>
> At present the only thing a domain action can do is to return a URL; this
> is then opened up (by the browser, via Ajax) in a separate tab).
>
> It isn't possible to set up cookies etc on this action, and I imagine that
> they would be needed in order to do the oauth interaction dance.
>
> AFAIK, cookies are not required for OAuth2 handshake. It simply requires
to redirect the browser to the authorization page provided by the OAuth2
service provider. RB here will be the OAuth2 consumer, requesting the
access token from the OAuth2 service provider (Google) to access the user's
gmail access. (I'm planning to use Gmail as the email service provider with
OAuth2 support.)

Further to perform all the email data related requests, I'm planning to use
context.io API:  http://context.io/.
It is a REST email API to integrate email data into applications.
Context.io internally supports OAuth2 authorization to connect mailboxes
with applications.

As a first iteration, I think it'd make most sense to implement this
> outside of the Isis wicket viewer (ie in a custom servlet).  This custom
> servlet could use the IsisSessionTemplate class to then interact with Isis,
> and shove the relevant credentials into an appropriate domain object.
>
>

>
> >  2. ReputationBox performs an initial reputation-analysis process to
> > build a reputation-index over the past emails imported as a batch. (This
> > initial reputation-index will be used as the training-data to analyse new
> > incoming emails)
> >
>
> Once the oauth credentials have been obtained and are held within the
> Isis-managed domain, then this looks like it should probably be done
> asynchronously.
>
> Yes, the initial reputation analysis process will be done asynchronously
and the backgroundService + Quartz scheduler suggestion you have given is
most suitable to implement this.


> You could the backgroundService + a Quartz scheduler to pick up a request
> to perform the initial reputation analysis process, and have it store its
> results in appropriate domain objects.
>
>
>
>
> >  3. New emails are polled/ pushed to ReputationBox server and
> > reputation-analysis is performed real-time to asses the reputation.
> >
>
> I prefer the idea of pushing emails to RB, but if that's the case, then why
> not use it everywhere (including the initial export?)  That'd remove the
> need for the oauth/custom servlet stuff in (1) above.
>
> Pushing emails to the RB requires some push-based mechanism from
email-server side. I'm not sure if that is possible with any email server
to push emails to a third party application/server. Please let me know if
that's possible, in which case above OAuth2 requirement to access gmail
from RB application will be removed as you have suggested. :)

Context.io uses a pull based mechanism to periodically retrieve email data.

>
>
> >  4. Email reputation data is stored as a special header in the email
> > itself OR stored in a special IMAP directory in the user's mailbox (need
> to
> > decide on the reputation data storage mechanism)
> >
>
> This also sounds like the email server should call out to the RB server.

 It'd be interesting to see how tools such as SpamAssassin [1] do this.
>

Thanks for the pointer, I will check that.
Reputation-storage can also be done at RB server, since Isis support
persistence storage. Each user will have a reputation-account and they can
be persisted in ISIS server. The other option is to store the
reputation-data in a special IMAP directory in the user's mailbox itself.

>
>
>
>
> > 5. ReputationBox client is either a web-app OR a plugin to an existing
> > web-mail client (eg :gmail) that represents the reputation data of the
> new
> > emails (based on the reputation data in the email the client could be
> > implemented as a priority-inbox, spam-filter, email categorizer etc)
> >
>
>
> The bit that's not clear to me is whether this is pull or push for both the
> initial analysis and the ongoing attachment of headers.


I'm thinking RB server will adopt a pull based model in general using
Context.io. Context.io also have a feature called webhooks which  provides
rule-based notifications pushed to the app through HTTP POST requests. I
will check the possibility of push-based implementation for RB from
context.io

>From your diagram
> on the ISIS-736 ticket it looks a bit like the RB client is actually a
> gmail plugin, and then the RB server is probably the domain surfaced by
> Isis' Restful Objects viewer.
>
> The Wicket viewer would then be an "administrative" console to also
> browse/amend reputation data.
>
> I'm thinking of developing the intial version of the RB client as a webapp
to browse the reputation-data with emails. That way ISIS will also have
another DEMO web-application with email data.

The final production level client will be a gmail plugin.


> Overall... think this is doable; just not sure if the oauth2 integration is
> actually required.
>
> Thanks and more suggestions are welcome. I will draft a proposal and send
to this mail thread for your review.

Regards,
Dileepa


> Dan
>
>
> [1] http://spamassassin.apache.org/
> [2]
>
> https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png
>
>
>
>
> >
> > Please see the high-level architecture diagram attached here to get a
> > better idea of my system architecture. I can also send a SRS draft if you
> > are interested.
> > The entities I have in my mind are : email-sender, email-message,
> > reputation-profile.
> >
> > Suggestions and ideas on how I can utilize Isis framework and it's tools
> > for my application are most welcome.
> >
> > Thanks,
> > Dileepa
> >
> >
> >> In particular, I'm interested to know what the entities are, and I'm
> also
> >> interested in about the integrations between the app and the users' meil
> >> client.  For example: how does the app get hold of these emails to
> assess
> >> reputation; is it a batch import, real-time, something else; and how
> does
> >> the Isis app then add in reputation scores later (does it interact with
> >> the
> >> email server, perhaps).
> >>
> >> Thx
> >> Dan
> >>
> >>
> >>
> >>
> >> On 14 March 2014 09:49, Dileepa Jayakody <di...@gmail.com>
> >> wrote:
> >>
> >> > Hi All,
> >> >
> >> > I'm Dileepa Jayakody a MSc research student from University of
> Moratuwa,
> >> > Sri Lanka. I'm doing my research project on the topic : reputation
> >> > assessment in emails.
> >> > My project goal is to introduce reputation data as a attribute to
> >> emails,
> >> > which could be used for various applications such as spam-filtering,
> >> > priority inboxes, social-networking, etc.
> >> >
> >> > I'm planning to adopt a prototype model to develop my application.
> And I
> >> > find Apache Isis a great framework to implement such applications
> mainly
> >> > focusing on my domain model. I'm interested in the GSoC idea : build a
> >> > "real-life" app in some suitable domain, along with a semi-academic
> >> > write-up of their learning [1] and wish to seek your opinion on
> whether
> >> > implementing my project using Apache Isis can be considered a GSoC
> >> project.
> >> > I'm willing to write a paper on the project implementation,
> highlighting
> >> > the features, usage details of the framework.
> >> >
> >> > As suggested by Dan I took a look at the thesis on  "Naked Objects",
> >> > chapter 7 on the implementation comparison of CarServ (conventional vs
> >> Isis
> >> > usage). In this GSoC project idea, do you  think the student must do 2
> >> > developments; one in conventional way and another using Isis? In that
> >> case
> >> > it might be difficult for a research project such as mine.
> >> >
> >> > WDYT?
> >> >
> >> > Thanks,
> >> > Dileepa
> >> >
> >> > [1] https://issues.apache.org/jira/browse/ISIS-736
> >> >
> >>
> >
> >
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com> wrote:

>
> My current idea is that it will follow below flow of operations.
>
>  1. User authorizes ReputationBox to connect to his mailbox to read email
> (probably using OAuth2)
>

I presume that this would be actioned from within ReputationBox (RB),
ultimately by invoking an action on some domain object or service?

At present the only thing a domain action can do is to return a URL; this
is then opened up (by the browser, via Ajax) in a separate tab).

It isn't possible to set up cookies etc on this action, and I imagine that
they would be needed in order to do the oauth interaction dance.

As a first iteration, I think it'd make most sense to implement this
outside of the Isis wicket viewer (ie in a custom servlet).  This custom
servlet could use the IsisSessionTemplate class to then interact with Isis,
and shove the relevant credentials into an appropriate domain object.

>  2. ReputationBox performs an initial reputation-analysis process to
> build a reputation-index over the past emails imported as a batch. (This
> initial reputation-index will be used as the training-data to analyse new
> incoming emails)
>

Once the oauth credentials have been obtained and are held within the
Isis-managed domain, then this looks like it should probably be done
asynchronously.

You could the backgroundService + a Quartz scheduler to pick up a request
to perform the initial reputation analysis process, and have it store its
results in appropriate domain objects.

>  3. New emails are polled/ pushed to ReputationBox server and
> reputation-analysis is performed real-time to asses the reputation.
>

I prefer the idea of pushing emails to RB, but if that's the case, then why
not use it everywhere (including the initial export?)  That'd remove the
need for the oauth/custom servlet stuff in (1) above.

>  4. Email reputation data is stored as a special header in the email
> itself OR stored in a special IMAP directory in the user's mailbox (need to
> decide on the reputation data storage mechanism)
>

This also sounds like the email server should call out to the RB server.
 It'd be interesting to see how tools such as SpamAssassin [1] do this.

> 5. ReputationBox client is either a web-app OR a plugin to an existing
> web-mail client (eg :gmail) that represents the reputation data of the new
> emails (based on the reputation data in the email the client could be
> implemented as a priority-inbox, spam-filter, email categorizer etc)
>

The bit that's not clear to me is whether this is pull or push for both the
initial analysis and the ongoing attachment of headers.  From your diagram
on the ISIS-736 ticket it looks a bit like the RB client is actually a
gmail plugin, and then the RB server is probably the domain surfaced by
Isis' Restful Objects viewer.

The Wicket viewer would then be an "administrative" console to also
browse/amend reputation data.

Overall... think this is doable; just not sure if the oauth2 integration is
actually required.

Dan

[1] http://spamassassin.apache.org/
[2]
https://issues.apache.org/jira/secure/attachment/12634802/EmailReputationSystem_v2.png

>
> Please see the high-level architecture diagram attached here to get a
> better idea of my system architecture. I can also send a SRS draft if you
> are interested.
> The entities I have in my mind are : email-sender, email-message,
> reputation-profile.
>
> Suggestions and ideas on how I can utilize Isis framework and it's tools
> for my application are most welcome.
>
> Thanks,
> Dileepa
>
>
>> In particular, I'm interested to know what the entities are, and I'm also
>> interested in about the integrations between the app and the users' meil
>> client.  For example: how does the app get hold of these emails to assess
>> reputation; is it a batch import, real-time, something else; and how does
>> the Isis app then add in reputation scores later (does it interact with
>> the
>> email server, perhaps).
>>
>> Thx
>> Dan
>>
>>
>>
>>
>> On 14 March 2014 09:49, Dileepa Jayakody <di...@gmail.com>
>> wrote:
>>
>> > Hi All,
>> >
>> > I'm Dileepa Jayakody a MSc research student from University of Moratuwa,
>> > Sri Lanka. I'm doing my research project on the topic : reputation
>> > assessment in emails.
>> > My project goal is to introduce reputation data as a attribute to
>> emails,
>> > which could be used for various applications such as spam-filtering,
>> > priority inboxes, social-networking, etc.
>> >
>> > I'm planning to adopt a prototype model to develop my application. And I
>> > find Apache Isis a great framework to implement such applications mainly
>> > focusing on my domain model. I'm interested in the GSoC idea : build a
>> > "real-life" app in some suitable domain, along with a semi-academic
>> > write-up of their learning [1] and wish to seek your opinion on whether
>> > implementing my project using Apache Isis can be considered a GSoC
>> project.
>> > I'm willing to write a paper on the project implementation, highlighting
>> > the features, usage details of the framework.
>> >
>> > As suggested by Dan I took a look at the thesis on  "Naked Objects",
>> > chapter 7 on the implementation comparison of CarServ (conventional vs
>> Isis
>> > usage). In this GSoC project idea, do you  think the student must do 2
>> > developments; one in conventional way and another using Isis? In that
>> case
>> > it might be difficult for a research project such as mine.
>> >
>> > WDYT?
>> >
>> > Thanks,
>> > Dileepa
>> >
>> > [1] https://issues.apache.org/jira/browse/ISIS-736
>> >
>>
>
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Hi Dan,

I have attached the architecture diagram to the Jira.
Suggestions, ideas are welcome.

Regards,
Dileepa


On Sat, Mar 15, 2014 at 12:45 AM, Dan Haywood
<da...@haywood-associates.co.uk>wrote:

> On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com>
> wrote:
>
> >
> > Please see the high-level architecture diagram attached here to get a
> > better idea of my system architecture.
> >
>
> will respond fully later, but the ASF mailing lists strip off any
> attachments, so could you attach instead to the original JIRA ticket (or
> put elsewhere and reference by link?)
>
> Thx
> Dan
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

On 14 March 2014 11:09, Dileepa Jayakody <di...@gmail.com> wrote:

>
> Please see the high-level architecture diagram attached here to get a
> better idea of my system architecture.
>

will respond fully later, but the ASF mailing lists strip off any
attachments, so could you attach instead to the original JIRA ticket (or
put elsewhere and reference by link?)

Thx
Dan

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dileepa Jayakody <di...@gmail.com>.

Hi Dan and All,

On Fri, Mar 14, 2014 at 3:28 PM, Dan Haywood
<da...@haywood-associates.co.uk>wrote:

> Hi Dileepa,
> Thanks for posting this on dev, as requested.
>
> No, I don't see it as mandatory that this project involves a comparison
> between Isis and some other "conventional" implementation.... that can be
> another project for someone else, if need be.  But I do like the idea that
> we'll get an academic write-up of your experiences with Isis; I think
> that'd be very useful for the community.
>

Yes I'm targeting to write a research paper by July end :).

>
> Can you tell us a bit more about the domain model and architecture.. I got
> the impression that you'd done some thinking on this topic already?
>
> Yes, I have come up with an initial architecture for my application :
ReputationBox.

My current idea is that it will follow below flow of operations.

1. User authorizes ReputationBox to connect to his mailbox to read email
(probably using OAuth2)
2. ReputationBox performs an initial reputation-analysis process to build a
reputation-index over the past emails imported as a batch. (This initial
reputation-index will be used as the training-data to analyse new incoming
emails)
3. New emails are polled/ pushed to ReputationBox server and
reputation-analysis is performed real-time to asses the reputation.
4. Email reputation data is stored as a special header in the email itself
OR stored in a special IMAP directory in the user's mailbox (need to decide
on the reputation data storage mechanism)
5. ReputationBox client is either a web-app OR a plugin to an existing
web-mail client (eg :gmail) that represents the reputation data of the new
emails (based on the reputation data in the email the client could be
implemented as a priority-inbox, spam-filter, email categorizer etc)

Please see the high-level architecture diagram attached here to get a
better idea of my system architecture. I can also send a SRS draft if you
are interested.
The entities I have in my mind are : email-sender, email-message,
reputation-profile.

Suggestions and ideas on how I can utilize Isis framework and it's tools
for my application are most welcome.

Thanks,
Dileepa

> In particular, I'm interested to know what the entities are, and I'm also
> interested in about the integrations between the app and the users' meil
> client.  For example: how does the app get hold of these emails to assess
> reputation; is it a batch import, real-time, something else; and how does
> the Isis app then add in reputation scores later (does it interact with the
> email server, perhaps).
>
> Thx
> Dan
>
>
>
>
> On 14 March 2014 09:49, Dileepa Jayakody <di...@gmail.com>
> wrote:
>
> > Hi All,
> >
> > I'm Dileepa Jayakody a MSc research student from University of Moratuwa,
> > Sri Lanka. I'm doing my research project on the topic : reputation
> > assessment in emails.
> > My project goal is to introduce reputation data as a attribute to emails,
> > which could be used for various applications such as spam-filtering,
> > priority inboxes, social-networking, etc.
> >
> > I'm planning to adopt a prototype model to develop my application. And I
> > find Apache Isis a great framework to implement such applications mainly
> > focusing on my domain model. I'm interested in the GSoC idea : build a
> > "real-life" app in some suitable domain, along with a semi-academic
> > write-up of their learning [1] and wish to seek your opinion on whether
> > implementing my project using Apache Isis can be considered a GSoC
> project.
> > I'm willing to write a paper on the project implementation, highlighting
> > the features, usage details of the framework.
> >
> > As suggested by Dan I took a look at the thesis on  "Naked Objects",
> > chapter 7 on the implementation comparison of CarServ (conventional vs
> Isis
> > usage). In this GSoC project idea, do you  think the student must do 2
> > developments; one in conventional way and another using Isis? In that
> case
> > it might be difficult for a research project such as mine.
> >
> > WDYT?
> >
> > Thanks,
> > Dileepa
> >
> > [1] https://issues.apache.org/jira/browse/ISIS-736
> >
>

Re: [GSOC] Building a "real-life" app in a suitable domain using Isis framework

Posted by Dan Haywood <da...@haywood-associates.co.uk>.

Hi Dileepa,
Thanks for posting this on dev, as requested.

No, I don't see it as mandatory that this project involves a comparison
between Isis and some other "conventional" implementation.... that can be
another project for someone else, if need be.  But I do like the idea that
we'll get an academic write-up of your experiences with Isis; I think
that'd be very useful for the community.

Can you tell us a bit more about the domain model and architecture.. I got
the impression that you'd done some thinking on this topic already?

In particular, I'm interested to know what the entities are, and I'm also
interested in about the integrations between the app and the users' meil
client.  For example: how does the app get hold of these emails to assess
reputation; is it a batch import, real-time, something else; and how does
the Isis app then add in reputation scores later (does it interact with the
email server, perhaps).

Thx
Dan

On 14 March 2014 09:49, Dileepa Jayakody <di...@gmail.com> wrote:

> Hi All,
>
> I'm Dileepa Jayakody a MSc research student from University of Moratuwa,
> Sri Lanka. I'm doing my research project on the topic : reputation
> assessment in emails.
> My project goal is to introduce reputation data as a attribute to emails,
> which could be used for various applications such as spam-filtering,
> priority inboxes, social-networking, etc.
>
> I'm planning to adopt a prototype model to develop my application. And I
> find Apache Isis a great framework to implement such applications mainly
> focusing on my domain model. I'm interested in the GSoC idea : build a
> "real-life" app in some suitable domain, along with a semi-academic
> write-up of their learning [1] and wish to seek your opinion on whether
> implementing my project using Apache Isis can be considered a GSoC project.
> I'm willing to write a paper on the project implementation, highlighting
> the features, usage details of the framework.
>
> As suggested by Dan I took a look at the thesis on  "Naked Objects",
> chapter 7 on the implementation comparison of CarServ (conventional vs Isis
> usage). In this GSoC project idea, do you  think the student must do 2
> developments; one in conventional way and another using Isis? In that case
> it might be difficult for a research project such as mine.
>
> WDYT?
>
> Thanks,
> Dileepa
>
> [1] https://issues.apache.org/jira/browse/ISIS-736
>