You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Adrian Crum <ad...@hlmksw.com> on 2008/07/09 19:46:48 UTC
Discussion: Mashups
I need to add mashup capability to my local OFBiz copy. My immediate
need is a web page scraper that will gather copy counts from our copiers
and store them as meter readings.
I would like to implement this in a generic way that might be fed back
into the project. I don't want to reinvent the wheel, however. So, I'm
opening up the topic for discussion.
Is there any interest in having something like a "mashup engine" or
"mashup services" in the framework? If yes, what are some strategies we
can use? Are there any open source libraries we can leverage in an
implementation?
-Adrian
Re: Discussion: Mashups
Posted by Adrian Crum <ad...@yahoo.com>.
I didn't implement it in the trunk code, it is in my custom application. Let me find a place to put it in the trunk, then I'll create a patch and upload it to Jira.
-Adrian
--- On Sat, 7/12/08, Jacques Le Roux <ja...@les7arts.com> wrote:
> From: Jacques Le Roux <ja...@les7arts.com>
> Subject: Re: Discussion: Mashups
> To: dev@ofbiz.apache.org, adrian.crum@yahoo.com
> Date: Saturday, July 12, 2008, 11:26 AM
> Adrian,
>
> Is there a reason why you did not create a Jira issue and
> attach a patch ?
>
> Thanks
>
> Jacques
>
> From: "Adrian Crum" <ad...@yahoo.com>
> >I created a basic service to retrieve external web
> pages and I also created a sample scraper - a form
> extraction service. It all
> >seems to work well.
> >
> > I implemented these as services so they can be used
> inside screen widget <actions> elements and in
> minilang.
> >
> > I want to wait a little while before committing so I
> can get additional feedback from the community.
> >
> > The sendExternalRequest service takes the following
> arguments:
> >
> > extRequestUrl - the external URL to invoke
> > extRequestType - get, post, or empty (for REST type
> URLs)
> > extRequestParams - request parameters Map
> >
> > It returns extResponse - the web page as a String.
> >
> > The getFormParametersFromExtResponse service takes the
> following arguments:
> >
> > extResponse - The response String
> > formId - The id attribute to search for
> > formName - The form name to search for
> >
> > It returns:
> >
> > extResponse - The original response String
> > formAction - The form's action attribute
> > formMethod - The form's method attribute
> > formFields - The form's input fields Map
> >
> > The idea being you could modify the contents of the
> formFields Map and then use the returned values in another
> call to
> > sendExternalRequest.
> >
> > I picture the Mashup services working like a chain -
> you invoke the initial request, then take the response and
> pass it through
> > various scrapers and/or XML element extractors.
> >
> > Comments are welcome.
> >
> > -Adrian
> >
> > --- On Thu, 7/10/08, Peter Ford
> <pe...@gmail.com> wrote:
> >
> >> From: Peter Ford <pe...@gmail.com>
> >> Subject: Re: Discussion: Mashups
> >> To: dev@ofbiz.apache.org
> >> Date: Thursday, July 10, 2008, 1:23 AM
> >> Adrian,
> >>
> >> Looking forward to have progress on it.
> >>
> >>
> >>
> >> On Wed, Jul 9, 2008 at 4:48 PM, Jacques Le Roux
> <
> >> jacques.le.roux@les7arts.com> wrote:
> >>
> >> > From: "Adrian Crum"
> >> <ad...@hlmksw.com>
> >> >
> >> >> Here's a good tutorial -
> >> >>
> >> >>
> >>
> http://java.sun.com/developer/technicalArticles/J2EE/mashup_1/
> >> >>
> >> >
> >> > Interesting, thanks
> >> > Jacques
> >> >
> >> >
> >> > -Adrian
> >> >>
> >> >> Jacques Le Roux wrote:
> >> >>
> >> >>> Sorry Adrian,
> >> >>>
> >> >>> What would be a "mashup
> engine" or
> >> "mashup services" ? Actually what is a
> >> >>> mashup capability ?
> >> >>>
> >> >>> Jacques
> >> >>>
> >> >>> From: "Adrian Crum"
> >> <ad...@hlmksw.com>
> >> >>>
> >> >>>> I need to add mashup capability
> to my
> >> local OFBiz copy. My immediate
> >> >>>> need is a web page scraper that
> will
> >> gather copy counts from our copiers and
> >> >>>> store them as meter readings.
> >> >>>>
> >> >>>> I would like to implement this in
> a
> >> generic way that might be fed back
> >> >>>> into the project. I don't
> want to
> >> reinvent the wheel, however. So, I'm
> >> >>>> opening up the topic for
> discussion.
> >> >>>>
> >> >>>> Is there any interest in having
> something
> >> like a "mashup engine" or
> >> >>>> "mashup services" in
> the
> >> framework? If yes, what are some strategies we can
> >> >>>> use? Are there any open source
> libraries
> >> we can leverage in an
> >> >>>> implementation?
> >> >>>>
> >> >>>> -Adrian
> >> >>>>
> >> >>>>
> >> >>>
> >> >>
> >>
> >>
> >> --
> >> Peter Ford
> >> USA
> >
> >
> >
> >
Re: Discussion: Mashups
Posted by Jacques Le Roux <ja...@les7arts.com>.
Adrian,
Is there a reason why you did not create a Jira issue and attach a patch ?
Thanks
Jacques
From: "Adrian Crum" <ad...@yahoo.com>
>I created a basic service to retrieve external web pages and I also created a sample scraper - a form extraction service. It all
>seems to work well.
>
> I implemented these as services so they can be used inside screen widget <actions> elements and in minilang.
>
> I want to wait a little while before committing so I can get additional feedback from the community.
>
> The sendExternalRequest service takes the following arguments:
>
> extRequestUrl - the external URL to invoke
> extRequestType - get, post, or empty (for REST type URLs)
> extRequestParams - request parameters Map
>
> It returns extResponse - the web page as a String.
>
> The getFormParametersFromExtResponse service takes the following arguments:
>
> extResponse - The response String
> formId - The id attribute to search for
> formName - The form name to search for
>
> It returns:
>
> extResponse - The original response String
> formAction - The form's action attribute
> formMethod - The form's method attribute
> formFields - The form's input fields Map
>
> The idea being you could modify the contents of the formFields Map and then use the returned values in another call to
> sendExternalRequest.
>
> I picture the Mashup services working like a chain - you invoke the initial request, then take the response and pass it through
> various scrapers and/or XML element extractors.
>
> Comments are welcome.
>
> -Adrian
>
> --- On Thu, 7/10/08, Peter Ford <pe...@gmail.com> wrote:
>
>> From: Peter Ford <pe...@gmail.com>
>> Subject: Re: Discussion: Mashups
>> To: dev@ofbiz.apache.org
>> Date: Thursday, July 10, 2008, 1:23 AM
>> Adrian,
>>
>> Looking forward to have progress on it.
>>
>>
>>
>> On Wed, Jul 9, 2008 at 4:48 PM, Jacques Le Roux <
>> jacques.le.roux@les7arts.com> wrote:
>>
>> > From: "Adrian Crum"
>> <ad...@hlmksw.com>
>> >
>> >> Here's a good tutorial -
>> >>
>> >>
>> http://java.sun.com/developer/technicalArticles/J2EE/mashup_1/
>> >>
>> >
>> > Interesting, thanks
>> > Jacques
>> >
>> >
>> > -Adrian
>> >>
>> >> Jacques Le Roux wrote:
>> >>
>> >>> Sorry Adrian,
>> >>>
>> >>> What would be a "mashup engine" or
>> "mashup services" ? Actually what is a
>> >>> mashup capability ?
>> >>>
>> >>> Jacques
>> >>>
>> >>> From: "Adrian Crum"
>> <ad...@hlmksw.com>
>> >>>
>> >>>> I need to add mashup capability to my
>> local OFBiz copy. My immediate
>> >>>> need is a web page scraper that will
>> gather copy counts from our copiers and
>> >>>> store them as meter readings.
>> >>>>
>> >>>> I would like to implement this in a
>> generic way that might be fed back
>> >>>> into the project. I don't want to
>> reinvent the wheel, however. So, I'm
>> >>>> opening up the topic for discussion.
>> >>>>
>> >>>> Is there any interest in having something
>> like a "mashup engine" or
>> >>>> "mashup services" in the
>> framework? If yes, what are some strategies we can
>> >>>> use? Are there any open source libraries
>> we can leverage in an
>> >>>> implementation?
>> >>>>
>> >>>> -Adrian
>> >>>>
>> >>>>
>> >>>
>> >>
>>
>>
>> --
>> Peter Ford
>> USA
>
>
>
>
Re: Discussion: Mashups
Posted by Adrian Crum <ad...@yahoo.com>.
I created a basic service to retrieve external web pages and I also created a sample scraper - a form extraction service. It all seems to work well.
I implemented these as services so they can be used inside screen widget <actions> elements and in minilang.
I want to wait a little while before committing so I can get additional feedback from the community.
The sendExternalRequest service takes the following arguments:
extRequestUrl - the external URL to invoke
extRequestType - get, post, or empty (for REST type URLs)
extRequestParams - request parameters Map
It returns extResponse - the web page as a String.
The getFormParametersFromExtResponse service takes the following arguments:
extResponse - The response String
formId - The id attribute to search for
formName - The form name to search for
It returns:
extResponse - The original response String
formAction - The form's action attribute
formMethod - The form's method attribute
formFields - The form's input fields Map
The idea being you could modify the contents of the formFields Map and then use the returned values in another call to sendExternalRequest.
I picture the Mashup services working like a chain - you invoke the initial request, then take the response and pass it through various scrapers and/or XML element extractors.
Comments are welcome.
-Adrian
--- On Thu, 7/10/08, Peter Ford <pe...@gmail.com> wrote:
> From: Peter Ford <pe...@gmail.com>
> Subject: Re: Discussion: Mashups
> To: dev@ofbiz.apache.org
> Date: Thursday, July 10, 2008, 1:23 AM
> Adrian,
>
> Looking forward to have progress on it.
>
>
>
> On Wed, Jul 9, 2008 at 4:48 PM, Jacques Le Roux <
> jacques.le.roux@les7arts.com> wrote:
>
> > From: "Adrian Crum"
> <ad...@hlmksw.com>
> >
> >> Here's a good tutorial -
> >>
> >>
> http://java.sun.com/developer/technicalArticles/J2EE/mashup_1/
> >>
> >
> > Interesting, thanks
> > Jacques
> >
> >
> > -Adrian
> >>
> >> Jacques Le Roux wrote:
> >>
> >>> Sorry Adrian,
> >>>
> >>> What would be a "mashup engine" or
> "mashup services" ? Actually what is a
> >>> mashup capability ?
> >>>
> >>> Jacques
> >>>
> >>> From: "Adrian Crum"
> <ad...@hlmksw.com>
> >>>
> >>>> I need to add mashup capability to my
> local OFBiz copy. My immediate
> >>>> need is a web page scraper that will
> gather copy counts from our copiers and
> >>>> store them as meter readings.
> >>>>
> >>>> I would like to implement this in a
> generic way that might be fed back
> >>>> into the project. I don't want to
> reinvent the wheel, however. So, I'm
> >>>> opening up the topic for discussion.
> >>>>
> >>>> Is there any interest in having something
> like a "mashup engine" or
> >>>> "mashup services" in the
> framework? If yes, what are some strategies we can
> >>>> use? Are there any open source libraries
> we can leverage in an
> >>>> implementation?
> >>>>
> >>>> -Adrian
> >>>>
> >>>>
> >>>
> >>
>
>
> --
> Peter Ford
> USA
Re: Discussion: Mashups
Posted by Peter Ford <pe...@gmail.com>.
Adrian,
Looking forward to have progress on it.
On Wed, Jul 9, 2008 at 4:48 PM, Jacques Le Roux <
jacques.le.roux@les7arts.com> wrote:
> From: "Adrian Crum" <ad...@hlmksw.com>
>
>> Here's a good tutorial -
>>
>> http://java.sun.com/developer/technicalArticles/J2EE/mashup_1/
>>
>
> Interesting, thanks
> Jacques
>
>
> -Adrian
>>
>> Jacques Le Roux wrote:
>>
>>> Sorry Adrian,
>>>
>>> What would be a "mashup engine" or "mashup services" ? Actually what is a
>>> mashup capability ?
>>>
>>> Jacques
>>>
>>> From: "Adrian Crum" <ad...@hlmksw.com>
>>>
>>>> I need to add mashup capability to my local OFBiz copy. My immediate
>>>> need is a web page scraper that will gather copy counts from our copiers and
>>>> store them as meter readings.
>>>>
>>>> I would like to implement this in a generic way that might be fed back
>>>> into the project. I don't want to reinvent the wheel, however. So, I'm
>>>> opening up the topic for discussion.
>>>>
>>>> Is there any interest in having something like a "mashup engine" or
>>>> "mashup services" in the framework? If yes, what are some strategies we can
>>>> use? Are there any open source libraries we can leverage in an
>>>> implementation?
>>>>
>>>> -Adrian
>>>>
>>>>
>>>
>>
--
Peter Ford
USA
Re: Discussion: Mashups
Posted by Jacques Le Roux <ja...@les7arts.com>.
From: "Adrian Crum" <ad...@hlmksw.com>
> Here's a good tutorial -
>
> http://java.sun.com/developer/technicalArticles/J2EE/mashup_1/
Interesting, thanks
Jacques
> -Adrian
>
> Jacques Le Roux wrote:
>> Sorry Adrian,
>>
>> What would be a "mashup engine" or "mashup services" ? Actually what is
>> a mashup capability ?
>>
>> Jacques
>>
>> From: "Adrian Crum" <ad...@hlmksw.com>
>>> I need to add mashup capability to my local OFBiz copy. My immediate
>>> need is a web page scraper that will gather copy counts from our
>>> copiers and store them as meter readings.
>>>
>>> I would like to implement this in a generic way that might be fed back
>>> into the project. I don't want to reinvent the wheel, however. So, I'm
>>> opening up the topic for discussion.
>>>
>>> Is there any interest in having something like a "mashup engine" or
>>> "mashup services" in the framework? If yes, what are some strategies
>>> we can use? Are there any open source libraries we can leverage in an
>>> implementation?
>>>
>>> -Adrian
>>>
>>
>
Re: Discussion: Mashups
Posted by Adrian Crum <ad...@hlmksw.com>.
Here's a good tutorial -
http://java.sun.com/developer/technicalArticles/J2EE/mashup_1/
-Adrian
Jacques Le Roux wrote:
> Sorry Adrian,
>
> What would be a "mashup engine" or "mashup services" ? Actually what is
> a mashup capability ?
>
> Jacques
>
> From: "Adrian Crum" <ad...@hlmksw.com>
>> I need to add mashup capability to my local OFBiz copy. My immediate
>> need is a web page scraper that will gather copy counts from our
>> copiers and store them as meter readings.
>>
>> I would like to implement this in a generic way that might be fed back
>> into the project. I don't want to reinvent the wheel, however. So, I'm
>> opening up the topic for discussion.
>>
>> Is there any interest in having something like a "mashup engine" or
>> "mashup services" in the framework? If yes, what are some strategies
>> we can use? Are there any open source libraries we can leverage in an
>> implementation?
>>
>> -Adrian
>>
>
Re: Discussion: Mashups
Posted by Jacques Le Roux <ja...@les7arts.com>.
Sorry Adrian,
What would be a "mashup engine" or "mashup services" ? Actually what is a mashup capability ?
Jacques
From: "Adrian Crum" <ad...@hlmksw.com>
>I need to add mashup capability to my local OFBiz copy. My immediate
> need is a web page scraper that will gather copy counts from our copiers
> and store them as meter readings.
>
> I would like to implement this in a generic way that might be fed back
> into the project. I don't want to reinvent the wheel, however. So, I'm
> opening up the topic for discussion.
>
> Is there any interest in having something like a "mashup engine" or
> "mashup services" in the framework? If yes, what are some strategies we
> can use? Are there any open source libraries we can leverage in an
> implementation?
>
> -Adrian
>