You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Suresh Marru <sm...@apache.org> on 2013/05/01 13:21:10 UTC
Re: Airavata GSoC 2013 Master Project
Hi Shameera,
Excellent proposal, great job. Would you mind to make your proposal public and post the link here? Your proposal should help others to look at it and learn from.
Again I emphasize to all students, please don't feel you will be competing with each others. If all of you write good proposals, there is a good chance all of you will be selected. But without a good proposal, we cannot help.
Suresh
On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <sh...@gmail.com> wrote:
> Hi,
>
> Yes it is not easy to solve all problems, But defining our own standard or
> adhere to any standard
> provided by third party library will solve the problem to some extend.
>
> Here i see two possible approaches,
>
> 1. Use existing third party library(we can find which is best) adhere to it
> standard and see how we change the
> backend to be inline with it.
>
> 2. Use our own convention with help of XMLSchema (The way i suggest).
>
> As Suresh mentioned we can do a POC with both approaches to compare
> performance
> and changes need to be done in server side. Then select the best one.
>
> Another question was, can we works with graph data in JSON format.
> There are few JS graph framworks[1] which provide that functionality.
> we can use one of them to show airavata monitoring data as graphs
>
> Thanks,
> Shameera.
>
> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/> ,
> Processing.js <http://processingjs.org> , Sencha
> Charts<http://www.sencha.com/products/extjs/>
>
>
> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org> wrote:
>
>> Hi Vijeyandra,
>>
>> Airavata Messaging is based on a pub-sub model and the events themselves
>> are xml (WS-Eventing [1]).
>>
>> The Messenger paper [2] should give you more information.
>>
>> Hi All (Especially those at WS02):
>>
>> Here is an old effort from a Morotuwa undergrad project, you may want to
>> read through these papers and chat with the authors to get experiences:
>>
>> http://dl.acm.org/citation.cfm?id=1890807
>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>> http://mooshabaya.wikidot.com/
>>
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>
>> Suresh
>> [1] - http://www.w3.org/Submission/WS-Eventing/
>> [2] -
>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>
>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>> vijayendra.sdm@gmail.com> wrote:
>>
>>> Hi Suresh
>>>
>>> I wanted to know more about the monitoring tool .
>>> Currently from where does the monitoring tool gets data . Is it from
>>> workflow interpreter ? or Is it from the WS Messenger ( that might
>> continuously
>>> send messages to monitoring tool as to tell how much is the progress
>>> and what are the variables getting changed) ?
>>>
>>> Again the how is the data being exchanged. I guess it must be xml ?
>>> It must be one way data exchange . I mean the data is TO the
>>> monitoring module.
>>> Then monitoring Tool is sending back this
>>> data to Xbaya for displaying to the user ? Please correct me if I am
>> wrong
>>>
>>> I have downloaded the source code from the trunk . can you please point
>>> me which part of code should I be code at for this module.
>>>
>>> Regards
>>> Vijayendra
>>>
>>>
>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>> vijayendra.sdm@gmail.com> wrote:
>>> Hi
>>>
>>> What i am suggesting is, we send the JSON message directly to Airavata
>>> Backend(or Registry)
>>> When the message gets build after the transport phase, convert JSON
>> message
>>> to SOAP(XML).
>>> From that point message will treated as SOAP message.
>>>
>>> If we look at the JSON <--> XML conversion there are set of third party
>>> libraries we
>>> can use for. But before selecting a one we need to think about problems
>>> having
>>>
>>> with JSON <--> XML and how these libraries handle those issues. Because
>> we
>>> need a robust
>>> way to do this conversions.
>>>
>>>
>>>
>>> Shameera what you are suggesting is sending the JSON message directly to
>> Registry.
>>> when the message gets built after the transport phase , convert it to
>> SOAP .
>>>
>>> If you are suggesting Registry will have JSON data instead of WSDL ,
>> Then this might
>>> complicate the things for us .
>>> The workflow interpreter needs wsdl(xml) to interpret the workflows and
>> for other details .
>>> Which means we might again have to do some changes with workflow
>> interpretor .
>>> Yesterday from what I heard in discussion is that , they do not want to
>> mess with workflow
>>> interpreter atleast for GSOC projects.
>>>
>>> What I want to suggest is , why carry the JSON data till Regisrty .
>> Build a interface
>>> before (Apache server API) where we can do the necessary conversion
>> (JSON to SOAP).
>>> In this way we can avoid messing up with Airavata server as a whole.
>>> Client ( using a we browser) is interacting with JSON (web service) but
>> the Apache server
>>> is interacting with SOAP.
>>>
>>>
>>>
>>> Secondly yesterday Suresh was speaking about validating the connections
>> of the workflow.
>>> for example , the workflow is expecting a file as input
>>> but user is giving a sting or int .
>>>
>>> Here what I suggest is , while creating wsdl in the registry for a
>> particular
>>> workflow , we can add extra information in the form of
>>> annotation as the kind of input/ output the workflow is accepting.
>>> Then we will be able to check these against users entry during execution.
>>> Please correct me if I am wrong.
>>>
>>> Regards
>>> Vijayendra
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <su...@gmail.com>
>> wrote:
>>> Well exactly, as long as you can define standard way of communication.
>> That
>>> is, you can define in advance what should be a string, array and what
>>> should be a integer etc. We have no problem.
>>>
>>> So, when you look at problems, with JSON <-> XML or the other way round,
>>> they talk of the very general case (where you no nothing about the data
>> you
>>> are converting other than it is valid XML/JSON). There are a myriad of
>>> problems in that case, which you pointed out.
>>>
>>> But when there is standard, there is only one way of doing things, and
>> not
>>> several. I think that is the way forward. So what I am proposing is maybe
>>> we all discuss and define this standard within the first week of GSoC
>>> starting and then actually move into coding. So as long as we work with
>> the
>>> presumption that this will be done, we really dont have to worry a lot
>>> about this.
>>>
>>> Cheers,
>>> Subho.
>>>
>>>
>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>> shameerainfo@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <su...@gmail.com>
>>>> wrote:
>>>>
>>>>> Some of these problems are very specific to what the XML is
>>>> representing,
>>>>> it might not be an actual problem in Airavata,
>>>>> maybe some one more experienced with the codebase can point this out.
>>>>>
>>>>
>>>> All issues pointed out in the paper is not directly valid to our
>>>> conversion, I didn't list the issues actually need to address in this
>> case
>>>> because thought it is worth to read that introduction part which
>> explain
>>>> the all the issues we have with this conversion and give us a solid
>>>> background of that.
>>>>
>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- I
>>>> really
>>>>> dont see these as problems, as long as you can agree that all
>> parts of
>>>>> airavata will treat the JSON in a standard (probably we have to
>> define
>>>>> this) way.
>>>>>
>>>>
>>>>
>>>> The issue with JSON array only comes when we try to convert XML to
>> JSON not
>>>> the other way. If we map with JSON, inputparameters and
>> outputparameters in
>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we
>> need to
>>>> solve this issue.
>>>>
>>>> JSON XML JSON
>>>> {"inputs":["test"]} --> <inputs>test<inputs> --> {"inputs":["test"]}
>> //
>>>> correct one
>>>> --> {"inputs":"test"} // incorrect one
>>>>
>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>
>>>>> Are separate namespaces used in Airavata? Only place I can see
>> this
>>>>> being
>>>>> used is probably in the WSDL, but if we can agree on another way
>>>>> of communicating registered applications' I/O parameters to the
>> front
>>>>> end
>>>>> (JSON based), then maybe we can work around this (minor) problem.
>> Are
>>>>> custom processing instructions to the Xbaya XML parse even used?
>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>
>>>>
>>>> Yes,attributes convertion will not be a big issues we can solve it. As
>>>> Lahiru mentioned in Hangout session namesapce handling is not a big
>> issue
>>>> with Airavata.
>>>>
>>>>
>>>>
>>>>>
>>>>> <array name="abc">
>>>>> <element>1</element>
>>>>> <element>2</element>
>>>>> <element>3</element>
>>>>> <element>4</element>
>>>>> </array>
>>>>>
>>>>> Can become
>>>>>
>>>>> {
>>>>>
>>>>> abc : ['1', '2', '3', '4']
>>>>>
>>>>> }
>>>>>
>>>>
>>>> With this example it show us we need to change the XML message format
>> of
>>>> server side, which require to change the all schemas, If we are going
>> to
>>>> change the schemas then we need to change the way it process it in
>> Ariavara
>>>> core. We have dropped our initial major requirement, which is keep the
>>>> Airavata Server side as it is.
>>>>
>>>> with this conversion we only deal with json strings, yes we can send
>> JSON
>>>> request with other formats supported by JSON like boolen, null, Number
>>>> etc.. But there is no way to get the same JSON from XML as XML only
>> deal
>>>> only with Strings. I think it is good if we can consume a this features
>>>> with JSON.
>>>>
>>>> let say i need to send a integer or float to the server using JSON then
>>>> proper way is to send {"<name>":123.45} this will works fine but
>> problem is
>>>> how we get the same output ?
>>>>
>>>> Thanks,
>>>> Shameera.
>>>>
>>>>
>>>>>
>>>>>
>>>>> Cheers,
>>>>> Subho.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Shameera Rathnayaka.
>>>>
>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>
>>>
>>>
>>
>>
>
>
> --
> Best Regards,
> Shameera Rathnayaka.
>
> Blog : http://shameerarathnayaka.blogspot.com/
Re: Airavata GSoC 2013 Master Project
Posted by Shameera Rathnayaka <sh...@gmail.com>.
H
i,
yes, Dead line is too close ,It is good to put your proposal to melange
ASAP and then edit it through the melange editor.
FYI, It is not an easy task to do formatting in melange and you can't
upload images directly to melange. If you need to attach images, first you
have to upload images to another hosting area and provide a link to melange
editor. I used
http://imgur.com/
which is a free image hosting site.
Thanks,
Shameera.
On Fri, May 3, 2013 at 4:26 PM, Suresh Marru <sm...@apache.org> wrote:
> Hi Subho,
>
> You did you select airavata as the project. Also, I assume you are still
> working on the proposal, it needs work and you are getting very close to
> deadline.
>
> Suresh
> On May 2, 2013, at 12:52 PM, Subho Banerjee <su...@gmail.com> wrote:
>
> > Hi Suresh,
> > I am in the process of writing the proposal... I should be done by
> tonight.
> > Will post it on the mailing list once I am done.
> >
> > Cheers,
> > Subho
> >
> >
> > On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
> >
> >> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
> >>
> >> Suresh
> >>
> >> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> >>
> >>> Hi Vijayendra,
> >>>
> >>> As you can see from Shameera's proposal, he proposed a JSON conversion
> >> in front of WS Messenger. Also Danuska has been proposing for the AMQP
> and
> >> idea and deliberating its advantages. So given all these, I would
> suggest
> >> for you to keep focused on the UI aspects of the monitoring and write
> into
> >> your proposal a plan for determining a good strategy for the plumbing to
> >> WS-Eventing based existing system. You can have the concrete
> deliverables
> >> of new UI to change colors based on executions (as it currently does in
> >> XBaya), double click and show error messages and so forth. And keep it
> >> exploratory for the actually messaging format.
> >>>
> >>> I do not have any opinion on the libraries you mentioned, but yaa a
> ajax
> >> based pub system might be the right way to go. Pending the content
> format
> >> (JSON or WS-Eventing or JMS or AMQP or something else)
> >>>
> >>> Suresh
> >>>
> >>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> >> vijayendra.sdm@gmail.com> wrote:
> >>>
> >>>> Hi Suresh
> >>>>
> >>>> I am writing proposal for monitoring tool . The monitoring tool is
> >> based on
> >>>> pub-sub model (ws-messenger).
> >>>> While writing proposal , I have to back it by technical stuff that
> tells
> >>>> how can we achieve our purpose.
> >>>> As this monitoring tool is supposed to be a web based , and we are
> >> thinking
> >>>> in the lines of
> >>>> developing it in javascript.
> >>>> I was looking into javascript libraries that can we used with
> >> ws-messenger
> >>>> in the monitoring module.
> >>>> Please correct me if I am wrong.
> >>>> I came across some of the libraries
> >>>>
> >>>> - jQuery custom
> >>>> events<
> >> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> >>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> >>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
> >>>> - js-signals <http://millermedeiros.github.com/js-signals/>
> >>>>
> >>>> please tell me am I thinking in right direction?
> >>>>
> >>>> Regards
> >>>> Vijayendra
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
> wrote:
> >>>>
> >>>>> Hi Shameera,
> >>>>>
> >>>>> This is great, I appreciate you sharing it, I realize this is still
> >>>>> working document, but I want other students to start seeing it and
> >> model
> >>>>> their proposals in a similar way.
> >>>>>
> >>>>> Airavata Mentors,
> >>>>>
> >>>>> Please provide feedback directly on the melange site and uncheck the
> >>>>> "private" box when you comment.
> >>>>>
> >>>>> Suresh
> >>>>>
> >>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> >> shameerainfo@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Suresh and All,
> >>>>>>
> >>>>>> Of course I am very much happy to share my proposal with everybody,
> >>>>>> actually i was going to update this thread with the melange link in
> >> few
> >>>>>> hours once i have done writing all the sections in the proposal. I
> >>>>> haven't
> >>>>>> yet added the milestone plan into it and now working on it.
> >>>>>>
> >>>>>> The sub area i am going to work from the Master project is '
> >>>>> Implementing
> >>>>>> a JSON interface to Airavata Client side and Registry component'.
> >> Here is
> >>>>>> the link
> >>>>>>
> >>>>>
> >>
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> >>>>>> .
> >>>>>>
> >>>>>>
> >>>>>> Please note that i haven't completed everything in this and still
> >> doing
> >>>>>> modifications .Therefore proposal content may be changed bit, need
> to
> >> add
> >>>>>> more technical details of the approach which explains it well.
> >>>>>>
> >>>>>> I would like to know the feedback from all of you regarding the
> >> proposal
> >>>>>> and will be modifying it if there is anything to be done. Also
> please
> >>>>>> contact me if you need any help and i am very much willing to share
> my
> >>>>>> thoughts with all.
> >>>>>>
> >>>>>> Thanks!
> >>>>>> Shameera
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> >> wrote:
> >>>>>>
> >>>>>>> Hi Shameera,
> >>>>>>>
> >>>>>>> Excellent proposal, great job. Would you mind to make your
> proposal
> >>>>>>> public and post the link here? Your proposal should help others to
> >> look
> >>>>> at
> >>>>>>> it and learn from.
> >>>>>>>
> >>>>>>> Again I emphasize to all students, please don't feel you will be
> >>>>> competing
> >>>>>>> with each others. If all of you write good proposals, there is a
> good
> >>>>>>> chance all of you will be selected. But without a good proposal, we
> >>>>> cannot
> >>>>>>> help.
> >>>>>>>
> >>>>>>> Suresh
> >>>>>>>
> >>>>>>>
> >>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> >>>>> shameerainfo@gmail.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> Yes it is not easy to solve all problems, But defining our own
> >> standard
> >>>>>>> or
> >>>>>>>> adhere to any standard
> >>>>>>>> provided by third party library will solve the problem to some
> >> extend.
> >>>>>>>>
> >>>>>>>> Here i see two possible approaches,
> >>>>>>>>
> >>>>>>>> 1. Use existing third party library(we can find which is best)
> >> adhere
> >>>>> to
> >>>>>>> it
> >>>>>>>> standard and see how we change the
> >>>>>>>> backend to be inline with it.
> >>>>>>>>
> >>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
> >> suggest).
> >>>>>>>>
> >>>>>>>> As Suresh mentioned we can do a POC with both approaches to
> compare
> >>>>>>>> performance
> >>>>>>>> and changes need to be done in server side. Then select the best
> >> one.
> >>>>>>>>
> >>>>>>>> Another question was, can we works with graph data in JSON format.
> >>>>>>>> There are few JS graph framworks[1] which provide that
> >> functionality.
> >>>>>>>> we can use one of them to show airavata monitoring data as graphs
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> Shameera.
> >>>>>>>>
> >>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
> http://d3js.org/>
> >> ,
> >>>>>>>> Processing.js <http://processingjs.org> , Sencha
> >>>>>>>> Charts<http://www.sencha.com/products/extjs/>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
> >>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Vijeyandra,
> >>>>>>>>>
> >>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
> >>>>> themselves
> >>>>>>>>> are xml (WS-Eventing [1]).
> >>>>>>>>>
> >>>>>>>>> The Messenger paper [2] should give you more information.
> >>>>>>>>>
> >>>>>>>>> Hi All (Especially those at WS02):
> >>>>>>>>>
> >>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
> >> want
> >>>>> to
> >>>>>>>>> read through these papers and chat with the authors to get
> >>>>> experiences:
> >>>>>>>>>
> >>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>>>>>>>
> >> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>>>>>>>
> >> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>>>>>>> http://mooshabaya.wikidot.com/
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>>>>>>>>
> >>>>>>>>> Suresh
> >>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>>>>>>> [2] -
> >>>>>>>>>
> >>>>>>>
> >>>>>
> >>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>>>>>>>>
> >>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Suresh
> >>>>>>>>>>
> >>>>>>>>>> I wanted to know more about the monitoring tool .
> >>>>>>>>>> Currently from where does the monitoring tool gets data . Is it
> >> from
> >>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
> might
> >>>>>>>>> continuously
> >>>>>>>>>> send messages to monitoring tool as to tell how much is the
> >> progress
> >>>>>>>>>> and what are the variables getting changed) ?
> >>>>>>>>>>
> >>>>>>>>>> Again the how is the data being exchanged. I guess it must be
> xml
> >> ?
> >>>>>>>>>> It must be one way data exchange . I mean the data is TO the
> >>>>>>>>>> monitoring module.
> >>>>>>>>>> Then monitoring Tool is sending back this
> >>>>>>>>>> data to Xbaya for displaying to the user ? Please correct me if
> I
> >> am
> >>>>>>>>> wrong
> >>>>>>>>>>
> >>>>>>>>>> I have downloaded the source code from the trunk . can you
> please
> >>>>> point
> >>>>>>>>>> me which part of code should I be code at for this module.
> >>>>>>>>>>
> >>>>>>>>>> Regards
> >>>>>>>>>> Vijayendra
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>> Hi
> >>>>>>>>>>
> >>>>>>>>>> What i am suggesting is, we send the JSON message directly to
> >>>>> Airavata
> >>>>>>>>>> Backend(or Registry)
> >>>>>>>>>> When the message gets build after the transport phase, convert
> >> JSON
> >>>>>>>>> message
> >>>>>>>>>> to SOAP(XML).
> >>>>>>>>>> From that point message will treated as SOAP message.
> >>>>>>>>>>
> >>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
> third
> >>>>> party
> >>>>>>>>>> libraries we
> >>>>>>>>>> can use for. But before selecting a one we need to think about
> >>>>> problems
> >>>>>>>>>> having
> >>>>>>>>>>
> >>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
> >>>>> Because
> >>>>>>>>> we
> >>>>>>>>>> need a robust
> >>>>>>>>>> way to do this conversions.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Shameera what you are suggesting is sending the JSON message
> >> directly
> >>>>>>> to
> >>>>>>>>> Registry.
> >>>>>>>>>> when the message gets built after the transport phase , convert
> >> it to
> >>>>>>>>> SOAP .
> >>>>>>>>>>
> >>>>>>>>>> If you are suggesting Registry will have JSON data instead of
> >> WSDL ,
> >>>>>>>>> Then this might
> >>>>>>>>>> complicate the things for us .
> >>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> >> workflows
> >>>>> and
> >>>>>>>>> for other details .
> >>>>>>>>>> Which means we might again have to do some changes with workflow
> >>>>>>>>> interpretor .
> >>>>>>>>>> Yesterday from what I heard in discussion is that , they do not
> >> want
> >>>>> to
> >>>>>>>>> mess with workflow
> >>>>>>>>>> interpreter atleast for GSOC projects.
> >>>>>>>>>>
> >>>>>>>>>> What I want to suggest is , why carry the JSON data till
> >> Regisrty .
> >>>>>>>>> Build a interface
> >>>>>>>>>> before (Apache server API) where we can do the necessary
> >> conversion
> >>>>>>>>> (JSON to SOAP).
> >>>>>>>>>> In this way we can avoid messing up with Airavata server as a
> >> whole.
> >>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
> >> service)
> >>>>> but
> >>>>>>>>> the Apache server
> >>>>>>>>>> is interacting with SOAP.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
> >>>>> connections
> >>>>>>>>> of the workflow.
> >>>>>>>>>> for example , the workflow is expecting a file as input
> >>>>>>>>>> but user is giving a sting or int .
> >>>>>>>>>>
> >>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
> for a
> >>>>>>>>> particular
> >>>>>>>>>> workflow , we can add extra information in the form of
> >>>>>>>>>> annotation as the kind of input/ output the workflow is
> >> accepting.
> >>>>>>>>>> Then we will be able to check these against users entry during
> >>>>>>> execution.
> >>>>>>>>>> Please correct me if I am wrong.
> >>>>>>>>>>
> >>>>>>>>>> Regards
> >>>>>>>>>> Vijayendra
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> >> subs.zero@gmail.com
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>> Well exactly, as long as you can define standard way of
> >>>>> communication.
> >>>>>>>>> That
> >>>>>>>>>> is, you can define in advance what should be a string, array and
> >> what
> >>>>>>>>>> should be a integer etc. We have no problem.
> >>>>>>>>>>
> >>>>>>>>>> So, when you look at problems, with JSON <-> XML or the other
> way
> >>>>>>> round,
> >>>>>>>>>> they talk of the very general case (where you no nothing about
> the
> >>>>> data
> >>>>>>>>> you
> >>>>>>>>>> are converting other than it is valid XML/JSON). There are a
> >> myriad
> >>>>> of
> >>>>>>>>>> problems in that case, which you pointed out.
> >>>>>>>>>>
> >>>>>>>>>> But when there is standard, there is only one way of doing
> things,
> >>>>> and
> >>>>>>>>> not
> >>>>>>>>>> several. I think that is the way forward. So what I am proposing
> >> is
> >>>>>>> maybe
> >>>>>>>>>> we all discuss and define this standard within the first week of
> >> GSoC
> >>>>>>>>>> starting and then actually move into coding. So as long as we
> work
> >>>>> with
> >>>>>>>>> the
> >>>>>>>>>> presumption that this will be done, we really dont have to
> worry a
> >>>>> lot
> >>>>>>>>>> about this.
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Subho.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>>>>>>> shameerainfo@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> >>>>> subs.zero@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Some of these problems are very specific to what the XML is
> >>>>>>>>>>> representing,
> >>>>>>>>>>>> it might not be an actual problem in Airavata,
> >>>>>>>>>>>> maybe some one more experienced with the codebase can point
> this
> >>>>> out.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> All issues pointed out in the paper is not directly valid to
> our
> >>>>>>>>>>> conversion, I didn't list the issues actually need to address
> in
> >>>>> this
> >>>>>>>>> case
> >>>>>>>>>>> because thought it is worth to read that introduction part
> which
> >>>>>>>>> explain
> >>>>>>>>>>> the all the issues we have with this conversion and give us a
> >> solid
> >>>>>>>>>>> background of that.
> >>>>>>>>>>>
> >>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets
> --
> >> I
> >>>>>>>>>>> really
> >>>>>>>>>>>> dont see these as problems, as long as you can agree that all
> >>>>>>>>> parts of
> >>>>>>>>>>>> airavata will treat the JSON in a standard (probably we have
> to
> >>>>>>>>> define
> >>>>>>>>>>>> this) way.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> The issue with JSON array only comes when we try to convert XML
> >> to
> >>>>>>>>> JSON not
> >>>>>>>>>>> the other way. If we map with JSON, inputparameters and
> >>>>>>>>> outputparameters in
> >>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
> >> we
> >>>>>>>>> need to
> >>>>>>>>>>> solve this issue.
> >>>>>>>>>>>
> >>>>>>>>>>> JSON XML JSON
> >>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> >>>>> {"inputs":["test"]}
> >>>>>>>>> //
> >>>>>>>>>>> correct one
> >>>>>>>>>>> --> {"inputs":"test"} // incorrect
> one
> >>>>>>>>>>>
> >>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>>>>>>
> >>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
> >>>>>>>>> this
> >>>>>>>>>>>> being
> >>>>>>>>>>>> used is probably in the WSDL, but if we can agree on another
> way
> >>>>>>>>>>>> of communicating registered applications' I/O parameters to
> the
> >>>>>>>>> front
> >>>>>>>>>>>> end
> >>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> >> problem.
> >>>>>>>>> Are
> >>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
> used?
> >>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
> >> it.
> >>>>> As
> >>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
> >> big
> >>>>>>>>> issue
> >>>>>>>>>>> with Airavata.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> <array name="abc">
> >>>>>>>>>>>> <element>1</element>
> >>>>>>>>>>>> <element>2</element>
> >>>>>>>>>>>> <element>3</element>
> >>>>>>>>>>>> <element>4</element>
> >>>>>>>>>>>> </array>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Can become
> >>>>>>>>>>>>
> >>>>>>>>>>>> {
> >>>>>>>>>>>>
> >>>>>>>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>>>>>>
> >>>>>>>>>>>> }
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> With this example it show us we need to change the XML message
> >>>>> format
> >>>>>>>>> of
> >>>>>>>>>>> server side, which require to change the all schemas, If we are
> >>>>> going
> >>>>>>>>> to
> >>>>>>>>>>> change the schemas then we need to change the way it process it
> >> in
> >>>>>>>>> Ariavara
> >>>>>>>>>>> core. We have dropped our initial major requirement, which is
> >> keep
> >>>>> the
> >>>>>>>>>>> Airavata Server side as it is.
> >>>>>>>>>>>
> >>>>>>>>>>> with this conversion we only deal with json strings, yes we can
> >> send
> >>>>>>>>> JSON
> >>>>>>>>>>> request with other formats supported by JSON like boolen, null,
> >>>>> Number
> >>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
> >> only
> >>>>>>>>> deal
> >>>>>>>>>>> only with Strings. I think it is good if we can consume a this
> >>>>>>> features
> >>>>>>>>>>> with JSON.
> >>>>>>>>>>>
> >>>>>>>>>>> let say i need to send a integer or float to the server using
> >> JSON
> >>>>>>> then
> >>>>>>>>>>> proper way is to send {"<name>":123.45} this will works fine
> but
> >>>>>>>>> problem is
> >>>>>>>>>>> how we get the same output ?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Shameera.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Cheers,
> >>>>>>>>>>>> Subho.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>>>
> >>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Best Regards,
> >>>>>>>> Shameera Rathnayaka.
> >>>>>>>>
> >>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards,
> >>>>>> Shameera Rathnayaka.
> >>>>>>
> >>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>>
> >>>
> >>
> >>
>
>
--
Best Regards,
Shameera Rathnayaka.
email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/
Re: Airavata GSoC 2013 Master Project
Posted by Suresh Marru <sm...@apache.org>.
Hi Subho,
You did you select airavata as the project. Also, I assume you are still working on the proposal, it needs work and you are getting very close to deadline.
Suresh
On May 2, 2013, at 12:52 PM, Subho Banerjee <su...@gmail.com> wrote:
> Hi Suresh,
> I am in the process of writing the proposal... I should be done by tonight.
> Will post it on the mailing list once I am done.
>
> Cheers,
> Subho
>
>
> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
>
>> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>>
>> Suresh
>>
>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
>>
>>> Hi Vijayendra,
>>>
>>> As you can see from Shameera's proposal, he proposed a JSON conversion
>> in front of WS Messenger. Also Danuska has been proposing for the AMQP and
>> idea and deliberating its advantages. So given all these, I would suggest
>> for you to keep focused on the UI aspects of the monitoring and write into
>> your proposal a plan for determining a good strategy for the plumbing to
>> WS-Eventing based existing system. You can have the concrete deliverables
>> of new UI to change colors based on executions (as it currently does in
>> XBaya), double click and show error messages and so forth. And keep it
>> exploratory for the actually messaging format.
>>>
>>> I do not have any opinion on the libraries you mentioned, but yaa a ajax
>> based pub system might be the right way to go. Pending the content format
>> (JSON or WS-Eventing or JMS or AMQP or something else)
>>>
>>> Suresh
>>>
>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
>> vijayendra.sdm@gmail.com> wrote:
>>>
>>>> Hi Suresh
>>>>
>>>> I am writing proposal for monitoring tool . The monitoring tool is
>> based on
>>>> pub-sub model (ws-messenger).
>>>> While writing proposal , I have to back it by technical stuff that tells
>>>> how can we achieve our purpose.
>>>> As this monitoring tool is supposed to be a web based , and we are
>> thinking
>>>> in the lines of
>>>> developing it in javascript.
>>>> I was looking into javascript libraries that can we used with
>> ws-messenger
>>>> in the monitoring module.
>>>> Please correct me if I am wrong.
>>>> I came across some of the libraries
>>>>
>>>> - jQuery custom
>>>> events<
>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
>>>>
>>>> please tell me am I thinking in right direction?
>>>>
>>>> Regards
>>>> Vijayendra
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org> wrote:
>>>>
>>>>> Hi Shameera,
>>>>>
>>>>> This is great, I appreciate you sharing it, I realize this is still
>>>>> working document, but I want other students to start seeing it and
>> model
>>>>> their proposals in a similar way.
>>>>>
>>>>> Airavata Mentors,
>>>>>
>>>>> Please provide feedback directly on the melange site and uncheck the
>>>>> "private" box when you comment.
>>>>>
>>>>> Suresh
>>>>>
>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
>> shameerainfo@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Suresh and All,
>>>>>>
>>>>>> Of course I am very much happy to share my proposal with everybody,
>>>>>> actually i was going to update this thread with the melange link in
>> few
>>>>>> hours once i have done writing all the sections in the proposal. I
>>>>> haven't
>>>>>> yet added the milestone plan into it and now working on it.
>>>>>>
>>>>>> The sub area i am going to work from the Master project is '
>>>>> Implementing
>>>>>> a JSON interface to Airavata Client side and Registry component'.
>> Here is
>>>>>> the link
>>>>>>
>>>>>
>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
>>>>>> .
>>>>>>
>>>>>>
>>>>>> Please note that i haven't completed everything in this and still
>> doing
>>>>>> modifications .Therefore proposal content may be changed bit, need to
>> add
>>>>>> more technical details of the approach which explains it well.
>>>>>>
>>>>>> I would like to know the feedback from all of you regarding the
>> proposal
>>>>>> and will be modifying it if there is anything to be done. Also please
>>>>>> contact me if you need any help and i am very much willing to share my
>>>>>> thoughts with all.
>>>>>>
>>>>>> Thanks!
>>>>>> Shameera
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
>> wrote:
>>>>>>
>>>>>>> Hi Shameera,
>>>>>>>
>>>>>>> Excellent proposal, great job. Would you mind to make your proposal
>>>>>>> public and post the link here? Your proposal should help others to
>> look
>>>>> at
>>>>>>> it and learn from.
>>>>>>>
>>>>>>> Again I emphasize to all students, please don't feel you will be
>>>>> competing
>>>>>>> with each others. If all of you write good proposals, there is a good
>>>>>>> chance all of you will be selected. But without a good proposal, we
>>>>> cannot
>>>>>>> help.
>>>>>>>
>>>>>>> Suresh
>>>>>>>
>>>>>>>
>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
>>>>> shameerainfo@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Yes it is not easy to solve all problems, But defining our own
>> standard
>>>>>>> or
>>>>>>>> adhere to any standard
>>>>>>>> provided by third party library will solve the problem to some
>> extend.
>>>>>>>>
>>>>>>>> Here i see two possible approaches,
>>>>>>>>
>>>>>>>> 1. Use existing third party library(we can find which is best)
>> adhere
>>>>> to
>>>>>>> it
>>>>>>>> standard and see how we change the
>>>>>>>> backend to be inline with it.
>>>>>>>>
>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
>> suggest).
>>>>>>>>
>>>>>>>> As Suresh mentioned we can do a POC with both approaches to compare
>>>>>>>> performance
>>>>>>>> and changes need to be done in server side. Then select the best
>> one.
>>>>>>>>
>>>>>>>> Another question was, can we works with graph data in JSON format.
>>>>>>>> There are few JS graph framworks[1] which provide that
>> functionality.
>>>>>>>> we can use one of them to show airavata monitoring data as graphs
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Shameera.
>>>>>>>>
>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/>
>> ,
>>>>>>>> Processing.js <http://processingjs.org> , Sencha
>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Vijeyandra,
>>>>>>>>>
>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
>>>>> themselves
>>>>>>>>> are xml (WS-Eventing [1]).
>>>>>>>>>
>>>>>>>>> The Messenger paper [2] should give you more information.
>>>>>>>>>
>>>>>>>>> Hi All (Especially those at WS02):
>>>>>>>>>
>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
>> want
>>>>> to
>>>>>>>>> read through these papers and chat with the authors to get
>>>>> experiences:
>>>>>>>>>
>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>>>>>>>
>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>>>>>>>
>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>>>>>>> http://mooshabaya.wikidot.com/
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>>>>>>>>
>>>>>>>>> Suresh
>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>>>>>>> [2] -
>>>>>>>>>
>>>>>>>
>>>>>
>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>>>>>>>>
>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Suresh
>>>>>>>>>>
>>>>>>>>>> I wanted to know more about the monitoring tool .
>>>>>>>>>> Currently from where does the monitoring tool gets data . Is it
>> from
>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
>>>>>>>>> continuously
>>>>>>>>>> send messages to monitoring tool as to tell how much is the
>> progress
>>>>>>>>>> and what are the variables getting changed) ?
>>>>>>>>>>
>>>>>>>>>> Again the how is the data being exchanged. I guess it must be xml
>> ?
>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
>>>>>>>>>> monitoring module.
>>>>>>>>>> Then monitoring Tool is sending back this
>>>>>>>>>> data to Xbaya for displaying to the user ? Please correct me if I
>> am
>>>>>>>>> wrong
>>>>>>>>>>
>>>>>>>>>> I have downloaded the source code from the trunk . can you please
>>>>> point
>>>>>>>>>> me which part of code should I be code at for this module.
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Vijayendra
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>> Hi
>>>>>>>>>>
>>>>>>>>>> What i am suggesting is, we send the JSON message directly to
>>>>> Airavata
>>>>>>>>>> Backend(or Registry)
>>>>>>>>>> When the message gets build after the transport phase, convert
>> JSON
>>>>>>>>> message
>>>>>>>>>> to SOAP(XML).
>>>>>>>>>> From that point message will treated as SOAP message.
>>>>>>>>>>
>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of third
>>>>> party
>>>>>>>>>> libraries we
>>>>>>>>>> can use for. But before selecting a one we need to think about
>>>>> problems
>>>>>>>>>> having
>>>>>>>>>>
>>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
>>>>> Because
>>>>>>>>> we
>>>>>>>>>> need a robust
>>>>>>>>>> way to do this conversions.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
>> directly
>>>>>>> to
>>>>>>>>> Registry.
>>>>>>>>>> when the message gets built after the transport phase , convert
>> it to
>>>>>>>>> SOAP .
>>>>>>>>>>
>>>>>>>>>> If you are suggesting Registry will have JSON data instead of
>> WSDL ,
>>>>>>>>> Then this might
>>>>>>>>>> complicate the things for us .
>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
>> workflows
>>>>> and
>>>>>>>>> for other details .
>>>>>>>>>> Which means we might again have to do some changes with workflow
>>>>>>>>> interpretor .
>>>>>>>>>> Yesterday from what I heard in discussion is that , they do not
>> want
>>>>> to
>>>>>>>>> mess with workflow
>>>>>>>>>> interpreter atleast for GSOC projects.
>>>>>>>>>>
>>>>>>>>>> What I want to suggest is , why carry the JSON data till
>> Regisrty .
>>>>>>>>> Build a interface
>>>>>>>>>> before (Apache server API) where we can do the necessary
>> conversion
>>>>>>>>> (JSON to SOAP).
>>>>>>>>>> In this way we can avoid messing up with Airavata server as a
>> whole.
>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
>> service)
>>>>> but
>>>>>>>>> the Apache server
>>>>>>>>>> is interacting with SOAP.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
>>>>> connections
>>>>>>>>> of the workflow.
>>>>>>>>>> for example , the workflow is expecting a file as input
>>>>>>>>>> but user is giving a sting or int .
>>>>>>>>>>
>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry for a
>>>>>>>>> particular
>>>>>>>>>> workflow , we can add extra information in the form of
>>>>>>>>>> annotation as the kind of input/ output the workflow is
>> accepting.
>>>>>>>>>> Then we will be able to check these against users entry during
>>>>>>> execution.
>>>>>>>>>> Please correct me if I am wrong.
>>>>>>>>>>
>>>>>>>>>> Regards
>>>>>>>>>> Vijayendra
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
>> subs.zero@gmail.com
>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>> Well exactly, as long as you can define standard way of
>>>>> communication.
>>>>>>>>> That
>>>>>>>>>> is, you can define in advance what should be a string, array and
>> what
>>>>>>>>>> should be a integer etc. We have no problem.
>>>>>>>>>>
>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the other way
>>>>>>> round,
>>>>>>>>>> they talk of the very general case (where you no nothing about the
>>>>> data
>>>>>>>>> you
>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
>> myriad
>>>>> of
>>>>>>>>>> problems in that case, which you pointed out.
>>>>>>>>>>
>>>>>>>>>> But when there is standard, there is only one way of doing things,
>>>>> and
>>>>>>>>> not
>>>>>>>>>> several. I think that is the way forward. So what I am proposing
>> is
>>>>>>> maybe
>>>>>>>>>> we all discuss and define this standard within the first week of
>> GSoC
>>>>>>>>>> starting and then actually move into coding. So as long as we work
>>>>> with
>>>>>>>>> the
>>>>>>>>>> presumption that this will be done, we really dont have to worry a
>>>>> lot
>>>>>>>>>> about this.
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Subho.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
>>>>> subs.zero@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Some of these problems are very specific to what the XML is
>>>>>>>>>>> representing,
>>>>>>>>>>>> it might not be an actual problem in Airavata,
>>>>>>>>>>>> maybe some one more experienced with the codebase can point this
>>>>> out.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> All issues pointed out in the paper is not directly valid to our
>>>>>>>>>>> conversion, I didn't list the issues actually need to address in
>>>>> this
>>>>>>>>> case
>>>>>>>>>>> because thought it is worth to read that introduction part which
>>>>>>>>> explain
>>>>>>>>>>> the all the issues we have with this conversion and give us a
>> solid
>>>>>>>>>>> background of that.
>>>>>>>>>>>
>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets --
>> I
>>>>>>>>>>> really
>>>>>>>>>>>> dont see these as problems, as long as you can agree that all
>>>>>>>>> parts of
>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we have to
>>>>>>>>> define
>>>>>>>>>>>> this) way.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> The issue with JSON array only comes when we try to convert XML
>> to
>>>>>>>>> JSON not
>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
>>>>>>>>> outputparameters in
>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
>> we
>>>>>>>>> need to
>>>>>>>>>>> solve this issue.
>>>>>>>>>>>
>>>>>>>>>>> JSON XML JSON
>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
>>>>> {"inputs":["test"]}
>>>>>>>>> //
>>>>>>>>>>> correct one
>>>>>>>>>>> --> {"inputs":"test"} // incorrect one
>>>>>>>>>>>
>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>>>>>>
>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
>>>>>>>>> this
>>>>>>>>>>>> being
>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on another way
>>>>>>>>>>>> of communicating registered applications' I/O parameters to the
>>>>>>>>> front
>>>>>>>>>>>> end
>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
>> problem.
>>>>>>>>> Are
>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even used?
>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
>> it.
>>>>> As
>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
>> big
>>>>>>>>> issue
>>>>>>>>>>> with Airavata.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> <array name="abc">
>>>>>>>>>>>> <element>1</element>
>>>>>>>>>>>> <element>2</element>
>>>>>>>>>>>> <element>3</element>
>>>>>>>>>>>> <element>4</element>
>>>>>>>>>>>> </array>
>>>>>>>>>>>>
>>>>>>>>>>>> Can become
>>>>>>>>>>>>
>>>>>>>>>>>> {
>>>>>>>>>>>>
>>>>>>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>>>>>>
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> With this example it show us we need to change the XML message
>>>>> format
>>>>>>>>> of
>>>>>>>>>>> server side, which require to change the all schemas, If we are
>>>>> going
>>>>>>>>> to
>>>>>>>>>>> change the schemas then we need to change the way it process it
>> in
>>>>>>>>> Ariavara
>>>>>>>>>>> core. We have dropped our initial major requirement, which is
>> keep
>>>>> the
>>>>>>>>>>> Airavata Server side as it is.
>>>>>>>>>>>
>>>>>>>>>>> with this conversion we only deal with json strings, yes we can
>> send
>>>>>>>>> JSON
>>>>>>>>>>> request with other formats supported by JSON like boolen, null,
>>>>> Number
>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
>> only
>>>>>>>>> deal
>>>>>>>>>>> only with Strings. I think it is good if we can consume a this
>>>>>>> features
>>>>>>>>>>> with JSON.
>>>>>>>>>>>
>>>>>>>>>>> let say i need to send a integer or float to the server using
>> JSON
>>>>>>> then
>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works fine but
>>>>>>>>> problem is
>>>>>>>>>>> how we get the same output ?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Shameera.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>> Subho.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>>>
>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards,
>>>>>>>> Shameera Rathnayaka.
>>>>>>>>
>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Shameera Rathnayaka.
>>>>>>
>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com
>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>
>>>>>
>>>
>>
>>
Re: Airavata GSoC 2013 Master Project
Posted by Amila Jayasekara <th...@gmail.com>.
Hi All,
We are at the edge of selecting GSOC projects.
For GSOC projects that will get accepted, we would like to follow below
process;
1. Before starting the project, post how you are planning to organize your
code (svn structure), how you plan to do unit tests, how you plan to do
build integration, etc ...
2. At start of each sprint update others with the deliverables expected at
the end of that sprint. - This should be a concise list of features/bugs
that you are planning to work during the sprint. (Create Jira for each
feature/issue and specify the sprint associated with it)
3. Organize a demo and a review of the deliverables at the end of each
sprint. - During the review session we will give feedback on
implementation.
4. If the deliverables of a sprint are sound, submit a patch through a Jira
ticket. (Also please make sure your patch contains test suite to verify
your implementation).
5. Work with your mentor to get your patch into the trunk.
Please feel free share your feedback on above approach.
Thanks
Amila
On Fri, May 3, 2013 at 11:10 PM, Danushka Menikkumbura <
danushka.menikkumbura@gmail.com> wrote:
> We basically have 3 approaches here as I understand.
>
> 1. Have protocol conversion tightly coupled to target back-end. For an
> instance extend registry API to have an JSON extension.
>
> 2. Have protocol conversion tightly coupled to UI layer so that each
> individual UI component have its own implementation. For an instance if we
> decide to have a Firefox plugin UI down the line, it will have a protocol
> conversion module on its own.
>
> 3. Have a loosely coupled protocol conversion layer so that any application
> can make use of relevant bits of it as required.
>
> To me, #3 makes more sense as it minimizes duplicating the same thing in
> multiple places.
>
> Apart from protocol conversion we have to think of many other things. For
> an instance the UI object model has to be unified so that a workflow
> composed using XBaya should be able to be opened using the Web UI and vice
> versa. In order to do that the object model generation needs refactoring as
> well as I understand.
>
> So, lets discuss all that in detail if we get the green light :-).
>
> Thanks,
> Danushka
>
>
> On Fri, May 3, 2013 at 8:11 PM, Suresh Marru <sm...@apache.org> wrote:
>
> > Good points Shameera. I would incline to option 2 as well, but do not
> know
> > the feasibility. I will let you guys come up with innovative approaches.
> >
> > Suresh
> >
> > On May 3, 2013, at 1:52 AM, Shameera Rathnayaka <sh...@gmail.com>
> > wrote:
> >
> > > Hi Suresh et al.
> > >
> > > When writing the proposal, I was thinking a better approach to
> implement
> > > Airavata Server API to use with web UI. I have come across two possible
> > > ways to do this.
> > >
> > > 1. Implement existing Airavata Server API to generate the JSON
> messages.
> > > 2. Write a new Client API with JavaScript to generate and handle JSON
> > > messages .
> > >
> > > In my opinion , Option two would be the best case here. I have added
> this
> > > parts to my proposal as well and it is now completed. There are few
> > > disadvantage with option one, i have explained it there, please go
> > through
> > > it. And i would like to know about your idea on this.If you have other
> > > alternative ways to do this we can discuss and select the best one.
> > >
> > > Thanks,
> > > Shameera.
> > >
> > >
> > > On Thu, May 2, 2013 at 10:22 PM, Subho Banerjee <su...@gmail.com>
> > wrote:
> > >
> > >> Hi Suresh,
> > >> I am in the process of writing the proposal... I should be done by
> > tonight.
> > >> Will post it on the mailing list once I am done.
> > >>
> > >> Cheers,
> > >> Subho
> > >>
> > >>
> > >> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org>
> wrote:
> > >>
> > >>> Ping!! No proposals yet beyond Shameera's and Danushka's place
> holder.
> > >>>
> > >>> Suresh
> > >>>
> > >>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> > >>>
> > >>>> Hi Vijayendra,
> > >>>>
> > >>>> As you can see from Shameera's proposal, he proposed a JSON
> conversion
> > >>> in front of WS Messenger. Also Danuska has been proposing for the
> AMQP
> > >> and
> > >>> idea and deliberating its advantages. So given all these, I would
> > suggest
> > >>> for you to keep focused on the UI aspects of the monitoring and write
> > >> into
> > >>> your proposal a plan for determining a good strategy for the plumbing
> > to
> > >>> WS-Eventing based existing system. You can have the concrete
> > deliverables
> > >>> of new UI to change colors based on executions (as it currently does
> in
> > >>> XBaya), double click and show error messages and so forth. And keep
> it
> > >>> exploratory for the actually messaging format.
> > >>>>
> > >>>> I do not have any opinion on the libraries you mentioned, but yaa a
> > >> ajax
> > >>> based pub system might be the right way to go. Pending the content
> > format
> > >>> (JSON or WS-Eventing or JMS or AMQP or something else)
> > >>>>
> > >>>> Suresh
> > >>>>
> > >>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> > >>> vijayendra.sdm@gmail.com> wrote:
> > >>>>
> > >>>>> Hi Suresh
> > >>>>>
> > >>>>> I am writing proposal for monitoring tool . The monitoring tool is
> > >>> based on
> > >>>>> pub-sub model (ws-messenger).
> > >>>>> While writing proposal , I have to back it by technical stuff that
> > >> tells
> > >>>>> how can we achieve our purpose.
> > >>>>> As this monitoring tool is supposed to be a web based , and we are
> > >>> thinking
> > >>>>> in the lines of
> > >>>>> developing it in javascript.
> > >>>>> I was looking into javascript libraries that can we used with
> > >>> ws-messenger
> > >>>>> in the monitoring module.
> > >>>>> Please correct me if I am wrong.
> > >>>>> I came across some of the libraries
> > >>>>>
> > >>>>> - jQuery custom
> > >>>>> events<
> > >>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> > >>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> > >>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
> > >>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
> > >>>>>
> > >>>>> please tell me am I thinking in right direction?
> > >>>>>
> > >>>>> Regards
> > >>>>> Vijayendra
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
> > >> wrote:
> > >>>>>
> > >>>>>> Hi Shameera,
> > >>>>>>
> > >>>>>> This is great, I appreciate you sharing it, I realize this is
> still
> > >>>>>> working document, but I want other students to start seeing it and
> > >>> model
> > >>>>>> their proposals in a similar way.
> > >>>>>>
> > >>>>>> Airavata Mentors,
> > >>>>>>
> > >>>>>> Please provide feedback directly on the melange site and uncheck
> the
> > >>>>>> "private" box when you comment.
> > >>>>>>
> > >>>>>> Suresh
> > >>>>>>
> > >>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> > >>> shameerainfo@gmail.com>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hi Suresh and All,
> > >>>>>>>
> > >>>>>>> Of course I am very much happy to share my proposal with
> everybody,
> > >>>>>>> actually i was going to update this thread with the melange link
> in
> > >>> few
> > >>>>>>> hours once i have done writing all the sections in the proposal.
> I
> > >>>>>> haven't
> > >>>>>>> yet added the milestone plan into it and now working on it.
> > >>>>>>>
> > >>>>>>> The sub area i am going to work from the Master project is '
> > >>>>>> Implementing
> > >>>>>>> a JSON interface to Airavata Client side and Registry component'.
> > >>> Here is
> > >>>>>>> the link
> > >>>>>>>
> > >>>>>>
> > >>>
> > >>
> >
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> > >>>>>>> .
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Please note that i haven't completed everything in this and still
> > >>> doing
> > >>>>>>> modifications .Therefore proposal content may be changed bit,
> need
> > >> to
> > >>> add
> > >>>>>>> more technical details of the approach which explains it well.
> > >>>>>>>
> > >>>>>>> I would like to know the feedback from all of you regarding the
> > >>> proposal
> > >>>>>>> and will be modifying it if there is anything to be done. Also
> > >> please
> > >>>>>>> contact me if you need any help and i am very much willing to
> share
> > >> my
> > >>>>>>> thoughts with all.
> > >>>>>>>
> > >>>>>>> Thanks!
> > >>>>>>> Shameera
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> > >>> wrote:
> > >>>>>>>
> > >>>>>>>> Hi Shameera,
> > >>>>>>>>
> > >>>>>>>> Excellent proposal, great job. Would you mind to make your
> > >> proposal
> > >>>>>>>> public and post the link here? Your proposal should help others
> to
> > >>> look
> > >>>>>> at
> > >>>>>>>> it and learn from.
> > >>>>>>>>
> > >>>>>>>> Again I emphasize to all students, please don't feel you will be
> > >>>>>> competing
> > >>>>>>>> with each others. If all of you write good proposals, there is a
> > >> good
> > >>>>>>>> chance all of you will be selected. But without a good proposal,
> > we
> > >>>>>> cannot
> > >>>>>>>> help.
> > >>>>>>>>
> > >>>>>>>> Suresh
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> > >>>>>> shameerainfo@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> Yes it is not easy to solve all problems, But defining our own
> > >>> standard
> > >>>>>>>> or
> > >>>>>>>>> adhere to any standard
> > >>>>>>>>> provided by third party library will solve the problem to some
> > >>> extend.
> > >>>>>>>>>
> > >>>>>>>>> Here i see two possible approaches,
> > >>>>>>>>>
> > >>>>>>>>> 1. Use existing third party library(we can find which is best)
> > >>> adhere
> > >>>>>> to
> > >>>>>>>> it
> > >>>>>>>>> standard and see how we change the
> > >>>>>>>>> backend to be inline with it.
> > >>>>>>>>>
> > >>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
> > >>> suggest).
> > >>>>>>>>>
> > >>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
> > >> compare
> > >>>>>>>>> performance
> > >>>>>>>>> and changes need to be done in server side. Then select the
> best
> > >>> one.
> > >>>>>>>>>
> > >>>>>>>>> Another question was, can we works with graph data in JSON
> > format.
> > >>>>>>>>> There are few JS graph framworks[1] which provide that
> > >>> functionality.
> > >>>>>>>>> we can use one of them to show airavata monitoring data as
> graphs
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> Shameera.
> > >>>>>>>>>
> > >>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
> > >> http://d3js.org/>
> > >>> ,
> > >>>>>>>>> Processing.js <http://processingjs.org> , Sencha
> > >>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <
> smarru@apache.org
> > >
> > >>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi Vijeyandra,
> > >>>>>>>>>>
> > >>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
> > >>>>>> themselves
> > >>>>>>>>>> are xml (WS-Eventing [1]).
> > >>>>>>>>>>
> > >>>>>>>>>> The Messenger paper [2] should give you more information.
> > >>>>>>>>>>
> > >>>>>>>>>> Hi All (Especially those at WS02):
> > >>>>>>>>>>
> > >>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you
> may
> > >>> want
> > >>>>>> to
> > >>>>>>>>>> read through these papers and chat with the authors to get
> > >>>>>> experiences:
> > >>>>>>>>>>
> > >>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> > >>>>>>>>>>
> > >>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> > >>>>>>>>>>
> > >>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> > >>>>>>>>>> http://mooshabaya.wikidot.com/
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>
> > >>
> >
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> > >>>>>>>>>>
> > >>>>>>>>>> Suresh
> > >>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> > >>>>>>>>>> [2] -
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>
> > http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> > >>>>>>>>>>
> > >>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> > >>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Hi Suresh
> > >>>>>>>>>>>
> > >>>>>>>>>>> I wanted to know more about the monitoring tool .
> > >>>>>>>>>>> Currently from where does the monitoring tool gets data . Is
> it
> > >>> from
> > >>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
> > >> might
> > >>>>>>>>>> continuously
> > >>>>>>>>>>> send messages to monitoring tool as to tell how much is the
> > >>> progress
> > >>>>>>>>>>> and what are the variables getting changed) ?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Again the how is the data being exchanged. I guess it must be
> > >> xml
> > >>> ?
> > >>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
> > >>>>>>>>>>> monitoring module.
> > >>>>>>>>>>> Then monitoring Tool is sending back this
> > >>>>>>>>>>> data to Xbaya for displaying to the user ? Please correct me
> if
> > >> I
> > >>> am
> > >>>>>>>>>> wrong
> > >>>>>>>>>>>
> > >>>>>>>>>>> I have downloaded the source code from the trunk . can you
> > >> please
> > >>>>>> point
> > >>>>>>>>>>> me which part of code should I be code at for this module.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Regards
> > >>>>>>>>>>> Vijayendra
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> > >>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>>>>> Hi
> > >>>>>>>>>>>
> > >>>>>>>>>>> What i am suggesting is, we send the JSON message directly to
> > >>>>>> Airavata
> > >>>>>>>>>>> Backend(or Registry)
> > >>>>>>>>>>> When the message gets build after the transport phase,
> convert
> > >>> JSON
> > >>>>>>>>>> message
> > >>>>>>>>>>> to SOAP(XML).
> > >>>>>>>>>>> From that point message will treated as SOAP message.
> > >>>>>>>>>>>
> > >>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
> > >> third
> > >>>>>> party
> > >>>>>>>>>>> libraries we
> > >>>>>>>>>>> can use for. But before selecting a one we need to think
> about
> > >>>>>> problems
> > >>>>>>>>>>> having
> > >>>>>>>>>>>
> > >>>>>>>>>>> with JSON <--> XML and how these libraries handle those
> issues.
> > >>>>>> Because
> > >>>>>>>>>> we
> > >>>>>>>>>>> need a robust
> > >>>>>>>>>>> way to do this conversions.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
> > >>> directly
> > >>>>>>>> to
> > >>>>>>>>>> Registry.
> > >>>>>>>>>>> when the message gets built after the transport phase ,
> convert
> > >>> it to
> > >>>>>>>>>> SOAP .
> > >>>>>>>>>>>
> > >>>>>>>>>>> If you are suggesting Registry will have JSON data instead of
> > >>> WSDL ,
> > >>>>>>>>>> Then this might
> > >>>>>>>>>>> complicate the things for us .
> > >>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> > >>> workflows
> > >>>>>> and
> > >>>>>>>>>> for other details .
> > >>>>>>>>>>> Which means we might again have to do some changes with
> > workflow
> > >>>>>>>>>> interpretor .
> > >>>>>>>>>>> Yesterday from what I heard in discussion is that , they do
> not
> > >>> want
> > >>>>>> to
> > >>>>>>>>>> mess with workflow
> > >>>>>>>>>>> interpreter atleast for GSOC projects.
> > >>>>>>>>>>>
> > >>>>>>>>>>> What I want to suggest is , why carry the JSON data till
> > >>> Regisrty .
> > >>>>>>>>>> Build a interface
> > >>>>>>>>>>> before (Apache server API) where we can do the necessary
> > >>> conversion
> > >>>>>>>>>> (JSON to SOAP).
> > >>>>>>>>>>> In this way we can avoid messing up with Airavata server as a
> > >>> whole.
> > >>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
> > >>> service)
> > >>>>>> but
> > >>>>>>>>>> the Apache server
> > >>>>>>>>>>> is interacting with SOAP.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
> > >>>>>> connections
> > >>>>>>>>>> of the workflow.
> > >>>>>>>>>>> for example , the workflow is expecting a file as input
> > >>>>>>>>>>> but user is giving a sting or int .
> > >>>>>>>>>>>
> > >>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
> > >> for a
> > >>>>>>>>>> particular
> > >>>>>>>>>>> workflow , we can add extra information in the form of
> > >>>>>>>>>>> annotation as the kind of input/ output the workflow is
> > >>> accepting.
> > >>>>>>>>>>> Then we will be able to check these against users entry
> during
> > >>>>>>>> execution.
> > >>>>>>>>>>> Please correct me if I am wrong.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Regards
> > >>>>>>>>>>> Vijayendra
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> > >>> subs.zero@gmail.com
> > >>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>> Well exactly, as long as you can define standard way of
> > >>>>>> communication.
> > >>>>>>>>>> That
> > >>>>>>>>>>> is, you can define in advance what should be a string, array
> > and
> > >>> what
> > >>>>>>>>>>> should be a integer etc. We have no problem.
> > >>>>>>>>>>>
> > >>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the other
> > >> way
> > >>>>>>>> round,
> > >>>>>>>>>>> they talk of the very general case (where you no nothing
> about
> > >> the
> > >>>>>> data
> > >>>>>>>>>> you
> > >>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
> > >>> myriad
> > >>>>>> of
> > >>>>>>>>>>> problems in that case, which you pointed out.
> > >>>>>>>>>>>
> > >>>>>>>>>>> But when there is standard, there is only one way of doing
> > >> things,
> > >>>>>> and
> > >>>>>>>>>> not
> > >>>>>>>>>>> several. I think that is the way forward. So what I am
> > proposing
> > >>> is
> > >>>>>>>> maybe
> > >>>>>>>>>>> we all discuss and define this standard within the first week
> > of
> > >>> GSoC
> > >>>>>>>>>>> starting and then actually move into coding. So as long as we
> > >> work
> > >>>>>> with
> > >>>>>>>>>> the
> > >>>>>>>>>>> presumption that this will be done, we really dont have to
> > >> worry a
> > >>>>>> lot
> > >>>>>>>>>>> about this.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Cheers,
> > >>>>>>>>>>> Subho.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> > >>>>>>>>>>> shameerainfo@gmail.com> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> > >>>>>> subs.zero@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Some of these problems are very specific to what the XML is
> > >>>>>>>>>>>> representing,
> > >>>>>>>>>>>>> it might not be an actual problem in Airavata,
> > >>>>>>>>>>>>> maybe some one more experienced with the codebase can point
> > >> this
> > >>>>>> out.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> All issues pointed out in the paper is not directly valid to
> > >> our
> > >>>>>>>>>>>> conversion, I didn't list the issues actually need to
> address
> > >> in
> > >>>>>> this
> > >>>>>>>>>> case
> > >>>>>>>>>>>> because thought it is worth to read that introduction part
> > >> which
> > >>>>>>>>>> explain
> > >>>>>>>>>>>> the all the issues we have with this conversion and give us
> a
> > >>> solid
> > >>>>>>>>>>>> background of that.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character
> sets
> > >> --
> > >>> I
> > >>>>>>>>>>>> really
> > >>>>>>>>>>>>> dont see these as problems, as long as you can agree that
> all
> > >>>>>>>>>> parts of
> > >>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we
> have
> > >> to
> > >>>>>>>>>> define
> > >>>>>>>>>>>>> this) way.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> The issue with JSON array only comes when we try to convert
> > XML
> > >>> to
> > >>>>>>>>>> JSON not
> > >>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
> > >>>>>>>>>> outputparameters in
> > >>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays.
> > Therefore
> > >>> we
> > >>>>>>>>>> need to
> > >>>>>>>>>>>> solve this issue.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> JSON XML JSON
> > >>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> > >>>>>> {"inputs":["test"]}
> > >>>>>>>>>> //
> > >>>>>>>>>>>> correct one
> > >>>>>>>>>>>> --> {"inputs":"test"} // incorrect
> > >> one
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can
> > see
> > >>>>>>>>>> this
> > >>>>>>>>>>>>> being
> > >>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on
> another
> > >> way
> > >>>>>>>>>>>>> of communicating registered applications' I/O parameters to
> > >> the
> > >>>>>>>>>> front
> > >>>>>>>>>>>>> end
> > >>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> > >>> problem.
> > >>>>>>>>>> Are
> > >>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
> > >> used?
> > >>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can
> > solve
> > >>> it.
> > >>>>>> As
> > >>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is
> not
> > a
> > >>> big
> > >>>>>>>>>> issue
> > >>>>>>>>>>>> with Airavata.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> <array name="abc">
> > >>>>>>>>>>>>> <element>1</element>
> > >>>>>>>>>>>>> <element>2</element>
> > >>>>>>>>>>>>> <element>3</element>
> > >>>>>>>>>>>>> <element>4</element>
> > >>>>>>>>>>>>> </array>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Can become
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> {
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> abc : ['1', '2', '3', '4']
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> }
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> With this example it show us we need to change the XML
> message
> > >>>>>> format
> > >>>>>>>>>> of
> > >>>>>>>>>>>> server side, which require to change the all schemas, If we
> > are
> > >>>>>> going
> > >>>>>>>>>> to
> > >>>>>>>>>>>> change the schemas then we need to change the way it process
> > it
> > >>> in
> > >>>>>>>>>> Ariavara
> > >>>>>>>>>>>> core. We have dropped our initial major requirement, which
> is
> > >>> keep
> > >>>>>> the
> > >>>>>>>>>>>> Airavata Server side as it is.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> with this conversion we only deal with json strings, yes we
> > can
> > >>> send
> > >>>>>>>>>> JSON
> > >>>>>>>>>>>> request with other formats supported by JSON like boolen,
> > null,
> > >>>>>> Number
> > >>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as
> XML
> > >>> only
> > >>>>>>>>>> deal
> > >>>>>>>>>>>> only with Strings. I think it is good if we can consume a
> this
> > >>>>>>>> features
> > >>>>>>>>>>>> with JSON.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> let say i need to send a integer or float to the server
> using
> > >>> JSON
> > >>>>>>>> then
> > >>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works fine
> > >> but
> > >>>>>>>>>> problem is
> > >>>>>>>>>>>> how we get the same output ?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>> Shameera.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>> Subho.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>> Shameera Rathnayaka.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Best Regards,
> > >>>>>>>>> Shameera Rathnayaka.
> > >>>>>>>>>
> > >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Best Regards,
> > >>>>>>> Shameera Rathnayaka.
> > >>>>>>>
> > >>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> > >>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >
> > >
> > >
> > > --
> > > Best Regards,
> > > Shameera Rathnayaka.
> > >
> > > email: shameera AT apache.org , shameerainfo AT gmail.com
> > > Blog : http://shameerarathnayaka.blogspot.com/
> >
> >
>
Re: Airavata GSoC 2013 Master Project
Posted by Danushka Menikkumbura <da...@gmail.com>.
We basically have 3 approaches here as I understand.
1. Have protocol conversion tightly coupled to target back-end. For an
instance extend registry API to have an JSON extension.
2. Have protocol conversion tightly coupled to UI layer so that each
individual UI component have its own implementation. For an instance if we
decide to have a Firefox plugin UI down the line, it will have a protocol
conversion module on its own.
3. Have a loosely coupled protocol conversion layer so that any application
can make use of relevant bits of it as required.
To me, #3 makes more sense as it minimizes duplicating the same thing in
multiple places.
Apart from protocol conversion we have to think of many other things. For
an instance the UI object model has to be unified so that a workflow
composed using XBaya should be able to be opened using the Web UI and vice
versa. In order to do that the object model generation needs refactoring as
well as I understand.
So, lets discuss all that in detail if we get the green light :-).
Thanks,
Danushka
On Fri, May 3, 2013 at 8:11 PM, Suresh Marru <sm...@apache.org> wrote:
> Good points Shameera. I would incline to option 2 as well, but do not know
> the feasibility. I will let you guys come up with innovative approaches.
>
> Suresh
>
> On May 3, 2013, at 1:52 AM, Shameera Rathnayaka <sh...@gmail.com>
> wrote:
>
> > Hi Suresh et al.
> >
> > When writing the proposal, I was thinking a better approach to implement
> > Airavata Server API to use with web UI. I have come across two possible
> > ways to do this.
> >
> > 1. Implement existing Airavata Server API to generate the JSON messages.
> > 2. Write a new Client API with JavaScript to generate and handle JSON
> > messages .
> >
> > In my opinion , Option two would be the best case here. I have added this
> > parts to my proposal as well and it is now completed. There are few
> > disadvantage with option one, i have explained it there, please go
> through
> > it. And i would like to know about your idea on this.If you have other
> > alternative ways to do this we can discuss and select the best one.
> >
> > Thanks,
> > Shameera.
> >
> >
> > On Thu, May 2, 2013 at 10:22 PM, Subho Banerjee <su...@gmail.com>
> wrote:
> >
> >> Hi Suresh,
> >> I am in the process of writing the proposal... I should be done by
> tonight.
> >> Will post it on the mailing list once I am done.
> >>
> >> Cheers,
> >> Subho
> >>
> >>
> >> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
> >>
> >>> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
> >>>
> >>> Suresh
> >>>
> >>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> >>>
> >>>> Hi Vijayendra,
> >>>>
> >>>> As you can see from Shameera's proposal, he proposed a JSON conversion
> >>> in front of WS Messenger. Also Danuska has been proposing for the AMQP
> >> and
> >>> idea and deliberating its advantages. So given all these, I would
> suggest
> >>> for you to keep focused on the UI aspects of the monitoring and write
> >> into
> >>> your proposal a plan for determining a good strategy for the plumbing
> to
> >>> WS-Eventing based existing system. You can have the concrete
> deliverables
> >>> of new UI to change colors based on executions (as it currently does in
> >>> XBaya), double click and show error messages and so forth. And keep it
> >>> exploratory for the actually messaging format.
> >>>>
> >>>> I do not have any opinion on the libraries you mentioned, but yaa a
> >> ajax
> >>> based pub system might be the right way to go. Pending the content
> format
> >>> (JSON or WS-Eventing or JMS or AMQP or something else)
> >>>>
> >>>> Suresh
> >>>>
> >>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> >>> vijayendra.sdm@gmail.com> wrote:
> >>>>
> >>>>> Hi Suresh
> >>>>>
> >>>>> I am writing proposal for monitoring tool . The monitoring tool is
> >>> based on
> >>>>> pub-sub model (ws-messenger).
> >>>>> While writing proposal , I have to back it by technical stuff that
> >> tells
> >>>>> how can we achieve our purpose.
> >>>>> As this monitoring tool is supposed to be a web based , and we are
> >>> thinking
> >>>>> in the lines of
> >>>>> developing it in javascript.
> >>>>> I was looking into javascript libraries that can we used with
> >>> ws-messenger
> >>>>> in the monitoring module.
> >>>>> Please correct me if I am wrong.
> >>>>> I came across some of the libraries
> >>>>>
> >>>>> - jQuery custom
> >>>>> events<
> >>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> >>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> >>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
> >>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
> >>>>>
> >>>>> please tell me am I thinking in right direction?
> >>>>>
> >>>>> Regards
> >>>>> Vijayendra
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
> >> wrote:
> >>>>>
> >>>>>> Hi Shameera,
> >>>>>>
> >>>>>> This is great, I appreciate you sharing it, I realize this is still
> >>>>>> working document, but I want other students to start seeing it and
> >>> model
> >>>>>> their proposals in a similar way.
> >>>>>>
> >>>>>> Airavata Mentors,
> >>>>>>
> >>>>>> Please provide feedback directly on the melange site and uncheck the
> >>>>>> "private" box when you comment.
> >>>>>>
> >>>>>> Suresh
> >>>>>>
> >>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> >>> shameerainfo@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Suresh and All,
> >>>>>>>
> >>>>>>> Of course I am very much happy to share my proposal with everybody,
> >>>>>>> actually i was going to update this thread with the melange link in
> >>> few
> >>>>>>> hours once i have done writing all the sections in the proposal. I
> >>>>>> haven't
> >>>>>>> yet added the milestone plan into it and now working on it.
> >>>>>>>
> >>>>>>> The sub area i am going to work from the Master project is '
> >>>>>> Implementing
> >>>>>>> a JSON interface to Airavata Client side and Registry component'.
> >>> Here is
> >>>>>>> the link
> >>>>>>>
> >>>>>>
> >>>
> >>
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> >>>>>>> .
> >>>>>>>
> >>>>>>>
> >>>>>>> Please note that i haven't completed everything in this and still
> >>> doing
> >>>>>>> modifications .Therefore proposal content may be changed bit, need
> >> to
> >>> add
> >>>>>>> more technical details of the approach which explains it well.
> >>>>>>>
> >>>>>>> I would like to know the feedback from all of you regarding the
> >>> proposal
> >>>>>>> and will be modifying it if there is anything to be done. Also
> >> please
> >>>>>>> contact me if you need any help and i am very much willing to share
> >> my
> >>>>>>> thoughts with all.
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>> Shameera
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> >>> wrote:
> >>>>>>>
> >>>>>>>> Hi Shameera,
> >>>>>>>>
> >>>>>>>> Excellent proposal, great job. Would you mind to make your
> >> proposal
> >>>>>>>> public and post the link here? Your proposal should help others to
> >>> look
> >>>>>> at
> >>>>>>>> it and learn from.
> >>>>>>>>
> >>>>>>>> Again I emphasize to all students, please don't feel you will be
> >>>>>> competing
> >>>>>>>> with each others. If all of you write good proposals, there is a
> >> good
> >>>>>>>> chance all of you will be selected. But without a good proposal,
> we
> >>>>>> cannot
> >>>>>>>> help.
> >>>>>>>>
> >>>>>>>> Suresh
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> >>>>>> shameerainfo@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Yes it is not easy to solve all problems, But defining our own
> >>> standard
> >>>>>>>> or
> >>>>>>>>> adhere to any standard
> >>>>>>>>> provided by third party library will solve the problem to some
> >>> extend.
> >>>>>>>>>
> >>>>>>>>> Here i see two possible approaches,
> >>>>>>>>>
> >>>>>>>>> 1. Use existing third party library(we can find which is best)
> >>> adhere
> >>>>>> to
> >>>>>>>> it
> >>>>>>>>> standard and see how we change the
> >>>>>>>>> backend to be inline with it.
> >>>>>>>>>
> >>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
> >>> suggest).
> >>>>>>>>>
> >>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
> >> compare
> >>>>>>>>> performance
> >>>>>>>>> and changes need to be done in server side. Then select the best
> >>> one.
> >>>>>>>>>
> >>>>>>>>> Another question was, can we works with graph data in JSON
> format.
> >>>>>>>>> There are few JS graph framworks[1] which provide that
> >>> functionality.
> >>>>>>>>> we can use one of them to show airavata monitoring data as graphs
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Shameera.
> >>>>>>>>>
> >>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
> >> http://d3js.org/>
> >>> ,
> >>>>>>>>> Processing.js <http://processingjs.org> , Sencha
> >>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <smarru@apache.org
> >
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Vijeyandra,
> >>>>>>>>>>
> >>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
> >>>>>> themselves
> >>>>>>>>>> are xml (WS-Eventing [1]).
> >>>>>>>>>>
> >>>>>>>>>> The Messenger paper [2] should give you more information.
> >>>>>>>>>>
> >>>>>>>>>> Hi All (Especially those at WS02):
> >>>>>>>>>>
> >>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
> >>> want
> >>>>>> to
> >>>>>>>>>> read through these papers and chat with the authors to get
> >>>>>> experiences:
> >>>>>>>>>>
> >>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>>>>>>>>
> >>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>>>>>>>>
> >>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>>>>>>>> http://mooshabaya.wikidot.com/
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>
> >>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>>>>>>>>>
> >>>>>>>>>> Suresh
> >>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>>>>>>>> [2] -
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>>>>>>>>>
> >>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Suresh
> >>>>>>>>>>>
> >>>>>>>>>>> I wanted to know more about the monitoring tool .
> >>>>>>>>>>> Currently from where does the monitoring tool gets data . Is it
> >>> from
> >>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
> >> might
> >>>>>>>>>> continuously
> >>>>>>>>>>> send messages to monitoring tool as to tell how much is the
> >>> progress
> >>>>>>>>>>> and what are the variables getting changed) ?
> >>>>>>>>>>>
> >>>>>>>>>>> Again the how is the data being exchanged. I guess it must be
> >> xml
> >>> ?
> >>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
> >>>>>>>>>>> monitoring module.
> >>>>>>>>>>> Then monitoring Tool is sending back this
> >>>>>>>>>>> data to Xbaya for displaying to the user ? Please correct me if
> >> I
> >>> am
> >>>>>>>>>> wrong
> >>>>>>>>>>>
> >>>>>>>>>>> I have downloaded the source code from the trunk . can you
> >> please
> >>>>>> point
> >>>>>>>>>>> me which part of code should I be code at for this module.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards
> >>>>>>>>>>> Vijayendra
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>>> Hi
> >>>>>>>>>>>
> >>>>>>>>>>> What i am suggesting is, we send the JSON message directly to
> >>>>>> Airavata
> >>>>>>>>>>> Backend(or Registry)
> >>>>>>>>>>> When the message gets build after the transport phase, convert
> >>> JSON
> >>>>>>>>>> message
> >>>>>>>>>>> to SOAP(XML).
> >>>>>>>>>>> From that point message will treated as SOAP message.
> >>>>>>>>>>>
> >>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
> >> third
> >>>>>> party
> >>>>>>>>>>> libraries we
> >>>>>>>>>>> can use for. But before selecting a one we need to think about
> >>>>>> problems
> >>>>>>>>>>> having
> >>>>>>>>>>>
> >>>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
> >>>>>> Because
> >>>>>>>>>> we
> >>>>>>>>>>> need a robust
> >>>>>>>>>>> way to do this conversions.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
> >>> directly
> >>>>>>>> to
> >>>>>>>>>> Registry.
> >>>>>>>>>>> when the message gets built after the transport phase , convert
> >>> it to
> >>>>>>>>>> SOAP .
> >>>>>>>>>>>
> >>>>>>>>>>> If you are suggesting Registry will have JSON data instead of
> >>> WSDL ,
> >>>>>>>>>> Then this might
> >>>>>>>>>>> complicate the things for us .
> >>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> >>> workflows
> >>>>>> and
> >>>>>>>>>> for other details .
> >>>>>>>>>>> Which means we might again have to do some changes with
> workflow
> >>>>>>>>>> interpretor .
> >>>>>>>>>>> Yesterday from what I heard in discussion is that , they do not
> >>> want
> >>>>>> to
> >>>>>>>>>> mess with workflow
> >>>>>>>>>>> interpreter atleast for GSOC projects.
> >>>>>>>>>>>
> >>>>>>>>>>> What I want to suggest is , why carry the JSON data till
> >>> Regisrty .
> >>>>>>>>>> Build a interface
> >>>>>>>>>>> before (Apache server API) where we can do the necessary
> >>> conversion
> >>>>>>>>>> (JSON to SOAP).
> >>>>>>>>>>> In this way we can avoid messing up with Airavata server as a
> >>> whole.
> >>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
> >>> service)
> >>>>>> but
> >>>>>>>>>> the Apache server
> >>>>>>>>>>> is interacting with SOAP.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
> >>>>>> connections
> >>>>>>>>>> of the workflow.
> >>>>>>>>>>> for example , the workflow is expecting a file as input
> >>>>>>>>>>> but user is giving a sting or int .
> >>>>>>>>>>>
> >>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
> >> for a
> >>>>>>>>>> particular
> >>>>>>>>>>> workflow , we can add extra information in the form of
> >>>>>>>>>>> annotation as the kind of input/ output the workflow is
> >>> accepting.
> >>>>>>>>>>> Then we will be able to check these against users entry during
> >>>>>>>> execution.
> >>>>>>>>>>> Please correct me if I am wrong.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards
> >>>>>>>>>>> Vijayendra
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> >>> subs.zero@gmail.com
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>> Well exactly, as long as you can define standard way of
> >>>>>> communication.
> >>>>>>>>>> That
> >>>>>>>>>>> is, you can define in advance what should be a string, array
> and
> >>> what
> >>>>>>>>>>> should be a integer etc. We have no problem.
> >>>>>>>>>>>
> >>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the other
> >> way
> >>>>>>>> round,
> >>>>>>>>>>> they talk of the very general case (where you no nothing about
> >> the
> >>>>>> data
> >>>>>>>>>> you
> >>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
> >>> myriad
> >>>>>> of
> >>>>>>>>>>> problems in that case, which you pointed out.
> >>>>>>>>>>>
> >>>>>>>>>>> But when there is standard, there is only one way of doing
> >> things,
> >>>>>> and
> >>>>>>>>>> not
> >>>>>>>>>>> several. I think that is the way forward. So what I am
> proposing
> >>> is
> >>>>>>>> maybe
> >>>>>>>>>>> we all discuss and define this standard within the first week
> of
> >>> GSoC
> >>>>>>>>>>> starting and then actually move into coding. So as long as we
> >> work
> >>>>>> with
> >>>>>>>>>> the
> >>>>>>>>>>> presumption that this will be done, we really dont have to
> >> worry a
> >>>>>> lot
> >>>>>>>>>>> about this.
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers,
> >>>>>>>>>>> Subho.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>>>>>>>> shameerainfo@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> >>>>>> subs.zero@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Some of these problems are very specific to what the XML is
> >>>>>>>>>>>> representing,
> >>>>>>>>>>>>> it might not be an actual problem in Airavata,
> >>>>>>>>>>>>> maybe some one more experienced with the codebase can point
> >> this
> >>>>>> out.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> All issues pointed out in the paper is not directly valid to
> >> our
> >>>>>>>>>>>> conversion, I didn't list the issues actually need to address
> >> in
> >>>>>> this
> >>>>>>>>>> case
> >>>>>>>>>>>> because thought it is worth to read that introduction part
> >> which
> >>>>>>>>>> explain
> >>>>>>>>>>>> the all the issues we have with this conversion and give us a
> >>> solid
> >>>>>>>>>>>> background of that.
> >>>>>>>>>>>>
> >>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets
> >> --
> >>> I
> >>>>>>>>>>>> really
> >>>>>>>>>>>>> dont see these as problems, as long as you can agree that all
> >>>>>>>>>> parts of
> >>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we have
> >> to
> >>>>>>>>>> define
> >>>>>>>>>>>>> this) way.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> The issue with JSON array only comes when we try to convert
> XML
> >>> to
> >>>>>>>>>> JSON not
> >>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
> >>>>>>>>>> outputparameters in
> >>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays.
> Therefore
> >>> we
> >>>>>>>>>> need to
> >>>>>>>>>>>> solve this issue.
> >>>>>>>>>>>>
> >>>>>>>>>>>> JSON XML JSON
> >>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> >>>>>> {"inputs":["test"]}
> >>>>>>>>>> //
> >>>>>>>>>>>> correct one
> >>>>>>>>>>>> --> {"inputs":"test"} // incorrect
> >> one
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can
> see
> >>>>>>>>>> this
> >>>>>>>>>>>>> being
> >>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on another
> >> way
> >>>>>>>>>>>>> of communicating registered applications' I/O parameters to
> >> the
> >>>>>>>>>> front
> >>>>>>>>>>>>> end
> >>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> >>> problem.
> >>>>>>>>>> Are
> >>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
> >> used?
> >>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can
> solve
> >>> it.
> >>>>>> As
> >>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not
> a
> >>> big
> >>>>>>>>>> issue
> >>>>>>>>>>>> with Airavata.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> <array name="abc">
> >>>>>>>>>>>>> <element>1</element>
> >>>>>>>>>>>>> <element>2</element>
> >>>>>>>>>>>>> <element>3</element>
> >>>>>>>>>>>>> <element>4</element>
> >>>>>>>>>>>>> </array>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Can become
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> {
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> }
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> With this example it show us we need to change the XML message
> >>>>>> format
> >>>>>>>>>> of
> >>>>>>>>>>>> server side, which require to change the all schemas, If we
> are
> >>>>>> going
> >>>>>>>>>> to
> >>>>>>>>>>>> change the schemas then we need to change the way it process
> it
> >>> in
> >>>>>>>>>> Ariavara
> >>>>>>>>>>>> core. We have dropped our initial major requirement, which is
> >>> keep
> >>>>>> the
> >>>>>>>>>>>> Airavata Server side as it is.
> >>>>>>>>>>>>
> >>>>>>>>>>>> with this conversion we only deal with json strings, yes we
> can
> >>> send
> >>>>>>>>>> JSON
> >>>>>>>>>>>> request with other formats supported by JSON like boolen,
> null,
> >>>>>> Number
> >>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
> >>> only
> >>>>>>>>>> deal
> >>>>>>>>>>>> only with Strings. I think it is good if we can consume a this
> >>>>>>>> features
> >>>>>>>>>>>> with JSON.
> >>>>>>>>>>>>
> >>>>>>>>>>>> let say i need to send a integer or float to the server using
> >>> JSON
> >>>>>>>> then
> >>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works fine
> >> but
> >>>>>>>>>> problem is
> >>>>>>>>>>>> how we get the same output ?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Shameera.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>> Subho.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best Regards,
> >>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>
> >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Best Regards,
> >>>>>>> Shameera Rathnayaka.
> >>>>>>>
> >>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> >>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Best Regards,
> > Shameera Rathnayaka.
> >
> > email: shameera AT apache.org , shameerainfo AT gmail.com
> > Blog : http://shameerarathnayaka.blogspot.com/
>
>
Re: Airavata GSoC 2013 Master Project
Posted by Suresh Marru <sm...@apache.org>.
Good points Shameera. I would incline to option 2 as well, but do not know the feasibility. I will let you guys come up with innovative approaches.
Suresh
On May 3, 2013, at 1:52 AM, Shameera Rathnayaka <sh...@gmail.com> wrote:
> Hi Suresh et al.
>
> When writing the proposal, I was thinking a better approach to implement
> Airavata Server API to use with web UI. I have come across two possible
> ways to do this.
>
> 1. Implement existing Airavata Server API to generate the JSON messages.
> 2. Write a new Client API with JavaScript to generate and handle JSON
> messages .
>
> In my opinion , Option two would be the best case here. I have added this
> parts to my proposal as well and it is now completed. There are few
> disadvantage with option one, i have explained it there, please go through
> it. And i would like to know about your idea on this.If you have other
> alternative ways to do this we can discuss and select the best one.
>
> Thanks,
> Shameera.
>
>
> On Thu, May 2, 2013 at 10:22 PM, Subho Banerjee <su...@gmail.com> wrote:
>
>> Hi Suresh,
>> I am in the process of writing the proposal... I should be done by tonight.
>> Will post it on the mailing list once I am done.
>>
>> Cheers,
>> Subho
>>
>>
>> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
>>
>>> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>>>
>>> Suresh
>>>
>>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
>>>
>>>> Hi Vijayendra,
>>>>
>>>> As you can see from Shameera's proposal, he proposed a JSON conversion
>>> in front of WS Messenger. Also Danuska has been proposing for the AMQP
>> and
>>> idea and deliberating its advantages. So given all these, I would suggest
>>> for you to keep focused on the UI aspects of the monitoring and write
>> into
>>> your proposal a plan for determining a good strategy for the plumbing to
>>> WS-Eventing based existing system. You can have the concrete deliverables
>>> of new UI to change colors based on executions (as it currently does in
>>> XBaya), double click and show error messages and so forth. And keep it
>>> exploratory for the actually messaging format.
>>>>
>>>> I do not have any opinion on the libraries you mentioned, but yaa a
>> ajax
>>> based pub system might be the right way to go. Pending the content format
>>> (JSON or WS-Eventing or JMS or AMQP or something else)
>>>>
>>>> Suresh
>>>>
>>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
>>> vijayendra.sdm@gmail.com> wrote:
>>>>
>>>>> Hi Suresh
>>>>>
>>>>> I am writing proposal for monitoring tool . The monitoring tool is
>>> based on
>>>>> pub-sub model (ws-messenger).
>>>>> While writing proposal , I have to back it by technical stuff that
>> tells
>>>>> how can we achieve our purpose.
>>>>> As this monitoring tool is supposed to be a web based , and we are
>>> thinking
>>>>> in the lines of
>>>>> developing it in javascript.
>>>>> I was looking into javascript libraries that can we used with
>>> ws-messenger
>>>>> in the monitoring module.
>>>>> Please correct me if I am wrong.
>>>>> I came across some of the libraries
>>>>>
>>>>> - jQuery custom
>>>>> events<
>>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
>>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
>>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
>>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
>>>>>
>>>>> please tell me am I thinking in right direction?
>>>>>
>>>>> Regards
>>>>> Vijayendra
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
>> wrote:
>>>>>
>>>>>> Hi Shameera,
>>>>>>
>>>>>> This is great, I appreciate you sharing it, I realize this is still
>>>>>> working document, but I want other students to start seeing it and
>>> model
>>>>>> their proposals in a similar way.
>>>>>>
>>>>>> Airavata Mentors,
>>>>>>
>>>>>> Please provide feedback directly on the melange site and uncheck the
>>>>>> "private" box when you comment.
>>>>>>
>>>>>> Suresh
>>>>>>
>>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
>>> shameerainfo@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Suresh and All,
>>>>>>>
>>>>>>> Of course I am very much happy to share my proposal with everybody,
>>>>>>> actually i was going to update this thread with the melange link in
>>> few
>>>>>>> hours once i have done writing all the sections in the proposal. I
>>>>>> haven't
>>>>>>> yet added the milestone plan into it and now working on it.
>>>>>>>
>>>>>>> The sub area i am going to work from the Master project is '
>>>>>> Implementing
>>>>>>> a JSON interface to Airavata Client side and Registry component'.
>>> Here is
>>>>>>> the link
>>>>>>>
>>>>>>
>>>
>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>> Please note that i haven't completed everything in this and still
>>> doing
>>>>>>> modifications .Therefore proposal content may be changed bit, need
>> to
>>> add
>>>>>>> more technical details of the approach which explains it well.
>>>>>>>
>>>>>>> I would like to know the feedback from all of you regarding the
>>> proposal
>>>>>>> and will be modifying it if there is anything to be done. Also
>> please
>>>>>>> contact me if you need any help and i am very much willing to share
>> my
>>>>>>> thoughts with all.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Shameera
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
>>> wrote:
>>>>>>>
>>>>>>>> Hi Shameera,
>>>>>>>>
>>>>>>>> Excellent proposal, great job. Would you mind to make your
>> proposal
>>>>>>>> public and post the link here? Your proposal should help others to
>>> look
>>>>>> at
>>>>>>>> it and learn from.
>>>>>>>>
>>>>>>>> Again I emphasize to all students, please don't feel you will be
>>>>>> competing
>>>>>>>> with each others. If all of you write good proposals, there is a
>> good
>>>>>>>> chance all of you will be selected. But without a good proposal, we
>>>>>> cannot
>>>>>>>> help.
>>>>>>>>
>>>>>>>> Suresh
>>>>>>>>
>>>>>>>>
>>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
>>>>>> shameerainfo@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Yes it is not easy to solve all problems, But defining our own
>>> standard
>>>>>>>> or
>>>>>>>>> adhere to any standard
>>>>>>>>> provided by third party library will solve the problem to some
>>> extend.
>>>>>>>>>
>>>>>>>>> Here i see two possible approaches,
>>>>>>>>>
>>>>>>>>> 1. Use existing third party library(we can find which is best)
>>> adhere
>>>>>> to
>>>>>>>> it
>>>>>>>>> standard and see how we change the
>>>>>>>>> backend to be inline with it.
>>>>>>>>>
>>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
>>> suggest).
>>>>>>>>>
>>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
>> compare
>>>>>>>>> performance
>>>>>>>>> and changes need to be done in server side. Then select the best
>>> one.
>>>>>>>>>
>>>>>>>>> Another question was, can we works with graph data in JSON format.
>>>>>>>>> There are few JS graph framworks[1] which provide that
>>> functionality.
>>>>>>>>> we can use one of them to show airavata monitoring data as graphs
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Shameera.
>>>>>>>>>
>>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
>> http://d3js.org/>
>>> ,
>>>>>>>>> Processing.js <http://processingjs.org> , Sencha
>>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Vijeyandra,
>>>>>>>>>>
>>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
>>>>>> themselves
>>>>>>>>>> are xml (WS-Eventing [1]).
>>>>>>>>>>
>>>>>>>>>> The Messenger paper [2] should give you more information.
>>>>>>>>>>
>>>>>>>>>> Hi All (Especially those at WS02):
>>>>>>>>>>
>>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
>>> want
>>>>>> to
>>>>>>>>>> read through these papers and chat with the authors to get
>>>>>> experiences:
>>>>>>>>>>
>>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>>>>>>>>
>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>>>>>>>>
>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>>>>>>>> http://mooshabaya.wikidot.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>>>>>>>>>
>>>>>>>>>> Suresh
>>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>>>>>>>> [2] -
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>>>>>>>>>
>>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Suresh
>>>>>>>>>>>
>>>>>>>>>>> I wanted to know more about the monitoring tool .
>>>>>>>>>>> Currently from where does the monitoring tool gets data . Is it
>>> from
>>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
>> might
>>>>>>>>>> continuously
>>>>>>>>>>> send messages to monitoring tool as to tell how much is the
>>> progress
>>>>>>>>>>> and what are the variables getting changed) ?
>>>>>>>>>>>
>>>>>>>>>>> Again the how is the data being exchanged. I guess it must be
>> xml
>>> ?
>>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
>>>>>>>>>>> monitoring module.
>>>>>>>>>>> Then monitoring Tool is sending back this
>>>>>>>>>>> data to Xbaya for displaying to the user ? Please correct me if
>> I
>>> am
>>>>>>>>>> wrong
>>>>>>>>>>>
>>>>>>>>>>> I have downloaded the source code from the trunk . can you
>> please
>>>>>> point
>>>>>>>>>>> me which part of code should I be code at for this module.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Vijayendra
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>> What i am suggesting is, we send the JSON message directly to
>>>>>> Airavata
>>>>>>>>>>> Backend(or Registry)
>>>>>>>>>>> When the message gets build after the transport phase, convert
>>> JSON
>>>>>>>>>> message
>>>>>>>>>>> to SOAP(XML).
>>>>>>>>>>> From that point message will treated as SOAP message.
>>>>>>>>>>>
>>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
>> third
>>>>>> party
>>>>>>>>>>> libraries we
>>>>>>>>>>> can use for. But before selecting a one we need to think about
>>>>>> problems
>>>>>>>>>>> having
>>>>>>>>>>>
>>>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
>>>>>> Because
>>>>>>>>>> we
>>>>>>>>>>> need a robust
>>>>>>>>>>> way to do this conversions.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
>>> directly
>>>>>>>> to
>>>>>>>>>> Registry.
>>>>>>>>>>> when the message gets built after the transport phase , convert
>>> it to
>>>>>>>>>> SOAP .
>>>>>>>>>>>
>>>>>>>>>>> If you are suggesting Registry will have JSON data instead of
>>> WSDL ,
>>>>>>>>>> Then this might
>>>>>>>>>>> complicate the things for us .
>>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
>>> workflows
>>>>>> and
>>>>>>>>>> for other details .
>>>>>>>>>>> Which means we might again have to do some changes with workflow
>>>>>>>>>> interpretor .
>>>>>>>>>>> Yesterday from what I heard in discussion is that , they do not
>>> want
>>>>>> to
>>>>>>>>>> mess with workflow
>>>>>>>>>>> interpreter atleast for GSOC projects.
>>>>>>>>>>>
>>>>>>>>>>> What I want to suggest is , why carry the JSON data till
>>> Regisrty .
>>>>>>>>>> Build a interface
>>>>>>>>>>> before (Apache server API) where we can do the necessary
>>> conversion
>>>>>>>>>> (JSON to SOAP).
>>>>>>>>>>> In this way we can avoid messing up with Airavata server as a
>>> whole.
>>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
>>> service)
>>>>>> but
>>>>>>>>>> the Apache server
>>>>>>>>>>> is interacting with SOAP.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
>>>>>> connections
>>>>>>>>>> of the workflow.
>>>>>>>>>>> for example , the workflow is expecting a file as input
>>>>>>>>>>> but user is giving a sting or int .
>>>>>>>>>>>
>>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
>> for a
>>>>>>>>>> particular
>>>>>>>>>>> workflow , we can add extra information in the form of
>>>>>>>>>>> annotation as the kind of input/ output the workflow is
>>> accepting.
>>>>>>>>>>> Then we will be able to check these against users entry during
>>>>>>>> execution.
>>>>>>>>>>> Please correct me if I am wrong.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Vijayendra
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
>>> subs.zero@gmail.com
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>> Well exactly, as long as you can define standard way of
>>>>>> communication.
>>>>>>>>>> That
>>>>>>>>>>> is, you can define in advance what should be a string, array and
>>> what
>>>>>>>>>>> should be a integer etc. We have no problem.
>>>>>>>>>>>
>>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the other
>> way
>>>>>>>> round,
>>>>>>>>>>> they talk of the very general case (where you no nothing about
>> the
>>>>>> data
>>>>>>>>>> you
>>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
>>> myriad
>>>>>> of
>>>>>>>>>>> problems in that case, which you pointed out.
>>>>>>>>>>>
>>>>>>>>>>> But when there is standard, there is only one way of doing
>> things,
>>>>>> and
>>>>>>>>>> not
>>>>>>>>>>> several. I think that is the way forward. So what I am proposing
>>> is
>>>>>>>> maybe
>>>>>>>>>>> we all discuss and define this standard within the first week of
>>> GSoC
>>>>>>>>>>> starting and then actually move into coding. So as long as we
>> work
>>>>>> with
>>>>>>>>>> the
>>>>>>>>>>> presumption that this will be done, we really dont have to
>> worry a
>>>>>> lot
>>>>>>>>>>> about this.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Subho.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
>>>>>> subs.zero@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Some of these problems are very specific to what the XML is
>>>>>>>>>>>> representing,
>>>>>>>>>>>>> it might not be an actual problem in Airavata,
>>>>>>>>>>>>> maybe some one more experienced with the codebase can point
>> this
>>>>>> out.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> All issues pointed out in the paper is not directly valid to
>> our
>>>>>>>>>>>> conversion, I didn't list the issues actually need to address
>> in
>>>>>> this
>>>>>>>>>> case
>>>>>>>>>>>> because thought it is worth to read that introduction part
>> which
>>>>>>>>>> explain
>>>>>>>>>>>> the all the issues we have with this conversion and give us a
>>> solid
>>>>>>>>>>>> background of that.
>>>>>>>>>>>>
>>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets
>> --
>>> I
>>>>>>>>>>>> really
>>>>>>>>>>>>> dont see these as problems, as long as you can agree that all
>>>>>>>>>> parts of
>>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we have
>> to
>>>>>>>>>> define
>>>>>>>>>>>>> this) way.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The issue with JSON array only comes when we try to convert XML
>>> to
>>>>>>>>>> JSON not
>>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
>>>>>>>>>> outputparameters in
>>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
>>> we
>>>>>>>>>> need to
>>>>>>>>>>>> solve this issue.
>>>>>>>>>>>>
>>>>>>>>>>>> JSON XML JSON
>>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
>>>>>> {"inputs":["test"]}
>>>>>>>>>> //
>>>>>>>>>>>> correct one
>>>>>>>>>>>> --> {"inputs":"test"} // incorrect
>> one
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>>>>>>>
>>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
>>>>>>>>>> this
>>>>>>>>>>>>> being
>>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on another
>> way
>>>>>>>>>>>>> of communicating registered applications' I/O parameters to
>> the
>>>>>>>>>> front
>>>>>>>>>>>>> end
>>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
>>> problem.
>>>>>>>>>> Are
>>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
>> used?
>>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
>>> it.
>>>>>> As
>>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
>>> big
>>>>>>>>>> issue
>>>>>>>>>>>> with Airavata.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> <array name="abc">
>>>>>>>>>>>>> <element>1</element>
>>>>>>>>>>>>> <element>2</element>
>>>>>>>>>>>>> <element>3</element>
>>>>>>>>>>>>> <element>4</element>
>>>>>>>>>>>>> </array>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can become
>>>>>>>>>>>>>
>>>>>>>>>>>>> {
>>>>>>>>>>>>>
>>>>>>>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> With this example it show us we need to change the XML message
>>>>>> format
>>>>>>>>>> of
>>>>>>>>>>>> server side, which require to change the all schemas, If we are
>>>>>> going
>>>>>>>>>> to
>>>>>>>>>>>> change the schemas then we need to change the way it process it
>>> in
>>>>>>>>>> Ariavara
>>>>>>>>>>>> core. We have dropped our initial major requirement, which is
>>> keep
>>>>>> the
>>>>>>>>>>>> Airavata Server side as it is.
>>>>>>>>>>>>
>>>>>>>>>>>> with this conversion we only deal with json strings, yes we can
>>> send
>>>>>>>>>> JSON
>>>>>>>>>>>> request with other formats supported by JSON like boolen, null,
>>>>>> Number
>>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
>>> only
>>>>>>>>>> deal
>>>>>>>>>>>> only with Strings. I think it is good if we can consume a this
>>>>>>>> features
>>>>>>>>>>>> with JSON.
>>>>>>>>>>>>
>>>>>>>>>>>> let say i need to send a integer or float to the server using
>>> JSON
>>>>>>>> then
>>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works fine
>> but
>>>>>>>>>> problem is
>>>>>>>>>>>> how we get the same output ?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Subho.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>>>>
>>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>
>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Shameera Rathnayaka.
>>>>>>>
>>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com
>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>
>>>>>>
>>>>
>>>
>>>
>>
>
>
>
> --
> Best Regards,
> Shameera Rathnayaka.
>
> email: shameera AT apache.org , shameerainfo AT gmail.com
> Blog : http://shameerarathnayaka.blogspot.com/
Re: Airavata GSoC 2013 Master Project
Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi Suresh et al.
When writing the proposal, I was thinking a better approach to implement
Airavata Server API to use with web UI. I have come across two possible
ways to do this.
1. Implement existing Airavata Server API to generate the JSON messages.
2. Write a new Client API with JavaScript to generate and handle JSON
messages .
In my opinion , Option two would be the best case here. I have added this
parts to my proposal as well and it is now completed. There are few
disadvantage with option one, i have explained it there, please go through
it. And i would like to know about your idea on this.If you have other
alternative ways to do this we can discuss and select the best one.
Thanks,
Shameera.
On Thu, May 2, 2013 at 10:22 PM, Subho Banerjee <su...@gmail.com> wrote:
> Hi Suresh,
> I am in the process of writing the proposal... I should be done by tonight.
> Will post it on the mailing list once I am done.
>
> Cheers,
> Subho
>
>
> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
>
> > Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
> >
> > Suresh
> >
> > On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> >
> > > Hi Vijayendra,
> > >
> > > As you can see from Shameera's proposal, he proposed a JSON conversion
> > in front of WS Messenger. Also Danuska has been proposing for the AMQP
> and
> > idea and deliberating its advantages. So given all these, I would suggest
> > for you to keep focused on the UI aspects of the monitoring and write
> into
> > your proposal a plan for determining a good strategy for the plumbing to
> > WS-Eventing based existing system. You can have the concrete deliverables
> > of new UI to change colors based on executions (as it currently does in
> > XBaya), double click and show error messages and so forth. And keep it
> > exploratory for the actually messaging format.
> > >
> > > I do not have any opinion on the libraries you mentioned, but yaa a
> ajax
> > based pub system might be the right way to go. Pending the content format
> > (JSON or WS-Eventing or JMS or AMQP or something else)
> > >
> > > Suresh
> > >
> > > On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> > vijayendra.sdm@gmail.com> wrote:
> > >
> > >> Hi Suresh
> > >>
> > >> I am writing proposal for monitoring tool . The monitoring tool is
> > based on
> > >> pub-sub model (ws-messenger).
> > >> While writing proposal , I have to back it by technical stuff that
> tells
> > >> how can we achieve our purpose.
> > >> As this monitoring tool is supposed to be a web based , and we are
> > thinking
> > >> in the lines of
> > >> developing it in javascript.
> > >> I was looking into javascript libraries that can we used with
> > ws-messenger
> > >> in the monitoring module.
> > >> Please correct me if I am wrong.
> > >> I came across some of the libraries
> > >>
> > >> - jQuery custom
> > >> events<
> > http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> > >> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> > >> - PubSubJS <https://github.com/mroderick/PubSubJS>
> > >> - js-signals <http://millermedeiros.github.com/js-signals/>
> > >>
> > >> please tell me am I thinking in right direction?
> > >>
> > >> Regards
> > >> Vijayendra
> > >>
> > >>
> > >>
> > >>
> > >> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
> wrote:
> > >>
> > >>> Hi Shameera,
> > >>>
> > >>> This is great, I appreciate you sharing it, I realize this is still
> > >>> working document, but I want other students to start seeing it and
> > model
> > >>> their proposals in a similar way.
> > >>>
> > >>> Airavata Mentors,
> > >>>
> > >>> Please provide feedback directly on the melange site and uncheck the
> > >>> "private" box when you comment.
> > >>>
> > >>> Suresh
> > >>>
> > >>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> > shameerainfo@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Suresh and All,
> > >>>>
> > >>>> Of course I am very much happy to share my proposal with everybody,
> > >>>> actually i was going to update this thread with the melange link in
> > few
> > >>>> hours once i have done writing all the sections in the proposal. I
> > >>> haven't
> > >>>> yet added the milestone plan into it and now working on it.
> > >>>>
> > >>>> The sub area i am going to work from the Master project is '
> > >>> Implementing
> > >>>> a JSON interface to Airavata Client side and Registry component'.
> > Here is
> > >>>> the link
> > >>>>
> > >>>
> >
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> > >>>> .
> > >>>>
> > >>>>
> > >>>> Please note that i haven't completed everything in this and still
> > doing
> > >>>> modifications .Therefore proposal content may be changed bit, need
> to
> > add
> > >>>> more technical details of the approach which explains it well.
> > >>>>
> > >>>> I would like to know the feedback from all of you regarding the
> > proposal
> > >>>> and will be modifying it if there is anything to be done. Also
> please
> > >>>> contact me if you need any help and i am very much willing to share
> my
> > >>>> thoughts with all.
> > >>>>
> > >>>> Thanks!
> > >>>> Shameera
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> > wrote:
> > >>>>
> > >>>>> Hi Shameera,
> > >>>>>
> > >>>>> Excellent proposal, great job. Would you mind to make your
> proposal
> > >>>>> public and post the link here? Your proposal should help others to
> > look
> > >>> at
> > >>>>> it and learn from.
> > >>>>>
> > >>>>> Again I emphasize to all students, please don't feel you will be
> > >>> competing
> > >>>>> with each others. If all of you write good proposals, there is a
> good
> > >>>>> chance all of you will be selected. But without a good proposal, we
> > >>> cannot
> > >>>>> help.
> > >>>>>
> > >>>>> Suresh
> > >>>>>
> > >>>>>
> > >>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> > >>> shameerainfo@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> Yes it is not easy to solve all problems, But defining our own
> > standard
> > >>>>> or
> > >>>>>> adhere to any standard
> > >>>>>> provided by third party library will solve the problem to some
> > extend.
> > >>>>>>
> > >>>>>> Here i see two possible approaches,
> > >>>>>>
> > >>>>>> 1. Use existing third party library(we can find which is best)
> > adhere
> > >>> to
> > >>>>> it
> > >>>>>> standard and see how we change the
> > >>>>>> backend to be inline with it.
> > >>>>>>
> > >>>>>> 2. Use our own convention with help of XMLSchema (The way i
> > suggest).
> > >>>>>>
> > >>>>>> As Suresh mentioned we can do a POC with both approaches to
> compare
> > >>>>>> performance
> > >>>>>> and changes need to be done in server side. Then select the best
> > one.
> > >>>>>>
> > >>>>>> Another question was, can we works with graph data in JSON format.
> > >>>>>> There are few JS graph framworks[1] which provide that
> > functionality.
> > >>>>>> we can use one of them to show airavata monitoring data as graphs
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Shameera.
> > >>>>>>
> > >>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
> http://d3js.org/>
> > ,
> > >>>>>> Processing.js <http://processingjs.org> , Sencha
> > >>>>>> Charts<http://www.sencha.com/products/extjs/>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> Hi Vijeyandra,
> > >>>>>>>
> > >>>>>>> Airavata Messaging is based on a pub-sub model and the events
> > >>> themselves
> > >>>>>>> are xml (WS-Eventing [1]).
> > >>>>>>>
> > >>>>>>> The Messenger paper [2] should give you more information.
> > >>>>>>>
> > >>>>>>> Hi All (Especially those at WS02):
> > >>>>>>>
> > >>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
> > want
> > >>> to
> > >>>>>>> read through these papers and chat with the authors to get
> > >>> experiences:
> > >>>>>>>
> > >>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> > >>>>>>>
> > http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> > >>>>>>>
> > http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> > >>>>>>> http://mooshabaya.wikidot.com/
> > >>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> > >>>>>>>
> > >>>>>>> Suresh
> > >>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> > >>>>>>> [2] -
> > >>>>>>>
> > >>>>>
> > >>>
> > http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> > >>>>>>>
> > >>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> > >>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> Hi Suresh
> > >>>>>>>>
> > >>>>>>>> I wanted to know more about the monitoring tool .
> > >>>>>>>> Currently from where does the monitoring tool gets data . Is it
> > from
> > >>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
> might
> > >>>>>>> continuously
> > >>>>>>>> send messages to monitoring tool as to tell how much is the
> > progress
> > >>>>>>>> and what are the variables getting changed) ?
> > >>>>>>>>
> > >>>>>>>> Again the how is the data being exchanged. I guess it must be
> xml
> > ?
> > >>>>>>>> It must be one way data exchange . I mean the data is TO the
> > >>>>>>>> monitoring module.
> > >>>>>>>> Then monitoring Tool is sending back this
> > >>>>>>>> data to Xbaya for displaying to the user ? Please correct me if
> I
> > am
> > >>>>>>> wrong
> > >>>>>>>>
> > >>>>>>>> I have downloaded the source code from the trunk . can you
> please
> > >>> point
> > >>>>>>>> me which part of code should I be code at for this module.
> > >>>>>>>>
> > >>>>>>>> Regards
> > >>>>>>>> Vijayendra
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> > >>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>> Hi
> > >>>>>>>>
> > >>>>>>>> What i am suggesting is, we send the JSON message directly to
> > >>> Airavata
> > >>>>>>>> Backend(or Registry)
> > >>>>>>>> When the message gets build after the transport phase, convert
> > JSON
> > >>>>>>> message
> > >>>>>>>> to SOAP(XML).
> > >>>>>>>> From that point message will treated as SOAP message.
> > >>>>>>>>
> > >>>>>>>> If we look at the JSON <--> XML conversion there are set of
> third
> > >>> party
> > >>>>>>>> libraries we
> > >>>>>>>> can use for. But before selecting a one we need to think about
> > >>> problems
> > >>>>>>>> having
> > >>>>>>>>
> > >>>>>>>> with JSON <--> XML and how these libraries handle those issues.
> > >>> Because
> > >>>>>>> we
> > >>>>>>>> need a robust
> > >>>>>>>> way to do this conversions.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Shameera what you are suggesting is sending the JSON message
> > directly
> > >>>>> to
> > >>>>>>> Registry.
> > >>>>>>>> when the message gets built after the transport phase , convert
> > it to
> > >>>>>>> SOAP .
> > >>>>>>>>
> > >>>>>>>> If you are suggesting Registry will have JSON data instead of
> > WSDL ,
> > >>>>>>> Then this might
> > >>>>>>>> complicate the things for us .
> > >>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> > workflows
> > >>> and
> > >>>>>>> for other details .
> > >>>>>>>> Which means we might again have to do some changes with workflow
> > >>>>>>> interpretor .
> > >>>>>>>> Yesterday from what I heard in discussion is that , they do not
> > want
> > >>> to
> > >>>>>>> mess with workflow
> > >>>>>>>> interpreter atleast for GSOC projects.
> > >>>>>>>>
> > >>>>>>>> What I want to suggest is , why carry the JSON data till
> > Regisrty .
> > >>>>>>> Build a interface
> > >>>>>>>> before (Apache server API) where we can do the necessary
> > conversion
> > >>>>>>> (JSON to SOAP).
> > >>>>>>>> In this way we can avoid messing up with Airavata server as a
> > whole.
> > >>>>>>>> Client ( using a we browser) is interacting with JSON (web
> > service)
> > >>> but
> > >>>>>>> the Apache server
> > >>>>>>>> is interacting with SOAP.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Secondly yesterday Suresh was speaking about validating the
> > >>> connections
> > >>>>>>> of the workflow.
> > >>>>>>>> for example , the workflow is expecting a file as input
> > >>>>>>>> but user is giving a sting or int .
> > >>>>>>>>
> > >>>>>>>> Here what I suggest is , while creating wsdl in the registry
> for a
> > >>>>>>> particular
> > >>>>>>>> workflow , we can add extra information in the form of
> > >>>>>>>> annotation as the kind of input/ output the workflow is
> > accepting.
> > >>>>>>>> Then we will be able to check these against users entry during
> > >>>>> execution.
> > >>>>>>>> Please correct me if I am wrong.
> > >>>>>>>>
> > >>>>>>>> Regards
> > >>>>>>>> Vijayendra
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> > subs.zero@gmail.com
> > >>>>
> > >>>>>>> wrote:
> > >>>>>>>> Well exactly, as long as you can define standard way of
> > >>> communication.
> > >>>>>>> That
> > >>>>>>>> is, you can define in advance what should be a string, array and
> > what
> > >>>>>>>> should be a integer etc. We have no problem.
> > >>>>>>>>
> > >>>>>>>> So, when you look at problems, with JSON <-> XML or the other
> way
> > >>>>> round,
> > >>>>>>>> they talk of the very general case (where you no nothing about
> the
> > >>> data
> > >>>>>>> you
> > >>>>>>>> are converting other than it is valid XML/JSON). There are a
> > myriad
> > >>> of
> > >>>>>>>> problems in that case, which you pointed out.
> > >>>>>>>>
> > >>>>>>>> But when there is standard, there is only one way of doing
> things,
> > >>> and
> > >>>>>>> not
> > >>>>>>>> several. I think that is the way forward. So what I am proposing
> > is
> > >>>>> maybe
> > >>>>>>>> we all discuss and define this standard within the first week of
> > GSoC
> > >>>>>>>> starting and then actually move into coding. So as long as we
> work
> > >>> with
> > >>>>>>> the
> > >>>>>>>> presumption that this will be done, we really dont have to
> worry a
> > >>> lot
> > >>>>>>>> about this.
> > >>>>>>>>
> > >>>>>>>> Cheers,
> > >>>>>>>> Subho.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> > >>>>>>>> shameerainfo@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> > >>> subs.zero@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Some of these problems are very specific to what the XML is
> > >>>>>>>>> representing,
> > >>>>>>>>>> it might not be an actual problem in Airavata,
> > >>>>>>>>>> maybe some one more experienced with the codebase can point
> this
> > >>> out.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> All issues pointed out in the paper is not directly valid to
> our
> > >>>>>>>>> conversion, I didn't list the issues actually need to address
> in
> > >>> this
> > >>>>>>> case
> > >>>>>>>>> because thought it is worth to read that introduction part
> which
> > >>>>>>> explain
> > >>>>>>>>> the all the issues we have with this conversion and give us a
> > solid
> > >>>>>>>>> background of that.
> > >>>>>>>>>
> > >>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets
> --
> > I
> > >>>>>>>>> really
> > >>>>>>>>>> dont see these as problems, as long as you can agree that all
> > >>>>>>> parts of
> > >>>>>>>>>> airavata will treat the JSON in a standard (probably we have
> to
> > >>>>>>> define
> > >>>>>>>>>> this) way.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> The issue with JSON array only comes when we try to convert XML
> > to
> > >>>>>>> JSON not
> > >>>>>>>>> the other way. If we map with JSON, inputparameters and
> > >>>>>>> outputparameters in
> > >>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
> > we
> > >>>>>>> need to
> > >>>>>>>>> solve this issue.
> > >>>>>>>>>
> > >>>>>>>>> JSON XML JSON
> > >>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> > >>> {"inputs":["test"]}
> > >>>>>>> //
> > >>>>>>>>> correct one
> > >>>>>>>>> --> {"inputs":"test"} // incorrect
> one
> > >>>>>>>>>
> > >>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> > >>>>>>>>>
> > >>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
> > >>>>>>> this
> > >>>>>>>>>> being
> > >>>>>>>>>> used is probably in the WSDL, but if we can agree on another
> way
> > >>>>>>>>>> of communicating registered applications' I/O parameters to
> the
> > >>>>>>> front
> > >>>>>>>>>> end
> > >>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> > problem.
> > >>>>>>> Are
> > >>>>>>>>>> custom processing instructions to the Xbaya XML parse even
> used?
> > >>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
> > it.
> > >>> As
> > >>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
> > big
> > >>>>>>> issue
> > >>>>>>>>> with Airavata.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> <array name="abc">
> > >>>>>>>>>> <element>1</element>
> > >>>>>>>>>> <element>2</element>
> > >>>>>>>>>> <element>3</element>
> > >>>>>>>>>> <element>4</element>
> > >>>>>>>>>> </array>
> > >>>>>>>>>>
> > >>>>>>>>>> Can become
> > >>>>>>>>>>
> > >>>>>>>>>> {
> > >>>>>>>>>>
> > >>>>>>>>>> abc : ['1', '2', '3', '4']
> > >>>>>>>>>>
> > >>>>>>>>>> }
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> With this example it show us we need to change the XML message
> > >>> format
> > >>>>>>> of
> > >>>>>>>>> server side, which require to change the all schemas, If we are
> > >>> going
> > >>>>>>> to
> > >>>>>>>>> change the schemas then we need to change the way it process it
> > in
> > >>>>>>> Ariavara
> > >>>>>>>>> core. We have dropped our initial major requirement, which is
> > keep
> > >>> the
> > >>>>>>>>> Airavata Server side as it is.
> > >>>>>>>>>
> > >>>>>>>>> with this conversion we only deal with json strings, yes we can
> > send
> > >>>>>>> JSON
> > >>>>>>>>> request with other formats supported by JSON like boolen, null,
> > >>> Number
> > >>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
> > only
> > >>>>>>> deal
> > >>>>>>>>> only with Strings. I think it is good if we can consume a this
> > >>>>> features
> > >>>>>>>>> with JSON.
> > >>>>>>>>>
> > >>>>>>>>> let say i need to send a integer or float to the server using
> > JSON
> > >>>>> then
> > >>>>>>>>> proper way is to send {"<name>":123.45} this will works fine
> but
> > >>>>>>> problem is
> > >>>>>>>>> how we get the same output ?
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> Shameera.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Cheers,
> > >>>>>>>>>> Subho.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Best Regards,
> > >>>>>>>>> Shameera Rathnayaka.
> > >>>>>>>>>
> > >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Best Regards,
> > >>>>>> Shameera Rathnayaka.
> > >>>>>>
> > >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Best Regards,
> > >>>> Shameera Rathnayaka.
> > >>>>
> > >>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> > >>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>
> > >>>
> > >
> >
> >
>
--
Best Regards,
Shameera Rathnayaka.
email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/
Re: Airavata GSoC 2013 Master Project
Posted by Subho Banerjee <su...@gmail.com>.
Hi Suresh,
I am in the process of writing the proposal... I should be done by tonight.
Will post it on the mailing list once I am done.
Cheers,
Subho
On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>
> Suresh
>
> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
>
> > Hi Vijayendra,
> >
> > As you can see from Shameera's proposal, he proposed a JSON conversion
> in front of WS Messenger. Also Danuska has been proposing for the AMQP and
> idea and deliberating its advantages. So given all these, I would suggest
> for you to keep focused on the UI aspects of the monitoring and write into
> your proposal a plan for determining a good strategy for the plumbing to
> WS-Eventing based existing system. You can have the concrete deliverables
> of new UI to change colors based on executions (as it currently does in
> XBaya), double click and show error messages and so forth. And keep it
> exploratory for the actually messaging format.
> >
> > I do not have any opinion on the libraries you mentioned, but yaa a ajax
> based pub system might be the right way to go. Pending the content format
> (JSON or WS-Eventing or JMS or AMQP or something else)
> >
> > Suresh
> >
> > On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> vijayendra.sdm@gmail.com> wrote:
> >
> >> Hi Suresh
> >>
> >> I am writing proposal for monitoring tool . The monitoring tool is
> based on
> >> pub-sub model (ws-messenger).
> >> While writing proposal , I have to back it by technical stuff that tells
> >> how can we achieve our purpose.
> >> As this monitoring tool is supposed to be a web based , and we are
> thinking
> >> in the lines of
> >> developing it in javascript.
> >> I was looking into javascript libraries that can we used with
> ws-messenger
> >> in the monitoring module.
> >> Please correct me if I am wrong.
> >> I came across some of the libraries
> >>
> >> - jQuery custom
> >> events<
> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> >> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> >> - PubSubJS <https://github.com/mroderick/PubSubJS>
> >> - js-signals <http://millermedeiros.github.com/js-signals/>
> >>
> >> please tell me am I thinking in right direction?
> >>
> >> Regards
> >> Vijayendra
> >>
> >>
> >>
> >>
> >> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org> wrote:
> >>
> >>> Hi Shameera,
> >>>
> >>> This is great, I appreciate you sharing it, I realize this is still
> >>> working document, but I want other students to start seeing it and
> model
> >>> their proposals in a similar way.
> >>>
> >>> Airavata Mentors,
> >>>
> >>> Please provide feedback directly on the melange site and uncheck the
> >>> "private" box when you comment.
> >>>
> >>> Suresh
> >>>
> >>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> shameerainfo@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi Suresh and All,
> >>>>
> >>>> Of course I am very much happy to share my proposal with everybody,
> >>>> actually i was going to update this thread with the melange link in
> few
> >>>> hours once i have done writing all the sections in the proposal. I
> >>> haven't
> >>>> yet added the milestone plan into it and now working on it.
> >>>>
> >>>> The sub area i am going to work from the Master project is '
> >>> Implementing
> >>>> a JSON interface to Airavata Client side and Registry component'.
> Here is
> >>>> the link
> >>>>
> >>>
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> >>>> .
> >>>>
> >>>>
> >>>> Please note that i haven't completed everything in this and still
> doing
> >>>> modifications .Therefore proposal content may be changed bit, need to
> add
> >>>> more technical details of the approach which explains it well.
> >>>>
> >>>> I would like to know the feedback from all of you regarding the
> proposal
> >>>> and will be modifying it if there is anything to be done. Also please
> >>>> contact me if you need any help and i am very much willing to share my
> >>>> thoughts with all.
> >>>>
> >>>> Thanks!
> >>>> Shameera
> >>>>
> >>>>
> >>>>
> >>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> wrote:
> >>>>
> >>>>> Hi Shameera,
> >>>>>
> >>>>> Excellent proposal, great job. Would you mind to make your proposal
> >>>>> public and post the link here? Your proposal should help others to
> look
> >>> at
> >>>>> it and learn from.
> >>>>>
> >>>>> Again I emphasize to all students, please don't feel you will be
> >>> competing
> >>>>> with each others. If all of you write good proposals, there is a good
> >>>>> chance all of you will be selected. But without a good proposal, we
> >>> cannot
> >>>>> help.
> >>>>>
> >>>>> Suresh
> >>>>>
> >>>>>
> >>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> >>> shameerainfo@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> Yes it is not easy to solve all problems, But defining our own
> standard
> >>>>> or
> >>>>>> adhere to any standard
> >>>>>> provided by third party library will solve the problem to some
> extend.
> >>>>>>
> >>>>>> Here i see two possible approaches,
> >>>>>>
> >>>>>> 1. Use existing third party library(we can find which is best)
> adhere
> >>> to
> >>>>> it
> >>>>>> standard and see how we change the
> >>>>>> backend to be inline with it.
> >>>>>>
> >>>>>> 2. Use our own convention with help of XMLSchema (The way i
> suggest).
> >>>>>>
> >>>>>> As Suresh mentioned we can do a POC with both approaches to compare
> >>>>>> performance
> >>>>>> and changes need to be done in server side. Then select the best
> one.
> >>>>>>
> >>>>>> Another question was, can we works with graph data in JSON format.
> >>>>>> There are few JS graph framworks[1] which provide that
> functionality.
> >>>>>> we can use one of them to show airavata monitoring data as graphs
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Shameera.
> >>>>>>
> >>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/>
> ,
> >>>>>> Processing.js <http://processingjs.org> , Sencha
> >>>>>> Charts<http://www.sencha.com/products/extjs/>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>>> Hi Vijeyandra,
> >>>>>>>
> >>>>>>> Airavata Messaging is based on a pub-sub model and the events
> >>> themselves
> >>>>>>> are xml (WS-Eventing [1]).
> >>>>>>>
> >>>>>>> The Messenger paper [2] should give you more information.
> >>>>>>>
> >>>>>>> Hi All (Especially those at WS02):
> >>>>>>>
> >>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
> want
> >>> to
> >>>>>>> read through these papers and chat with the authors to get
> >>> experiences:
> >>>>>>>
> >>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>>>>>
> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>>>>>
> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>>>>> http://mooshabaya.wikidot.com/
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>>>>>>
> >>>>>>> Suresh
> >>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>>>>> [2] -
> >>>>>>>
> >>>>>
> >>>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>>>>>>
> >>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Hi Suresh
> >>>>>>>>
> >>>>>>>> I wanted to know more about the monitoring tool .
> >>>>>>>> Currently from where does the monitoring tool gets data . Is it
> from
> >>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
> >>>>>>> continuously
> >>>>>>>> send messages to monitoring tool as to tell how much is the
> progress
> >>>>>>>> and what are the variables getting changed) ?
> >>>>>>>>
> >>>>>>>> Again the how is the data being exchanged. I guess it must be xml
> ?
> >>>>>>>> It must be one way data exchange . I mean the data is TO the
> >>>>>>>> monitoring module.
> >>>>>>>> Then monitoring Tool is sending back this
> >>>>>>>> data to Xbaya for displaying to the user ? Please correct me if I
> am
> >>>>>>> wrong
> >>>>>>>>
> >>>>>>>> I have downloaded the source code from the trunk . can you please
> >>> point
> >>>>>>>> me which part of code should I be code at for this module.
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Vijayendra
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>> Hi
> >>>>>>>>
> >>>>>>>> What i am suggesting is, we send the JSON message directly to
> >>> Airavata
> >>>>>>>> Backend(or Registry)
> >>>>>>>> When the message gets build after the transport phase, convert
> JSON
> >>>>>>> message
> >>>>>>>> to SOAP(XML).
> >>>>>>>> From that point message will treated as SOAP message.
> >>>>>>>>
> >>>>>>>> If we look at the JSON <--> XML conversion there are set of third
> >>> party
> >>>>>>>> libraries we
> >>>>>>>> can use for. But before selecting a one we need to think about
> >>> problems
> >>>>>>>> having
> >>>>>>>>
> >>>>>>>> with JSON <--> XML and how these libraries handle those issues.
> >>> Because
> >>>>>>> we
> >>>>>>>> need a robust
> >>>>>>>> way to do this conversions.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Shameera what you are suggesting is sending the JSON message
> directly
> >>>>> to
> >>>>>>> Registry.
> >>>>>>>> when the message gets built after the transport phase , convert
> it to
> >>>>>>> SOAP .
> >>>>>>>>
> >>>>>>>> If you are suggesting Registry will have JSON data instead of
> WSDL ,
> >>>>>>> Then this might
> >>>>>>>> complicate the things for us .
> >>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> workflows
> >>> and
> >>>>>>> for other details .
> >>>>>>>> Which means we might again have to do some changes with workflow
> >>>>>>> interpretor .
> >>>>>>>> Yesterday from what I heard in discussion is that , they do not
> want
> >>> to
> >>>>>>> mess with workflow
> >>>>>>>> interpreter atleast for GSOC projects.
> >>>>>>>>
> >>>>>>>> What I want to suggest is , why carry the JSON data till
> Regisrty .
> >>>>>>> Build a interface
> >>>>>>>> before (Apache server API) where we can do the necessary
> conversion
> >>>>>>> (JSON to SOAP).
> >>>>>>>> In this way we can avoid messing up with Airavata server as a
> whole.
> >>>>>>>> Client ( using a we browser) is interacting with JSON (web
> service)
> >>> but
> >>>>>>> the Apache server
> >>>>>>>> is interacting with SOAP.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Secondly yesterday Suresh was speaking about validating the
> >>> connections
> >>>>>>> of the workflow.
> >>>>>>>> for example , the workflow is expecting a file as input
> >>>>>>>> but user is giving a sting or int .
> >>>>>>>>
> >>>>>>>> Here what I suggest is , while creating wsdl in the registry for a
> >>>>>>> particular
> >>>>>>>> workflow , we can add extra information in the form of
> >>>>>>>> annotation as the kind of input/ output the workflow is
> accepting.
> >>>>>>>> Then we will be able to check these against users entry during
> >>>>> execution.
> >>>>>>>> Please correct me if I am wrong.
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Vijayendra
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> subs.zero@gmail.com
> >>>>
> >>>>>>> wrote:
> >>>>>>>> Well exactly, as long as you can define standard way of
> >>> communication.
> >>>>>>> That
> >>>>>>>> is, you can define in advance what should be a string, array and
> what
> >>>>>>>> should be a integer etc. We have no problem.
> >>>>>>>>
> >>>>>>>> So, when you look at problems, with JSON <-> XML or the other way
> >>>>> round,
> >>>>>>>> they talk of the very general case (where you no nothing about the
> >>> data
> >>>>>>> you
> >>>>>>>> are converting other than it is valid XML/JSON). There are a
> myriad
> >>> of
> >>>>>>>> problems in that case, which you pointed out.
> >>>>>>>>
> >>>>>>>> But when there is standard, there is only one way of doing things,
> >>> and
> >>>>>>> not
> >>>>>>>> several. I think that is the way forward. So what I am proposing
> is
> >>>>> maybe
> >>>>>>>> we all discuss and define this standard within the first week of
> GSoC
> >>>>>>>> starting and then actually move into coding. So as long as we work
> >>> with
> >>>>>>> the
> >>>>>>>> presumption that this will be done, we really dont have to worry a
> >>> lot
> >>>>>>>> about this.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Subho.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>>>>> shameerainfo@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> >>> subs.zero@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Some of these problems are very specific to what the XML is
> >>>>>>>>> representing,
> >>>>>>>>>> it might not be an actual problem in Airavata,
> >>>>>>>>>> maybe some one more experienced with the codebase can point this
> >>> out.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> All issues pointed out in the paper is not directly valid to our
> >>>>>>>>> conversion, I didn't list the issues actually need to address in
> >>> this
> >>>>>>> case
> >>>>>>>>> because thought it is worth to read that introduction part which
> >>>>>>> explain
> >>>>>>>>> the all the issues we have with this conversion and give us a
> solid
> >>>>>>>>> background of that.
> >>>>>>>>>
> >>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets --
> I
> >>>>>>>>> really
> >>>>>>>>>> dont see these as problems, as long as you can agree that all
> >>>>>>> parts of
> >>>>>>>>>> airavata will treat the JSON in a standard (probably we have to
> >>>>>>> define
> >>>>>>>>>> this) way.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> The issue with JSON array only comes when we try to convert XML
> to
> >>>>>>> JSON not
> >>>>>>>>> the other way. If we map with JSON, inputparameters and
> >>>>>>> outputparameters in
> >>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
> we
> >>>>>>> need to
> >>>>>>>>> solve this issue.
> >>>>>>>>>
> >>>>>>>>> JSON XML JSON
> >>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> >>> {"inputs":["test"]}
> >>>>>>> //
> >>>>>>>>> correct one
> >>>>>>>>> --> {"inputs":"test"} // incorrect one
> >>>>>>>>>
> >>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>>>>
> >>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
> >>>>>>> this
> >>>>>>>>>> being
> >>>>>>>>>> used is probably in the WSDL, but if we can agree on another way
> >>>>>>>>>> of communicating registered applications' I/O parameters to the
> >>>>>>> front
> >>>>>>>>>> end
> >>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> problem.
> >>>>>>> Are
> >>>>>>>>>> custom processing instructions to the Xbaya XML parse even used?
> >>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
> it.
> >>> As
> >>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
> big
> >>>>>>> issue
> >>>>>>>>> with Airavata.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> <array name="abc">
> >>>>>>>>>> <element>1</element>
> >>>>>>>>>> <element>2</element>
> >>>>>>>>>> <element>3</element>
> >>>>>>>>>> <element>4</element>
> >>>>>>>>>> </array>
> >>>>>>>>>>
> >>>>>>>>>> Can become
> >>>>>>>>>>
> >>>>>>>>>> {
> >>>>>>>>>>
> >>>>>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>>>>
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> With this example it show us we need to change the XML message
> >>> format
> >>>>>>> of
> >>>>>>>>> server side, which require to change the all schemas, If we are
> >>> going
> >>>>>>> to
> >>>>>>>>> change the schemas then we need to change the way it process it
> in
> >>>>>>> Ariavara
> >>>>>>>>> core. We have dropped our initial major requirement, which is
> keep
> >>> the
> >>>>>>>>> Airavata Server side as it is.
> >>>>>>>>>
> >>>>>>>>> with this conversion we only deal with json strings, yes we can
> send
> >>>>>>> JSON
> >>>>>>>>> request with other formats supported by JSON like boolen, null,
> >>> Number
> >>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
> only
> >>>>>>> deal
> >>>>>>>>> only with Strings. I think it is good if we can consume a this
> >>>>> features
> >>>>>>>>> with JSON.
> >>>>>>>>>
> >>>>>>>>> let say i need to send a integer or float to the server using
> JSON
> >>>>> then
> >>>>>>>>> proper way is to send {"<name>":123.45} this will works fine but
> >>>>>>> problem is
> >>>>>>>>> how we get the same output ?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Shameera.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Subho.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best Regards,
> >>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>
> >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards,
> >>>>>> Shameera Rathnayaka.
> >>>>>>
> >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>> Shameera Rathnayaka.
> >>>>
> >>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> >>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>>
> >
>
>
Re: Airavata GSoC 2013 Master Project
Posted by Shameera Rathnayaka <sh...@gmail.com>.
@Viknes
You can easily add a topic which explain what you have proposed in you
proposal in few words.
Thanks,
Shameera.
On Fri, May 3, 2013 at 10:47 PM, Raminder Singh <ra...@gmail.com>wrote:
> Yes this need to be done in melange system. Mentors can do that. I will
> take a look to tag Airavata proposals.
>
> Thanks
> Raminder
>
> On May 3, 2013, at 1:12 PM, Danushka Menikkumbura wrote:
>
> > Well I did not do anything specifically. Maybe a reviewer hooked it up?.
> > Not sure. All I did was to use the same name that was used in the
> > corresponding JIRA ticket.
> >
> > Thanks,
> > Danushka
> >
> >
> > On Fri, May 3, 2013 at 9:13 PM, Viknes Balasubramanee <viknesb@msn.com
> >wrote:
> >
> >> Hey Guys,
> >>
> >> Can you please share in the list how to associate our proposal with a
> >> specific project.
> >>
> >> Thanks
> >> Viknes
> >>
> >> -----Original Message-----
> >> From: Suresh Marru [mailto:smarru@apache.org]
> >> Sent: Friday, May 03, 2013 9:11 AM
> >> To: dev@airavata.apache.org
> >> Subject: Re: Airavata GSoC 2013 Master Project
> >>
> >> Hi Subho,
> >>
> >> I do not see your proposal associated with Airavata. May be Danushka or
> >> Shameera can tell you how they tagged it to airavata.
> >>
> >> Suresh
> >>
> >> On May 3, 2013, at 8:43 AM, Subho Banerjee <su...@gmail.com> wrote:
> >>
> >>> Hi,
> >>> You can find a rough draft of my proposal at
> >>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sub
> >>> hobanerjee/13001
> >>>
> >>> I am now working against the clock (6 hours left) to iron out the
> >>> edges and to put in any content I am missing.
> >>>
> >>> Any comments would be welcome.
> >>>
> >>> Cheers,
> >>> Subho,
> >>>
> >>>
> >>> On Fri, May 3, 2013 at 5:24 PM, Danushka Menikkumbura <
> >>> danushka.menikkumbura@gmail.com> wrote:
> >>>
> >>>> Please find the proposal for "AMQP Messaging protocol support for
> >>>> Airavata WS-Messenger" at [1]
> >>>>
> >>>> Thanks,
> >>>> Danushka
> >>>>
> >>>> [1] -
> >>>>
> >>>> https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc20
> >>>> 13/danushka/1#
> >>>>
> >>>>
> >>>> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org>
> wrote:
> >>>>
> >>>>> Ping!! No proposals yet beyond Shameera's and Danushka's place
> holder.
> >>>>>
> >>>>> Suresh
> >>>>>
> >>>>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> >>>>>
> >>>>>> Hi Vijayendra,
> >>>>>>
> >>>>>> As you can see from Shameera's proposal, he proposed a JSON
> >>>>>> conversion
> >>>>> in front of WS Messenger. Also Danuska has been proposing for the
> >>>>> AMQP
> >>>> and
> >>>>> idea and deliberating its advantages. So given all these, I would
> >>>>> suggest for you to keep focused on the UI aspects of the monitoring
> >>>>> and write
> >>>> into
> >>>>> your proposal a plan for determining a good strategy for the
> >>>>> plumbing to WS-Eventing based existing system. You can have the
> >>>>> concrete deliverables of new UI to change colors based on executions
> >>>>> (as it currently does in XBaya), double click and show error
> >>>>> messages and so forth. And keep it exploratory for the actually
> >> messaging format.
> >>>>>>
> >>>>>> I do not have any opinion on the libraries you mentioned, but yaa a
> >>>> ajax
> >>>>> based pub system might be the right way to go. Pending the content
> >>>>> format (JSON or WS-Eventing or JMS or AMQP or something else)
> >>>>>>
> >>>>>> Suresh
> >>>>>>
> >>>>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> >>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hi Suresh
> >>>>>>>
> >>>>>>> I am writing proposal for monitoring tool . The monitoring tool is
> >>>>> based on
> >>>>>>> pub-sub model (ws-messenger).
> >>>>>>> While writing proposal , I have to back it by technical stuff that
> >>>> tells
> >>>>>>> how can we achieve our purpose.
> >>>>>>> As this monitoring tool is supposed to be a web based , and we are
> >>>>> thinking
> >>>>>>> in the lines of
> >>>>>>> developing it in javascript.
> >>>>>>> I was looking into javascript libraries that can we used with
> >>>>> ws-messenger
> >>>>>>> in the monitoring module.
> >>>>>>> Please correct me if I am wrong.
> >>>>>>> I came across some of the libraries
> >>>>>>>
> >>>>>>> - jQuery custom
> >>>>>>> events<
> >>>>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> >>>>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> >>>>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
> >>>>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
> >>>>>>>
> >>>>>>> please tell me am I thinking in right direction?
> >>>>>>>
> >>>>>>> Regards
> >>>>>>> Vijayendra
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Shameera,
> >>>>>>>>
> >>>>>>>> This is great, I appreciate you sharing it, I realize this is
> >>>>>>>> still working document, but I want other students to start seeing
> >>>>>>>> it and
> >>>>> model
> >>>>>>>> their proposals in a similar way.
> >>>>>>>>
> >>>>>>>> Airavata Mentors,
> >>>>>>>>
> >>>>>>>> Please provide feedback directly on the melange site and uncheck
> >>>>>>>> the "private" box when you comment.
> >>>>>>>>
> >>>>>>>> Suresh
> >>>>>>>>
> >>>>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> >>>>> shameerainfo@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi Suresh and All,
> >>>>>>>>>
> >>>>>>>>> Of course I am very much happy to share my proposal with
> >>>>>>>>> everybody, actually i was going to update this thread with the
> >>>>>>>>> melange link in
> >>>>> few
> >>>>>>>>> hours once i have done writing all the sections in the proposal.
> >>>>>>>>> I
> >>>>>>>> haven't
> >>>>>>>>> yet added the milestone plan into it and now working on it.
> >>>>>>>>>
> >>>>>>>>> The sub area i am going to work from the Master project is '
> >>>>>>>> Implementing
> >>>>>>>>> a JSON interface to Airavata Client side and Registry component'.
> >>>>> Here is
> >>>>>>>>> the link
> >>>>>>>>>
> >>>>>>>>
> >>>>>
> >>>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sh
> >>>> ameera/60002#
> >>>>>>>>> .
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Please note that i haven't completed everything in this and
> >>>>>>>>> still
> >>>>> doing
> >>>>>>>>> modifications .Therefore proposal content may be changed bit,
> >>>>>>>>> need
> >>>> to
> >>>>> add
> >>>>>>>>> more technical details of the approach which explains it well.
> >>>>>>>>>
> >>>>>>>>> I would like to know the feedback from all of you regarding the
> >>>>> proposal
> >>>>>>>>> and will be modifying it if there is anything to be done. Also
> >>>> please
> >>>>>>>>> contact me if you need any help and i am very much willing to
> >>>>>>>>> share
> >>>> my
> >>>>>>>>> thoughts with all.
> >>>>>>>>>
> >>>>>>>>> Thanks!
> >>>>>>>>> Shameera
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> >>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Shameera,
> >>>>>>>>>>
> >>>>>>>>>> Excellent proposal, great job. Would you mind to make your
> >>>> proposal
> >>>>>>>>>> public and post the link here? Your proposal should help others
> >>>>>>>>>> to
> >>>>> look
> >>>>>>>> at
> >>>>>>>>>> it and learn from.
> >>>>>>>>>>
> >>>>>>>>>> Again I emphasize to all students, please don't feel you will
> >>>>>>>>>> be
> >>>>>>>> competing
> >>>>>>>>>> with each others. If all of you write good proposals, there is
> >>>>>>>>>> a
> >>>> good
> >>>>>>>>>> chance all of you will be selected. But without a good
> >>>>>>>>>> proposal, we
> >>>>>>>> cannot
> >>>>>>>>>> help.
> >>>>>>>>>>
> >>>>>>>>>> Suresh
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> >>>>>>>> shameerainfo@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi,
> >>>>>>>>>>>
> >>>>>>>>>>> Yes it is not easy to solve all problems, But defining our own
> >>>>> standard
> >>>>>>>>>> or
> >>>>>>>>>>> adhere to any standard
> >>>>>>>>>>> provided by third party library will solve the problem to some
> >>>>> extend.
> >>>>>>>>>>>
> >>>>>>>>>>> Here i see two possible approaches,
> >>>>>>>>>>>
> >>>>>>>>>>> 1. Use existing third party library(we can find which is best)
> >>>>> adhere
> >>>>>>>> to
> >>>>>>>>>> it
> >>>>>>>>>>> standard and see how we change the backend to be inline with
> >>>>>>>>>>> it.
> >>>>>>>>>>>
> >>>>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
> >>>>> suggest).
> >>>>>>>>>>>
> >>>>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
> >>>> compare
> >>>>>>>>>>> performance
> >>>>>>>>>>> and changes need to be done in server side. Then select the
> >>>>>>>>>>> best
> >>>>> one.
> >>>>>>>>>>>
> >>>>>>>>>>> Another question was, can we works with graph data in JSON
> >> format.
> >>>>>>>>>>> There are few JS graph framworks[1] which provide that
> >>>>> functionality.
> >>>>>>>>>>> we can use one of them to show airavata monitoring data as
> >>>>>>>>>>> graphs
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> Shameera.
> >>>>>>>>>>>
> >>>>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
> >>>> http://d3js.org/>
> >>>>> ,
> >>>>>>>>>>> Processing.js <http://processingjs.org> , Sencha
> >>>>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru
> >>>>>>>>>>> <sm...@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi Vijeyandra,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
> >>>>>>>> themselves
> >>>>>>>>>>>> are xml (WS-Eventing [1]).
> >>>>>>>>>>>>
> >>>>>>>>>>>> The Messenger paper [2] should give you more information.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Hi All (Especially those at WS02):
> >>>>>>>>>>>>
> >>>>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you
> >>>>>>>>>>>> may
> >>>>> want
> >>>>>>>> to
> >>>>>>>>>>>> read through these papers and chat with the authors to get
> >>>>>>>> experiences:
> >>>>>>>>>>>>
> >>>>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>>>>>>>>>>
> >>>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>>>>>>>>>>
> >>>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>>>>>>>>>> http://mooshabaya.wikidot.com/
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>
> >>>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-ma
> >>>> shups-from-workflows/
> >>>>>>>>>>>>
> >>>>>>>>>>>> Suresh
> >>>>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>>>>>>>>>> [2] -
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger
> >>>>> .pdf
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Suresh
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I wanted to know more about the monitoring tool .
> >>>>>>>>>>>>> Currently from where does the monitoring tool gets data . Is
> >>>>>>>>>>>>> it
> >>>>> from
> >>>>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
> >>>> might
> >>>>>>>>>>>> continuously
> >>>>>>>>>>>>> send messages to monitoring tool as to tell how much is the
> >>>>> progress
> >>>>>>>>>>>>> and what are the variables getting changed) ?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Again the how is the data being exchanged. I guess it must
> >>>>>>>>>>>>> be
> >>>> xml
> >>>>> ?
> >>>>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
> >>>>>>>>>>>>> monitoring module.
> >>>>>>>>>>>>> Then monitoring Tool is sending back this data to Xbaya for
> >>>>>>>>>>>>> displaying to the user ? Please correct me if
> >>>> I
> >>>>> am
> >>>>>>>>>>>> wrong
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I have downloaded the source code from the trunk . can you
> >>>> please
> >>>>>>>> point
> >>>>>>>>>>>>> me which part of code should I be code at for this module.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards
> >>>>>>>>>>>>> Vijayendra
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>>>>> Hi
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What i am suggesting is, we send the JSON message directly
> >>>>>>>>>>>>> to
> >>>>>>>> Airavata
> >>>>>>>>>>>>> Backend(or Registry)
> >>>>>>>>>>>>> When the message gets build after the transport phase,
> >>>>>>>>>>>>> convert
> >>>>> JSON
> >>>>>>>>>>>> message
> >>>>>>>>>>>>> to SOAP(XML).
> >>>>>>>>>>>>> From that point message will treated as SOAP message.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
> >>>> third
> >>>>>>>> party
> >>>>>>>>>>>>> libraries we
> >>>>>>>>>>>>> can use for. But before selecting a one we need to think
> >>>>>>>>>>>>> about
> >>>>>>>> problems
> >>>>>>>>>>>>> having
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> with JSON <--> XML and how these libraries handle those
> issues.
> >>>>>>>> Because
> >>>>>>>>>>>> we
> >>>>>>>>>>>>> need a robust
> >>>>>>>>>>>>> way to do this conversions.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
> >>>>> directly
> >>>>>>>>>> to
> >>>>>>>>>>>> Registry.
> >>>>>>>>>>>>> when the message gets built after the transport phase ,
> >>>>>>>>>>>>> convert
> >>>>> it to
> >>>>>>>>>>>> SOAP .
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If you are suggesting Registry will have JSON data instead
> >>>>>>>>>>>>> of
> >>>>> WSDL ,
> >>>>>>>>>>>> Then this might
> >>>>>>>>>>>>> complicate the things for us .
> >>>>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> >>>>> workflows
> >>>>>>>> and
> >>>>>>>>>>>> for other details .
> >>>>>>>>>>>>> Which means we might again have to do some changes with
> >>>>>>>>>>>>> workflow
> >>>>>>>>>>>> interpretor .
> >>>>>>>>>>>>> Yesterday from what I heard in discussion is that , they do
> >>>>>>>>>>>>> not
> >>>>> want
> >>>>>>>> to
> >>>>>>>>>>>> mess with workflow
> >>>>>>>>>>>>> interpreter atleast for GSOC projects.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What I want to suggest is , why carry the JSON data till
> >>>>> Regisrty .
> >>>>>>>>>>>> Build a interface
> >>>>>>>>>>>>> before (Apache server API) where we can do the necessary
> >>>>> conversion
> >>>>>>>>>>>> (JSON to SOAP).
> >>>>>>>>>>>>> In this way we can avoid messing up with Airavata server as
> >>>>>>>>>>>>> a
> >>>>> whole.
> >>>>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
> >>>>> service)
> >>>>>>>> but
> >>>>>>>>>>>> the Apache server
> >>>>>>>>>>>>> is interacting with SOAP.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
> >>>>>>>> connections
> >>>>>>>>>>>> of the workflow.
> >>>>>>>>>>>>> for example , the workflow is expecting a file as input but
> >>>>>>>>>>>>> user is giving a sting or int .
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
> >>>> for a
> >>>>>>>>>>>> particular
> >>>>>>>>>>>>> workflow , we can add extra information in the form of
> >>>>>>>>>>>>> annotation as the kind of input/ output the workflow is
> >>>>> accepting.
> >>>>>>>>>>>>> Then we will be able to check these against users entry
> >>>>>>>>>>>>> during
> >>>>>>>>>> execution.
> >>>>>>>>>>>>> Please correct me if I am wrong.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards
> >>>>>>>>>>>>> Vijayendra
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> >>>>> subs.zero@gmail.com
> >>>>>>>>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>> Well exactly, as long as you can define standard way of
> >>>>>>>> communication.
> >>>>>>>>>>>> That
> >>>>>>>>>>>>> is, you can define in advance what should be a string, array
> >>>>>>>>>>>>> and
> >>>>> what
> >>>>>>>>>>>>> should be a integer etc. We have no problem.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the
> >>>>>>>>>>>>> other
> >>>> way
> >>>>>>>>>> round,
> >>>>>>>>>>>>> they talk of the very general case (where you no nothing
> >>>>>>>>>>>>> about
> >>>> the
> >>>>>>>> data
> >>>>>>>>>>>> you
> >>>>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
> >>>>> myriad
> >>>>>>>> of
> >>>>>>>>>>>>> problems in that case, which you pointed out.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> But when there is standard, there is only one way of doing
> >>>> things,
> >>>>>>>> and
> >>>>>>>>>>>> not
> >>>>>>>>>>>>> several. I think that is the way forward. So what I am
> >>>>>>>>>>>>> proposing
> >>>>> is
> >>>>>>>>>> maybe
> >>>>>>>>>>>>> we all discuss and define this standard within the first
> >>>>>>>>>>>>> week of
> >>>>> GSoC
> >>>>>>>>>>>>> starting and then actually move into coding. So as long as
> >>>>>>>>>>>>> we
> >>>> work
> >>>>>>>> with
> >>>>>>>>>>>> the
> >>>>>>>>>>>>> presumption that this will be done, we really dont have to
> >>>> worry a
> >>>>>>>> lot
> >>>>>>>>>>>>> about this.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>> Subho.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> >>>>>>>> subs.zero@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Some of these problems are very specific to what the XML
> >>>>>>>>>>>>>>> is
> >>>>>>>>>>>>>> representing,
> >>>>>>>>>>>>>>> it might not be an actual problem in Airavata, maybe some
> >>>>>>>>>>>>>>> one more experienced with the codebase can point
> >>>> this
> >>>>>>>> out.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> All issues pointed out in the paper is not directly valid
> >>>>>>>>>>>>>> to
> >>>> our
> >>>>>>>>>>>>>> conversion, I didn't list the issues actually need to
> >>>>>>>>>>>>>> address
> >>>> in
> >>>>>>>> this
> >>>>>>>>>>>> case
> >>>>>>>>>>>>>> because thought it is worth to read that introduction part
> >>>> which
> >>>>>>>>>>>> explain
> >>>>>>>>>>>>>> the all the issues we have with this conversion and give us
> >>>>>>>>>>>>>> a
> >>>>> solid
> >>>>>>>>>>>>>> background of that.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character
> >>>>>>>>>>>>>>> sets
> >>>> --
> >>>>> I
> >>>>>>>>>>>>>> really
> >>>>>>>>>>>>>>> dont see these as problems, as long as you can agree that
> >>>>>>>>>>>>>>> all
> >>>>>>>>>>>> parts of
> >>>>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we
> >>>>>>>>>>>>>>> have
> >>>> to
> >>>>>>>>>>>> define
> >>>>>>>>>>>>>>> this) way.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> The issue with JSON array only comes when we try to convert
> >>>>>>>>>>>>>> XML
> >>>>> to
> >>>>>>>>>>>> JSON not
> >>>>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
> >>>>>>>>>>>> outputparameters in
> >>>>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays.
> >>>>>>>>>>>>>> Therefore
> >>>>> we
> >>>>>>>>>>>> need to
> >>>>>>>>>>>>>> solve this issue.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> JSON XML JSON
> >>>>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> >>>>>>>> {"inputs":["test"]}
> >>>>>>>>>>>> //
> >>>>>>>>>>>>>> correct one
> >>>>>>>>>>>>>> --> {"inputs":"test"} // incorrect
> >>>> one
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can
> >>>>>>>>>>>>>>> see
> >>>>>>>>>>>> this
> >>>>>>>>>>>>>>> being
> >>>>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on
> >>>>>>>>>>>>>>> another
> >>>> way
> >>>>>>>>>>>>>>> of communicating registered applications' I/O parameters
> >>>>>>>>>>>>>>> to
> >>>> the
> >>>>>>>>>>>> front
> >>>>>>>>>>>>>>> end
> >>>>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> >>>>> problem.
> >>>>>>>>>>>> Are
> >>>>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
> >>>> used?
> >>>>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can
> >>>>>>>>>>>>>> solve
> >>>>> it.
> >>>>>>>> As
> >>>>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is
> >>>>>>>>>>>>>> not a
> >>>>> big
> >>>>>>>>>>>> issue
> >>>>>>>>>>>>>> with Airavata.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> <array name="abc">
> >>>>>>>>>>>>>>> <element>1</element>
> >>>>>>>>>>>>>>> <element>2</element>
> >>>>>>>>>>>>>>> <element>3</element>
> >>>>>>>>>>>>>>> <element>4</element>
> >>>>>>>>>>>>>>> </array>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Can become
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> {
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> With this example it show us we need to change the XML
> >>>>>>>>>>>>>> message
> >>>>>>>> format
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>> server side, which require to change the all schemas, If we
> >>>>>>>>>>>>>> are
> >>>>>>>> going
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>> change the schemas then we need to change the way it
> >>>>>>>>>>>>>> process it
> >>>>> in
> >>>>>>>>>>>> Ariavara
> >>>>>>>>>>>>>> core. We have dropped our initial major requirement, which
> >>>>>>>>>>>>>> is
> >>>>> keep
> >>>>>>>> the
> >>>>>>>>>>>>>> Airavata Server side as it is.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> with this conversion we only deal with json strings, yes we
> >>>>>>>>>>>>>> can
> >>>>> send
> >>>>>>>>>>>> JSON
> >>>>>>>>>>>>>> request with other formats supported by JSON like boolen,
> >>>>>>>>>>>>>> null,
> >>>>>>>> Number
> >>>>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as
> >>>>>>>>>>>>>> XML
> >>>>> only
> >>>>>>>>>>>> deal
> >>>>>>>>>>>>>> only with Strings. I think it is good if we can consume a
> >>>>>>>>>>>>>> this
> >>>>>>>>>> features
> >>>>>>>>>>>>>> with JSON.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> let say i need to send a integer or float to the server
> >>>>>>>>>>>>>> using
> >>>>> JSON
> >>>>>>>>>> then
> >>>>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works
> >>>>>>>>>>>>>> fine
> >>>> but
> >>>>>>>>>>>> problem is
> >>>>>>>>>>>>>> how we get the same output ?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> Shameera.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>> Subho.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>>>
> >>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best Regards,
> >>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>
> >>>>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com Blog :
> >>>>>>>>> http://shameerarathnayaka.blogspot.com/
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
--
Best Regards,
Shameera Rathnayaka.
email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/
Re: Airavata GSoC 2013 Master Project
Posted by Raminder Singh <ra...@gmail.com>.
Yes this need to be done in melange system. Mentors can do that. I will take a look to tag Airavata proposals.
Thanks
Raminder
On May 3, 2013, at 1:12 PM, Danushka Menikkumbura wrote:
> Well I did not do anything specifically. Maybe a reviewer hooked it up?.
> Not sure. All I did was to use the same name that was used in the
> corresponding JIRA ticket.
>
> Thanks,
> Danushka
>
>
> On Fri, May 3, 2013 at 9:13 PM, Viknes Balasubramanee <vi...@msn.com>wrote:
>
>> Hey Guys,
>>
>> Can you please share in the list how to associate our proposal with a
>> specific project.
>>
>> Thanks
>> Viknes
>>
>> -----Original Message-----
>> From: Suresh Marru [mailto:smarru@apache.org]
>> Sent: Friday, May 03, 2013 9:11 AM
>> To: dev@airavata.apache.org
>> Subject: Re: Airavata GSoC 2013 Master Project
>>
>> Hi Subho,
>>
>> I do not see your proposal associated with Airavata. May be Danushka or
>> Shameera can tell you how they tagged it to airavata.
>>
>> Suresh
>>
>> On May 3, 2013, at 8:43 AM, Subho Banerjee <su...@gmail.com> wrote:
>>
>>> Hi,
>>> You can find a rough draft of my proposal at
>>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sub
>>> hobanerjee/13001
>>>
>>> I am now working against the clock (6 hours left) to iron out the
>>> edges and to put in any content I am missing.
>>>
>>> Any comments would be welcome.
>>>
>>> Cheers,
>>> Subho,
>>>
>>>
>>> On Fri, May 3, 2013 at 5:24 PM, Danushka Menikkumbura <
>>> danushka.menikkumbura@gmail.com> wrote:
>>>
>>>> Please find the proposal for "AMQP Messaging protocol support for
>>>> Airavata WS-Messenger" at [1]
>>>>
>>>> Thanks,
>>>> Danushka
>>>>
>>>> [1] -
>>>>
>>>> https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc20
>>>> 13/danushka/1#
>>>>
>>>>
>>>> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
>>>>
>>>>> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>>>>>
>>>>> Suresh
>>>>>
>>>>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
>>>>>
>>>>>> Hi Vijayendra,
>>>>>>
>>>>>> As you can see from Shameera's proposal, he proposed a JSON
>>>>>> conversion
>>>>> in front of WS Messenger. Also Danuska has been proposing for the
>>>>> AMQP
>>>> and
>>>>> idea and deliberating its advantages. So given all these, I would
>>>>> suggest for you to keep focused on the UI aspects of the monitoring
>>>>> and write
>>>> into
>>>>> your proposal a plan for determining a good strategy for the
>>>>> plumbing to WS-Eventing based existing system. You can have the
>>>>> concrete deliverables of new UI to change colors based on executions
>>>>> (as it currently does in XBaya), double click and show error
>>>>> messages and so forth. And keep it exploratory for the actually
>> messaging format.
>>>>>>
>>>>>> I do not have any opinion on the libraries you mentioned, but yaa a
>>>> ajax
>>>>> based pub system might be the right way to go. Pending the content
>>>>> format (JSON or WS-Eventing or JMS or AMQP or something else)
>>>>>>
>>>>>> Suresh
>>>>>>
>>>>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Suresh
>>>>>>>
>>>>>>> I am writing proposal for monitoring tool . The monitoring tool is
>>>>> based on
>>>>>>> pub-sub model (ws-messenger).
>>>>>>> While writing proposal , I have to back it by technical stuff that
>>>> tells
>>>>>>> how can we achieve our purpose.
>>>>>>> As this monitoring tool is supposed to be a web based , and we are
>>>>> thinking
>>>>>>> in the lines of
>>>>>>> developing it in javascript.
>>>>>>> I was looking into javascript libraries that can we used with
>>>>> ws-messenger
>>>>>>> in the monitoring module.
>>>>>>> Please correct me if I am wrong.
>>>>>>> I came across some of the libraries
>>>>>>>
>>>>>>> - jQuery custom
>>>>>>> events<
>>>>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
>>>>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
>>>>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
>>>>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
>>>>>>>
>>>>>>> please tell me am I thinking in right direction?
>>>>>>>
>>>>>>> Regards
>>>>>>> Vijayendra
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
>>>> wrote:
>>>>>>>
>>>>>>>> Hi Shameera,
>>>>>>>>
>>>>>>>> This is great, I appreciate you sharing it, I realize this is
>>>>>>>> still working document, but I want other students to start seeing
>>>>>>>> it and
>>>>> model
>>>>>>>> their proposals in a similar way.
>>>>>>>>
>>>>>>>> Airavata Mentors,
>>>>>>>>
>>>>>>>> Please provide feedback directly on the melange site and uncheck
>>>>>>>> the "private" box when you comment.
>>>>>>>>
>>>>>>>> Suresh
>>>>>>>>
>>>>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
>>>>> shameerainfo@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Suresh and All,
>>>>>>>>>
>>>>>>>>> Of course I am very much happy to share my proposal with
>>>>>>>>> everybody, actually i was going to update this thread with the
>>>>>>>>> melange link in
>>>>> few
>>>>>>>>> hours once i have done writing all the sections in the proposal.
>>>>>>>>> I
>>>>>>>> haven't
>>>>>>>>> yet added the milestone plan into it and now working on it.
>>>>>>>>>
>>>>>>>>> The sub area i am going to work from the Master project is '
>>>>>>>> Implementing
>>>>>>>>> a JSON interface to Airavata Client side and Registry component'.
>>>>> Here is
>>>>>>>>> the link
>>>>>>>>>
>>>>>>>>
>>>>>
>>>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sh
>>>> ameera/60002#
>>>>>>>>> .
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Please note that i haven't completed everything in this and
>>>>>>>>> still
>>>>> doing
>>>>>>>>> modifications .Therefore proposal content may be changed bit,
>>>>>>>>> need
>>>> to
>>>>> add
>>>>>>>>> more technical details of the approach which explains it well.
>>>>>>>>>
>>>>>>>>> I would like to know the feedback from all of you regarding the
>>>>> proposal
>>>>>>>>> and will be modifying it if there is anything to be done. Also
>>>> please
>>>>>>>>> contact me if you need any help and i am very much willing to
>>>>>>>>> share
>>>> my
>>>>>>>>> thoughts with all.
>>>>>>>>>
>>>>>>>>> Thanks!
>>>>>>>>> Shameera
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Shameera,
>>>>>>>>>>
>>>>>>>>>> Excellent proposal, great job. Would you mind to make your
>>>> proposal
>>>>>>>>>> public and post the link here? Your proposal should help others
>>>>>>>>>> to
>>>>> look
>>>>>>>> at
>>>>>>>>>> it and learn from.
>>>>>>>>>>
>>>>>>>>>> Again I emphasize to all students, please don't feel you will
>>>>>>>>>> be
>>>>>>>> competing
>>>>>>>>>> with each others. If all of you write good proposals, there is
>>>>>>>>>> a
>>>> good
>>>>>>>>>> chance all of you will be selected. But without a good
>>>>>>>>>> proposal, we
>>>>>>>> cannot
>>>>>>>>>> help.
>>>>>>>>>>
>>>>>>>>>> Suresh
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
>>>>>>>> shameerainfo@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Yes it is not easy to solve all problems, But defining our own
>>>>> standard
>>>>>>>>>> or
>>>>>>>>>>> adhere to any standard
>>>>>>>>>>> provided by third party library will solve the problem to some
>>>>> extend.
>>>>>>>>>>>
>>>>>>>>>>> Here i see two possible approaches,
>>>>>>>>>>>
>>>>>>>>>>> 1. Use existing third party library(we can find which is best)
>>>>> adhere
>>>>>>>> to
>>>>>>>>>> it
>>>>>>>>>>> standard and see how we change the backend to be inline with
>>>>>>>>>>> it.
>>>>>>>>>>>
>>>>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
>>>>> suggest).
>>>>>>>>>>>
>>>>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
>>>> compare
>>>>>>>>>>> performance
>>>>>>>>>>> and changes need to be done in server side. Then select the
>>>>>>>>>>> best
>>>>> one.
>>>>>>>>>>>
>>>>>>>>>>> Another question was, can we works with graph data in JSON
>> format.
>>>>>>>>>>> There are few JS graph framworks[1] which provide that
>>>>> functionality.
>>>>>>>>>>> we can use one of them to show airavata monitoring data as
>>>>>>>>>>> graphs
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Shameera.
>>>>>>>>>>>
>>>>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
>>>> http://d3js.org/>
>>>>> ,
>>>>>>>>>>> Processing.js <http://processingjs.org> , Sencha
>>>>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru
>>>>>>>>>>> <sm...@apache.org>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Vijeyandra,
>>>>>>>>>>>>
>>>>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
>>>>>>>> themselves
>>>>>>>>>>>> are xml (WS-Eventing [1]).
>>>>>>>>>>>>
>>>>>>>>>>>> The Messenger paper [2] should give you more information.
>>>>>>>>>>>>
>>>>>>>>>>>> Hi All (Especially those at WS02):
>>>>>>>>>>>>
>>>>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you
>>>>>>>>>>>> may
>>>>> want
>>>>>>>> to
>>>>>>>>>>>> read through these papers and chat with the authors to get
>>>>>>>> experiences:
>>>>>>>>>>>>
>>>>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>>>>>>>>>>
>>>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>>>>>>>>>>
>>>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>>>>>>>>>> http://mooshabaya.wikidot.com/
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>
>>>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-ma
>>>> shups-from-workflows/
>>>>>>>>>>>>
>>>>>>>>>>>> Suresh
>>>>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>>>>>>>>>> [2] -
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger
>>>>> .pdf
>>>>>>>>>>>>
>>>>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Suresh
>>>>>>>>>>>>>
>>>>>>>>>>>>> I wanted to know more about the monitoring tool .
>>>>>>>>>>>>> Currently from where does the monitoring tool gets data . Is
>>>>>>>>>>>>> it
>>>>> from
>>>>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
>>>> might
>>>>>>>>>>>> continuously
>>>>>>>>>>>>> send messages to monitoring tool as to tell how much is the
>>>>> progress
>>>>>>>>>>>>> and what are the variables getting changed) ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Again the how is the data being exchanged. I guess it must
>>>>>>>>>>>>> be
>>>> xml
>>>>> ?
>>>>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
>>>>>>>>>>>>> monitoring module.
>>>>>>>>>>>>> Then monitoring Tool is sending back this data to Xbaya for
>>>>>>>>>>>>> displaying to the user ? Please correct me if
>>>> I
>>>>> am
>>>>>>>>>>>> wrong
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have downloaded the source code from the trunk . can you
>>>> please
>>>>>>>> point
>>>>>>>>>>>>> me which part of code should I be code at for this module.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Vijayendra
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>> What i am suggesting is, we send the JSON message directly
>>>>>>>>>>>>> to
>>>>>>>> Airavata
>>>>>>>>>>>>> Backend(or Registry)
>>>>>>>>>>>>> When the message gets build after the transport phase,
>>>>>>>>>>>>> convert
>>>>> JSON
>>>>>>>>>>>> message
>>>>>>>>>>>>> to SOAP(XML).
>>>>>>>>>>>>> From that point message will treated as SOAP message.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
>>>> third
>>>>>>>> party
>>>>>>>>>>>>> libraries we
>>>>>>>>>>>>> can use for. But before selecting a one we need to think
>>>>>>>>>>>>> about
>>>>>>>> problems
>>>>>>>>>>>>> having
>>>>>>>>>>>>>
>>>>>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
>>>>>>>> Because
>>>>>>>>>>>> we
>>>>>>>>>>>>> need a robust
>>>>>>>>>>>>> way to do this conversions.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
>>>>> directly
>>>>>>>>>> to
>>>>>>>>>>>> Registry.
>>>>>>>>>>>>> when the message gets built after the transport phase ,
>>>>>>>>>>>>> convert
>>>>> it to
>>>>>>>>>>>> SOAP .
>>>>>>>>>>>>>
>>>>>>>>>>>>> If you are suggesting Registry will have JSON data instead
>>>>>>>>>>>>> of
>>>>> WSDL ,
>>>>>>>>>>>> Then this might
>>>>>>>>>>>>> complicate the things for us .
>>>>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
>>>>> workflows
>>>>>>>> and
>>>>>>>>>>>> for other details .
>>>>>>>>>>>>> Which means we might again have to do some changes with
>>>>>>>>>>>>> workflow
>>>>>>>>>>>> interpretor .
>>>>>>>>>>>>> Yesterday from what I heard in discussion is that , they do
>>>>>>>>>>>>> not
>>>>> want
>>>>>>>> to
>>>>>>>>>>>> mess with workflow
>>>>>>>>>>>>> interpreter atleast for GSOC projects.
>>>>>>>>>>>>>
>>>>>>>>>>>>> What I want to suggest is , why carry the JSON data till
>>>>> Regisrty .
>>>>>>>>>>>> Build a interface
>>>>>>>>>>>>> before (Apache server API) where we can do the necessary
>>>>> conversion
>>>>>>>>>>>> (JSON to SOAP).
>>>>>>>>>>>>> In this way we can avoid messing up with Airavata server as
>>>>>>>>>>>>> a
>>>>> whole.
>>>>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
>>>>> service)
>>>>>>>> but
>>>>>>>>>>>> the Apache server
>>>>>>>>>>>>> is interacting with SOAP.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
>>>>>>>> connections
>>>>>>>>>>>> of the workflow.
>>>>>>>>>>>>> for example , the workflow is expecting a file as input but
>>>>>>>>>>>>> user is giving a sting or int .
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
>>>> for a
>>>>>>>>>>>> particular
>>>>>>>>>>>>> workflow , we can add extra information in the form of
>>>>>>>>>>>>> annotation as the kind of input/ output the workflow is
>>>>> accepting.
>>>>>>>>>>>>> Then we will be able to check these against users entry
>>>>>>>>>>>>> during
>>>>>>>>>> execution.
>>>>>>>>>>>>> Please correct me if I am wrong.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Vijayendra
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
>>>>> subs.zero@gmail.com
>>>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> Well exactly, as long as you can define standard way of
>>>>>>>> communication.
>>>>>>>>>>>> That
>>>>>>>>>>>>> is, you can define in advance what should be a string, array
>>>>>>>>>>>>> and
>>>>> what
>>>>>>>>>>>>> should be a integer etc. We have no problem.
>>>>>>>>>>>>>
>>>>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the
>>>>>>>>>>>>> other
>>>> way
>>>>>>>>>> round,
>>>>>>>>>>>>> they talk of the very general case (where you no nothing
>>>>>>>>>>>>> about
>>>> the
>>>>>>>> data
>>>>>>>>>>>> you
>>>>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
>>>>> myriad
>>>>>>>> of
>>>>>>>>>>>>> problems in that case, which you pointed out.
>>>>>>>>>>>>>
>>>>>>>>>>>>> But when there is standard, there is only one way of doing
>>>> things,
>>>>>>>> and
>>>>>>>>>>>> not
>>>>>>>>>>>>> several. I think that is the way forward. So what I am
>>>>>>>>>>>>> proposing
>>>>> is
>>>>>>>>>> maybe
>>>>>>>>>>>>> we all discuss and define this standard within the first
>>>>>>>>>>>>> week of
>>>>> GSoC
>>>>>>>>>>>>> starting and then actually move into coding. So as long as
>>>>>>>>>>>>> we
>>>> work
>>>>>>>> with
>>>>>>>>>>>> the
>>>>>>>>>>>>> presumption that this will be done, we really dont have to
>>>> worry a
>>>>>>>> lot
>>>>>>>>>>>>> about this.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Subho.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
>>>>>>>> subs.zero@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Some of these problems are very specific to what the XML
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> representing,
>>>>>>>>>>>>>>> it might not be an actual problem in Airavata, maybe some
>>>>>>>>>>>>>>> one more experienced with the codebase can point
>>>> this
>>>>>>>> out.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> All issues pointed out in the paper is not directly valid
>>>>>>>>>>>>>> to
>>>> our
>>>>>>>>>>>>>> conversion, I didn't list the issues actually need to
>>>>>>>>>>>>>> address
>>>> in
>>>>>>>> this
>>>>>>>>>>>> case
>>>>>>>>>>>>>> because thought it is worth to read that introduction part
>>>> which
>>>>>>>>>>>> explain
>>>>>>>>>>>>>> the all the issues we have with this conversion and give us
>>>>>>>>>>>>>> a
>>>>> solid
>>>>>>>>>>>>>> background of that.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character
>>>>>>>>>>>>>>> sets
>>>> --
>>>>> I
>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>> dont see these as problems, as long as you can agree that
>>>>>>>>>>>>>>> all
>>>>>>>>>>>> parts of
>>>>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we
>>>>>>>>>>>>>>> have
>>>> to
>>>>>>>>>>>> define
>>>>>>>>>>>>>>> this) way.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The issue with JSON array only comes when we try to convert
>>>>>>>>>>>>>> XML
>>>>> to
>>>>>>>>>>>> JSON not
>>>>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
>>>>>>>>>>>> outputparameters in
>>>>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays.
>>>>>>>>>>>>>> Therefore
>>>>> we
>>>>>>>>>>>> need to
>>>>>>>>>>>>>> solve this issue.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> JSON XML JSON
>>>>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
>>>>>>>> {"inputs":["test"]}
>>>>>>>>>>>> //
>>>>>>>>>>>>>> correct one
>>>>>>>>>>>>>> --> {"inputs":"test"} // incorrect
>>>> one
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can
>>>>>>>>>>>>>>> see
>>>>>>>>>>>> this
>>>>>>>>>>>>>>> being
>>>>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on
>>>>>>>>>>>>>>> another
>>>> way
>>>>>>>>>>>>>>> of communicating registered applications' I/O parameters
>>>>>>>>>>>>>>> to
>>>> the
>>>>>>>>>>>> front
>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
>>>>> problem.
>>>>>>>>>>>> Are
>>>>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
>>>> used?
>>>>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can
>>>>>>>>>>>>>> solve
>>>>> it.
>>>>>>>> As
>>>>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is
>>>>>>>>>>>>>> not a
>>>>> big
>>>>>>>>>>>> issue
>>>>>>>>>>>>>> with Airavata.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> <array name="abc">
>>>>>>>>>>>>>>> <element>1</element>
>>>>>>>>>>>>>>> <element>2</element>
>>>>>>>>>>>>>>> <element>3</element>
>>>>>>>>>>>>>>> <element>4</element>
>>>>>>>>>>>>>>> </array>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can become
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> With this example it show us we need to change the XML
>>>>>>>>>>>>>> message
>>>>>>>> format
>>>>>>>>>>>> of
>>>>>>>>>>>>>> server side, which require to change the all schemas, If we
>>>>>>>>>>>>>> are
>>>>>>>> going
>>>>>>>>>>>> to
>>>>>>>>>>>>>> change the schemas then we need to change the way it
>>>>>>>>>>>>>> process it
>>>>> in
>>>>>>>>>>>> Ariavara
>>>>>>>>>>>>>> core. We have dropped our initial major requirement, which
>>>>>>>>>>>>>> is
>>>>> keep
>>>>>>>> the
>>>>>>>>>>>>>> Airavata Server side as it is.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> with this conversion we only deal with json strings, yes we
>>>>>>>>>>>>>> can
>>>>> send
>>>>>>>>>>>> JSON
>>>>>>>>>>>>>> request with other formats supported by JSON like boolen,
>>>>>>>>>>>>>> null,
>>>>>>>> Number
>>>>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as
>>>>>>>>>>>>>> XML
>>>>> only
>>>>>>>>>>>> deal
>>>>>>>>>>>>>> only with Strings. I think it is good if we can consume a
>>>>>>>>>>>>>> this
>>>>>>>>>> features
>>>>>>>>>>>>>> with JSON.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> let say i need to send a integer or float to the server
>>>>>>>>>>>>>> using
>>>>> JSON
>>>>>>>>>> then
>>>>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works
>>>>>>>>>>>>>> fine
>>>> but
>>>>>>>>>>>> problem is
>>>>>>>>>>>>>> how we get the same output ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Subho.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>>>
>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>
>>>>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com Blog :
>>>>>>>>> http://shameerarathnayaka.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>
>>
Re: Airavata GSoC 2013 Master Project
Posted by Danushka Menikkumbura <da...@gmail.com>.
Well I did not do anything specifically. Maybe a reviewer hooked it up?.
Not sure. All I did was to use the same name that was used in the
corresponding JIRA ticket.
Thanks,
Danushka
On Fri, May 3, 2013 at 9:13 PM, Viknes Balasubramanee <vi...@msn.com>wrote:
> Hey Guys,
>
> Can you please share in the list how to associate our proposal with a
> specific project.
>
> Thanks
> Viknes
>
> -----Original Message-----
> From: Suresh Marru [mailto:smarru@apache.org]
> Sent: Friday, May 03, 2013 9:11 AM
> To: dev@airavata.apache.org
> Subject: Re: Airavata GSoC 2013 Master Project
>
> Hi Subho,
>
> I do not see your proposal associated with Airavata. May be Danushka or
> Shameera can tell you how they tagged it to airavata.
>
> Suresh
>
> On May 3, 2013, at 8:43 AM, Subho Banerjee <su...@gmail.com> wrote:
>
> > Hi,
> > You can find a rough draft of my proposal at
> > http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sub
> > hobanerjee/13001
> >
> > I am now working against the clock (6 hours left) to iron out the
> > edges and to put in any content I am missing.
> >
> > Any comments would be welcome.
> >
> > Cheers,
> > Subho,
> >
> >
> > On Fri, May 3, 2013 at 5:24 PM, Danushka Menikkumbura <
> > danushka.menikkumbura@gmail.com> wrote:
> >
> >> Please find the proposal for "AMQP Messaging protocol support for
> >> Airavata WS-Messenger" at [1]
> >>
> >> Thanks,
> >> Danushka
> >>
> >> [1] -
> >>
> >> https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc20
> >> 13/danushka/1#
> >>
> >>
> >> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
> >>
> >>> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
> >>>
> >>> Suresh
> >>>
> >>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> >>>
> >>>> Hi Vijayendra,
> >>>>
> >>>> As you can see from Shameera's proposal, he proposed a JSON
> >>>> conversion
> >>> in front of WS Messenger. Also Danuska has been proposing for the
> >>> AMQP
> >> and
> >>> idea and deliberating its advantages. So given all these, I would
> >>> suggest for you to keep focused on the UI aspects of the monitoring
> >>> and write
> >> into
> >>> your proposal a plan for determining a good strategy for the
> >>> plumbing to WS-Eventing based existing system. You can have the
> >>> concrete deliverables of new UI to change colors based on executions
> >>> (as it currently does in XBaya), double click and show error
> >>> messages and so forth. And keep it exploratory for the actually
> messaging format.
> >>>>
> >>>> I do not have any opinion on the libraries you mentioned, but yaa a
> >> ajax
> >>> based pub system might be the right way to go. Pending the content
> >>> format (JSON or WS-Eventing or JMS or AMQP or something else)
> >>>>
> >>>> Suresh
> >>>>
> >>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> >>> vijayendra.sdm@gmail.com> wrote:
> >>>>
> >>>>> Hi Suresh
> >>>>>
> >>>>> I am writing proposal for monitoring tool . The monitoring tool is
> >>> based on
> >>>>> pub-sub model (ws-messenger).
> >>>>> While writing proposal , I have to back it by technical stuff that
> >> tells
> >>>>> how can we achieve our purpose.
> >>>>> As this monitoring tool is supposed to be a web based , and we are
> >>> thinking
> >>>>> in the lines of
> >>>>> developing it in javascript.
> >>>>> I was looking into javascript libraries that can we used with
> >>> ws-messenger
> >>>>> in the monitoring module.
> >>>>> Please correct me if I am wrong.
> >>>>> I came across some of the libraries
> >>>>>
> >>>>> - jQuery custom
> >>>>> events<
> >>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> >>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> >>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
> >>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
> >>>>>
> >>>>> please tell me am I thinking in right direction?
> >>>>>
> >>>>> Regards
> >>>>> Vijayendra
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
> >> wrote:
> >>>>>
> >>>>>> Hi Shameera,
> >>>>>>
> >>>>>> This is great, I appreciate you sharing it, I realize this is
> >>>>>> still working document, but I want other students to start seeing
> >>>>>> it and
> >>> model
> >>>>>> their proposals in a similar way.
> >>>>>>
> >>>>>> Airavata Mentors,
> >>>>>>
> >>>>>> Please provide feedback directly on the melange site and uncheck
> >>>>>> the "private" box when you comment.
> >>>>>>
> >>>>>> Suresh
> >>>>>>
> >>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> >>> shameerainfo@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Suresh and All,
> >>>>>>>
> >>>>>>> Of course I am very much happy to share my proposal with
> >>>>>>> everybody, actually i was going to update this thread with the
> >>>>>>> melange link in
> >>> few
> >>>>>>> hours once i have done writing all the sections in the proposal.
> >>>>>>> I
> >>>>>> haven't
> >>>>>>> yet added the milestone plan into it and now working on it.
> >>>>>>>
> >>>>>>> The sub area i am going to work from the Master project is '
> >>>>>> Implementing
> >>>>>>> a JSON interface to Airavata Client side and Registry component'.
> >>> Here is
> >>>>>>> the link
> >>>>>>>
> >>>>>>
> >>>
> >> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sh
> >> ameera/60002#
> >>>>>>> .
> >>>>>>>
> >>>>>>>
> >>>>>>> Please note that i haven't completed everything in this and
> >>>>>>> still
> >>> doing
> >>>>>>> modifications .Therefore proposal content may be changed bit,
> >>>>>>> need
> >> to
> >>> add
> >>>>>>> more technical details of the approach which explains it well.
> >>>>>>>
> >>>>>>> I would like to know the feedback from all of you regarding the
> >>> proposal
> >>>>>>> and will be modifying it if there is anything to be done. Also
> >> please
> >>>>>>> contact me if you need any help and i am very much willing to
> >>>>>>> share
> >> my
> >>>>>>> thoughts with all.
> >>>>>>>
> >>>>>>> Thanks!
> >>>>>>> Shameera
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> >>> wrote:
> >>>>>>>
> >>>>>>>> Hi Shameera,
> >>>>>>>>
> >>>>>>>> Excellent proposal, great job. Would you mind to make your
> >> proposal
> >>>>>>>> public and post the link here? Your proposal should help others
> >>>>>>>> to
> >>> look
> >>>>>> at
> >>>>>>>> it and learn from.
> >>>>>>>>
> >>>>>>>> Again I emphasize to all students, please don't feel you will
> >>>>>>>> be
> >>>>>> competing
> >>>>>>>> with each others. If all of you write good proposals, there is
> >>>>>>>> a
> >> good
> >>>>>>>> chance all of you will be selected. But without a good
> >>>>>>>> proposal, we
> >>>>>> cannot
> >>>>>>>> help.
> >>>>>>>>
> >>>>>>>> Suresh
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> >>>>>> shameerainfo@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> Yes it is not easy to solve all problems, But defining our own
> >>> standard
> >>>>>>>> or
> >>>>>>>>> adhere to any standard
> >>>>>>>>> provided by third party library will solve the problem to some
> >>> extend.
> >>>>>>>>>
> >>>>>>>>> Here i see two possible approaches,
> >>>>>>>>>
> >>>>>>>>> 1. Use existing third party library(we can find which is best)
> >>> adhere
> >>>>>> to
> >>>>>>>> it
> >>>>>>>>> standard and see how we change the backend to be inline with
> >>>>>>>>> it.
> >>>>>>>>>
> >>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
> >>> suggest).
> >>>>>>>>>
> >>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
> >> compare
> >>>>>>>>> performance
> >>>>>>>>> and changes need to be done in server side. Then select the
> >>>>>>>>> best
> >>> one.
> >>>>>>>>>
> >>>>>>>>> Another question was, can we works with graph data in JSON
> format.
> >>>>>>>>> There are few JS graph framworks[1] which provide that
> >>> functionality.
> >>>>>>>>> we can use one of them to show airavata monitoring data as
> >>>>>>>>> graphs
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Shameera.
> >>>>>>>>>
> >>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
> >> http://d3js.org/>
> >>> ,
> >>>>>>>>> Processing.js <http://processingjs.org> , Sencha
> >>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru
> >>>>>>>>> <sm...@apache.org>
> >>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Vijeyandra,
> >>>>>>>>>>
> >>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
> >>>>>> themselves
> >>>>>>>>>> are xml (WS-Eventing [1]).
> >>>>>>>>>>
> >>>>>>>>>> The Messenger paper [2] should give you more information.
> >>>>>>>>>>
> >>>>>>>>>> Hi All (Especially those at WS02):
> >>>>>>>>>>
> >>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you
> >>>>>>>>>> may
> >>> want
> >>>>>> to
> >>>>>>>>>> read through these papers and chat with the authors to get
> >>>>>> experiences:
> >>>>>>>>>>
> >>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>>>>>>>>
> >>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>>>>>>>>
> >>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>>>>>>>> http://mooshabaya.wikidot.com/
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>
> >> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-ma
> >> shups-from-workflows/
> >>>>>>>>>>
> >>>>>>>>>> Suresh
> >>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>>>>>>>> [2] -
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger
> >>> .pdf
> >>>>>>>>>>
> >>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Suresh
> >>>>>>>>>>>
> >>>>>>>>>>> I wanted to know more about the monitoring tool .
> >>>>>>>>>>> Currently from where does the monitoring tool gets data . Is
> >>>>>>>>>>> it
> >>> from
> >>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
> >> might
> >>>>>>>>>> continuously
> >>>>>>>>>>> send messages to monitoring tool as to tell how much is the
> >>> progress
> >>>>>>>>>>> and what are the variables getting changed) ?
> >>>>>>>>>>>
> >>>>>>>>>>> Again the how is the data being exchanged. I guess it must
> >>>>>>>>>>> be
> >> xml
> >>> ?
> >>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
> >>>>>>>>>>> monitoring module.
> >>>>>>>>>>> Then monitoring Tool is sending back this data to Xbaya for
> >>>>>>>>>>> displaying to the user ? Please correct me if
> >> I
> >>> am
> >>>>>>>>>> wrong
> >>>>>>>>>>>
> >>>>>>>>>>> I have downloaded the source code from the trunk . can you
> >> please
> >>>>>> point
> >>>>>>>>>>> me which part of code should I be code at for this module.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards
> >>>>>>>>>>> Vijayendra
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>>>>> Hi
> >>>>>>>>>>>
> >>>>>>>>>>> What i am suggesting is, we send the JSON message directly
> >>>>>>>>>>> to
> >>>>>> Airavata
> >>>>>>>>>>> Backend(or Registry)
> >>>>>>>>>>> When the message gets build after the transport phase,
> >>>>>>>>>>> convert
> >>> JSON
> >>>>>>>>>> message
> >>>>>>>>>>> to SOAP(XML).
> >>>>>>>>>>> From that point message will treated as SOAP message.
> >>>>>>>>>>>
> >>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
> >> third
> >>>>>> party
> >>>>>>>>>>> libraries we
> >>>>>>>>>>> can use for. But before selecting a one we need to think
> >>>>>>>>>>> about
> >>>>>> problems
> >>>>>>>>>>> having
> >>>>>>>>>>>
> >>>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
> >>>>>> Because
> >>>>>>>>>> we
> >>>>>>>>>>> need a robust
> >>>>>>>>>>> way to do this conversions.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
> >>> directly
> >>>>>>>> to
> >>>>>>>>>> Registry.
> >>>>>>>>>>> when the message gets built after the transport phase ,
> >>>>>>>>>>> convert
> >>> it to
> >>>>>>>>>> SOAP .
> >>>>>>>>>>>
> >>>>>>>>>>> If you are suggesting Registry will have JSON data instead
> >>>>>>>>>>> of
> >>> WSDL ,
> >>>>>>>>>> Then this might
> >>>>>>>>>>> complicate the things for us .
> >>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> >>> workflows
> >>>>>> and
> >>>>>>>>>> for other details .
> >>>>>>>>>>> Which means we might again have to do some changes with
> >>>>>>>>>>> workflow
> >>>>>>>>>> interpretor .
> >>>>>>>>>>> Yesterday from what I heard in discussion is that , they do
> >>>>>>>>>>> not
> >>> want
> >>>>>> to
> >>>>>>>>>> mess with workflow
> >>>>>>>>>>> interpreter atleast for GSOC projects.
> >>>>>>>>>>>
> >>>>>>>>>>> What I want to suggest is , why carry the JSON data till
> >>> Regisrty .
> >>>>>>>>>> Build a interface
> >>>>>>>>>>> before (Apache server API) where we can do the necessary
> >>> conversion
> >>>>>>>>>> (JSON to SOAP).
> >>>>>>>>>>> In this way we can avoid messing up with Airavata server as
> >>>>>>>>>>> a
> >>> whole.
> >>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
> >>> service)
> >>>>>> but
> >>>>>>>>>> the Apache server
> >>>>>>>>>>> is interacting with SOAP.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
> >>>>>> connections
> >>>>>>>>>> of the workflow.
> >>>>>>>>>>> for example , the workflow is expecting a file as input but
> >>>>>>>>>>> user is giving a sting or int .
> >>>>>>>>>>>
> >>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
> >> for a
> >>>>>>>>>> particular
> >>>>>>>>>>> workflow , we can add extra information in the form of
> >>>>>>>>>>> annotation as the kind of input/ output the workflow is
> >>> accepting.
> >>>>>>>>>>> Then we will be able to check these against users entry
> >>>>>>>>>>> during
> >>>>>>>> execution.
> >>>>>>>>>>> Please correct me if I am wrong.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards
> >>>>>>>>>>> Vijayendra
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> >>> subs.zero@gmail.com
> >>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>> Well exactly, as long as you can define standard way of
> >>>>>> communication.
> >>>>>>>>>> That
> >>>>>>>>>>> is, you can define in advance what should be a string, array
> >>>>>>>>>>> and
> >>> what
> >>>>>>>>>>> should be a integer etc. We have no problem.
> >>>>>>>>>>>
> >>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the
> >>>>>>>>>>> other
> >> way
> >>>>>>>> round,
> >>>>>>>>>>> they talk of the very general case (where you no nothing
> >>>>>>>>>>> about
> >> the
> >>>>>> data
> >>>>>>>>>> you
> >>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
> >>> myriad
> >>>>>> of
> >>>>>>>>>>> problems in that case, which you pointed out.
> >>>>>>>>>>>
> >>>>>>>>>>> But when there is standard, there is only one way of doing
> >> things,
> >>>>>> and
> >>>>>>>>>> not
> >>>>>>>>>>> several. I think that is the way forward. So what I am
> >>>>>>>>>>> proposing
> >>> is
> >>>>>>>> maybe
> >>>>>>>>>>> we all discuss and define this standard within the first
> >>>>>>>>>>> week of
> >>> GSoC
> >>>>>>>>>>> starting and then actually move into coding. So as long as
> >>>>>>>>>>> we
> >> work
> >>>>>> with
> >>>>>>>>>> the
> >>>>>>>>>>> presumption that this will be done, we really dont have to
> >> worry a
> >>>>>> lot
> >>>>>>>>>>> about this.
> >>>>>>>>>>>
> >>>>>>>>>>> Cheers,
> >>>>>>>>>>> Subho.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>>>>>>>> shameerainfo@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi,
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> >>>>>> subs.zero@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Some of these problems are very specific to what the XML
> >>>>>>>>>>>>> is
> >>>>>>>>>>>> representing,
> >>>>>>>>>>>>> it might not be an actual problem in Airavata, maybe some
> >>>>>>>>>>>>> one more experienced with the codebase can point
> >> this
> >>>>>> out.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> All issues pointed out in the paper is not directly valid
> >>>>>>>>>>>> to
> >> our
> >>>>>>>>>>>> conversion, I didn't list the issues actually need to
> >>>>>>>>>>>> address
> >> in
> >>>>>> this
> >>>>>>>>>> case
> >>>>>>>>>>>> because thought it is worth to read that introduction part
> >> which
> >>>>>>>>>> explain
> >>>>>>>>>>>> the all the issues we have with this conversion and give us
> >>>>>>>>>>>> a
> >>> solid
> >>>>>>>>>>>> background of that.
> >>>>>>>>>>>>
> >>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character
> >>>>>>>>>>>>> sets
> >> --
> >>> I
> >>>>>>>>>>>> really
> >>>>>>>>>>>>> dont see these as problems, as long as you can agree that
> >>>>>>>>>>>>> all
> >>>>>>>>>> parts of
> >>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we
> >>>>>>>>>>>>> have
> >> to
> >>>>>>>>>> define
> >>>>>>>>>>>>> this) way.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> The issue with JSON array only comes when we try to convert
> >>>>>>>>>>>> XML
> >>> to
> >>>>>>>>>> JSON not
> >>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
> >>>>>>>>>> outputparameters in
> >>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays.
> >>>>>>>>>>>> Therefore
> >>> we
> >>>>>>>>>> need to
> >>>>>>>>>>>> solve this issue.
> >>>>>>>>>>>>
> >>>>>>>>>>>> JSON XML JSON
> >>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> >>>>>> {"inputs":["test"]}
> >>>>>>>>>> //
> >>>>>>>>>>>> correct one
> >>>>>>>>>>>> --> {"inputs":"test"} // incorrect
> >> one
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can
> >>>>>>>>>>>>> see
> >>>>>>>>>> this
> >>>>>>>>>>>>> being
> >>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on
> >>>>>>>>>>>>> another
> >> way
> >>>>>>>>>>>>> of communicating registered applications' I/O parameters
> >>>>>>>>>>>>> to
> >> the
> >>>>>>>>>> front
> >>>>>>>>>>>>> end
> >>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> >>> problem.
> >>>>>>>>>> Are
> >>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
> >> used?
> >>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can
> >>>>>>>>>>>> solve
> >>> it.
> >>>>>> As
> >>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is
> >>>>>>>>>>>> not a
> >>> big
> >>>>>>>>>> issue
> >>>>>>>>>>>> with Airavata.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> <array name="abc">
> >>>>>>>>>>>>> <element>1</element>
> >>>>>>>>>>>>> <element>2</element>
> >>>>>>>>>>>>> <element>3</element>
> >>>>>>>>>>>>> <element>4</element>
> >>>>>>>>>>>>> </array>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Can become
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> {
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> }
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> With this example it show us we need to change the XML
> >>>>>>>>>>>> message
> >>>>>> format
> >>>>>>>>>> of
> >>>>>>>>>>>> server side, which require to change the all schemas, If we
> >>>>>>>>>>>> are
> >>>>>> going
> >>>>>>>>>> to
> >>>>>>>>>>>> change the schemas then we need to change the way it
> >>>>>>>>>>>> process it
> >>> in
> >>>>>>>>>> Ariavara
> >>>>>>>>>>>> core. We have dropped our initial major requirement, which
> >>>>>>>>>>>> is
> >>> keep
> >>>>>> the
> >>>>>>>>>>>> Airavata Server side as it is.
> >>>>>>>>>>>>
> >>>>>>>>>>>> with this conversion we only deal with json strings, yes we
> >>>>>>>>>>>> can
> >>> send
> >>>>>>>>>> JSON
> >>>>>>>>>>>> request with other formats supported by JSON like boolen,
> >>>>>>>>>>>> null,
> >>>>>> Number
> >>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as
> >>>>>>>>>>>> XML
> >>> only
> >>>>>>>>>> deal
> >>>>>>>>>>>> only with Strings. I think it is good if we can consume a
> >>>>>>>>>>>> this
> >>>>>>>> features
> >>>>>>>>>>>> with JSON.
> >>>>>>>>>>>>
> >>>>>>>>>>>> let say i need to send a integer or float to the server
> >>>>>>>>>>>> using
> >>> JSON
> >>>>>>>> then
> >>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works
> >>>>>>>>>>>> fine
> >> but
> >>>>>>>>>> problem is
> >>>>>>>>>>>> how we get the same output ?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Shameera.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>> Subho.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best Regards,
> >>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>
> >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Best Regards,
> >>>>>>> Shameera Rathnayaka.
> >>>>>>>
> >>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com Blog :
> >>>>>>> http://shameerarathnayaka.blogspot.com/
> >>>>>>
> >>>>>>
> >>>>
> >>>
> >>>
> >>
>
>
RE: Airavata GSoC 2013 Master Project
Posted by Viknes Balasubramanee <vi...@msn.com>.
Hey Guys,
Can you please share in the list how to associate our proposal with a
specific project.
Thanks
Viknes
-----Original Message-----
From: Suresh Marru [mailto:smarru@apache.org]
Sent: Friday, May 03, 2013 9:11 AM
To: dev@airavata.apache.org
Subject: Re: Airavata GSoC 2013 Master Project
Hi Subho,
I do not see your proposal associated with Airavata. May be Danushka or
Shameera can tell you how they tagged it to airavata.
Suresh
On May 3, 2013, at 8:43 AM, Subho Banerjee <su...@gmail.com> wrote:
> Hi,
> You can find a rough draft of my proposal at
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sub
> hobanerjee/13001
>
> I am now working against the clock (6 hours left) to iron out the
> edges and to put in any content I am missing.
>
> Any comments would be welcome.
>
> Cheers,
> Subho,
>
>
> On Fri, May 3, 2013 at 5:24 PM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
>> Please find the proposal for "AMQP Messaging protocol support for
>> Airavata WS-Messenger" at [1]
>>
>> Thanks,
>> Danushka
>>
>> [1] -
>>
>> https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc20
>> 13/danushka/1#
>>
>>
>> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
>>
>>> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>>>
>>> Suresh
>>>
>>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
>>>
>>>> Hi Vijayendra,
>>>>
>>>> As you can see from Shameera's proposal, he proposed a JSON
>>>> conversion
>>> in front of WS Messenger. Also Danuska has been proposing for the
>>> AMQP
>> and
>>> idea and deliberating its advantages. So given all these, I would
>>> suggest for you to keep focused on the UI aspects of the monitoring
>>> and write
>> into
>>> your proposal a plan for determining a good strategy for the
>>> plumbing to WS-Eventing based existing system. You can have the
>>> concrete deliverables of new UI to change colors based on executions
>>> (as it currently does in XBaya), double click and show error
>>> messages and so forth. And keep it exploratory for the actually
messaging format.
>>>>
>>>> I do not have any opinion on the libraries you mentioned, but yaa a
>> ajax
>>> based pub system might be the right way to go. Pending the content
>>> format (JSON or WS-Eventing or JMS or AMQP or something else)
>>>>
>>>> Suresh
>>>>
>>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
>>> vijayendra.sdm@gmail.com> wrote:
>>>>
>>>>> Hi Suresh
>>>>>
>>>>> I am writing proposal for monitoring tool . The monitoring tool is
>>> based on
>>>>> pub-sub model (ws-messenger).
>>>>> While writing proposal , I have to back it by technical stuff that
>> tells
>>>>> how can we achieve our purpose.
>>>>> As this monitoring tool is supposed to be a web based , and we are
>>> thinking
>>>>> in the lines of
>>>>> developing it in javascript.
>>>>> I was looking into javascript libraries that can we used with
>>> ws-messenger
>>>>> in the monitoring module.
>>>>> Please correct me if I am wrong.
>>>>> I came across some of the libraries
>>>>>
>>>>> - jQuery custom
>>>>> events<
>>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
>>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
>>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
>>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
>>>>>
>>>>> please tell me am I thinking in right direction?
>>>>>
>>>>> Regards
>>>>> Vijayendra
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
>> wrote:
>>>>>
>>>>>> Hi Shameera,
>>>>>>
>>>>>> This is great, I appreciate you sharing it, I realize this is
>>>>>> still working document, but I want other students to start seeing
>>>>>> it and
>>> model
>>>>>> their proposals in a similar way.
>>>>>>
>>>>>> Airavata Mentors,
>>>>>>
>>>>>> Please provide feedback directly on the melange site and uncheck
>>>>>> the "private" box when you comment.
>>>>>>
>>>>>> Suresh
>>>>>>
>>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
>>> shameerainfo@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Suresh and All,
>>>>>>>
>>>>>>> Of course I am very much happy to share my proposal with
>>>>>>> everybody, actually i was going to update this thread with the
>>>>>>> melange link in
>>> few
>>>>>>> hours once i have done writing all the sections in the proposal.
>>>>>>> I
>>>>>> haven't
>>>>>>> yet added the milestone plan into it and now working on it.
>>>>>>>
>>>>>>> The sub area i am going to work from the Master project is '
>>>>>> Implementing
>>>>>>> a JSON interface to Airavata Client side and Registry component'.
>>> Here is
>>>>>>> the link
>>>>>>>
>>>>>>
>>>
>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/sh
>> ameera/60002#
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>> Please note that i haven't completed everything in this and
>>>>>>> still
>>> doing
>>>>>>> modifications .Therefore proposal content may be changed bit,
>>>>>>> need
>> to
>>> add
>>>>>>> more technical details of the approach which explains it well.
>>>>>>>
>>>>>>> I would like to know the feedback from all of you regarding the
>>> proposal
>>>>>>> and will be modifying it if there is anything to be done. Also
>> please
>>>>>>> contact me if you need any help and i am very much willing to
>>>>>>> share
>> my
>>>>>>> thoughts with all.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Shameera
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
>>> wrote:
>>>>>>>
>>>>>>>> Hi Shameera,
>>>>>>>>
>>>>>>>> Excellent proposal, great job. Would you mind to make your
>> proposal
>>>>>>>> public and post the link here? Your proposal should help others
>>>>>>>> to
>>> look
>>>>>> at
>>>>>>>> it and learn from.
>>>>>>>>
>>>>>>>> Again I emphasize to all students, please don't feel you will
>>>>>>>> be
>>>>>> competing
>>>>>>>> with each others. If all of you write good proposals, there is
>>>>>>>> a
>> good
>>>>>>>> chance all of you will be selected. But without a good
>>>>>>>> proposal, we
>>>>>> cannot
>>>>>>>> help.
>>>>>>>>
>>>>>>>> Suresh
>>>>>>>>
>>>>>>>>
>>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
>>>>>> shameerainfo@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Yes it is not easy to solve all problems, But defining our own
>>> standard
>>>>>>>> or
>>>>>>>>> adhere to any standard
>>>>>>>>> provided by third party library will solve the problem to some
>>> extend.
>>>>>>>>>
>>>>>>>>> Here i see two possible approaches,
>>>>>>>>>
>>>>>>>>> 1. Use existing third party library(we can find which is best)
>>> adhere
>>>>>> to
>>>>>>>> it
>>>>>>>>> standard and see how we change the backend to be inline with
>>>>>>>>> it.
>>>>>>>>>
>>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
>>> suggest).
>>>>>>>>>
>>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
>> compare
>>>>>>>>> performance
>>>>>>>>> and changes need to be done in server side. Then select the
>>>>>>>>> best
>>> one.
>>>>>>>>>
>>>>>>>>> Another question was, can we works with graph data in JSON format.
>>>>>>>>> There are few JS graph framworks[1] which provide that
>>> functionality.
>>>>>>>>> we can use one of them to show airavata monitoring data as
>>>>>>>>> graphs
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Shameera.
>>>>>>>>>
>>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
>> http://d3js.org/>
>>> ,
>>>>>>>>> Processing.js <http://processingjs.org> , Sencha
>>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru
>>>>>>>>> <sm...@apache.org>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Vijeyandra,
>>>>>>>>>>
>>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
>>>>>> themselves
>>>>>>>>>> are xml (WS-Eventing [1]).
>>>>>>>>>>
>>>>>>>>>> The Messenger paper [2] should give you more information.
>>>>>>>>>>
>>>>>>>>>> Hi All (Especially those at WS02):
>>>>>>>>>>
>>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you
>>>>>>>>>> may
>>> want
>>>>>> to
>>>>>>>>>> read through these papers and chat with the authors to get
>>>>>> experiences:
>>>>>>>>>>
>>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>>>>>>>>
>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>>>>>>>>
>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>>>>>>>> http://mooshabaya.wikidot.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-ma
>> shups-from-workflows/
>>>>>>>>>>
>>>>>>>>>> Suresh
>>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>>>>>>>> [2] -
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger
>>> .pdf
>>>>>>>>>>
>>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Suresh
>>>>>>>>>>>
>>>>>>>>>>> I wanted to know more about the monitoring tool .
>>>>>>>>>>> Currently from where does the monitoring tool gets data . Is
>>>>>>>>>>> it
>>> from
>>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
>> might
>>>>>>>>>> continuously
>>>>>>>>>>> send messages to monitoring tool as to tell how much is the
>>> progress
>>>>>>>>>>> and what are the variables getting changed) ?
>>>>>>>>>>>
>>>>>>>>>>> Again the how is the data being exchanged. I guess it must
>>>>>>>>>>> be
>> xml
>>> ?
>>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
>>>>>>>>>>> monitoring module.
>>>>>>>>>>> Then monitoring Tool is sending back this data to Xbaya for
>>>>>>>>>>> displaying to the user ? Please correct me if
>> I
>>> am
>>>>>>>>>> wrong
>>>>>>>>>>>
>>>>>>>>>>> I have downloaded the source code from the trunk . can you
>> please
>>>>>> point
>>>>>>>>>>> me which part of code should I be code at for this module.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Vijayendra
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>> What i am suggesting is, we send the JSON message directly
>>>>>>>>>>> to
>>>>>> Airavata
>>>>>>>>>>> Backend(or Registry)
>>>>>>>>>>> When the message gets build after the transport phase,
>>>>>>>>>>> convert
>>> JSON
>>>>>>>>>> message
>>>>>>>>>>> to SOAP(XML).
>>>>>>>>>>> From that point message will treated as SOAP message.
>>>>>>>>>>>
>>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
>> third
>>>>>> party
>>>>>>>>>>> libraries we
>>>>>>>>>>> can use for. But before selecting a one we need to think
>>>>>>>>>>> about
>>>>>> problems
>>>>>>>>>>> having
>>>>>>>>>>>
>>>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
>>>>>> Because
>>>>>>>>>> we
>>>>>>>>>>> need a robust
>>>>>>>>>>> way to do this conversions.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
>>> directly
>>>>>>>> to
>>>>>>>>>> Registry.
>>>>>>>>>>> when the message gets built after the transport phase ,
>>>>>>>>>>> convert
>>> it to
>>>>>>>>>> SOAP .
>>>>>>>>>>>
>>>>>>>>>>> If you are suggesting Registry will have JSON data instead
>>>>>>>>>>> of
>>> WSDL ,
>>>>>>>>>> Then this might
>>>>>>>>>>> complicate the things for us .
>>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
>>> workflows
>>>>>> and
>>>>>>>>>> for other details .
>>>>>>>>>>> Which means we might again have to do some changes with
>>>>>>>>>>> workflow
>>>>>>>>>> interpretor .
>>>>>>>>>>> Yesterday from what I heard in discussion is that , they do
>>>>>>>>>>> not
>>> want
>>>>>> to
>>>>>>>>>> mess with workflow
>>>>>>>>>>> interpreter atleast for GSOC projects.
>>>>>>>>>>>
>>>>>>>>>>> What I want to suggest is , why carry the JSON data till
>>> Regisrty .
>>>>>>>>>> Build a interface
>>>>>>>>>>> before (Apache server API) where we can do the necessary
>>> conversion
>>>>>>>>>> (JSON to SOAP).
>>>>>>>>>>> In this way we can avoid messing up with Airavata server as
>>>>>>>>>>> a
>>> whole.
>>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
>>> service)
>>>>>> but
>>>>>>>>>> the Apache server
>>>>>>>>>>> is interacting with SOAP.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
>>>>>> connections
>>>>>>>>>> of the workflow.
>>>>>>>>>>> for example , the workflow is expecting a file as input but
>>>>>>>>>>> user is giving a sting or int .
>>>>>>>>>>>
>>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
>> for a
>>>>>>>>>> particular
>>>>>>>>>>> workflow , we can add extra information in the form of
>>>>>>>>>>> annotation as the kind of input/ output the workflow is
>>> accepting.
>>>>>>>>>>> Then we will be able to check these against users entry
>>>>>>>>>>> during
>>>>>>>> execution.
>>>>>>>>>>> Please correct me if I am wrong.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Vijayendra
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
>>> subs.zero@gmail.com
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>> Well exactly, as long as you can define standard way of
>>>>>> communication.
>>>>>>>>>> That
>>>>>>>>>>> is, you can define in advance what should be a string, array
>>>>>>>>>>> and
>>> what
>>>>>>>>>>> should be a integer etc. We have no problem.
>>>>>>>>>>>
>>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the
>>>>>>>>>>> other
>> way
>>>>>>>> round,
>>>>>>>>>>> they talk of the very general case (where you no nothing
>>>>>>>>>>> about
>> the
>>>>>> data
>>>>>>>>>> you
>>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
>>> myriad
>>>>>> of
>>>>>>>>>>> problems in that case, which you pointed out.
>>>>>>>>>>>
>>>>>>>>>>> But when there is standard, there is only one way of doing
>> things,
>>>>>> and
>>>>>>>>>> not
>>>>>>>>>>> several. I think that is the way forward. So what I am
>>>>>>>>>>> proposing
>>> is
>>>>>>>> maybe
>>>>>>>>>>> we all discuss and define this standard within the first
>>>>>>>>>>> week of
>>> GSoC
>>>>>>>>>>> starting and then actually move into coding. So as long as
>>>>>>>>>>> we
>> work
>>>>>> with
>>>>>>>>>> the
>>>>>>>>>>> presumption that this will be done, we really dont have to
>> worry a
>>>>>> lot
>>>>>>>>>>> about this.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Subho.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
>>>>>> subs.zero@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Some of these problems are very specific to what the XML
>>>>>>>>>>>>> is
>>>>>>>>>>>> representing,
>>>>>>>>>>>>> it might not be an actual problem in Airavata, maybe some
>>>>>>>>>>>>> one more experienced with the codebase can point
>> this
>>>>>> out.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> All issues pointed out in the paper is not directly valid
>>>>>>>>>>>> to
>> our
>>>>>>>>>>>> conversion, I didn't list the issues actually need to
>>>>>>>>>>>> address
>> in
>>>>>> this
>>>>>>>>>> case
>>>>>>>>>>>> because thought it is worth to read that introduction part
>> which
>>>>>>>>>> explain
>>>>>>>>>>>> the all the issues we have with this conversion and give us
>>>>>>>>>>>> a
>>> solid
>>>>>>>>>>>> background of that.
>>>>>>>>>>>>
>>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character
>>>>>>>>>>>>> sets
>> --
>>> I
>>>>>>>>>>>> really
>>>>>>>>>>>>> dont see these as problems, as long as you can agree that
>>>>>>>>>>>>> all
>>>>>>>>>> parts of
>>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we
>>>>>>>>>>>>> have
>> to
>>>>>>>>>> define
>>>>>>>>>>>>> this) way.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The issue with JSON array only comes when we try to convert
>>>>>>>>>>>> XML
>>> to
>>>>>>>>>> JSON not
>>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
>>>>>>>>>> outputparameters in
>>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays.
>>>>>>>>>>>> Therefore
>>> we
>>>>>>>>>> need to
>>>>>>>>>>>> solve this issue.
>>>>>>>>>>>>
>>>>>>>>>>>> JSON XML JSON
>>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
>>>>>> {"inputs":["test"]}
>>>>>>>>>> //
>>>>>>>>>>>> correct one
>>>>>>>>>>>> --> {"inputs":"test"} // incorrect
>> one
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>>>>>>>
>>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can
>>>>>>>>>>>>> see
>>>>>>>>>> this
>>>>>>>>>>>>> being
>>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on
>>>>>>>>>>>>> another
>> way
>>>>>>>>>>>>> of communicating registered applications' I/O parameters
>>>>>>>>>>>>> to
>> the
>>>>>>>>>> front
>>>>>>>>>>>>> end
>>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
>>> problem.
>>>>>>>>>> Are
>>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
>> used?
>>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can
>>>>>>>>>>>> solve
>>> it.
>>>>>> As
>>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is
>>>>>>>>>>>> not a
>>> big
>>>>>>>>>> issue
>>>>>>>>>>>> with Airavata.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> <array name="abc">
>>>>>>>>>>>>> <element>1</element>
>>>>>>>>>>>>> <element>2</element>
>>>>>>>>>>>>> <element>3</element>
>>>>>>>>>>>>> <element>4</element>
>>>>>>>>>>>>> </array>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can become
>>>>>>>>>>>>>
>>>>>>>>>>>>> {
>>>>>>>>>>>>>
>>>>>>>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> With this example it show us we need to change the XML
>>>>>>>>>>>> message
>>>>>> format
>>>>>>>>>> of
>>>>>>>>>>>> server side, which require to change the all schemas, If we
>>>>>>>>>>>> are
>>>>>> going
>>>>>>>>>> to
>>>>>>>>>>>> change the schemas then we need to change the way it
>>>>>>>>>>>> process it
>>> in
>>>>>>>>>> Ariavara
>>>>>>>>>>>> core. We have dropped our initial major requirement, which
>>>>>>>>>>>> is
>>> keep
>>>>>> the
>>>>>>>>>>>> Airavata Server side as it is.
>>>>>>>>>>>>
>>>>>>>>>>>> with this conversion we only deal with json strings, yes we
>>>>>>>>>>>> can
>>> send
>>>>>>>>>> JSON
>>>>>>>>>>>> request with other formats supported by JSON like boolen,
>>>>>>>>>>>> null,
>>>>>> Number
>>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as
>>>>>>>>>>>> XML
>>> only
>>>>>>>>>> deal
>>>>>>>>>>>> only with Strings. I think it is good if we can consume a
>>>>>>>>>>>> this
>>>>>>>> features
>>>>>>>>>>>> with JSON.
>>>>>>>>>>>>
>>>>>>>>>>>> let say i need to send a integer or float to the server
>>>>>>>>>>>> using
>>> JSON
>>>>>>>> then
>>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works
>>>>>>>>>>>> fine
>> but
>>>>>>>>>> problem is
>>>>>>>>>>>> how we get the same output ?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Subho.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>>>>
>>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>
>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Shameera Rathnayaka.
>>>>>>>
>>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com Blog :
>>>>>>> http://shameerarathnayaka.blogspot.com/
>>>>>>
>>>>>>
>>>>
>>>
>>>
>>
Re: Airavata GSoC 2013 Master Project
Posted by Suresh Marru <sm...@apache.org>.
Hi Subho,
I do not see your proposal associated with Airavata. May be Danushka or Shameera can tell you how they tagged it to airavata.
Suresh
On May 3, 2013, at 8:43 AM, Subho Banerjee <su...@gmail.com> wrote:
> Hi,
> You can find a rough draft of my proposal at
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/subhobanerjee/13001
>
> I am now working against the clock (6 hours left) to iron out the edges and
> to put in any content I am missing.
>
> Any comments would be welcome.
>
> Cheers,
> Subho,
>
>
> On Fri, May 3, 2013 at 5:24 PM, Danushka Menikkumbura <
> danushka.menikkumbura@gmail.com> wrote:
>
>> Please find the proposal for "AMQP Messaging protocol support for Airavata
>> WS-Messenger" at [1]
>>
>> Thanks,
>> Danushka
>>
>> [1] -
>>
>> https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/danushka/1#
>>
>>
>> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
>>
>>> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>>>
>>> Suresh
>>>
>>> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
>>>
>>>> Hi Vijayendra,
>>>>
>>>> As you can see from Shameera's proposal, he proposed a JSON conversion
>>> in front of WS Messenger. Also Danuska has been proposing for the AMQP
>> and
>>> idea and deliberating its advantages. So given all these, I would suggest
>>> for you to keep focused on the UI aspects of the monitoring and write
>> into
>>> your proposal a plan for determining a good strategy for the plumbing to
>>> WS-Eventing based existing system. You can have the concrete deliverables
>>> of new UI to change colors based on executions (as it currently does in
>>> XBaya), double click and show error messages and so forth. And keep it
>>> exploratory for the actually messaging format.
>>>>
>>>> I do not have any opinion on the libraries you mentioned, but yaa a
>> ajax
>>> based pub system might be the right way to go. Pending the content format
>>> (JSON or WS-Eventing or JMS or AMQP or something else)
>>>>
>>>> Suresh
>>>>
>>>> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
>>> vijayendra.sdm@gmail.com> wrote:
>>>>
>>>>> Hi Suresh
>>>>>
>>>>> I am writing proposal for monitoring tool . The monitoring tool is
>>> based on
>>>>> pub-sub model (ws-messenger).
>>>>> While writing proposal , I have to back it by technical stuff that
>> tells
>>>>> how can we achieve our purpose.
>>>>> As this monitoring tool is supposed to be a web based , and we are
>>> thinking
>>>>> in the lines of
>>>>> developing it in javascript.
>>>>> I was looking into javascript libraries that can we used with
>>> ws-messenger
>>>>> in the monitoring module.
>>>>> Please correct me if I am wrong.
>>>>> I came across some of the libraries
>>>>>
>>>>> - jQuery custom
>>>>> events<
>>> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
>>>>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
>>>>> - PubSubJS <https://github.com/mroderick/PubSubJS>
>>>>> - js-signals <http://millermedeiros.github.com/js-signals/>
>>>>>
>>>>> please tell me am I thinking in right direction?
>>>>>
>>>>> Regards
>>>>> Vijayendra
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
>> wrote:
>>>>>
>>>>>> Hi Shameera,
>>>>>>
>>>>>> This is great, I appreciate you sharing it, I realize this is still
>>>>>> working document, but I want other students to start seeing it and
>>> model
>>>>>> their proposals in a similar way.
>>>>>>
>>>>>> Airavata Mentors,
>>>>>>
>>>>>> Please provide feedback directly on the melange site and uncheck the
>>>>>> "private" box when you comment.
>>>>>>
>>>>>> Suresh
>>>>>>
>>>>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
>>> shameerainfo@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Suresh and All,
>>>>>>>
>>>>>>> Of course I am very much happy to share my proposal with everybody,
>>>>>>> actually i was going to update this thread with the melange link in
>>> few
>>>>>>> hours once i have done writing all the sections in the proposal. I
>>>>>> haven't
>>>>>>> yet added the milestone plan into it and now working on it.
>>>>>>>
>>>>>>> The sub area i am going to work from the Master project is '
>>>>>> Implementing
>>>>>>> a JSON interface to Airavata Client side and Registry component'.
>>> Here is
>>>>>>> the link
>>>>>>>
>>>>>>
>>>
>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>> Please note that i haven't completed everything in this and still
>>> doing
>>>>>>> modifications .Therefore proposal content may be changed bit, need
>> to
>>> add
>>>>>>> more technical details of the approach which explains it well.
>>>>>>>
>>>>>>> I would like to know the feedback from all of you regarding the
>>> proposal
>>>>>>> and will be modifying it if there is anything to be done. Also
>> please
>>>>>>> contact me if you need any help and i am very much willing to share
>> my
>>>>>>> thoughts with all.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Shameera
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
>>> wrote:
>>>>>>>
>>>>>>>> Hi Shameera,
>>>>>>>>
>>>>>>>> Excellent proposal, great job. Would you mind to make your
>> proposal
>>>>>>>> public and post the link here? Your proposal should help others to
>>> look
>>>>>> at
>>>>>>>> it and learn from.
>>>>>>>>
>>>>>>>> Again I emphasize to all students, please don't feel you will be
>>>>>> competing
>>>>>>>> with each others. If all of you write good proposals, there is a
>> good
>>>>>>>> chance all of you will be selected. But without a good proposal, we
>>>>>> cannot
>>>>>>>> help.
>>>>>>>>
>>>>>>>> Suresh
>>>>>>>>
>>>>>>>>
>>>>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
>>>>>> shameerainfo@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Yes it is not easy to solve all problems, But defining our own
>>> standard
>>>>>>>> or
>>>>>>>>> adhere to any standard
>>>>>>>>> provided by third party library will solve the problem to some
>>> extend.
>>>>>>>>>
>>>>>>>>> Here i see two possible approaches,
>>>>>>>>>
>>>>>>>>> 1. Use existing third party library(we can find which is best)
>>> adhere
>>>>>> to
>>>>>>>> it
>>>>>>>>> standard and see how we change the
>>>>>>>>> backend to be inline with it.
>>>>>>>>>
>>>>>>>>> 2. Use our own convention with help of XMLSchema (The way i
>>> suggest).
>>>>>>>>>
>>>>>>>>> As Suresh mentioned we can do a POC with both approaches to
>> compare
>>>>>>>>> performance
>>>>>>>>> and changes need to be done in server side. Then select the best
>>> one.
>>>>>>>>>
>>>>>>>>> Another question was, can we works with graph data in JSON format.
>>>>>>>>> There are few JS graph framworks[1] which provide that
>>> functionality.
>>>>>>>>> we can use one of them to show airavata monitoring data as graphs
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Shameera.
>>>>>>>>>
>>>>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
>> http://d3js.org/>
>>> ,
>>>>>>>>> Processing.js <http://processingjs.org> , Sencha
>>>>>>>>> Charts<http://www.sencha.com/products/extjs/>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Vijeyandra,
>>>>>>>>>>
>>>>>>>>>> Airavata Messaging is based on a pub-sub model and the events
>>>>>> themselves
>>>>>>>>>> are xml (WS-Eventing [1]).
>>>>>>>>>>
>>>>>>>>>> The Messenger paper [2] should give you more information.
>>>>>>>>>>
>>>>>>>>>> Hi All (Especially those at WS02):
>>>>>>>>>>
>>>>>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
>>> want
>>>>>> to
>>>>>>>>>> read through these papers and chat with the authors to get
>>>>>> experiences:
>>>>>>>>>>
>>>>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>>>>>>>>
>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>>>>>>>>
>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>>>>>>>> http://mooshabaya.wikidot.com/
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>>>>>>>>>
>>>>>>>>>> Suresh
>>>>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>>>>>>>> [2] -
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>>>>>>>>>
>>>>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Suresh
>>>>>>>>>>>
>>>>>>>>>>> I wanted to know more about the monitoring tool .
>>>>>>>>>>> Currently from where does the monitoring tool gets data . Is it
>>> from
>>>>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
>> might
>>>>>>>>>> continuously
>>>>>>>>>>> send messages to monitoring tool as to tell how much is the
>>> progress
>>>>>>>>>>> and what are the variables getting changed) ?
>>>>>>>>>>>
>>>>>>>>>>> Again the how is the data being exchanged. I guess it must be
>> xml
>>> ?
>>>>>>>>>>> It must be one way data exchange . I mean the data is TO the
>>>>>>>>>>> monitoring module.
>>>>>>>>>>> Then monitoring Tool is sending back this
>>>>>>>>>>> data to Xbaya for displaying to the user ? Please correct me if
>> I
>>> am
>>>>>>>>>> wrong
>>>>>>>>>>>
>>>>>>>>>>> I have downloaded the source code from the trunk . can you
>> please
>>>>>> point
>>>>>>>>>>> me which part of code should I be code at for this module.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Vijayendra
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>>>>> Hi
>>>>>>>>>>>
>>>>>>>>>>> What i am suggesting is, we send the JSON message directly to
>>>>>> Airavata
>>>>>>>>>>> Backend(or Registry)
>>>>>>>>>>> When the message gets build after the transport phase, convert
>>> JSON
>>>>>>>>>> message
>>>>>>>>>>> to SOAP(XML).
>>>>>>>>>>> From that point message will treated as SOAP message.
>>>>>>>>>>>
>>>>>>>>>>> If we look at the JSON <--> XML conversion there are set of
>> third
>>>>>> party
>>>>>>>>>>> libraries we
>>>>>>>>>>> can use for. But before selecting a one we need to think about
>>>>>> problems
>>>>>>>>>>> having
>>>>>>>>>>>
>>>>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
>>>>>> Because
>>>>>>>>>> we
>>>>>>>>>>> need a robust
>>>>>>>>>>> way to do this conversions.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Shameera what you are suggesting is sending the JSON message
>>> directly
>>>>>>>> to
>>>>>>>>>> Registry.
>>>>>>>>>>> when the message gets built after the transport phase , convert
>>> it to
>>>>>>>>>> SOAP .
>>>>>>>>>>>
>>>>>>>>>>> If you are suggesting Registry will have JSON data instead of
>>> WSDL ,
>>>>>>>>>> Then this might
>>>>>>>>>>> complicate the things for us .
>>>>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
>>> workflows
>>>>>> and
>>>>>>>>>> for other details .
>>>>>>>>>>> Which means we might again have to do some changes with workflow
>>>>>>>>>> interpretor .
>>>>>>>>>>> Yesterday from what I heard in discussion is that , they do not
>>> want
>>>>>> to
>>>>>>>>>> mess with workflow
>>>>>>>>>>> interpreter atleast for GSOC projects.
>>>>>>>>>>>
>>>>>>>>>>> What I want to suggest is , why carry the JSON data till
>>> Regisrty .
>>>>>>>>>> Build a interface
>>>>>>>>>>> before (Apache server API) where we can do the necessary
>>> conversion
>>>>>>>>>> (JSON to SOAP).
>>>>>>>>>>> In this way we can avoid messing up with Airavata server as a
>>> whole.
>>>>>>>>>>> Client ( using a we browser) is interacting with JSON (web
>>> service)
>>>>>> but
>>>>>>>>>> the Apache server
>>>>>>>>>>> is interacting with SOAP.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Secondly yesterday Suresh was speaking about validating the
>>>>>> connections
>>>>>>>>>> of the workflow.
>>>>>>>>>>> for example , the workflow is expecting a file as input
>>>>>>>>>>> but user is giving a sting or int .
>>>>>>>>>>>
>>>>>>>>>>> Here what I suggest is , while creating wsdl in the registry
>> for a
>>>>>>>>>> particular
>>>>>>>>>>> workflow , we can add extra information in the form of
>>>>>>>>>>> annotation as the kind of input/ output the workflow is
>>> accepting.
>>>>>>>>>>> Then we will be able to check these against users entry during
>>>>>>>> execution.
>>>>>>>>>>> Please correct me if I am wrong.
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Vijayendra
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
>>> subs.zero@gmail.com
>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>> Well exactly, as long as you can define standard way of
>>>>>> communication.
>>>>>>>>>> That
>>>>>>>>>>> is, you can define in advance what should be a string, array and
>>> what
>>>>>>>>>>> should be a integer etc. We have no problem.
>>>>>>>>>>>
>>>>>>>>>>> So, when you look at problems, with JSON <-> XML or the other
>> way
>>>>>>>> round,
>>>>>>>>>>> they talk of the very general case (where you no nothing about
>> the
>>>>>> data
>>>>>>>>>> you
>>>>>>>>>>> are converting other than it is valid XML/JSON). There are a
>>> myriad
>>>>>> of
>>>>>>>>>>> problems in that case, which you pointed out.
>>>>>>>>>>>
>>>>>>>>>>> But when there is standard, there is only one way of doing
>> things,
>>>>>> and
>>>>>>>>>> not
>>>>>>>>>>> several. I think that is the way forward. So what I am proposing
>>> is
>>>>>>>> maybe
>>>>>>>>>>> we all discuss and define this standard within the first week of
>>> GSoC
>>>>>>>>>>> starting and then actually move into coding. So as long as we
>> work
>>>>>> with
>>>>>>>>>> the
>>>>>>>>>>> presumption that this will be done, we really dont have to
>> worry a
>>>>>> lot
>>>>>>>>>>> about this.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Subho.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
>>>>>> subs.zero@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Some of these problems are very specific to what the XML is
>>>>>>>>>>>> representing,
>>>>>>>>>>>>> it might not be an actual problem in Airavata,
>>>>>>>>>>>>> maybe some one more experienced with the codebase can point
>> this
>>>>>> out.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> All issues pointed out in the paper is not directly valid to
>> our
>>>>>>>>>>>> conversion, I didn't list the issues actually need to address
>> in
>>>>>> this
>>>>>>>>>> case
>>>>>>>>>>>> because thought it is worth to read that introduction part
>> which
>>>>>>>>>> explain
>>>>>>>>>>>> the all the issues we have with this conversion and give us a
>>> solid
>>>>>>>>>>>> background of that.
>>>>>>>>>>>>
>>>>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets
>> --
>>> I
>>>>>>>>>>>> really
>>>>>>>>>>>>> dont see these as problems, as long as you can agree that all
>>>>>>>>>> parts of
>>>>>>>>>>>>> airavata will treat the JSON in a standard (probably we have
>> to
>>>>>>>>>> define
>>>>>>>>>>>>> this) way.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> The issue with JSON array only comes when we try to convert XML
>>> to
>>>>>>>>>> JSON not
>>>>>>>>>>>> the other way. If we map with JSON, inputparameters and
>>>>>>>>>> outputparameters in
>>>>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
>>> we
>>>>>>>>>> need to
>>>>>>>>>>>> solve this issue.
>>>>>>>>>>>>
>>>>>>>>>>>> JSON XML JSON
>>>>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
>>>>>> {"inputs":["test"]}
>>>>>>>>>> //
>>>>>>>>>>>> correct one
>>>>>>>>>>>> --> {"inputs":"test"} // incorrect
>> one
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>>>>>>>
>>>>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
>>>>>>>>>> this
>>>>>>>>>>>>> being
>>>>>>>>>>>>> used is probably in the WSDL, but if we can agree on another
>> way
>>>>>>>>>>>>> of communicating registered applications' I/O parameters to
>> the
>>>>>>>>>> front
>>>>>>>>>>>>> end
>>>>>>>>>>>>> (JSON based), then maybe we can work around this (minor)
>>> problem.
>>>>>>>>>> Are
>>>>>>>>>>>>> custom processing instructions to the Xbaya XML parse even
>> used?
>>>>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
>>> it.
>>>>>> As
>>>>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
>>> big
>>>>>>>>>> issue
>>>>>>>>>>>> with Airavata.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> <array name="abc">
>>>>>>>>>>>>> <element>1</element>
>>>>>>>>>>>>> <element>2</element>
>>>>>>>>>>>>> <element>3</element>
>>>>>>>>>>>>> <element>4</element>
>>>>>>>>>>>>> </array>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can become
>>>>>>>>>>>>>
>>>>>>>>>>>>> {
>>>>>>>>>>>>>
>>>>>>>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>>>>>>>
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> With this example it show us we need to change the XML message
>>>>>> format
>>>>>>>>>> of
>>>>>>>>>>>> server side, which require to change the all schemas, If we are
>>>>>> going
>>>>>>>>>> to
>>>>>>>>>>>> change the schemas then we need to change the way it process it
>>> in
>>>>>>>>>> Ariavara
>>>>>>>>>>>> core. We have dropped our initial major requirement, which is
>>> keep
>>>>>> the
>>>>>>>>>>>> Airavata Server side as it is.
>>>>>>>>>>>>
>>>>>>>>>>>> with this conversion we only deal with json strings, yes we can
>>> send
>>>>>>>>>> JSON
>>>>>>>>>>>> request with other formats supported by JSON like boolen, null,
>>>>>> Number
>>>>>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
>>> only
>>>>>>>>>> deal
>>>>>>>>>>>> only with Strings. I think it is good if we can consume a this
>>>>>>>> features
>>>>>>>>>>>> with JSON.
>>>>>>>>>>>>
>>>>>>>>>>>> let say i need to send a integer or float to the server using
>>> JSON
>>>>>>>> then
>>>>>>>>>>>> proper way is to send {"<name>":123.45} this will works fine
>> but
>>>>>>>>>> problem is
>>>>>>>>>>>> how we get the same output ?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Shameera.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Subho.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>>>>
>>>>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>
>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards,
>>>>>>> Shameera Rathnayaka.
>>>>>>>
>>>>>>> email: shameera AT apache.org , shameerainfo AT gmail.com
>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>
>>>>>>
>>>>
>>>
>>>
>>
Re: Airavata GSoC 2013 Master Project
Posted by Subho Banerjee <su...@gmail.com>.
Hi,
You can find a rough draft of my proposal at
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/subhobanerjee/13001
I am now working against the clock (6 hours left) to iron out the edges and
to put in any content I am missing.
Any comments would be welcome.
Cheers,
Subho,
On Fri, May 3, 2013 at 5:24 PM, Danushka Menikkumbura <
danushka.menikkumbura@gmail.com> wrote:
> Please find the proposal for "AMQP Messaging protocol support for Airavata
> WS-Messenger" at [1]
>
> Thanks,
> Danushka
>
> [1] -
>
> https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/danushka/1#
>
>
> On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
>
> > Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
> >
> > Suresh
> >
> > On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> >
> > > Hi Vijayendra,
> > >
> > > As you can see from Shameera's proposal, he proposed a JSON conversion
> > in front of WS Messenger. Also Danuska has been proposing for the AMQP
> and
> > idea and deliberating its advantages. So given all these, I would suggest
> > for you to keep focused on the UI aspects of the monitoring and write
> into
> > your proposal a plan for determining a good strategy for the plumbing to
> > WS-Eventing based existing system. You can have the concrete deliverables
> > of new UI to change colors based on executions (as it currently does in
> > XBaya), double click and show error messages and so forth. And keep it
> > exploratory for the actually messaging format.
> > >
> > > I do not have any opinion on the libraries you mentioned, but yaa a
> ajax
> > based pub system might be the right way to go. Pending the content format
> > (JSON or WS-Eventing or JMS or AMQP or something else)
> > >
> > > Suresh
> > >
> > > On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> > vijayendra.sdm@gmail.com> wrote:
> > >
> > >> Hi Suresh
> > >>
> > >> I am writing proposal for monitoring tool . The monitoring tool is
> > based on
> > >> pub-sub model (ws-messenger).
> > >> While writing proposal , I have to back it by technical stuff that
> tells
> > >> how can we achieve our purpose.
> > >> As this monitoring tool is supposed to be a web based , and we are
> > thinking
> > >> in the lines of
> > >> developing it in javascript.
> > >> I was looking into javascript libraries that can we used with
> > ws-messenger
> > >> in the monitoring module.
> > >> Please correct me if I am wrong.
> > >> I came across some of the libraries
> > >>
> > >> - jQuery custom
> > >> events<
> > http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> > >> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> > >> - PubSubJS <https://github.com/mroderick/PubSubJS>
> > >> - js-signals <http://millermedeiros.github.com/js-signals/>
> > >>
> > >> please tell me am I thinking in right direction?
> > >>
> > >> Regards
> > >> Vijayendra
> > >>
> > >>
> > >>
> > >>
> > >> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org>
> wrote:
> > >>
> > >>> Hi Shameera,
> > >>>
> > >>> This is great, I appreciate you sharing it, I realize this is still
> > >>> working document, but I want other students to start seeing it and
> > model
> > >>> their proposals in a similar way.
> > >>>
> > >>> Airavata Mentors,
> > >>>
> > >>> Please provide feedback directly on the melange site and uncheck the
> > >>> "private" box when you comment.
> > >>>
> > >>> Suresh
> > >>>
> > >>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> > shameerainfo@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Suresh and All,
> > >>>>
> > >>>> Of course I am very much happy to share my proposal with everybody,
> > >>>> actually i was going to update this thread with the melange link in
> > few
> > >>>> hours once i have done writing all the sections in the proposal. I
> > >>> haven't
> > >>>> yet added the milestone plan into it and now working on it.
> > >>>>
> > >>>> The sub area i am going to work from the Master project is '
> > >>> Implementing
> > >>>> a JSON interface to Airavata Client side and Registry component'.
> > Here is
> > >>>> the link
> > >>>>
> > >>>
> >
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> > >>>> .
> > >>>>
> > >>>>
> > >>>> Please note that i haven't completed everything in this and still
> > doing
> > >>>> modifications .Therefore proposal content may be changed bit, need
> to
> > add
> > >>>> more technical details of the approach which explains it well.
> > >>>>
> > >>>> I would like to know the feedback from all of you regarding the
> > proposal
> > >>>> and will be modifying it if there is anything to be done. Also
> please
> > >>>> contact me if you need any help and i am very much willing to share
> my
> > >>>> thoughts with all.
> > >>>>
> > >>>> Thanks!
> > >>>> Shameera
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> > wrote:
> > >>>>
> > >>>>> Hi Shameera,
> > >>>>>
> > >>>>> Excellent proposal, great job. Would you mind to make your
> proposal
> > >>>>> public and post the link here? Your proposal should help others to
> > look
> > >>> at
> > >>>>> it and learn from.
> > >>>>>
> > >>>>> Again I emphasize to all students, please don't feel you will be
> > >>> competing
> > >>>>> with each others. If all of you write good proposals, there is a
> good
> > >>>>> chance all of you will be selected. But without a good proposal, we
> > >>> cannot
> > >>>>> help.
> > >>>>>
> > >>>>> Suresh
> > >>>>>
> > >>>>>
> > >>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> > >>> shameerainfo@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> Yes it is not easy to solve all problems, But defining our own
> > standard
> > >>>>> or
> > >>>>>> adhere to any standard
> > >>>>>> provided by third party library will solve the problem to some
> > extend.
> > >>>>>>
> > >>>>>> Here i see two possible approaches,
> > >>>>>>
> > >>>>>> 1. Use existing third party library(we can find which is best)
> > adhere
> > >>> to
> > >>>>> it
> > >>>>>> standard and see how we change the
> > >>>>>> backend to be inline with it.
> > >>>>>>
> > >>>>>> 2. Use our own convention with help of XMLSchema (The way i
> > suggest).
> > >>>>>>
> > >>>>>> As Suresh mentioned we can do a POC with both approaches to
> compare
> > >>>>>> performance
> > >>>>>> and changes need to be done in server side. Then select the best
> > one.
> > >>>>>>
> > >>>>>> Another question was, can we works with graph data in JSON format.
> > >>>>>> There are few JS graph framworks[1] which provide that
> > functionality.
> > >>>>>> we can use one of them to show airavata monitoring data as graphs
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Shameera.
> > >>>>>>
> > >>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <
> http://d3js.org/>
> > ,
> > >>>>>> Processing.js <http://processingjs.org> , Sencha
> > >>>>>> Charts<http://www.sencha.com/products/extjs/>
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> Hi Vijeyandra,
> > >>>>>>>
> > >>>>>>> Airavata Messaging is based on a pub-sub model and the events
> > >>> themselves
> > >>>>>>> are xml (WS-Eventing [1]).
> > >>>>>>>
> > >>>>>>> The Messenger paper [2] should give you more information.
> > >>>>>>>
> > >>>>>>> Hi All (Especially those at WS02):
> > >>>>>>>
> > >>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
> > want
> > >>> to
> > >>>>>>> read through these papers and chat with the authors to get
> > >>> experiences:
> > >>>>>>>
> > >>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> > >>>>>>>
> > http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> > >>>>>>>
> > http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> > >>>>>>> http://mooshabaya.wikidot.com/
> > >>>>>>>
> > >>>>>>>
> > >>>>>
> > >>>
> >
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> > >>>>>>>
> > >>>>>>> Suresh
> > >>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> > >>>>>>> [2] -
> > >>>>>>>
> > >>>>>
> > >>>
> > http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> > >>>>>>>
> > >>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> > >>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>
> > >>>>>>>> Hi Suresh
> > >>>>>>>>
> > >>>>>>>> I wanted to know more about the monitoring tool .
> > >>>>>>>> Currently from where does the monitoring tool gets data . Is it
> > from
> > >>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that
> might
> > >>>>>>> continuously
> > >>>>>>>> send messages to monitoring tool as to tell how much is the
> > progress
> > >>>>>>>> and what are the variables getting changed) ?
> > >>>>>>>>
> > >>>>>>>> Again the how is the data being exchanged. I guess it must be
> xml
> > ?
> > >>>>>>>> It must be one way data exchange . I mean the data is TO the
> > >>>>>>>> monitoring module.
> > >>>>>>>> Then monitoring Tool is sending back this
> > >>>>>>>> data to Xbaya for displaying to the user ? Please correct me if
> I
> > am
> > >>>>>>> wrong
> > >>>>>>>>
> > >>>>>>>> I have downloaded the source code from the trunk . can you
> please
> > >>> point
> > >>>>>>>> me which part of code should I be code at for this module.
> > >>>>>>>>
> > >>>>>>>> Regards
> > >>>>>>>> Vijayendra
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> > >>>>>>> vijayendra.sdm@gmail.com> wrote:
> > >>>>>>>> Hi
> > >>>>>>>>
> > >>>>>>>> What i am suggesting is, we send the JSON message directly to
> > >>> Airavata
> > >>>>>>>> Backend(or Registry)
> > >>>>>>>> When the message gets build after the transport phase, convert
> > JSON
> > >>>>>>> message
> > >>>>>>>> to SOAP(XML).
> > >>>>>>>> From that point message will treated as SOAP message.
> > >>>>>>>>
> > >>>>>>>> If we look at the JSON <--> XML conversion there are set of
> third
> > >>> party
> > >>>>>>>> libraries we
> > >>>>>>>> can use for. But before selecting a one we need to think about
> > >>> problems
> > >>>>>>>> having
> > >>>>>>>>
> > >>>>>>>> with JSON <--> XML and how these libraries handle those issues.
> > >>> Because
> > >>>>>>> we
> > >>>>>>>> need a robust
> > >>>>>>>> way to do this conversions.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Shameera what you are suggesting is sending the JSON message
> > directly
> > >>>>> to
> > >>>>>>> Registry.
> > >>>>>>>> when the message gets built after the transport phase , convert
> > it to
> > >>>>>>> SOAP .
> > >>>>>>>>
> > >>>>>>>> If you are suggesting Registry will have JSON data instead of
> > WSDL ,
> > >>>>>>> Then this might
> > >>>>>>>> complicate the things for us .
> > >>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> > workflows
> > >>> and
> > >>>>>>> for other details .
> > >>>>>>>> Which means we might again have to do some changes with workflow
> > >>>>>>> interpretor .
> > >>>>>>>> Yesterday from what I heard in discussion is that , they do not
> > want
> > >>> to
> > >>>>>>> mess with workflow
> > >>>>>>>> interpreter atleast for GSOC projects.
> > >>>>>>>>
> > >>>>>>>> What I want to suggest is , why carry the JSON data till
> > Regisrty .
> > >>>>>>> Build a interface
> > >>>>>>>> before (Apache server API) where we can do the necessary
> > conversion
> > >>>>>>> (JSON to SOAP).
> > >>>>>>>> In this way we can avoid messing up with Airavata server as a
> > whole.
> > >>>>>>>> Client ( using a we browser) is interacting with JSON (web
> > service)
> > >>> but
> > >>>>>>> the Apache server
> > >>>>>>>> is interacting with SOAP.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Secondly yesterday Suresh was speaking about validating the
> > >>> connections
> > >>>>>>> of the workflow.
> > >>>>>>>> for example , the workflow is expecting a file as input
> > >>>>>>>> but user is giving a sting or int .
> > >>>>>>>>
> > >>>>>>>> Here what I suggest is , while creating wsdl in the registry
> for a
> > >>>>>>> particular
> > >>>>>>>> workflow , we can add extra information in the form of
> > >>>>>>>> annotation as the kind of input/ output the workflow is
> > accepting.
> > >>>>>>>> Then we will be able to check these against users entry during
> > >>>>> execution.
> > >>>>>>>> Please correct me if I am wrong.
> > >>>>>>>>
> > >>>>>>>> Regards
> > >>>>>>>> Vijayendra
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> > subs.zero@gmail.com
> > >>>>
> > >>>>>>> wrote:
> > >>>>>>>> Well exactly, as long as you can define standard way of
> > >>> communication.
> > >>>>>>> That
> > >>>>>>>> is, you can define in advance what should be a string, array and
> > what
> > >>>>>>>> should be a integer etc. We have no problem.
> > >>>>>>>>
> > >>>>>>>> So, when you look at problems, with JSON <-> XML or the other
> way
> > >>>>> round,
> > >>>>>>>> they talk of the very general case (where you no nothing about
> the
> > >>> data
> > >>>>>>> you
> > >>>>>>>> are converting other than it is valid XML/JSON). There are a
> > myriad
> > >>> of
> > >>>>>>>> problems in that case, which you pointed out.
> > >>>>>>>>
> > >>>>>>>> But when there is standard, there is only one way of doing
> things,
> > >>> and
> > >>>>>>> not
> > >>>>>>>> several. I think that is the way forward. So what I am proposing
> > is
> > >>>>> maybe
> > >>>>>>>> we all discuss and define this standard within the first week of
> > GSoC
> > >>>>>>>> starting and then actually move into coding. So as long as we
> work
> > >>> with
> > >>>>>>> the
> > >>>>>>>> presumption that this will be done, we really dont have to
> worry a
> > >>> lot
> > >>>>>>>> about this.
> > >>>>>>>>
> > >>>>>>>> Cheers,
> > >>>>>>>> Subho.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> > >>>>>>>> shameerainfo@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hi,
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> > >>> subs.zero@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Some of these problems are very specific to what the XML is
> > >>>>>>>>> representing,
> > >>>>>>>>>> it might not be an actual problem in Airavata,
> > >>>>>>>>>> maybe some one more experienced with the codebase can point
> this
> > >>> out.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> All issues pointed out in the paper is not directly valid to
> our
> > >>>>>>>>> conversion, I didn't list the issues actually need to address
> in
> > >>> this
> > >>>>>>> case
> > >>>>>>>>> because thought it is worth to read that introduction part
> which
> > >>>>>>> explain
> > >>>>>>>>> the all the issues we have with this conversion and give us a
> > solid
> > >>>>>>>>> background of that.
> > >>>>>>>>>
> > >>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets
> --
> > I
> > >>>>>>>>> really
> > >>>>>>>>>> dont see these as problems, as long as you can agree that all
> > >>>>>>> parts of
> > >>>>>>>>>> airavata will treat the JSON in a standard (probably we have
> to
> > >>>>>>> define
> > >>>>>>>>>> this) way.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> The issue with JSON array only comes when we try to convert XML
> > to
> > >>>>>>> JSON not
> > >>>>>>>>> the other way. If we map with JSON, inputparameters and
> > >>>>>>> outputparameters in
> > >>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
> > we
> > >>>>>>> need to
> > >>>>>>>>> solve this issue.
> > >>>>>>>>>
> > >>>>>>>>> JSON XML JSON
> > >>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> > >>> {"inputs":["test"]}
> > >>>>>>> //
> > >>>>>>>>> correct one
> > >>>>>>>>> --> {"inputs":"test"} // incorrect
> one
> > >>>>>>>>>
> > >>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> > >>>>>>>>>
> > >>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
> > >>>>>>> this
> > >>>>>>>>>> being
> > >>>>>>>>>> used is probably in the WSDL, but if we can agree on another
> way
> > >>>>>>>>>> of communicating registered applications' I/O parameters to
> the
> > >>>>>>> front
> > >>>>>>>>>> end
> > >>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> > problem.
> > >>>>>>> Are
> > >>>>>>>>>> custom processing instructions to the Xbaya XML parse even
> used?
> > >>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
> > it.
> > >>> As
> > >>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
> > big
> > >>>>>>> issue
> > >>>>>>>>> with Airavata.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> <array name="abc">
> > >>>>>>>>>> <element>1</element>
> > >>>>>>>>>> <element>2</element>
> > >>>>>>>>>> <element>3</element>
> > >>>>>>>>>> <element>4</element>
> > >>>>>>>>>> </array>
> > >>>>>>>>>>
> > >>>>>>>>>> Can become
> > >>>>>>>>>>
> > >>>>>>>>>> {
> > >>>>>>>>>>
> > >>>>>>>>>> abc : ['1', '2', '3', '4']
> > >>>>>>>>>>
> > >>>>>>>>>> }
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> With this example it show us we need to change the XML message
> > >>> format
> > >>>>>>> of
> > >>>>>>>>> server side, which require to change the all schemas, If we are
> > >>> going
> > >>>>>>> to
> > >>>>>>>>> change the schemas then we need to change the way it process it
> > in
> > >>>>>>> Ariavara
> > >>>>>>>>> core. We have dropped our initial major requirement, which is
> > keep
> > >>> the
> > >>>>>>>>> Airavata Server side as it is.
> > >>>>>>>>>
> > >>>>>>>>> with this conversion we only deal with json strings, yes we can
> > send
> > >>>>>>> JSON
> > >>>>>>>>> request with other formats supported by JSON like boolen, null,
> > >>> Number
> > >>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
> > only
> > >>>>>>> deal
> > >>>>>>>>> only with Strings. I think it is good if we can consume a this
> > >>>>> features
> > >>>>>>>>> with JSON.
> > >>>>>>>>>
> > >>>>>>>>> let say i need to send a integer or float to the server using
> > JSON
> > >>>>> then
> > >>>>>>>>> proper way is to send {"<name>":123.45} this will works fine
> but
> > >>>>>>> problem is
> > >>>>>>>>> how we get the same output ?
> > >>>>>>>>>
> > >>>>>>>>> Thanks,
> > >>>>>>>>> Shameera.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Cheers,
> > >>>>>>>>>> Subho.
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Best Regards,
> > >>>>>>>>> Shameera Rathnayaka.
> > >>>>>>>>>
> > >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Best Regards,
> > >>>>>> Shameera Rathnayaka.
> > >>>>>>
> > >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Best Regards,
> > >>>> Shameera Rathnayaka.
> > >>>>
> > >>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> > >>>> Blog : http://shameerarathnayaka.blogspot.com/
> > >>>
> > >>>
> > >
> >
> >
>
Re: Airavata GSoC 2013 Master Project
Posted by Danushka Menikkumbura <da...@gmail.com>.
Please find the proposal for "AMQP Messaging protocol support for Airavata
WS-Messenger" at [1]
Thanks,
Danushka
[1] -
https://google-melange.appspot.com/gsoc/proposal/review/google/gsoc2013/danushka/1#
On Thu, May 2, 2013 at 7:56 PM, Suresh Marru <sm...@apache.org> wrote:
> Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
>
> Suresh
>
> On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
>
> > Hi Vijayendra,
> >
> > As you can see from Shameera's proposal, he proposed a JSON conversion
> in front of WS Messenger. Also Danuska has been proposing for the AMQP and
> idea and deliberating its advantages. So given all these, I would suggest
> for you to keep focused on the UI aspects of the monitoring and write into
> your proposal a plan for determining a good strategy for the plumbing to
> WS-Eventing based existing system. You can have the concrete deliverables
> of new UI to change colors based on executions (as it currently does in
> XBaya), double click and show error messages and so forth. And keep it
> exploratory for the actually messaging format.
> >
> > I do not have any opinion on the libraries you mentioned, but yaa a ajax
> based pub system might be the right way to go. Pending the content format
> (JSON or WS-Eventing or JMS or AMQP or something else)
> >
> > Suresh
> >
> > On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <
> vijayendra.sdm@gmail.com> wrote:
> >
> >> Hi Suresh
> >>
> >> I am writing proposal for monitoring tool . The monitoring tool is
> based on
> >> pub-sub model (ws-messenger).
> >> While writing proposal , I have to back it by technical stuff that tells
> >> how can we achieve our purpose.
> >> As this monitoring tool is supposed to be a web based , and we are
> thinking
> >> in the lines of
> >> developing it in javascript.
> >> I was looking into javascript libraries that can we used with
> ws-messenger
> >> in the monitoring module.
> >> Please correct me if I am wrong.
> >> I came across some of the libraries
> >>
> >> - jQuery custom
> >> events<
> http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> >> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> >> - PubSubJS <https://github.com/mroderick/PubSubJS>
> >> - js-signals <http://millermedeiros.github.com/js-signals/>
> >>
> >> please tell me am I thinking in right direction?
> >>
> >> Regards
> >> Vijayendra
> >>
> >>
> >>
> >>
> >> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org> wrote:
> >>
> >>> Hi Shameera,
> >>>
> >>> This is great, I appreciate you sharing it, I realize this is still
> >>> working document, but I want other students to start seeing it and
> model
> >>> their proposals in a similar way.
> >>>
> >>> Airavata Mentors,
> >>>
> >>> Please provide feedback directly on the melange site and uncheck the
> >>> "private" box when you comment.
> >>>
> >>> Suresh
> >>>
> >>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <
> shameerainfo@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi Suresh and All,
> >>>>
> >>>> Of course I am very much happy to share my proposal with everybody,
> >>>> actually i was going to update this thread with the melange link in
> few
> >>>> hours once i have done writing all the sections in the proposal. I
> >>> haven't
> >>>> yet added the milestone plan into it and now working on it.
> >>>>
> >>>> The sub area i am going to work from the Master project is '
> >>> Implementing
> >>>> a JSON interface to Airavata Client side and Registry component'.
> Here is
> >>>> the link
> >>>>
> >>>
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> >>>> .
> >>>>
> >>>>
> >>>> Please note that i haven't completed everything in this and still
> doing
> >>>> modifications .Therefore proposal content may be changed bit, need to
> add
> >>>> more technical details of the approach which explains it well.
> >>>>
> >>>> I would like to know the feedback from all of you regarding the
> proposal
> >>>> and will be modifying it if there is anything to be done. Also please
> >>>> contact me if you need any help and i am very much willing to share my
> >>>> thoughts with all.
> >>>>
> >>>> Thanks!
> >>>> Shameera
> >>>>
> >>>>
> >>>>
> >>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org>
> wrote:
> >>>>
> >>>>> Hi Shameera,
> >>>>>
> >>>>> Excellent proposal, great job. Would you mind to make your proposal
> >>>>> public and post the link here? Your proposal should help others to
> look
> >>> at
> >>>>> it and learn from.
> >>>>>
> >>>>> Again I emphasize to all students, please don't feel you will be
> >>> competing
> >>>>> with each others. If all of you write good proposals, there is a good
> >>>>> chance all of you will be selected. But without a good proposal, we
> >>> cannot
> >>>>> help.
> >>>>>
> >>>>> Suresh
> >>>>>
> >>>>>
> >>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> >>> shameerainfo@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> Yes it is not easy to solve all problems, But defining our own
> standard
> >>>>> or
> >>>>>> adhere to any standard
> >>>>>> provided by third party library will solve the problem to some
> extend.
> >>>>>>
> >>>>>> Here i see two possible approaches,
> >>>>>>
> >>>>>> 1. Use existing third party library(we can find which is best)
> adhere
> >>> to
> >>>>> it
> >>>>>> standard and see how we change the
> >>>>>> backend to be inline with it.
> >>>>>>
> >>>>>> 2. Use our own convention with help of XMLSchema (The way i
> suggest).
> >>>>>>
> >>>>>> As Suresh mentioned we can do a POC with both approaches to compare
> >>>>>> performance
> >>>>>> and changes need to be done in server side. Then select the best
> one.
> >>>>>>
> >>>>>> Another question was, can we works with graph data in JSON format.
> >>>>>> There are few JS graph framworks[1] which provide that
> functionality.
> >>>>>> we can use one of them to show airavata monitoring data as graphs
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Shameera.
> >>>>>>
> >>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/>
> ,
> >>>>>> Processing.js <http://processingjs.org> , Sencha
> >>>>>> Charts<http://www.sencha.com/products/extjs/>
> >>>>>>
> >>>>>>
> >>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
> >>> wrote:
> >>>>>>
> >>>>>>> Hi Vijeyandra,
> >>>>>>>
> >>>>>>> Airavata Messaging is based on a pub-sub model and the events
> >>> themselves
> >>>>>>> are xml (WS-Eventing [1]).
> >>>>>>>
> >>>>>>> The Messenger paper [2] should give you more information.
> >>>>>>>
> >>>>>>> Hi All (Especially those at WS02):
> >>>>>>>
> >>>>>>> Here is an old effort from a Morotuwa undergrad project, you may
> want
> >>> to
> >>>>>>> read through these papers and chat with the authors to get
> >>> experiences:
> >>>>>>>
> >>>>>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>>>>>
> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>>>>>
> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>>>>> http://mooshabaya.wikidot.com/
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>>>>>>
> >>>>>>> Suresh
> >>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>>>>> [2] -
> >>>>>>>
> >>>>>
> >>>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>>>>>>
> >>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>
> >>>>>>>> Hi Suresh
> >>>>>>>>
> >>>>>>>> I wanted to know more about the monitoring tool .
> >>>>>>>> Currently from where does the monitoring tool gets data . Is it
> from
> >>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
> >>>>>>> continuously
> >>>>>>>> send messages to monitoring tool as to tell how much is the
> progress
> >>>>>>>> and what are the variables getting changed) ?
> >>>>>>>>
> >>>>>>>> Again the how is the data being exchanged. I guess it must be xml
> ?
> >>>>>>>> It must be one way data exchange . I mean the data is TO the
> >>>>>>>> monitoring module.
> >>>>>>>> Then monitoring Tool is sending back this
> >>>>>>>> data to Xbaya for displaying to the user ? Please correct me if I
> am
> >>>>>>> wrong
> >>>>>>>>
> >>>>>>>> I have downloaded the source code from the trunk . can you please
> >>> point
> >>>>>>>> me which part of code should I be code at for this module.
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Vijayendra
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>>>>> vijayendra.sdm@gmail.com> wrote:
> >>>>>>>> Hi
> >>>>>>>>
> >>>>>>>> What i am suggesting is, we send the JSON message directly to
> >>> Airavata
> >>>>>>>> Backend(or Registry)
> >>>>>>>> When the message gets build after the transport phase, convert
> JSON
> >>>>>>> message
> >>>>>>>> to SOAP(XML).
> >>>>>>>> From that point message will treated as SOAP message.
> >>>>>>>>
> >>>>>>>> If we look at the JSON <--> XML conversion there are set of third
> >>> party
> >>>>>>>> libraries we
> >>>>>>>> can use for. But before selecting a one we need to think about
> >>> problems
> >>>>>>>> having
> >>>>>>>>
> >>>>>>>> with JSON <--> XML and how these libraries handle those issues.
> >>> Because
> >>>>>>> we
> >>>>>>>> need a robust
> >>>>>>>> way to do this conversions.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Shameera what you are suggesting is sending the JSON message
> directly
> >>>>> to
> >>>>>>> Registry.
> >>>>>>>> when the message gets built after the transport phase , convert
> it to
> >>>>>>> SOAP .
> >>>>>>>>
> >>>>>>>> If you are suggesting Registry will have JSON data instead of
> WSDL ,
> >>>>>>> Then this might
> >>>>>>>> complicate the things for us .
> >>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the
> workflows
> >>> and
> >>>>>>> for other details .
> >>>>>>>> Which means we might again have to do some changes with workflow
> >>>>>>> interpretor .
> >>>>>>>> Yesterday from what I heard in discussion is that , they do not
> want
> >>> to
> >>>>>>> mess with workflow
> >>>>>>>> interpreter atleast for GSOC projects.
> >>>>>>>>
> >>>>>>>> What I want to suggest is , why carry the JSON data till
> Regisrty .
> >>>>>>> Build a interface
> >>>>>>>> before (Apache server API) where we can do the necessary
> conversion
> >>>>>>> (JSON to SOAP).
> >>>>>>>> In this way we can avoid messing up with Airavata server as a
> whole.
> >>>>>>>> Client ( using a we browser) is interacting with JSON (web
> service)
> >>> but
> >>>>>>> the Apache server
> >>>>>>>> is interacting with SOAP.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Secondly yesterday Suresh was speaking about validating the
> >>> connections
> >>>>>>> of the workflow.
> >>>>>>>> for example , the workflow is expecting a file as input
> >>>>>>>> but user is giving a sting or int .
> >>>>>>>>
> >>>>>>>> Here what I suggest is , while creating wsdl in the registry for a
> >>>>>>> particular
> >>>>>>>> workflow , we can add extra information in the form of
> >>>>>>>> annotation as the kind of input/ output the workflow is
> accepting.
> >>>>>>>> Then we will be able to check these against users entry during
> >>>>> execution.
> >>>>>>>> Please correct me if I am wrong.
> >>>>>>>>
> >>>>>>>> Regards
> >>>>>>>> Vijayendra
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <
> subs.zero@gmail.com
> >>>>
> >>>>>>> wrote:
> >>>>>>>> Well exactly, as long as you can define standard way of
> >>> communication.
> >>>>>>> That
> >>>>>>>> is, you can define in advance what should be a string, array and
> what
> >>>>>>>> should be a integer etc. We have no problem.
> >>>>>>>>
> >>>>>>>> So, when you look at problems, with JSON <-> XML or the other way
> >>>>> round,
> >>>>>>>> they talk of the very general case (where you no nothing about the
> >>> data
> >>>>>>> you
> >>>>>>>> are converting other than it is valid XML/JSON). There are a
> myriad
> >>> of
> >>>>>>>> problems in that case, which you pointed out.
> >>>>>>>>
> >>>>>>>> But when there is standard, there is only one way of doing things,
> >>> and
> >>>>>>> not
> >>>>>>>> several. I think that is the way forward. So what I am proposing
> is
> >>>>> maybe
> >>>>>>>> we all discuss and define this standard within the first week of
> GSoC
> >>>>>>>> starting and then actually move into coding. So as long as we work
> >>> with
> >>>>>>> the
> >>>>>>>> presumption that this will be done, we really dont have to worry a
> >>> lot
> >>>>>>>> about this.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Subho.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>>>>> shameerainfo@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hi,
> >>>>>>>>>
> >>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> >>> subs.zero@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Some of these problems are very specific to what the XML is
> >>>>>>>>> representing,
> >>>>>>>>>> it might not be an actual problem in Airavata,
> >>>>>>>>>> maybe some one more experienced with the codebase can point this
> >>> out.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> All issues pointed out in the paper is not directly valid to our
> >>>>>>>>> conversion, I didn't list the issues actually need to address in
> >>> this
> >>>>>>> case
> >>>>>>>>> because thought it is worth to read that introduction part which
> >>>>>>> explain
> >>>>>>>>> the all the issues we have with this conversion and give us a
> solid
> >>>>>>>>> background of that.
> >>>>>>>>>
> >>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets --
> I
> >>>>>>>>> really
> >>>>>>>>>> dont see these as problems, as long as you can agree that all
> >>>>>>> parts of
> >>>>>>>>>> airavata will treat the JSON in a standard (probably we have to
> >>>>>>> define
> >>>>>>>>>> this) way.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> The issue with JSON array only comes when we try to convert XML
> to
> >>>>>>> JSON not
> >>>>>>>>> the other way. If we map with JSON, inputparameters and
> >>>>>>> outputparameters in
> >>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore
> we
> >>>>>>> need to
> >>>>>>>>> solve this issue.
> >>>>>>>>>
> >>>>>>>>> JSON XML JSON
> >>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> >>> {"inputs":["test"]}
> >>>>>>> //
> >>>>>>>>> correct one
> >>>>>>>>> --> {"inputs":"test"} // incorrect one
> >>>>>>>>>
> >>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>>>>
> >>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
> >>>>>>> this
> >>>>>>>>>> being
> >>>>>>>>>> used is probably in the WSDL, but if we can agree on another way
> >>>>>>>>>> of communicating registered applications' I/O parameters to the
> >>>>>>> front
> >>>>>>>>>> end
> >>>>>>>>>> (JSON based), then maybe we can work around this (minor)
> problem.
> >>>>>>> Are
> >>>>>>>>>> custom processing instructions to the Xbaya XML parse even used?
> >>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Yes,attributes convertion will not be a big issues we can solve
> it.
> >>> As
> >>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a
> big
> >>>>>>> issue
> >>>>>>>>> with Airavata.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> <array name="abc">
> >>>>>>>>>> <element>1</element>
> >>>>>>>>>> <element>2</element>
> >>>>>>>>>> <element>3</element>
> >>>>>>>>>> <element>4</element>
> >>>>>>>>>> </array>
> >>>>>>>>>>
> >>>>>>>>>> Can become
> >>>>>>>>>>
> >>>>>>>>>> {
> >>>>>>>>>>
> >>>>>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>>>>
> >>>>>>>>>> }
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> With this example it show us we need to change the XML message
> >>> format
> >>>>>>> of
> >>>>>>>>> server side, which require to change the all schemas, If we are
> >>> going
> >>>>>>> to
> >>>>>>>>> change the schemas then we need to change the way it process it
> in
> >>>>>>> Ariavara
> >>>>>>>>> core. We have dropped our initial major requirement, which is
> keep
> >>> the
> >>>>>>>>> Airavata Server side as it is.
> >>>>>>>>>
> >>>>>>>>> with this conversion we only deal with json strings, yes we can
> send
> >>>>>>> JSON
> >>>>>>>>> request with other formats supported by JSON like boolen, null,
> >>> Number
> >>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML
> only
> >>>>>>> deal
> >>>>>>>>> only with Strings. I think it is good if we can consume a this
> >>>>> features
> >>>>>>>>> with JSON.
> >>>>>>>>>
> >>>>>>>>> let say i need to send a integer or float to the server using
> JSON
> >>>>> then
> >>>>>>>>> proper way is to send {"<name>":123.45} this will works fine but
> >>>>>>> problem is
> >>>>>>>>> how we get the same output ?
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Shameera.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Cheers,
> >>>>>>>>>> Subho.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best Regards,
> >>>>>>>>> Shameera Rathnayaka.
> >>>>>>>>>
> >>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards,
> >>>>>> Shameera Rathnayaka.
> >>>>>>
> >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>> Shameera Rathnayaka.
> >>>>
> >>>> email: shameera AT apache.org , shameerainfo AT gmail.com
> >>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>
> >>>
> >
>
>
Re: Airavata GSoC 2013 Master Project
Posted by Suresh Marru <sm...@apache.org>.
Ping!! No proposals yet beyond Shameera's and Danushka's place holder.
Suresh
On May 1, 2013, at 5:13 PM, Suresh Marru <sm...@apache.org> wrote:
> Hi Vijayendra,
>
> As you can see from Shameera's proposal, he proposed a JSON conversion in front of WS Messenger. Also Danuska has been proposing for the AMQP and idea and deliberating its advantages. So given all these, I would suggest for you to keep focused on the UI aspects of the monitoring and write into your proposal a plan for determining a good strategy for the plumbing to WS-Eventing based existing system. You can have the concrete deliverables of new UI to change colors based on executions (as it currently does in XBaya), double click and show error messages and so forth. And keep it exploratory for the actually messaging format.
>
> I do not have any opinion on the libraries you mentioned, but yaa a ajax based pub system might be the right way to go. Pending the content format (JSON or WS-Eventing or JMS or AMQP or something else)
>
> Suresh
>
> On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <vi...@gmail.com> wrote:
>
>> Hi Suresh
>>
>> I am writing proposal for monitoring tool . The monitoring tool is based on
>> pub-sub model (ws-messenger).
>> While writing proposal , I have to back it by technical stuff that tells
>> how can we achieve our purpose.
>> As this monitoring tool is supposed to be a web based , and we are thinking
>> in the lines of
>> developing it in javascript.
>> I was looking into javascript libraries that can we used with ws-messenger
>> in the monitoring module.
>> Please correct me if I am wrong.
>> I came across some of the libraries
>>
>> - jQuery custom
>> events<http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
>> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
>> - PubSubJS <https://github.com/mroderick/PubSubJS>
>> - js-signals <http://millermedeiros.github.com/js-signals/>
>>
>> please tell me am I thinking in right direction?
>>
>> Regards
>> Vijayendra
>>
>>
>>
>>
>> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org> wrote:
>>
>>> Hi Shameera,
>>>
>>> This is great, I appreciate you sharing it, I realize this is still
>>> working document, but I want other students to start seeing it and model
>>> their proposals in a similar way.
>>>
>>> Airavata Mentors,
>>>
>>> Please provide feedback directly on the melange site and uncheck the
>>> "private" box when you comment.
>>>
>>> Suresh
>>>
>>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <sh...@gmail.com>
>>> wrote:
>>>
>>>> Hi Suresh and All,
>>>>
>>>> Of course I am very much happy to share my proposal with everybody,
>>>> actually i was going to update this thread with the melange link in few
>>>> hours once i have done writing all the sections in the proposal. I
>>> haven't
>>>> yet added the milestone plan into it and now working on it.
>>>>
>>>> The sub area i am going to work from the Master project is '
>>> Implementing
>>>> a JSON interface to Airavata Client side and Registry component'. Here is
>>>> the link
>>>>
>>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
>>>> .
>>>>
>>>>
>>>> Please note that i haven't completed everything in this and still doing
>>>> modifications .Therefore proposal content may be changed bit, need to add
>>>> more technical details of the approach which explains it well.
>>>>
>>>> I would like to know the feedback from all of you regarding the proposal
>>>> and will be modifying it if there is anything to be done. Also please
>>>> contact me if you need any help and i am very much willing to share my
>>>> thoughts with all.
>>>>
>>>> Thanks!
>>>> Shameera
>>>>
>>>>
>>>>
>>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org> wrote:
>>>>
>>>>> Hi Shameera,
>>>>>
>>>>> Excellent proposal, great job. Would you mind to make your proposal
>>>>> public and post the link here? Your proposal should help others to look
>>> at
>>>>> it and learn from.
>>>>>
>>>>> Again I emphasize to all students, please don't feel you will be
>>> competing
>>>>> with each others. If all of you write good proposals, there is a good
>>>>> chance all of you will be selected. But without a good proposal, we
>>> cannot
>>>>> help.
>>>>>
>>>>> Suresh
>>>>>
>>>>>
>>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
>>> shameerainfo@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Yes it is not easy to solve all problems, But defining our own standard
>>>>> or
>>>>>> adhere to any standard
>>>>>> provided by third party library will solve the problem to some extend.
>>>>>>
>>>>>> Here i see two possible approaches,
>>>>>>
>>>>>> 1. Use existing third party library(we can find which is best) adhere
>>> to
>>>>> it
>>>>>> standard and see how we change the
>>>>>> backend to be inline with it.
>>>>>>
>>>>>> 2. Use our own convention with help of XMLSchema (The way i suggest).
>>>>>>
>>>>>> As Suresh mentioned we can do a POC with both approaches to compare
>>>>>> performance
>>>>>> and changes need to be done in server side. Then select the best one.
>>>>>>
>>>>>> Another question was, can we works with graph data in JSON format.
>>>>>> There are few JS graph framworks[1] which provide that functionality.
>>>>>> we can use one of them to show airavata monitoring data as graphs
>>>>>>
>>>>>> Thanks,
>>>>>> Shameera.
>>>>>>
>>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/> ,
>>>>>> Processing.js <http://processingjs.org> , Sencha
>>>>>> Charts<http://www.sencha.com/products/extjs/>
>>>>>>
>>>>>>
>>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
>>> wrote:
>>>>>>
>>>>>>> Hi Vijeyandra,
>>>>>>>
>>>>>>> Airavata Messaging is based on a pub-sub model and the events
>>> themselves
>>>>>>> are xml (WS-Eventing [1]).
>>>>>>>
>>>>>>> The Messenger paper [2] should give you more information.
>>>>>>>
>>>>>>> Hi All (Especially those at WS02):
>>>>>>>
>>>>>>> Here is an old effort from a Morotuwa undergrad project, you may want
>>> to
>>>>>>> read through these papers and chat with the authors to get
>>> experiences:
>>>>>>>
>>>>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>>>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>>>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>>>>> http://mooshabaya.wikidot.com/
>>>>>>>
>>>>>>>
>>>>>
>>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>>>>>>
>>>>>>> Suresh
>>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>>>>> [2] -
>>>>>>>
>>>>>
>>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>>>>>>
>>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Suresh
>>>>>>>>
>>>>>>>> I wanted to know more about the monitoring tool .
>>>>>>>> Currently from where does the monitoring tool gets data . Is it from
>>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
>>>>>>> continuously
>>>>>>>> send messages to monitoring tool as to tell how much is the progress
>>>>>>>> and what are the variables getting changed) ?
>>>>>>>>
>>>>>>>> Again the how is the data being exchanged. I guess it must be xml ?
>>>>>>>> It must be one way data exchange . I mean the data is TO the
>>>>>>>> monitoring module.
>>>>>>>> Then monitoring Tool is sending back this
>>>>>>>> data to Xbaya for displaying to the user ? Please correct me if I am
>>>>>>> wrong
>>>>>>>>
>>>>>>>> I have downloaded the source code from the trunk . can you please
>>> point
>>>>>>>> me which part of code should I be code at for this module.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Vijayendra
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> What i am suggesting is, we send the JSON message directly to
>>> Airavata
>>>>>>>> Backend(or Registry)
>>>>>>>> When the message gets build after the transport phase, convert JSON
>>>>>>> message
>>>>>>>> to SOAP(XML).
>>>>>>>> From that point message will treated as SOAP message.
>>>>>>>>
>>>>>>>> If we look at the JSON <--> XML conversion there are set of third
>>> party
>>>>>>>> libraries we
>>>>>>>> can use for. But before selecting a one we need to think about
>>> problems
>>>>>>>> having
>>>>>>>>
>>>>>>>> with JSON <--> XML and how these libraries handle those issues.
>>> Because
>>>>>>> we
>>>>>>>> need a robust
>>>>>>>> way to do this conversions.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Shameera what you are suggesting is sending the JSON message directly
>>>>> to
>>>>>>> Registry.
>>>>>>>> when the message gets built after the transport phase , convert it to
>>>>>>> SOAP .
>>>>>>>>
>>>>>>>> If you are suggesting Registry will have JSON data instead of WSDL ,
>>>>>>> Then this might
>>>>>>>> complicate the things for us .
>>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the workflows
>>> and
>>>>>>> for other details .
>>>>>>>> Which means we might again have to do some changes with workflow
>>>>>>> interpretor .
>>>>>>>> Yesterday from what I heard in discussion is that , they do not want
>>> to
>>>>>>> mess with workflow
>>>>>>>> interpreter atleast for GSOC projects.
>>>>>>>>
>>>>>>>> What I want to suggest is , why carry the JSON data till Regisrty .
>>>>>>> Build a interface
>>>>>>>> before (Apache server API) where we can do the necessary conversion
>>>>>>> (JSON to SOAP).
>>>>>>>> In this way we can avoid messing up with Airavata server as a whole.
>>>>>>>> Client ( using a we browser) is interacting with JSON (web service)
>>> but
>>>>>>> the Apache server
>>>>>>>> is interacting with SOAP.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Secondly yesterday Suresh was speaking about validating the
>>> connections
>>>>>>> of the workflow.
>>>>>>>> for example , the workflow is expecting a file as input
>>>>>>>> but user is giving a sting or int .
>>>>>>>>
>>>>>>>> Here what I suggest is , while creating wsdl in the registry for a
>>>>>>> particular
>>>>>>>> workflow , we can add extra information in the form of
>>>>>>>> annotation as the kind of input/ output the workflow is accepting.
>>>>>>>> Then we will be able to check these against users entry during
>>>>> execution.
>>>>>>>> Please correct me if I am wrong.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Vijayendra
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <subs.zero@gmail.com
>>>>
>>>>>>> wrote:
>>>>>>>> Well exactly, as long as you can define standard way of
>>> communication.
>>>>>>> That
>>>>>>>> is, you can define in advance what should be a string, array and what
>>>>>>>> should be a integer etc. We have no problem.
>>>>>>>>
>>>>>>>> So, when you look at problems, with JSON <-> XML or the other way
>>>>> round,
>>>>>>>> they talk of the very general case (where you no nothing about the
>>> data
>>>>>>> you
>>>>>>>> are converting other than it is valid XML/JSON). There are a myriad
>>> of
>>>>>>>> problems in that case, which you pointed out.
>>>>>>>>
>>>>>>>> But when there is standard, there is only one way of doing things,
>>> and
>>>>>>> not
>>>>>>>> several. I think that is the way forward. So what I am proposing is
>>>>> maybe
>>>>>>>> we all discuss and define this standard within the first week of GSoC
>>>>>>>> starting and then actually move into coding. So as long as we work
>>> with
>>>>>>> the
>>>>>>>> presumption that this will be done, we really dont have to worry a
>>> lot
>>>>>>>> about this.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Subho.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
>>> subs.zero@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Some of these problems are very specific to what the XML is
>>>>>>>>> representing,
>>>>>>>>>> it might not be an actual problem in Airavata,
>>>>>>>>>> maybe some one more experienced with the codebase can point this
>>> out.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> All issues pointed out in the paper is not directly valid to our
>>>>>>>>> conversion, I didn't list the issues actually need to address in
>>> this
>>>>>>> case
>>>>>>>>> because thought it is worth to read that introduction part which
>>>>>>> explain
>>>>>>>>> the all the issues we have with this conversion and give us a solid
>>>>>>>>> background of that.
>>>>>>>>>
>>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- I
>>>>>>>>> really
>>>>>>>>>> dont see these as problems, as long as you can agree that all
>>>>>>> parts of
>>>>>>>>>> airavata will treat the JSON in a standard (probably we have to
>>>>>>> define
>>>>>>>>>> this) way.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The issue with JSON array only comes when we try to convert XML to
>>>>>>> JSON not
>>>>>>>>> the other way. If we map with JSON, inputparameters and
>>>>>>> outputparameters in
>>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we
>>>>>>> need to
>>>>>>>>> solve this issue.
>>>>>>>>>
>>>>>>>>> JSON XML JSON
>>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
>>> {"inputs":["test"]}
>>>>>>> //
>>>>>>>>> correct one
>>>>>>>>> --> {"inputs":"test"} // incorrect one
>>>>>>>>>
>>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>>>>
>>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
>>>>>>> this
>>>>>>>>>> being
>>>>>>>>>> used is probably in the WSDL, but if we can agree on another way
>>>>>>>>>> of communicating registered applications' I/O parameters to the
>>>>>>> front
>>>>>>>>>> end
>>>>>>>>>> (JSON based), then maybe we can work around this (minor) problem.
>>>>>>> Are
>>>>>>>>>> custom processing instructions to the Xbaya XML parse even used?
>>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes,attributes convertion will not be a big issues we can solve it.
>>> As
>>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a big
>>>>>>> issue
>>>>>>>>> with Airavata.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <array name="abc">
>>>>>>>>>> <element>1</element>
>>>>>>>>>> <element>2</element>
>>>>>>>>>> <element>3</element>
>>>>>>>>>> <element>4</element>
>>>>>>>>>> </array>
>>>>>>>>>>
>>>>>>>>>> Can become
>>>>>>>>>>
>>>>>>>>>> {
>>>>>>>>>>
>>>>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>>>>
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> With this example it show us we need to change the XML message
>>> format
>>>>>>> of
>>>>>>>>> server side, which require to change the all schemas, If we are
>>> going
>>>>>>> to
>>>>>>>>> change the schemas then we need to change the way it process it in
>>>>>>> Ariavara
>>>>>>>>> core. We have dropped our initial major requirement, which is keep
>>> the
>>>>>>>>> Airavata Server side as it is.
>>>>>>>>>
>>>>>>>>> with this conversion we only deal with json strings, yes we can send
>>>>>>> JSON
>>>>>>>>> request with other formats supported by JSON like boolen, null,
>>> Number
>>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML only
>>>>>>> deal
>>>>>>>>> only with Strings. I think it is good if we can consume a this
>>>>> features
>>>>>>>>> with JSON.
>>>>>>>>>
>>>>>>>>> let say i need to send a integer or float to the server using JSON
>>>>> then
>>>>>>>>> proper way is to send {"<name>":123.45} this will works fine but
>>>>>>> problem is
>>>>>>>>> how we get the same output ?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Shameera.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Subho.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Shameera Rathnayaka.
>>>>>>>>>
>>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Shameera Rathnayaka.
>>>>>>
>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>> Shameera Rathnayaka.
>>>>
>>>> email: shameera AT apache.org , shameerainfo AT gmail.com
>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>
>>>
>
Re: Airavata GSoC 2013 Master Project
Posted by Suresh Marru <sm...@apache.org>.
Hi Vijayendra,
As you can see from Shameera's proposal, he proposed a JSON conversion in front of WS Messenger. Also Danuska has been proposing for the AMQP and idea and deliberating its advantages. So given all these, I would suggest for you to keep focused on the UI aspects of the monitoring and write into your proposal a plan for determining a good strategy for the plumbing to WS-Eventing based existing system. You can have the concrete deliverables of new UI to change colors based on executions (as it currently does in XBaya), double click and show error messages and so forth. And keep it exploratory for the actually messaging format.
I do not have any opinion on the libraries you mentioned, but yaa a ajax based pub system might be the right way to go. Pending the content format (JSON or WS-Eventing or JMS or AMQP or something else)
Suresh
On May 1, 2013, at 4:13 PM, Vijayendra Grampurohit <vi...@gmail.com> wrote:
> Hi Suresh
>
> I am writing proposal for monitoring tool . The monitoring tool is based on
> pub-sub model (ws-messenger).
> While writing proposal , I have to back it by technical stuff that tells
> how can we achieve our purpose.
> As this monitoring tool is supposed to be a web based , and we are thinking
> in the lines of
> developing it in javascript.
> I was looking into javascript libraries that can we used with ws-messenger
> in the monitoring module.
> Please correct me if I am wrong.
> I came across some of the libraries
>
> - jQuery custom
> events<http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
> - AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
> - PubSubJS <https://github.com/mroderick/PubSubJS>
> - js-signals <http://millermedeiros.github.com/js-signals/>
>
> please tell me am I thinking in right direction?
>
> Regards
> Vijayendra
>
>
>
>
> On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org> wrote:
>
>> Hi Shameera,
>>
>> This is great, I appreciate you sharing it, I realize this is still
>> working document, but I want other students to start seeing it and model
>> their proposals in a similar way.
>>
>> Airavata Mentors,
>>
>> Please provide feedback directly on the melange site and uncheck the
>> "private" box when you comment.
>>
>> Suresh
>>
>> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <sh...@gmail.com>
>> wrote:
>>
>>> Hi Suresh and All,
>>>
>>> Of course I am very much happy to share my proposal with everybody,
>>> actually i was going to update this thread with the melange link in few
>>> hours once i have done writing all the sections in the proposal. I
>> haven't
>>> yet added the milestone plan into it and now working on it.
>>>
>>> The sub area i am going to work from the Master project is '
>> Implementing
>>> a JSON interface to Airavata Client side and Registry component'. Here is
>>> the link
>>>
>> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
>>> .
>>>
>>>
>>> Please note that i haven't completed everything in this and still doing
>>> modifications .Therefore proposal content may be changed bit, need to add
>>> more technical details of the approach which explains it well.
>>>
>>> I would like to know the feedback from all of you regarding the proposal
>>> and will be modifying it if there is anything to be done. Also please
>>> contact me if you need any help and i am very much willing to share my
>>> thoughts with all.
>>>
>>> Thanks!
>>> Shameera
>>>
>>>
>>>
>>> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org> wrote:
>>>
>>>> Hi Shameera,
>>>>
>>>> Excellent proposal, great job. Would you mind to make your proposal
>>>> public and post the link here? Your proposal should help others to look
>> at
>>>> it and learn from.
>>>>
>>>> Again I emphasize to all students, please don't feel you will be
>> competing
>>>> with each others. If all of you write good proposals, there is a good
>>>> chance all of you will be selected. But without a good proposal, we
>> cannot
>>>> help.
>>>>
>>>> Suresh
>>>>
>>>>
>>>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
>> shameerainfo@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Yes it is not easy to solve all problems, But defining our own standard
>>>> or
>>>>> adhere to any standard
>>>>> provided by third party library will solve the problem to some extend.
>>>>>
>>>>> Here i see two possible approaches,
>>>>>
>>>>> 1. Use existing third party library(we can find which is best) adhere
>> to
>>>> it
>>>>> standard and see how we change the
>>>>> backend to be inline with it.
>>>>>
>>>>> 2. Use our own convention with help of XMLSchema (The way i suggest).
>>>>>
>>>>> As Suresh mentioned we can do a POC with both approaches to compare
>>>>> performance
>>>>> and changes need to be done in server side. Then select the best one.
>>>>>
>>>>> Another question was, can we works with graph data in JSON format.
>>>>> There are few JS graph framworks[1] which provide that functionality.
>>>>> we can use one of them to show airavata monitoring data as graphs
>>>>>
>>>>> Thanks,
>>>>> Shameera.
>>>>>
>>>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/> ,
>>>>> Processing.js <http://processingjs.org> , Sencha
>>>>> Charts<http://www.sencha.com/products/extjs/>
>>>>>
>>>>>
>>>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
>> wrote:
>>>>>
>>>>>> Hi Vijeyandra,
>>>>>>
>>>>>> Airavata Messaging is based on a pub-sub model and the events
>> themselves
>>>>>> are xml (WS-Eventing [1]).
>>>>>>
>>>>>> The Messenger paper [2] should give you more information.
>>>>>>
>>>>>> Hi All (Especially those at WS02):
>>>>>>
>>>>>> Here is an old effort from a Morotuwa undergrad project, you may want
>> to
>>>>>> read through these papers and chat with the authors to get
>> experiences:
>>>>>>
>>>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>>>> http://mooshabaya.wikidot.com/
>>>>>>
>>>>>>
>>>>
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>>>>>
>>>>>> Suresh
>>>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>>>> [2] -
>>>>>>
>>>>
>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>>>>>
>>>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Suresh
>>>>>>>
>>>>>>> I wanted to know more about the monitoring tool .
>>>>>>> Currently from where does the monitoring tool gets data . Is it from
>>>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
>>>>>> continuously
>>>>>>> send messages to monitoring tool as to tell how much is the progress
>>>>>>> and what are the variables getting changed) ?
>>>>>>>
>>>>>>> Again the how is the data being exchanged. I guess it must be xml ?
>>>>>>> It must be one way data exchange . I mean the data is TO the
>>>>>>> monitoring module.
>>>>>>> Then monitoring Tool is sending back this
>>>>>>> data to Xbaya for displaying to the user ? Please correct me if I am
>>>>>> wrong
>>>>>>>
>>>>>>> I have downloaded the source code from the trunk . can you please
>> point
>>>>>>> me which part of code should I be code at for this module.
>>>>>>>
>>>>>>> Regards
>>>>>>> Vijayendra
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>>>> vijayendra.sdm@gmail.com> wrote:
>>>>>>> Hi
>>>>>>>
>>>>>>> What i am suggesting is, we send the JSON message directly to
>> Airavata
>>>>>>> Backend(or Registry)
>>>>>>> When the message gets build after the transport phase, convert JSON
>>>>>> message
>>>>>>> to SOAP(XML).
>>>>>>> From that point message will treated as SOAP message.
>>>>>>>
>>>>>>> If we look at the JSON <--> XML conversion there are set of third
>> party
>>>>>>> libraries we
>>>>>>> can use for. But before selecting a one we need to think about
>> problems
>>>>>>> having
>>>>>>>
>>>>>>> with JSON <--> XML and how these libraries handle those issues.
>> Because
>>>>>> we
>>>>>>> need a robust
>>>>>>> way to do this conversions.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Shameera what you are suggesting is sending the JSON message directly
>>>> to
>>>>>> Registry.
>>>>>>> when the message gets built after the transport phase , convert it to
>>>>>> SOAP .
>>>>>>>
>>>>>>> If you are suggesting Registry will have JSON data instead of WSDL ,
>>>>>> Then this might
>>>>>>> complicate the things for us .
>>>>>>> The workflow interpreter needs wsdl(xml) to interpret the workflows
>> and
>>>>>> for other details .
>>>>>>> Which means we might again have to do some changes with workflow
>>>>>> interpretor .
>>>>>>> Yesterday from what I heard in discussion is that , they do not want
>> to
>>>>>> mess with workflow
>>>>>>> interpreter atleast for GSOC projects.
>>>>>>>
>>>>>>> What I want to suggest is , why carry the JSON data till Regisrty .
>>>>>> Build a interface
>>>>>>> before (Apache server API) where we can do the necessary conversion
>>>>>> (JSON to SOAP).
>>>>>>> In this way we can avoid messing up with Airavata server as a whole.
>>>>>>> Client ( using a we browser) is interacting with JSON (web service)
>> but
>>>>>> the Apache server
>>>>>>> is interacting with SOAP.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Secondly yesterday Suresh was speaking about validating the
>> connections
>>>>>> of the workflow.
>>>>>>> for example , the workflow is expecting a file as input
>>>>>>> but user is giving a sting or int .
>>>>>>>
>>>>>>> Here what I suggest is , while creating wsdl in the registry for a
>>>>>> particular
>>>>>>> workflow , we can add extra information in the form of
>>>>>>> annotation as the kind of input/ output the workflow is accepting.
>>>>>>> Then we will be able to check these against users entry during
>>>> execution.
>>>>>>> Please correct me if I am wrong.
>>>>>>>
>>>>>>> Regards
>>>>>>> Vijayendra
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <subs.zero@gmail.com
>>>
>>>>>> wrote:
>>>>>>> Well exactly, as long as you can define standard way of
>> communication.
>>>>>> That
>>>>>>> is, you can define in advance what should be a string, array and what
>>>>>>> should be a integer etc. We have no problem.
>>>>>>>
>>>>>>> So, when you look at problems, with JSON <-> XML or the other way
>>>> round,
>>>>>>> they talk of the very general case (where you no nothing about the
>> data
>>>>>> you
>>>>>>> are converting other than it is valid XML/JSON). There are a myriad
>> of
>>>>>>> problems in that case, which you pointed out.
>>>>>>>
>>>>>>> But when there is standard, there is only one way of doing things,
>> and
>>>>>> not
>>>>>>> several. I think that is the way forward. So what I am proposing is
>>>> maybe
>>>>>>> we all discuss and define this standard within the first week of GSoC
>>>>>>> starting and then actually move into coding. So as long as we work
>> with
>>>>>> the
>>>>>>> presumption that this will be done, we really dont have to worry a
>> lot
>>>>>>> about this.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Subho.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>>>> shameerainfo@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
>> subs.zero@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Some of these problems are very specific to what the XML is
>>>>>>>> representing,
>>>>>>>>> it might not be an actual problem in Airavata,
>>>>>>>>> maybe some one more experienced with the codebase can point this
>> out.
>>>>>>>>>
>>>>>>>>
>>>>>>>> All issues pointed out in the paper is not directly valid to our
>>>>>>>> conversion, I didn't list the issues actually need to address in
>> this
>>>>>> case
>>>>>>>> because thought it is worth to read that introduction part which
>>>>>> explain
>>>>>>>> the all the issues we have with this conversion and give us a solid
>>>>>>>> background of that.
>>>>>>>>
>>>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- I
>>>>>>>> really
>>>>>>>>> dont see these as problems, as long as you can agree that all
>>>>>> parts of
>>>>>>>>> airavata will treat the JSON in a standard (probably we have to
>>>>>> define
>>>>>>>>> this) way.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The issue with JSON array only comes when we try to convert XML to
>>>>>> JSON not
>>>>>>>> the other way. If we map with JSON, inputparameters and
>>>>>> outputparameters in
>>>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we
>>>>>> need to
>>>>>>>> solve this issue.
>>>>>>>>
>>>>>>>> JSON XML JSON
>>>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
>> {"inputs":["test"]}
>>>>>> //
>>>>>>>> correct one
>>>>>>>> --> {"inputs":"test"} // incorrect one
>>>>>>>>
>>>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>>>
>>>>>>>>> Are separate namespaces used in Airavata? Only place I can see
>>>>>> this
>>>>>>>>> being
>>>>>>>>> used is probably in the WSDL, but if we can agree on another way
>>>>>>>>> of communicating registered applications' I/O parameters to the
>>>>>> front
>>>>>>>>> end
>>>>>>>>> (JSON based), then maybe we can work around this (minor) problem.
>>>>>> Are
>>>>>>>>> custom processing instructions to the Xbaya XML parse even used?
>>>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yes,attributes convertion will not be a big issues we can solve it.
>> As
>>>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a big
>>>>>> issue
>>>>>>>> with Airavata.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> <array name="abc">
>>>>>>>>> <element>1</element>
>>>>>>>>> <element>2</element>
>>>>>>>>> <element>3</element>
>>>>>>>>> <element>4</element>
>>>>>>>>> </array>
>>>>>>>>>
>>>>>>>>> Can become
>>>>>>>>>
>>>>>>>>> {
>>>>>>>>>
>>>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>>>
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>
>>>>>>>> With this example it show us we need to change the XML message
>> format
>>>>>> of
>>>>>>>> server side, which require to change the all schemas, If we are
>> going
>>>>>> to
>>>>>>>> change the schemas then we need to change the way it process it in
>>>>>> Ariavara
>>>>>>>> core. We have dropped our initial major requirement, which is keep
>> the
>>>>>>>> Airavata Server side as it is.
>>>>>>>>
>>>>>>>> with this conversion we only deal with json strings, yes we can send
>>>>>> JSON
>>>>>>>> request with other formats supported by JSON like boolen, null,
>> Number
>>>>>>>> etc.. But there is no way to get the same JSON from XML as XML only
>>>>>> deal
>>>>>>>> only with Strings. I think it is good if we can consume a this
>>>> features
>>>>>>>> with JSON.
>>>>>>>>
>>>>>>>> let say i need to send a integer or float to the server using JSON
>>>> then
>>>>>>>> proper way is to send {"<name>":123.45} this will works fine but
>>>>>> problem is
>>>>>>>> how we get the same output ?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Shameera.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Subho.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best Regards,
>>>>>>>> Shameera Rathnayaka.
>>>>>>>>
>>>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>> Shameera Rathnayaka.
>>>>>
>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Shameera Rathnayaka.
>>>
>>> email: shameera AT apache.org , shameerainfo AT gmail.com
>>> Blog : http://shameerarathnayaka.blogspot.com/
>>
>>
Re: Airavata GSoC 2013 Master Project
Posted by Vijayendra Grampurohit <vi...@gmail.com>.
Hi Suresh
I am writing proposal for monitoring tool . The monitoring tool is based on
pub-sub model (ws-messenger).
While writing proposal , I have to back it by technical stuff that tells
how can we achieve our purpose.
As this monitoring tool is supposed to be a web based , and we are thinking
in the lines of
developing it in javascript.
I was looking into javascript libraries that can we used with ws-messenger
in the monitoring module.
Please correct me if I am wrong.
I came across some of the libraries
- jQuery custom
events<http://weblog.bocoup.com/publishsubscribe-with-jquery-custom-events>
- AmplifyJS Pub/Sub <http://amplifyjs.com/api/pubsub/>
- PubSubJS <https://github.com/mroderick/PubSubJS>
- js-signals <http://millermedeiros.github.com/js-signals/>
please tell me am I thinking in right direction?
Regards
Vijayendra
On Wed, May 1, 2013 at 5:30 PM, Suresh Marru <sm...@apache.org> wrote:
> Hi Shameera,
>
> This is great, I appreciate you sharing it, I realize this is still
> working document, but I want other students to start seeing it and model
> their proposals in a similar way.
>
> Airavata Mentors,
>
> Please provide feedback directly on the melange site and uncheck the
> "private" box when you comment.
>
> Suresh
>
> On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <sh...@gmail.com>
> wrote:
>
> > Hi Suresh and All,
> >
> > Of course I am very much happy to share my proposal with everybody,
> > actually i was going to update this thread with the melange link in few
> > hours once i have done writing all the sections in the proposal. I
> haven't
> > yet added the milestone plan into it and now working on it.
> >
> > The sub area i am going to work from the Master project is '
> Implementing
> > a JSON interface to Airavata Client side and Registry component'. Here is
> > the link
> >
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> > .
> >
> >
> > Please note that i haven't completed everything in this and still doing
> > modifications .Therefore proposal content may be changed bit, need to add
> > more technical details of the approach which explains it well.
> >
> > I would like to know the feedback from all of you regarding the proposal
> > and will be modifying it if there is anything to be done. Also please
> > contact me if you need any help and i am very much willing to share my
> > thoughts with all.
> >
> > Thanks!
> > Shameera
> >
> >
> >
> > On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org> wrote:
> >
> >> Hi Shameera,
> >>
> >> Excellent proposal, great job. Would you mind to make your proposal
> >> public and post the link here? Your proposal should help others to look
> at
> >> it and learn from.
> >>
> >> Again I emphasize to all students, please don't feel you will be
> competing
> >> with each others. If all of you write good proposals, there is a good
> >> chance all of you will be selected. But without a good proposal, we
> cannot
> >> help.
> >>
> >> Suresh
> >>
> >>
> >> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <
> shameerainfo@gmail.com>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> Yes it is not easy to solve all problems, But defining our own standard
> >> or
> >>> adhere to any standard
> >>> provided by third party library will solve the problem to some extend.
> >>>
> >>> Here i see two possible approaches,
> >>>
> >>> 1. Use existing third party library(we can find which is best) adhere
> to
> >> it
> >>> standard and see how we change the
> >>> backend to be inline with it.
> >>>
> >>> 2. Use our own convention with help of XMLSchema (The way i suggest).
> >>>
> >>> As Suresh mentioned we can do a POC with both approaches to compare
> >>> performance
> >>> and changes need to be done in server side. Then select the best one.
> >>>
> >>> Another question was, can we works with graph data in JSON format.
> >>> There are few JS graph framworks[1] which provide that functionality.
> >>> we can use one of them to show airavata monitoring data as graphs
> >>>
> >>> Thanks,
> >>> Shameera.
> >>>
> >>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/> ,
> >>> Processing.js <http://processingjs.org> , Sencha
> >>> Charts<http://www.sencha.com/products/extjs/>
> >>>
> >>>
> >>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org>
> wrote:
> >>>
> >>>> Hi Vijeyandra,
> >>>>
> >>>> Airavata Messaging is based on a pub-sub model and the events
> themselves
> >>>> are xml (WS-Eventing [1]).
> >>>>
> >>>> The Messenger paper [2] should give you more information.
> >>>>
> >>>> Hi All (Especially those at WS02):
> >>>>
> >>>> Here is an old effort from a Morotuwa undergrad project, you may want
> to
> >>>> read through these papers and chat with the authors to get
> experiences:
> >>>>
> >>>> http://dl.acm.org/citation.cfm?id=1890807
> >>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >>>> http://mooshabaya.wikidot.com/
> >>>>
> >>>>
> >>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>>>
> >>>> Suresh
> >>>> [1] - http://www.w3.org/Submission/WS-Eventing/
> >>>> [2] -
> >>>>
> >>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>>>
> >>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >>>> vijayendra.sdm@gmail.com> wrote:
> >>>>
> >>>>> Hi Suresh
> >>>>>
> >>>>> I wanted to know more about the monitoring tool .
> >>>>> Currently from where does the monitoring tool gets data . Is it from
> >>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
> >>>> continuously
> >>>>> send messages to monitoring tool as to tell how much is the progress
> >>>>> and what are the variables getting changed) ?
> >>>>>
> >>>>> Again the how is the data being exchanged. I guess it must be xml ?
> >>>>> It must be one way data exchange . I mean the data is TO the
> >>>>> monitoring module.
> >>>>> Then monitoring Tool is sending back this
> >>>>> data to Xbaya for displaying to the user ? Please correct me if I am
> >>>> wrong
> >>>>>
> >>>>> I have downloaded the source code from the trunk . can you please
> point
> >>>>> me which part of code should I be code at for this module.
> >>>>>
> >>>>> Regards
> >>>>> Vijayendra
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >>>> vijayendra.sdm@gmail.com> wrote:
> >>>>> Hi
> >>>>>
> >>>>> What i am suggesting is, we send the JSON message directly to
> Airavata
> >>>>> Backend(or Registry)
> >>>>> When the message gets build after the transport phase, convert JSON
> >>>> message
> >>>>> to SOAP(XML).
> >>>>> From that point message will treated as SOAP message.
> >>>>>
> >>>>> If we look at the JSON <--> XML conversion there are set of third
> party
> >>>>> libraries we
> >>>>> can use for. But before selecting a one we need to think about
> problems
> >>>>> having
> >>>>>
> >>>>> with JSON <--> XML and how these libraries handle those issues.
> Because
> >>>> we
> >>>>> need a robust
> >>>>> way to do this conversions.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Shameera what you are suggesting is sending the JSON message directly
> >> to
> >>>> Registry.
> >>>>> when the message gets built after the transport phase , convert it to
> >>>> SOAP .
> >>>>>
> >>>>> If you are suggesting Registry will have JSON data instead of WSDL ,
> >>>> Then this might
> >>>>> complicate the things for us .
> >>>>> The workflow interpreter needs wsdl(xml) to interpret the workflows
> and
> >>>> for other details .
> >>>>> Which means we might again have to do some changes with workflow
> >>>> interpretor .
> >>>>> Yesterday from what I heard in discussion is that , they do not want
> to
> >>>> mess with workflow
> >>>>> interpreter atleast for GSOC projects.
> >>>>>
> >>>>> What I want to suggest is , why carry the JSON data till Regisrty .
> >>>> Build a interface
> >>>>> before (Apache server API) where we can do the necessary conversion
> >>>> (JSON to SOAP).
> >>>>> In this way we can avoid messing up with Airavata server as a whole.
> >>>>> Client ( using a we browser) is interacting with JSON (web service)
> but
> >>>> the Apache server
> >>>>> is interacting with SOAP.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Secondly yesterday Suresh was speaking about validating the
> connections
> >>>> of the workflow.
> >>>>> for example , the workflow is expecting a file as input
> >>>>> but user is giving a sting or int .
> >>>>>
> >>>>> Here what I suggest is , while creating wsdl in the registry for a
> >>>> particular
> >>>>> workflow , we can add extra information in the form of
> >>>>> annotation as the kind of input/ output the workflow is accepting.
> >>>>> Then we will be able to check these against users entry during
> >> execution.
> >>>>> Please correct me if I am wrong.
> >>>>>
> >>>>> Regards
> >>>>> Vijayendra
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <subs.zero@gmail.com
> >
> >>>> wrote:
> >>>>> Well exactly, as long as you can define standard way of
> communication.
> >>>> That
> >>>>> is, you can define in advance what should be a string, array and what
> >>>>> should be a integer etc. We have no problem.
> >>>>>
> >>>>> So, when you look at problems, with JSON <-> XML or the other way
> >> round,
> >>>>> they talk of the very general case (where you no nothing about the
> data
> >>>> you
> >>>>> are converting other than it is valid XML/JSON). There are a myriad
> of
> >>>>> problems in that case, which you pointed out.
> >>>>>
> >>>>> But when there is standard, there is only one way of doing things,
> and
> >>>> not
> >>>>> several. I think that is the way forward. So what I am proposing is
> >> maybe
> >>>>> we all discuss and define this standard within the first week of GSoC
> >>>>> starting and then actually move into coding. So as long as we work
> with
> >>>> the
> >>>>> presumption that this will be done, we really dont have to worry a
> lot
> >>>>> about this.
> >>>>>
> >>>>> Cheers,
> >>>>> Subho.
> >>>>>
> >>>>>
> >>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>>>> shameerainfo@gmail.com> wrote:
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <
> subs.zero@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Some of these problems are very specific to what the XML is
> >>>>>> representing,
> >>>>>>> it might not be an actual problem in Airavata,
> >>>>>>> maybe some one more experienced with the codebase can point this
> out.
> >>>>>>>
> >>>>>>
> >>>>>> All issues pointed out in the paper is not directly valid to our
> >>>>>> conversion, I didn't list the issues actually need to address in
> this
> >>>> case
> >>>>>> because thought it is worth to read that introduction part which
> >>>> explain
> >>>>>> the all the issues we have with this conversion and give us a solid
> >>>>>> background of that.
> >>>>>>
> >>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- I
> >>>>>> really
> >>>>>>> dont see these as problems, as long as you can agree that all
> >>>> parts of
> >>>>>>> airavata will treat the JSON in a standard (probably we have to
> >>>> define
> >>>>>>> this) way.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>> The issue with JSON array only comes when we try to convert XML to
> >>>> JSON not
> >>>>>> the other way. If we map with JSON, inputparameters and
> >>>> outputparameters in
> >>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we
> >>>> need to
> >>>>>> solve this issue.
> >>>>>>
> >>>>>> JSON XML JSON
> >>>>>> {"inputs":["test"]} --> <inputs>test<inputs> -->
> {"inputs":["test"]}
> >>>> //
> >>>>>> correct one
> >>>>>> --> {"inputs":"test"} // incorrect one
> >>>>>>
> >>>>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>>>
> >>>>>>> Are separate namespaces used in Airavata? Only place I can see
> >>>> this
> >>>>>>> being
> >>>>>>> used is probably in the WSDL, but if we can agree on another way
> >>>>>>> of communicating registered applications' I/O parameters to the
> >>>> front
> >>>>>>> end
> >>>>>>> (JSON based), then maybe we can work around this (minor) problem.
> >>>> Are
> >>>>>>> custom processing instructions to the Xbaya XML parse even used?
> >>>>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>>>
> >>>>>>
> >>>>>> Yes,attributes convertion will not be a big issues we can solve it.
> As
> >>>>>> Lahiru mentioned in Hangout session namesapce handling is not a big
> >>>> issue
> >>>>>> with Airavata.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> <array name="abc">
> >>>>>>> <element>1</element>
> >>>>>>> <element>2</element>
> >>>>>>> <element>3</element>
> >>>>>>> <element>4</element>
> >>>>>>> </array>
> >>>>>>>
> >>>>>>> Can become
> >>>>>>>
> >>>>>>> {
> >>>>>>>
> >>>>>>> abc : ['1', '2', '3', '4']
> >>>>>>>
> >>>>>>> }
> >>>>>>>
> >>>>>>
> >>>>>> With this example it show us we need to change the XML message
> format
> >>>> of
> >>>>>> server side, which require to change the all schemas, If we are
> going
> >>>> to
> >>>>>> change the schemas then we need to change the way it process it in
> >>>> Ariavara
> >>>>>> core. We have dropped our initial major requirement, which is keep
> the
> >>>>>> Airavata Server side as it is.
> >>>>>>
> >>>>>> with this conversion we only deal with json strings, yes we can send
> >>>> JSON
> >>>>>> request with other formats supported by JSON like boolen, null,
> Number
> >>>>>> etc.. But there is no way to get the same JSON from XML as XML only
> >>>> deal
> >>>>>> only with Strings. I think it is good if we can consume a this
> >> features
> >>>>>> with JSON.
> >>>>>>
> >>>>>> let say i need to send a integer or float to the server using JSON
> >> then
> >>>>>> proper way is to send {"<name>":123.45} this will works fine but
> >>>> problem is
> >>>>>> how we get the same output ?
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Shameera.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>> Subho.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best Regards,
> >>>>>> Shameera Rathnayaka.
> >>>>>>
> >>>>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>>
> >>> --
> >>> Best Regards,
> >>> Shameera Rathnayaka.
> >>>
> >>> Blog : http://shameerarathnayaka.blogspot.com/
> >>
> >>
> >
> >
> > --
> > Best Regards,
> > Shameera Rathnayaka.
> >
> > email: shameera AT apache.org , shameerainfo AT gmail.com
> > Blog : http://shameerarathnayaka.blogspot.com/
>
>
Re: Airavata GSoC 2013 Master Project
Posted by Suresh Marru <sm...@apache.org>.
Hi Shameera,
This is great, I appreciate you sharing it, I realize this is still working document, but I want other students to start seeing it and model their proposals in a similar way.
Airavata Mentors,
Please provide feedback directly on the melange site and uncheck the "private" box when you comment.
Suresh
On May 1, 2013, at 7:52 AM, Shameera Rathnayaka <sh...@gmail.com> wrote:
> Hi Suresh and All,
>
> Of course I am very much happy to share my proposal with everybody,
> actually i was going to update this thread with the melange link in few
> hours once i have done writing all the sections in the proposal. I haven't
> yet added the milestone plan into it and now working on it.
>
> The sub area i am going to work from the Master project is ' Implementing
> a JSON interface to Airavata Client side and Registry component'. Here is
> the link
> http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
> .
>
>
> Please note that i haven't completed everything in this and still doing
> modifications .Therefore proposal content may be changed bit, need to add
> more technical details of the approach which explains it well.
>
> I would like to know the feedback from all of you regarding the proposal
> and will be modifying it if there is anything to be done. Also please
> contact me if you need any help and i am very much willing to share my
> thoughts with all.
>
> Thanks!
> Shameera
>
>
>
> On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org> wrote:
>
>> Hi Shameera,
>>
>> Excellent proposal, great job. Would you mind to make your proposal
>> public and post the link here? Your proposal should help others to look at
>> it and learn from.
>>
>> Again I emphasize to all students, please don't feel you will be competing
>> with each others. If all of you write good proposals, there is a good
>> chance all of you will be selected. But without a good proposal, we cannot
>> help.
>>
>> Suresh
>>
>>
>> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <sh...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Yes it is not easy to solve all problems, But defining our own standard
>> or
>>> adhere to any standard
>>> provided by third party library will solve the problem to some extend.
>>>
>>> Here i see two possible approaches,
>>>
>>> 1. Use existing third party library(we can find which is best) adhere to
>> it
>>> standard and see how we change the
>>> backend to be inline with it.
>>>
>>> 2. Use our own convention with help of XMLSchema (The way i suggest).
>>>
>>> As Suresh mentioned we can do a POC with both approaches to compare
>>> performance
>>> and changes need to be done in server side. Then select the best one.
>>>
>>> Another question was, can we works with graph data in JSON format.
>>> There are few JS graph framworks[1] which provide that functionality.
>>> we can use one of them to show airavata monitoring data as graphs
>>>
>>> Thanks,
>>> Shameera.
>>>
>>> [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/> ,
>>> Processing.js <http://processingjs.org> , Sencha
>>> Charts<http://www.sencha.com/products/extjs/>
>>>
>>>
>>> On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org> wrote:
>>>
>>>> Hi Vijeyandra,
>>>>
>>>> Airavata Messaging is based on a pub-sub model and the events themselves
>>>> are xml (WS-Eventing [1]).
>>>>
>>>> The Messenger paper [2] should give you more information.
>>>>
>>>> Hi All (Especially those at WS02):
>>>>
>>>> Here is an old effort from a Morotuwa undergrad project, you may want to
>>>> read through these papers and chat with the authors to get experiences:
>>>>
>>>> http://dl.acm.org/citation.cfm?id=1890807
>>>> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
>>>> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
>>>> http://mooshabaya.wikidot.com/
>>>>
>>>>
>> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
>>>>
>>>> Suresh
>>>> [1] - http://www.w3.org/Submission/WS-Eventing/
>>>> [2] -
>>>>
>> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
>>>>
>>>> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
>>>> vijayendra.sdm@gmail.com> wrote:
>>>>
>>>>> Hi Suresh
>>>>>
>>>>> I wanted to know more about the monitoring tool .
>>>>> Currently from where does the monitoring tool gets data . Is it from
>>>>> workflow interpreter ? or Is it from the WS Messenger ( that might
>>>> continuously
>>>>> send messages to monitoring tool as to tell how much is the progress
>>>>> and what are the variables getting changed) ?
>>>>>
>>>>> Again the how is the data being exchanged. I guess it must be xml ?
>>>>> It must be one way data exchange . I mean the data is TO the
>>>>> monitoring module.
>>>>> Then monitoring Tool is sending back this
>>>>> data to Xbaya for displaying to the user ? Please correct me if I am
>>>> wrong
>>>>>
>>>>> I have downloaded the source code from the trunk . can you please point
>>>>> me which part of code should I be code at for this module.
>>>>>
>>>>> Regards
>>>>> Vijayendra
>>>>>
>>>>>
>>>>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
>>>> vijayendra.sdm@gmail.com> wrote:
>>>>> Hi
>>>>>
>>>>> What i am suggesting is, we send the JSON message directly to Airavata
>>>>> Backend(or Registry)
>>>>> When the message gets build after the transport phase, convert JSON
>>>> message
>>>>> to SOAP(XML).
>>>>> From that point message will treated as SOAP message.
>>>>>
>>>>> If we look at the JSON <--> XML conversion there are set of third party
>>>>> libraries we
>>>>> can use for. But before selecting a one we need to think about problems
>>>>> having
>>>>>
>>>>> with JSON <--> XML and how these libraries handle those issues. Because
>>>> we
>>>>> need a robust
>>>>> way to do this conversions.
>>>>>
>>>>>
>>>>>
>>>>> Shameera what you are suggesting is sending the JSON message directly
>> to
>>>> Registry.
>>>>> when the message gets built after the transport phase , convert it to
>>>> SOAP .
>>>>>
>>>>> If you are suggesting Registry will have JSON data instead of WSDL ,
>>>> Then this might
>>>>> complicate the things for us .
>>>>> The workflow interpreter needs wsdl(xml) to interpret the workflows and
>>>> for other details .
>>>>> Which means we might again have to do some changes with workflow
>>>> interpretor .
>>>>> Yesterday from what I heard in discussion is that , they do not want to
>>>> mess with workflow
>>>>> interpreter atleast for GSOC projects.
>>>>>
>>>>> What I want to suggest is , why carry the JSON data till Regisrty .
>>>> Build a interface
>>>>> before (Apache server API) where we can do the necessary conversion
>>>> (JSON to SOAP).
>>>>> In this way we can avoid messing up with Airavata server as a whole.
>>>>> Client ( using a we browser) is interacting with JSON (web service) but
>>>> the Apache server
>>>>> is interacting with SOAP.
>>>>>
>>>>>
>>>>>
>>>>> Secondly yesterday Suresh was speaking about validating the connections
>>>> of the workflow.
>>>>> for example , the workflow is expecting a file as input
>>>>> but user is giving a sting or int .
>>>>>
>>>>> Here what I suggest is , while creating wsdl in the registry for a
>>>> particular
>>>>> workflow , we can add extra information in the form of
>>>>> annotation as the kind of input/ output the workflow is accepting.
>>>>> Then we will be able to check these against users entry during
>> execution.
>>>>> Please correct me if I am wrong.
>>>>>
>>>>> Regards
>>>>> Vijayendra
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <su...@gmail.com>
>>>> wrote:
>>>>> Well exactly, as long as you can define standard way of communication.
>>>> That
>>>>> is, you can define in advance what should be a string, array and what
>>>>> should be a integer etc. We have no problem.
>>>>>
>>>>> So, when you look at problems, with JSON <-> XML or the other way
>> round,
>>>>> they talk of the very general case (where you no nothing about the data
>>>> you
>>>>> are converting other than it is valid XML/JSON). There are a myriad of
>>>>> problems in that case, which you pointed out.
>>>>>
>>>>> But when there is standard, there is only one way of doing things, and
>>>> not
>>>>> several. I think that is the way forward. So what I am proposing is
>> maybe
>>>>> we all discuss and define this standard within the first week of GSoC
>>>>> starting and then actually move into coding. So as long as we work with
>>>> the
>>>>> presumption that this will be done, we really dont have to worry a lot
>>>>> about this.
>>>>>
>>>>> Cheers,
>>>>> Subho.
>>>>>
>>>>>
>>>>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
>>>>> shameerainfo@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <su...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Some of these problems are very specific to what the XML is
>>>>>> representing,
>>>>>>> it might not be an actual problem in Airavata,
>>>>>>> maybe some one more experienced with the codebase can point this out.
>>>>>>>
>>>>>>
>>>>>> All issues pointed out in the paper is not directly valid to our
>>>>>> conversion, I didn't list the issues actually need to address in this
>>>> case
>>>>>> because thought it is worth to read that introduction part which
>>>> explain
>>>>>> the all the issues we have with this conversion and give us a solid
>>>>>> background of that.
>>>>>>
>>>>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- I
>>>>>> really
>>>>>>> dont see these as problems, as long as you can agree that all
>>>> parts of
>>>>>>> airavata will treat the JSON in a standard (probably we have to
>>>> define
>>>>>>> this) way.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> The issue with JSON array only comes when we try to convert XML to
>>>> JSON not
>>>>>> the other way. If we map with JSON, inputparameters and
>>>> outputparameters in
>>>>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we
>>>> need to
>>>>>> solve this issue.
>>>>>>
>>>>>> JSON XML JSON
>>>>>> {"inputs":["test"]} --> <inputs>test<inputs> --> {"inputs":["test"]}
>>>> //
>>>>>> correct one
>>>>>> --> {"inputs":"test"} // incorrect one
>>>>>>
>>>>>> 2. Namespaces, Processing Instructions -- Is this required?
>>>>>>
>>>>>>> Are separate namespaces used in Airavata? Only place I can see
>>>> this
>>>>>>> being
>>>>>>> used is probably in the WSDL, but if we can agree on another way
>>>>>>> of communicating registered applications' I/O parameters to the
>>>> front
>>>>>>> end
>>>>>>> (JSON based), then maybe we can work around this (minor) problem.
>>>> Are
>>>>>>> custom processing instructions to the Xbaya XML parse even used?
>>>>>>> 3. Attributes -- Again, this can be fixed easily
>>>>>>>
>>>>>>
>>>>>> Yes,attributes convertion will not be a big issues we can solve it. As
>>>>>> Lahiru mentioned in Hangout session namesapce handling is not a big
>>>> issue
>>>>>> with Airavata.
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> <array name="abc">
>>>>>>> <element>1</element>
>>>>>>> <element>2</element>
>>>>>>> <element>3</element>
>>>>>>> <element>4</element>
>>>>>>> </array>
>>>>>>>
>>>>>>> Can become
>>>>>>>
>>>>>>> {
>>>>>>>
>>>>>>> abc : ['1', '2', '3', '4']
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>
>>>>>> With this example it show us we need to change the XML message format
>>>> of
>>>>>> server side, which require to change the all schemas, If we are going
>>>> to
>>>>>> change the schemas then we need to change the way it process it in
>>>> Ariavara
>>>>>> core. We have dropped our initial major requirement, which is keep the
>>>>>> Airavata Server side as it is.
>>>>>>
>>>>>> with this conversion we only deal with json strings, yes we can send
>>>> JSON
>>>>>> request with other formats supported by JSON like boolen, null, Number
>>>>>> etc.. But there is no way to get the same JSON from XML as XML only
>>>> deal
>>>>>> only with Strings. I think it is good if we can consume a this
>> features
>>>>>> with JSON.
>>>>>>
>>>>>> let say i need to send a integer or float to the server using JSON
>> then
>>>>>> proper way is to send {"<name>":123.45} this will works fine but
>>>> problem is
>>>>>> how we get the same output ?
>>>>>>
>>>>>> Thanks,
>>>>>> Shameera.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Subho.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Shameera Rathnayaka.
>>>>>>
>>>>>> Blog : http://shameerarathnayaka.blogspot.com/
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Shameera Rathnayaka.
>>>
>>> Blog : http://shameerarathnayaka.blogspot.com/
>>
>>
>
>
> --
> Best Regards,
> Shameera Rathnayaka.
>
> email: shameera AT apache.org , shameerainfo AT gmail.com
> Blog : http://shameerarathnayaka.blogspot.com/
Re: Airavata GSoC 2013 Master Project
Posted by Shameera Rathnayaka <sh...@gmail.com>.
Hi Suresh and All,
Of course I am very much happy to share my proposal with everybody,
actually i was going to update this thread with the melange link in few
hours once i have done writing all the sections in the proposal. I haven't
yet added the milestone plan into it and now working on it.
The sub area i am going to work from the Master project is ' Implementing
a JSON interface to Airavata Client side and Registry component'. Here is
the link
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2013/shameera/60002#
.
Please note that i haven't completed everything in this and still doing
modifications .Therefore proposal content may be changed bit, need to add
more technical details of the approach which explains it well.
I would like to know the feedback from all of you regarding the proposal
and will be modifying it if there is anything to be done. Also please
contact me if you need any help and i am very much willing to share my
thoughts with all.
Thanks!
Shameera
On Wed, May 1, 2013 at 4:51 PM, Suresh Marru <sm...@apache.org> wrote:
> Hi Shameera,
>
> Excellent proposal, great job. Would you mind to make your proposal
> public and post the link here? Your proposal should help others to look at
> it and learn from.
>
> Again I emphasize to all students, please don't feel you will be competing
> with each others. If all of you write good proposals, there is a good
> chance all of you will be selected. But without a good proposal, we cannot
> help.
>
> Suresh
>
>
> On Apr 23, 2013, at 1:22 PM, Shameera Rathnayaka <sh...@gmail.com>
> wrote:
>
> > Hi,
> >
> > Yes it is not easy to solve all problems, But defining our own standard
> or
> > adhere to any standard
> > provided by third party library will solve the problem to some extend.
> >
> > Here i see two possible approaches,
> >
> > 1. Use existing third party library(we can find which is best) adhere to
> it
> > standard and see how we change the
> > backend to be inline with it.
> >
> > 2. Use our own convention with help of XMLSchema (The way i suggest).
> >
> > As Suresh mentioned we can do a POC with both approaches to compare
> > performance
> > and changes need to be done in server side. Then select the best one.
> >
> > Another question was, can we works with graph data in JSON format.
> > There are few JS graph framworks[1] which provide that functionality.
> > we can use one of them to show airavata monitoring data as graphs
> >
> > Thanks,
> > Shameera.
> >
> > [1] jqPlot <http://www.jqplot.com/index.php> , D3 <http://d3js.org/> ,
> > Processing.js <http://processingjs.org> , Sencha
> > Charts<http://www.sencha.com/products/extjs/>
> >
> >
> > On Tue, Apr 23, 2013 at 5:44 PM, Suresh Marru <sm...@apache.org> wrote:
> >
> >> Hi Vijeyandra,
> >>
> >> Airavata Messaging is based on a pub-sub model and the events themselves
> >> are xml (WS-Eventing [1]).
> >>
> >> The Messenger paper [2] should give you more information.
> >>
> >> Hi All (Especially those at WS02):
> >>
> >> Here is an old effort from a Morotuwa undergrad project, you may want to
> >> read through these papers and chat with the authors to get experiences:
> >>
> >> http://dl.acm.org/citation.cfm?id=1890807
> >> http://mgc2010.lncc.br/slides-pdf/Mooshabaya_Final_Presentation.pdf
> >> http://kkpradeeban.blogspot.com/2010/09/mooshabaya-story-behind.html
> >> http://mooshabaya.wikidot.com/
> >>
> >>
> http://chamibuddhika.wordpress.com/2009/10/06/mooshabaya-generates-mashups-from-workflows/
> >>
> >> Suresh
> >> [1] - http://www.w3.org/Submission/WS-Eventing/
> >> [2] -
> >>
> http://www.extreme.indiana.edu/xgws/messenger/doc/HuangY-WSMessenger.pdf
> >>
> >> On Apr 23, 2013, at 6:20 AM, Vijayendra Grampurohit <
> >> vijayendra.sdm@gmail.com> wrote:
> >>
> >>> Hi Suresh
> >>>
> >>> I wanted to know more about the monitoring tool .
> >>> Currently from where does the monitoring tool gets data . Is it from
> >>> workflow interpreter ? or Is it from the WS Messenger ( that might
> >> continuously
> >>> send messages to monitoring tool as to tell how much is the progress
> >>> and what are the variables getting changed) ?
> >>>
> >>> Again the how is the data being exchanged. I guess it must be xml ?
> >>> It must be one way data exchange . I mean the data is TO the
> >>> monitoring module.
> >>> Then monitoring Tool is sending back this
> >>> data to Xbaya for displaying to the user ? Please correct me if I am
> >> wrong
> >>>
> >>> I have downloaded the source code from the trunk . can you please point
> >>> me which part of code should I be code at for this module.
> >>>
> >>> Regards
> >>> Vijayendra
> >>>
> >>>
> >>> On Tue, Apr 23, 2013 at 3:16 PM, Vijayendra Grampurohit <
> >> vijayendra.sdm@gmail.com> wrote:
> >>> Hi
> >>>
> >>> What i am suggesting is, we send the JSON message directly to Airavata
> >>> Backend(or Registry)
> >>> When the message gets build after the transport phase, convert JSON
> >> message
> >>> to SOAP(XML).
> >>> From that point message will treated as SOAP message.
> >>>
> >>> If we look at the JSON <--> XML conversion there are set of third party
> >>> libraries we
> >>> can use for. But before selecting a one we need to think about problems
> >>> having
> >>>
> >>> with JSON <--> XML and how these libraries handle those issues. Because
> >> we
> >>> need a robust
> >>> way to do this conversions.
> >>>
> >>>
> >>>
> >>> Shameera what you are suggesting is sending the JSON message directly
> to
> >> Registry.
> >>> when the message gets built after the transport phase , convert it to
> >> SOAP .
> >>>
> >>> If you are suggesting Registry will have JSON data instead of WSDL ,
> >> Then this might
> >>> complicate the things for us .
> >>> The workflow interpreter needs wsdl(xml) to interpret the workflows and
> >> for other details .
> >>> Which means we might again have to do some changes with workflow
> >> interpretor .
> >>> Yesterday from what I heard in discussion is that , they do not want to
> >> mess with workflow
> >>> interpreter atleast for GSOC projects.
> >>>
> >>> What I want to suggest is , why carry the JSON data till Regisrty .
> >> Build a interface
> >>> before (Apache server API) where we can do the necessary conversion
> >> (JSON to SOAP).
> >>> In this way we can avoid messing up with Airavata server as a whole.
> >>> Client ( using a we browser) is interacting with JSON (web service) but
> >> the Apache server
> >>> is interacting with SOAP.
> >>>
> >>>
> >>>
> >>> Secondly yesterday Suresh was speaking about validating the connections
> >> of the workflow.
> >>> for example , the workflow is expecting a file as input
> >>> but user is giving a sting or int .
> >>>
> >>> Here what I suggest is , while creating wsdl in the registry for a
> >> particular
> >>> workflow , we can add extra information in the form of
> >>> annotation as the kind of input/ output the workflow is accepting.
> >>> Then we will be able to check these against users entry during
> execution.
> >>> Please correct me if I am wrong.
> >>>
> >>> Regards
> >>> Vijayendra
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Tue, Apr 23, 2013 at 1:13 PM, Subho Banerjee <su...@gmail.com>
> >> wrote:
> >>> Well exactly, as long as you can define standard way of communication.
> >> That
> >>> is, you can define in advance what should be a string, array and what
> >>> should be a integer etc. We have no problem.
> >>>
> >>> So, when you look at problems, with JSON <-> XML or the other way
> round,
> >>> they talk of the very general case (where you no nothing about the data
> >> you
> >>> are converting other than it is valid XML/JSON). There are a myriad of
> >>> problems in that case, which you pointed out.
> >>>
> >>> But when there is standard, there is only one way of doing things, and
> >> not
> >>> several. I think that is the way forward. So what I am proposing is
> maybe
> >>> we all discuss and define this standard within the first week of GSoC
> >>> starting and then actually move into coding. So as long as we work with
> >> the
> >>> presumption that this will be done, we really dont have to worry a lot
> >>> about this.
> >>>
> >>> Cheers,
> >>> Subho.
> >>>
> >>>
> >>> On Tue, Apr 23, 2013 at 11:52 AM, Shameera Rathnayaka <
> >>> shameerainfo@gmail.com> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> On Tue, Apr 23, 2013 at 2:25 AM, Subho Banerjee <su...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Some of these problems are very specific to what the XML is
> >>>> representing,
> >>>>> it might not be an actual problem in Airavata,
> >>>>> maybe some one more experienced with the codebase can point this out.
> >>>>>
> >>>>
> >>>> All issues pointed out in the paper is not directly valid to our
> >>>> conversion, I didn't list the issues actually need to address in this
> >> case
> >>>> because thought it is worth to read that introduction part which
> >> explain
> >>>> the all the issues we have with this conversion and give us a solid
> >>>> background of that.
> >>>>
> >>>>> 1. Anonymous values, Arrays, Implicit Typing, Character sets -- I
> >>>> really
> >>>>> dont see these as problems, as long as you can agree that all
> >> parts of
> >>>>> airavata will treat the JSON in a standard (probably we have to
> >> define
> >>>>> this) way.
> >>>>>
> >>>>
> >>>>
> >>>> The issue with JSON array only comes when we try to convert XML to
> >> JSON not
> >>>> the other way. If we map with JSON, inputparameters and
> >> outputparameters in
> >>>> the ServiceDescription.xsd will map with JSON Arrays. Therefore we
> >> need to
> >>>> solve this issue.
> >>>>
> >>>> JSON XML JSON
> >>>> {"inputs":["test"]} --> <inputs>test<inputs> --> {"inputs":["test"]}
> >> //
> >>>> correct one
> >>>> --> {"inputs":"test"} // incorrect one
> >>>>
> >>>> 2. Namespaces, Processing Instructions -- Is this required?
> >>>>
> >>>>> Are separate namespaces used in Airavata? Only place I can see
> >> this
> >>>>> being
> >>>>> used is probably in the WSDL, but if we can agree on another way
> >>>>> of communicating registered applications' I/O parameters to the
> >> front
> >>>>> end
> >>>>> (JSON based), then maybe we can work around this (minor) problem.
> >> Are
> >>>>> custom processing instructions to the Xbaya XML parse even used?
> >>>>> 3. Attributes -- Again, this can be fixed easily
> >>>>>
> >>>>
> >>>> Yes,attributes convertion will not be a big issues we can solve it. As
> >>>> Lahiru mentioned in Hangout session namesapce handling is not a big
> >> issue
> >>>> with Airavata.
> >>>>
> >>>>
> >>>>
> >>>>>
> >>>>> <array name="abc">
> >>>>> <element>1</element>
> >>>>> <element>2</element>
> >>>>> <element>3</element>
> >>>>> <element>4</element>
> >>>>> </array>
> >>>>>
> >>>>> Can become
> >>>>>
> >>>>> {
> >>>>>
> >>>>> abc : ['1', '2', '3', '4']
> >>>>>
> >>>>> }
> >>>>>
> >>>>
> >>>> With this example it show us we need to change the XML message format
> >> of
> >>>> server side, which require to change the all schemas, If we are going
> >> to
> >>>> change the schemas then we need to change the way it process it in
> >> Ariavara
> >>>> core. We have dropped our initial major requirement, which is keep the
> >>>> Airavata Server side as it is.
> >>>>
> >>>> with this conversion we only deal with json strings, yes we can send
> >> JSON
> >>>> request with other formats supported by JSON like boolen, null, Number
> >>>> etc.. But there is no way to get the same JSON from XML as XML only
> >> deal
> >>>> only with Strings. I think it is good if we can consume a this
> features
> >>>> with JSON.
> >>>>
> >>>> let say i need to send a integer or float to the server using JSON
> then
> >>>> proper way is to send {"<name>":123.45} this will works fine but
> >> problem is
> >>>> how we get the same output ?
> >>>>
> >>>> Thanks,
> >>>> Shameera.
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>> Cheers,
> >>>>> Subho.
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>> Shameera Rathnayaka.
> >>>>
> >>>> Blog : http://shameerarathnayaka.blogspot.com/
> >>>>
> >>>
> >>>
> >>
> >>
> >
> >
> > --
> > Best Regards,
> > Shameera Rathnayaka.
> >
> > Blog : http://shameerarathnayaka.blogspot.com/
>
>
--
Best Regards,
Shameera Rathnayaka.
email: shameera AT apache.org , shameerainfo AT gmail.com
Blog : http://shameerarathnayaka.blogspot.com/