You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airavata.apache.org by Saminda Wijeratne <sa...@gmail.com> on 2013/05/20 23:05:22 UTC
Updates to Airavata API version 0.8
Following updates will be present Airavata API from 0.8 release.
1. *Error message from the immediate exception used as the error message
for the AiravataAPInvocationException*. Earlier it was just "Error
invoking API". *This change was made so that the users of the API can
show the error from the exception without having to dig in to the inner
exception to get a better error message.*
2. *Saving and retrieving experiment execution error details*. Earlier
we were not persisting any execution error details of the workflow. Error
details are only available only if the developers are monitoring experiment
at the time of error occurs. *This is not feasible if users would only
want to go through the error details later without doing monitoring from
the beginning. Therefore we are now persisting the error details (not as
part of provenance data) of executing workflows (i.e. experiments)* in
to the registry and also retrieving from registry through Airavata API
(which calls the Registry API).
For now we have the functions for saving and retrieving errors in the
ExecutionManager[1]. We need to decide if this is the correct place to put
these functions. Your thoughts in this matter are most welcome...
Thanks,
Saminda
1.
https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
Re: Updates to Airavata API version 0.8
Posted by Saminda Wijeratne <sa...@gmail.com>.
Thanks Raman for the feedback.
On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
<ra...@gmail.com>wrote:
> Thank, these are very useful for API users. Only thing i found confusing
> for the user is API managers. ProvenanceManager is the one used to get
> experiment status. Users will get FINISHED or FAILED status from the data
> object. On failure, i was trying to find a method to get the error data but
> i was not able to find as those methods exist in ExecutionManager.
> According to me, experiment status and in case of failures get error need
> to be part of one manager. May be a ExperimentManager? Getting output data
> can be left as part of provenance manages as only few client may want to
> get the data in few cases. Getting error detail in case of error is obvious
> step.
>
I totally agree with you. We need to determine where to place the functions
to retrieve errors that is logical and driven to locate it through
intuition of the user.
>
> Other thing can you please explain the difference between
> WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError,
> GFacExecutionError on a Wiki page for the users. Also add a method to get
> JobID as that is required to get GFacExecutionError.
>
Will do Raman. I created a Jira[2] for this incase I forget.
2. https://issues.apache.org/jira/browse/AIRAVATA-858
>
> I tested the WorkflowExecutionError method and its not returning any data.
> Can you please test that as well?
>
> ExperimentData data =
> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> List<WorkflowExecutionError> experimentExecutionErrors =
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> experimentID);
>
> Thanks
> Raminder
>
>
> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
>
> > Following updates will be present Airavata API from 0.8 release.
> >
> >
> > 1. *Error message from the immediate exception used as the error
> message
> > for the AiravataAPInvocationException*. Earlier it was just "Error
> > invoking API". *This change was made so that the users of the API can
> > show the error from the exception without having to dig in to the inner
> > exception to get a better error message.*
> > 2. *Saving and retrieving experiment execution error details*. Earlier
> > we were not persisting any execution error details of the workflow.
> Error
> > details are only available only if the developers are monitoring
> experiment
> > at the time of error occurs. *This is not feasible if users would only
> > want to go through the error details later without doing monitoring
> from
> > the beginning. Therefore we are now persisting the error details (not
> as
> > part of provenance data) of executing workflows (i.e. experiments)* in
> > to the registry and also retrieving from registry through Airavata API
> > (which calls the Registry API).
> >
> > For now we have the functions for saving and retrieving errors in the
> > ExecutionManager[1]. We need to decide if this is the correct place to
> put
> > these functions. Your thoughts in this matter are most welcome...
> >
> >
> > Thanks,
> > Saminda
> >
> > 1.
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>
>
Re: Updates to Airavata API version 0.8
Posted by Saminda Wijeratne <sa...@gmail.com>.
Except for HadoopProvider I've completed job data persistance for all the
other 5 providers. (We should revisit it after the release to verify the
implementation)
On Fri, Jun 14, 2013 at 11:13 AM, Suresh Marru <sm...@apache.org> wrote:
> On Jun 14, 2013, at 10:49 AM, Lahiru Gunathilake <gl...@gmail.com>
> wrote:
>
> > Hi Marlon,
> >
> > I have no update about the release, since I've been working with Gary in
> > Ultrascan project. I found some issues to fix for ultrascan, now working
> on
> > them, so we have to wait until Gary gives the green light to do the
> release.
>
> Lahiru, thanks for all your work on the release blocks and for being the
> RM for this release.
>
> No biggie for this case but want to caution:
>
> Just a friendly reminder to all of us - 'if it did not happen in the
> mailing list, it did not happen'. So please do not assume context with
> these discussions and explicitly state the issues. Also, ultrascan cannot
> block airavata release. If Gary or others report issues to you offline from
> a RC-* testing, please alert the list or raise a JIRA (thanks for doing it
> for this issue).
>
> Suresh
>
> >
> > WDYT ?
> >
> > Regards
> > Lahiru
> >
> >
> > On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce <ma...@iu.edu> wrote:
> >
> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> Hash: SHA1
> >>
> >> What's the status of the 0.8 release?
> >>
> >>
> >> Marlon
> >>
> >>
> >> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> >>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
> >>> <th...@gmail.com>wrote:
> >>>
> >>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
> >>>> <samindaw@gmail.com
> >>>>> wrote:
> >>>>
> >>>>> Latest update on AiravataAPI updates for 0.8 version
> >>>>>
> >>>>> 1. *Error message from the immediate exception used as the
> >>>>> error
> >>>> message
> >>>>> for the AiravataAPInvocationException*. Earlier it was just
> >>>>> "Error invoking API". *This change was made so that the users
> >>>>> of the API can show the error from the exception without having
> >>>>> to dig in to the
> >>>> inner
> >>>>> exception to get a better error message.* 2. *Saving and
> >>>>> retrieving experiment execution error details*. Earlier we were
> >>>>> not persisting any execution error details of the workflow.
> >>>>> Error details are only available only if the developers are
> >>>>> monitoring experiment at the time of error occurs. *This is not
> >>>>> feasible if users would only want to go through the error
> >>>>> details later without doing monitoring
> >>>> from
> >>>>> the beginning. Therefore we are now persisting the error
> >>>>> details (not
> >>>> as
> >>>>> part of provenance data) of executing workflows (i.e.
> >>>>> experiments) in
> >>>> to
> >>>>> the registry and also retrieving from registry through Airavata
> >>>>> API (which calls the Registry API). These functions are
> >>>>> available in the "ExecutionManager" of the AiravataAPI and
> >>>>> "ProvenanceRegistry" in the RegistryAPI. * 3. Persisting &
> >>>>> retrieving GFac job data. There are many benefits of saving the
> >>>>> GRAM data, EC2 data etc. such as for debugging, analysis
> >>>> etc.
> >>>>> Currently these data are available only through the message
> >>>>> notifications. *It is not feasible for the same reasons
> >>>>> mentioned for API change for saving error data. Hence the
> >>>>> introduction of functions in the API to save & retrieve GFac
> >>>>> job data.** These functions are available as
> >>>>> get/setApplicationJob***(...) functions in the
> >>>>> "ProvenanceManager" of the AiravataAPI and "ProvenanceRegistry"
> >>>>> in the RegistryAPI.*
> >>>>>
> >>>>
> >>>> May I ask why we need setApplicationJob****() methods in the API
> >>>> ? As far as I understood job information is populated by GFac and
> >>>> other internal components. So why do we need API methods to set
> >>>> those data ? (OR am I missing something ?)
> >>>>
> >>> GFac and Internal components also use the AiravataAPI to populate
> >>> the job data in the registry.
> >>>
> >>>
> >>>> Thanks Amila
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> Saminda
> >>>>>
> >>>>>
> >>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> >>>> raminderjsingh@gmail.com
> >>>>>> wrote:
> >>>>>
> >>>>>> Thanks Chathuri, I will test and let you know.
> >>>>>>
> >>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Raman,
> >>>>>>>
> >>>>>>> The method Saminda mentioned will work for your case. I
> >>>>>>> tested with
> >>>>> your
> >>>>>>> test class. I fixed deserialization issues from REST
> >>>>>>> service side.
> >>>> You
> >>>>>> will
> >>>>>>> probably have to update both server and client.
> >>>>>>>
> >>>>>>> Regards, Chathuri
> >>>>>>>
> >>>>>>>
> >>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
> >>>> samindaw@gmail.com
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> So far what you are saying is true. If the client knows
> >>>>>>>> which node
> >>>>>> failed
> >>>>>>>> he/she can get the list of errors for that node using
> >>>>>>>> the getExecutionError(...) function. Use the "type" as
> >>>>>>>> ALL and leave the gfacJobId as null. I couldn;t think of
> >>>>>>>> doing this any other better
> >>>> way
> >>>>>> than
> >>>>>>>> introducing a whole lot of functions which could be more
> >>>>>> annoying/confusing
> >>>>>>>> to figure-out. I'm welcome for suggestions for
> >>>>>>>> improvements.
> >>>>>>>>
> >>>>>>>> Saminda
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> >>>>>> raminderjsingh@gmail.com
> >>>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Thanks Chathuri, the WorkflowExecutionError works.
> >>>>>>>>> Integrating to
> >>>>>> Error
> >>>>>>>>> API brought few questions. We have ErrorTypes as :
> >>>>>>>> WorkflowExecutionError,
> >>>>>>>>> ExperimentExecutionError, NodeExecutionError,
> >>>> GFacJobExecutionError,
> >>>>>>>>> ExecutionError . How do we want clients to know which
> >>>>>>>>> error type to
> >>>>>>>> query?
> >>>>>>>>> I ran a job and it failed, now i want to get error.
> >>>>>>>>> Error occurred
> >>>> at
> >>>>>>>> GFac
> >>>>>>>>> provider level and client does not have any
> >>>>>>>>> information. Do we
> >>>> expect
> >>>>>>>>> client to query all different types?
> >>>>>>>>>
> >>>>>>>>> I thing we can have is a method to get all error and
> >>>>>>>>> return the
> >>>> Error
> >>>>>>>> type
> >>>>>>>>> part of the object to help the client to classify the
> >>>>>>>>> error
> >>>> location.
> >>>>>>>>> Other suggestions?
> >>>>>>>>>
> >>>>>>>>> Raminder
> >>>>>>>>>
> >>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Raman,
> >>>>>>>>>>
> >>>>>>>>>> WorkflowExecutionError method not returning any data
> >>>>>>>>>> issue should
> >>>> be
> >>>>>>>>> fixed
> >>>>>>>>>> now. Can you get an update and check.
> >>>>>>>>>>
> >>>>>>>>>> Regards, Chathuri
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> >>>>>>>>>> <ra...@gmail.com>wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Thank, these are very useful for API users. Only
> >>>>>>>>>>> thing i found
> >>>>>>>> confusing
> >>>>>>>>>>> for the user is API managers. ProvenanceManager is
> >>>>>>>>>>> the one used
> >>>> to
> >>>>>>>> get
> >>>>>>>>>>> experiment status. Users will get FINISHED or
> >>>>>>>>>>> FAILED status from
> >>>>> the
> >>>>>>>>> data
> >>>>>>>>>>> object. On failure, i was trying to find a method
> >>>>>>>>>>> to get the
> >>>> error
> >>>>>>>> data
> >>>>>>>>> but
> >>>>>>>>>>> i was not able to find as those methods exist in
> >>>> ExecutionManager.
> >>>>>>>>>>> According to me, experiment status and in case of
> >>>>>>>>>>> failures get
> >>>>> error
> >>>>>>>>> need
> >>>>>>>>>>> to be part of one manager. May be a
> >>>>>>>>>>> ExperimentManager? Getting
> >>>>>> output
> >>>>>>>>> data
> >>>>>>>>>>> can be left as part of provenance manages as only
> >>>>>>>>>>> few client may
> >>>>> want
> >>>>>>>> to
> >>>>>>>>>>> get the data in few cases. Getting error detail in
> >>>>>>>>>>> case of error
> >>>> is
> >>>>>>>>> obvious
> >>>>>>>>>>> step.
> >>>>>>>>>>>
> >>>>>>>>>>> Other thing can you please explain the difference
> >>>>>>>>>>> between WorkflowExecutionError,
> >>>>>>>>>>> ExperimentExecutionError,
> >>>>> NodeExecutionError,
> >>>>>>>>>>> GFacExecutionError on a Wiki page for the users.
> >>>>>>>>>>> Also add a
> >>>> method
> >>>>> to
> >>>>>>>>> get
> >>>>>>>>>>> JobID as that is required to get
> >>>>>>>>>>> GFacExecutionError.
> >>>>>>>>>>>
> >>>>>>>>>>> I tested the WorkflowExecutionError method and its
> >>>>>>>>>>> not returning
> >>>>> any
> >>>>>>>>> data.
> >>>>>>>>>>> Can you please test that as well?
> >>>>>>>>>>>
> >>>>>>>>>>> ExperimentData data =
> >>>>>>>>>>>
> >>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >>>>>>>>>>>
> >>>>
> >> List<WorkflowExecutionError> experimentExecutionErrors =
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >>>>>>>>>>>
> >>>>
> >> experimentID);
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks Raminder
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Following updates will be present Airavata API
> >>>>>>>>>>>> from 0.8 release.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1. *Error message from the immediate exception
> >>>>>>>>>>>> used as the error
> >>>>>>>>>>> message
> >>>>>>>>>>>> for the AiravataAPInvocationException*. Earlier
> >>>>>>>>>>>> it was just
> >>>> "Error
> >>>>>>>>>>>> invoking API". *This change was made so that the
> >>>>>>>>>>>> users of the
> >>>> API
> >>>>>>>> can
> >>>>>>>>>>>> show the error from the exception without having
> >>>>>>>>>>>> to dig in to
> >>>> the
> >>>>>>>>> inner
> >>>>>>>>>>>> exception to get a better error message.* 2.
> >>>>>>>>>>>> *Saving and retrieving experiment execution error
> >>>>>>>>>>>> details*.
> >>>>>>>> Earlier
> >>>>>>>>>>>> we were not persisting any execution error
> >>>>>>>>>>>> details of the
> >>>>> workflow.
> >>>>>>>>>>> Error
> >>>>>>>>>>>> details are only available only if the developers
> >>>>>>>>>>>> are monitoring
> >>>>>>>>>>> experiment
> >>>>>>>>>>>> at the time of error occurs. *This is not
> >>>>>>>>>>>> feasible if users
> >>>> would
> >>>>>>>> only
> >>>>>>>>>>>> want to go through the error details later
> >>>>>>>>>>>> without doing
> >>>>> monitoring
> >>>>>>>>>>> from
> >>>>>>>>>>>> the beginning. Therefore we are now persisting
> >>>>>>>>>>>> the error details
> >>>>>>>> (not
> >>>>>>>>>>> as
> >>>>>>>>>>>> part of provenance data) of executing workflows
> >>>>>>>>>>>> (i.e.
> >>>>> experiments)*
> >>>>>>>> in
> >>>>>>>>>>>> to the registry and also retrieving from registry
> >>>>>>>>>>>> through
> >>>> Airavata
> >>>>>>>> API
> >>>>>>>>>>>> (which calls the Registry API).
> >>>>>>>>>>>>
> >>>>>>>>>>>> For now we have the functions for saving and
> >>>>>>>>>>>> retrieving errors
> >>>> in
> >>>>>> the
> >>>>>>>>>>>> ExecutionManager[1]. We need to decide if this is
> >>>>>>>>>>>> the correct
> >>>>> place
> >>>>>>>> to
> >>>>>>>>>>> put
> >>>>>>>>>>>> these functions. Your thoughts in this matter are
> >>>>>>>>>>>> most
> >>>> welcome...
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks, Saminda
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >> -----BEGIN PGP SIGNATURE-----
> >> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> >> Comment: GPGTools - http://gpgtools.org
> >> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
> >>
> >> iQEcBAEBAgAGBQJRuyxqAAoJEOEgD2XReDo54nwIAKjDjVqPk6YldnJD+T91k2HU
> >> Yj/VcvH/XAxvoZOu9zGpcUFNuLPbSiJEJes9Tx3Wk63k8v+l4rxTTRB5rWihbyYn
> >> xPEKtSnD8sCAiuFXlRHWLygzE80OVhC9Q43xkrnjjubKaWKp3ZgU1uF1ZL6LiTZb
> >> R83+Yl2WCdAAckX1W+567APk202nw93GLKGjmRcuj2mBPKgWXzzD7AX3cQX6pt9e
> >> g93PKGWJWdk/kb6YlhjkEfKH2RxJS/AgZvUYSAmf1ELL+siGqxEEOg7bwWwpfLNd
> >> NsjUr/rN88EcGfUKxYb4fSh1zGceqp02rWG2TwufZEIVaF8WjztY6CsE2TgfS0M=
> >> =vpMl
> >> -----END PGP SIGNATURE-----
> >>
> >
> >
> >
> > --
> > System Analyst Programmer
> > PTI Lab
> > Indiana University
>
>
Re: Updates to Airavata API version 0.8
Posted by Suresh Marru <sm...@apache.org>.
On Jun 14, 2013, at 10:49 AM, Lahiru Gunathilake <gl...@gmail.com> wrote:
> Hi Marlon,
>
> I have no update about the release, since I've been working with Gary in
> Ultrascan project. I found some issues to fix for ultrascan, now working on
> them, so we have to wait until Gary gives the green light to do the release.
Lahiru, thanks for all your work on the release blocks and for being the RM for this release.
No biggie for this case but want to caution:
Just a friendly reminder to all of us - 'if it did not happen in the mailing list, it did not happen'. So please do not assume context with these discussions and explicitly state the issues. Also, ultrascan cannot block airavata release. If Gary or others report issues to you offline from a RC-* testing, please alert the list or raise a JIRA (thanks for doing it for this issue).
Suresh
>
> WDYT ?
>
> Regards
> Lahiru
>
>
> On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce <ma...@iu.edu> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> What's the status of the 0.8 release?
>>
>>
>> Marlon
>>
>>
>> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
>>> <th...@gmail.com>wrote:
>>>
>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
>>>> <samindaw@gmail.com
>>>>> wrote:
>>>>
>>>>> Latest update on AiravataAPI updates for 0.8 version
>>>>>
>>>>> 1. *Error message from the immediate exception used as the
>>>>> error
>>>> message
>>>>> for the AiravataAPInvocationException*. Earlier it was just
>>>>> "Error invoking API". *This change was made so that the users
>>>>> of the API can show the error from the exception without having
>>>>> to dig in to the
>>>> inner
>>>>> exception to get a better error message.* 2. *Saving and
>>>>> retrieving experiment execution error details*. Earlier we were
>>>>> not persisting any execution error details of the workflow.
>>>>> Error details are only available only if the developers are
>>>>> monitoring experiment at the time of error occurs. *This is not
>>>>> feasible if users would only want to go through the error
>>>>> details later without doing monitoring
>>>> from
>>>>> the beginning. Therefore we are now persisting the error
>>>>> details (not
>>>> as
>>>>> part of provenance data) of executing workflows (i.e.
>>>>> experiments) in
>>>> to
>>>>> the registry and also retrieving from registry through Airavata
>>>>> API (which calls the Registry API). These functions are
>>>>> available in the "ExecutionManager" of the AiravataAPI and
>>>>> "ProvenanceRegistry" in the RegistryAPI. * 3. Persisting &
>>>>> retrieving GFac job data. There are many benefits of saving the
>>>>> GRAM data, EC2 data etc. such as for debugging, analysis
>>>> etc.
>>>>> Currently these data are available only through the message
>>>>> notifications. *It is not feasible for the same reasons
>>>>> mentioned for API change for saving error data. Hence the
>>>>> introduction of functions in the API to save & retrieve GFac
>>>>> job data.** These functions are available as
>>>>> get/setApplicationJob***(...) functions in the
>>>>> "ProvenanceManager" of the AiravataAPI and "ProvenanceRegistry"
>>>>> in the RegistryAPI.*
>>>>>
>>>>
>>>> May I ask why we need setApplicationJob****() methods in the API
>>>> ? As far as I understood job information is populated by GFac and
>>>> other internal components. So why do we need API methods to set
>>>> those data ? (OR am I missing something ?)
>>>>
>>> GFac and Internal components also use the AiravataAPI to populate
>>> the job data in the registry.
>>>
>>>
>>>> Thanks Amila
>>>>
>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Saminda
>>>>>
>>>>>
>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
>>>> raminderjsingh@gmail.com
>>>>>> wrote:
>>>>>
>>>>>> Thanks Chathuri, I will test and let you know.
>>>>>>
>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Raman,
>>>>>>>
>>>>>>> The method Saminda mentioned will work for your case. I
>>>>>>> tested with
>>>>> your
>>>>>>> test class. I fixed deserialization issues from REST
>>>>>>> service side.
>>>> You
>>>>>> will
>>>>>>> probably have to update both server and client.
>>>>>>>
>>>>>>> Regards, Chathuri
>>>>>>>
>>>>>>>
>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
>>>> samindaw@gmail.com
>>>>>>> wrote:
>>>>>>>
>>>>>>>> So far what you are saying is true. If the client knows
>>>>>>>> which node
>>>>>> failed
>>>>>>>> he/she can get the list of errors for that node using
>>>>>>>> the getExecutionError(...) function. Use the "type" as
>>>>>>>> ALL and leave the gfacJobId as null. I couldn;t think of
>>>>>>>> doing this any other better
>>>> way
>>>>>> than
>>>>>>>> introducing a whole lot of functions which could be more
>>>>>> annoying/confusing
>>>>>>>> to figure-out. I'm welcome for suggestions for
>>>>>>>> improvements.
>>>>>>>>
>>>>>>>> Saminda
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
>>>>>> raminderjsingh@gmail.com
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks Chathuri, the WorkflowExecutionError works.
>>>>>>>>> Integrating to
>>>>>> Error
>>>>>>>>> API brought few questions. We have ErrorTypes as :
>>>>>>>> WorkflowExecutionError,
>>>>>>>>> ExperimentExecutionError, NodeExecutionError,
>>>> GFacJobExecutionError,
>>>>>>>>> ExecutionError . How do we want clients to know which
>>>>>>>>> error type to
>>>>>>>> query?
>>>>>>>>> I ran a job and it failed, now i want to get error.
>>>>>>>>> Error occurred
>>>> at
>>>>>>>> GFac
>>>>>>>>> provider level and client does not have any
>>>>>>>>> information. Do we
>>>> expect
>>>>>>>>> client to query all different types?
>>>>>>>>>
>>>>>>>>> I thing we can have is a method to get all error and
>>>>>>>>> return the
>>>> Error
>>>>>>>> type
>>>>>>>>> part of the object to help the client to classify the
>>>>>>>>> error
>>>> location.
>>>>>>>>> Other suggestions?
>>>>>>>>>
>>>>>>>>> Raminder
>>>>>>>>>
>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Raman,
>>>>>>>>>>
>>>>>>>>>> WorkflowExecutionError method not returning any data
>>>>>>>>>> issue should
>>>> be
>>>>>>>>> fixed
>>>>>>>>>> now. Can you get an update and check.
>>>>>>>>>>
>>>>>>>>>> Regards, Chathuri
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
>>>>>>>>>> <ra...@gmail.com>wrote:
>>>>>>>>>>
>>>>>>>>>>> Thank, these are very useful for API users. Only
>>>>>>>>>>> thing i found
>>>>>>>> confusing
>>>>>>>>>>> for the user is API managers. ProvenanceManager is
>>>>>>>>>>> the one used
>>>> to
>>>>>>>> get
>>>>>>>>>>> experiment status. Users will get FINISHED or
>>>>>>>>>>> FAILED status from
>>>>> the
>>>>>>>>> data
>>>>>>>>>>> object. On failure, i was trying to find a method
>>>>>>>>>>> to get the
>>>> error
>>>>>>>> data
>>>>>>>>> but
>>>>>>>>>>> i was not able to find as those methods exist in
>>>> ExecutionManager.
>>>>>>>>>>> According to me, experiment status and in case of
>>>>>>>>>>> failures get
>>>>> error
>>>>>>>>> need
>>>>>>>>>>> to be part of one manager. May be a
>>>>>>>>>>> ExperimentManager? Getting
>>>>>> output
>>>>>>>>> data
>>>>>>>>>>> can be left as part of provenance manages as only
>>>>>>>>>>> few client may
>>>>> want
>>>>>>>> to
>>>>>>>>>>> get the data in few cases. Getting error detail in
>>>>>>>>>>> case of error
>>>> is
>>>>>>>>> obvious
>>>>>>>>>>> step.
>>>>>>>>>>>
>>>>>>>>>>> Other thing can you please explain the difference
>>>>>>>>>>> between WorkflowExecutionError,
>>>>>>>>>>> ExperimentExecutionError,
>>>>> NodeExecutionError,
>>>>>>>>>>> GFacExecutionError on a Wiki page for the users.
>>>>>>>>>>> Also add a
>>>> method
>>>>> to
>>>>>>>>> get
>>>>>>>>>>> JobID as that is required to get
>>>>>>>>>>> GFacExecutionError.
>>>>>>>>>>>
>>>>>>>>>>> I tested the WorkflowExecutionError method and its
>>>>>>>>>>> not returning
>>>>> any
>>>>>>>>> data.
>>>>>>>>>>> Can you please test that as well?
>>>>>>>>>>>
>>>>>>>>>>> ExperimentData data =
>>>>>>>>>>>
>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>>>>>>>>>>>
>>>>
>> List<WorkflowExecutionError> experimentExecutionErrors =
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>>>>>>>>>>>
>>>>
>> experimentID);
>>>>>>>>>>>
>>>>>>>>>>> Thanks Raminder
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Following updates will be present Airavata API
>>>>>>>>>>>> from 0.8 release.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 1. *Error message from the immediate exception
>>>>>>>>>>>> used as the error
>>>>>>>>>>> message
>>>>>>>>>>>> for the AiravataAPInvocationException*. Earlier
>>>>>>>>>>>> it was just
>>>> "Error
>>>>>>>>>>>> invoking API". *This change was made so that the
>>>>>>>>>>>> users of the
>>>> API
>>>>>>>> can
>>>>>>>>>>>> show the error from the exception without having
>>>>>>>>>>>> to dig in to
>>>> the
>>>>>>>>> inner
>>>>>>>>>>>> exception to get a better error message.* 2.
>>>>>>>>>>>> *Saving and retrieving experiment execution error
>>>>>>>>>>>> details*.
>>>>>>>> Earlier
>>>>>>>>>>>> we were not persisting any execution error
>>>>>>>>>>>> details of the
>>>>> workflow.
>>>>>>>>>>> Error
>>>>>>>>>>>> details are only available only if the developers
>>>>>>>>>>>> are monitoring
>>>>>>>>>>> experiment
>>>>>>>>>>>> at the time of error occurs. *This is not
>>>>>>>>>>>> feasible if users
>>>> would
>>>>>>>> only
>>>>>>>>>>>> want to go through the error details later
>>>>>>>>>>>> without doing
>>>>> monitoring
>>>>>>>>>>> from
>>>>>>>>>>>> the beginning. Therefore we are now persisting
>>>>>>>>>>>> the error details
>>>>>>>> (not
>>>>>>>>>>> as
>>>>>>>>>>>> part of provenance data) of executing workflows
>>>>>>>>>>>> (i.e.
>>>>> experiments)*
>>>>>>>> in
>>>>>>>>>>>> to the registry and also retrieving from registry
>>>>>>>>>>>> through
>>>> Airavata
>>>>>>>> API
>>>>>>>>>>>> (which calls the Registry API).
>>>>>>>>>>>>
>>>>>>>>>>>> For now we have the functions for saving and
>>>>>>>>>>>> retrieving errors
>>>> in
>>>>>> the
>>>>>>>>>>>> ExecutionManager[1]. We need to decide if this is
>>>>>>>>>>>> the correct
>>>>> place
>>>>>>>> to
>>>>>>>>>>> put
>>>>>>>>>>>> these functions. Your thoughts in this matter are
>>>>>>>>>>>> most
>>>> welcome...
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks, Saminda
>>>>>>>>>>>>
>>>>>>>>>>>> 1.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQEcBAEBAgAGBQJRuyxqAAoJEOEgD2XReDo54nwIAKjDjVqPk6YldnJD+T91k2HU
>> Yj/VcvH/XAxvoZOu9zGpcUFNuLPbSiJEJes9Tx3Wk63k8v+l4rxTTRB5rWihbyYn
>> xPEKtSnD8sCAiuFXlRHWLygzE80OVhC9Q43xkrnjjubKaWKp3ZgU1uF1ZL6LiTZb
>> R83+Yl2WCdAAckX1W+567APk202nw93GLKGjmRcuj2mBPKgWXzzD7AX3cQX6pt9e
>> g93PKGWJWdk/kb6YlhjkEfKH2RxJS/AgZvUYSAmf1ELL+siGqxEEOg7bwWwpfLNd
>> NsjUr/rN88EcGfUKxYb4fSh1zGceqp02rWG2TwufZEIVaF8WjztY6CsE2TgfS0M=
>> =vpMl
>> -----END PGP SIGNATURE-----
>>
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
Re: Updates to Airavata API version 0.8
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Here is the issue[1].
[1]https://issues.apache.org/jira/browse/AIRAVATA-869
Lahiru
On Fri, Jun 14, 2013 at 10:59 AM, Lahiru Gunathilake <gl...@gmail.com>wrote:
> I got to know about this yesterday, sorry I couldn't create a jira yet.
> Will do now !
>
> Lahiru
>
>
> On Fri, Jun 14, 2013 at 10:56 AM, Marlon Pierce <ma...@iu.edu> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> OK--which Jira task? Did it get marked as a blocker for 0.8?
>>
>>
>> Marlon
>>
>>
>> On 6/14/13 10:54 AM, Lahiru Gunathilake wrote:
>> > The one I am working is blocker !
>> >
>> > Lahiru
>> >
>> >
>> > On Fri, Jun 14, 2013 at 10:52 AM, Marlon Pierce <ma...@iu.edu>
>> > wrote:
>> >
>> > Are the issues blockers for 0.8 release?
>> >
>> >
>> > Marlon
>> >
>> >
>> > On 6/14/13 10:49 AM, Lahiru Gunathilake wrote:
>> >>>> Hi Marlon,
>> >>>>
>> >>>> I have no update about the release, since I've been working
>> >>>> with Gary in Ultrascan project. I found some issues to fix
>> >>>> for ultrascan, now working on them, so we have to wait until
>> >>>> Gary gives the green light to do the release.
>> >>>>
>> >>>> WDYT ?
>> >>>>
>> >>>> Regards Lahiru
>> >>>>
>> >>>>
>> >>>> On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce
>> >>>> <ma...@iu.edu> wrote:
>> >>>>
>> >>>> What's the status of the 0.8 release?
>> >>>>
>> >>>>
>> >>>> Marlon
>> >>>>
>> >>>>
>> >>>> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
>> >>>>>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
>> >>>>>>> <th...@gmail.com>wrote:
>> >>>>>>>
>> >>>>>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
>> >>>>>>>> <samindaw@gmail.com
>> >>>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Latest update on AiravataAPI updates for 0.8
>> >>>>>>>>> version
>> >>>>>>>>>
>> >>>>>>>>> 1. *Error message from the immediate exception used
>> >>>>>>>>> as the error
>> >>>>>>>> message
>> >>>>>>>>> for the AiravataAPInvocationException*. Earlier it
>> >>>>>>>>> was just "Error invoking API". *This change was
>> >>>>>>>>> made so that the users of the API can show the
>> >>>>>>>>> error from the exception without having to dig in
>> >>>>>>>>> to the
>> >>>>>>>> inner
>> >>>>>>>>> exception to get a better error message.* 2.
>> >>>>>>>>> *Saving and retrieving experiment execution error
>> >>>>>>>>> details*. Earlier we were not persisting any
>> >>>>>>>>> execution error details of the workflow. Error
>> >>>>>>>>> details are only available only if the developers
>> >>>>>>>>> are monitoring experiment at the time of error
>> >>>>>>>>> occurs. *This is not feasible if users would only
>> >>>>>>>>> want to go through the error details later without
>> >>>>>>>>> doing monitoring
>> >>>>>>>> from
>> >>>>>>>>> the beginning. Therefore we are now persisting the
>> >>>>>>>>> error details (not
>> >>>>>>>> as
>> >>>>>>>>> part of provenance data) of executing workflows
>> >>>>>>>>> (i.e. experiments) in
>> >>>>>>>> to
>> >>>>>>>>> the registry and also retrieving from registry
>> >>>>>>>>> through Airavata API (which calls the Registry
>> >>>>>>>>> API). These functions are available in the
>> >>>>>>>>> "ExecutionManager" of the AiravataAPI and
>> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI. * 3.
>> >>>>>>>>> Persisting & retrieving GFac job data. There are
>> >>>>>>>>> many benefits of saving the GRAM data, EC2 data
>> >>>>>>>>> etc. such as for debugging, analysis
>> >>>>>>>> etc.
>> >>>>>>>>> Currently these data are available only through
>> >>>>>>>>> the message notifications. *It is not feasible for
>> >>>>>>>>> the same reasons mentioned for API change for
>> >>>>>>>>> saving error data. Hence the introduction of
>> >>>>>>>>> functions in the API to save & retrieve GFac job
>> >>>>>>>>> data.** These functions are available as
>> >>>>>>>>> get/setApplicationJob***(...) functions in the
>> >>>>>>>>> "ProvenanceManager" of the AiravataAPI and
>> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI.*
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> May I ask why we need setApplicationJob****() methods
>> >>>>>>>> in the API ? As far as I understood job information
>> >>>>>>>> is populated by GFac and other internal components.
>> >>>>>>>> So why do we need API methods to set those data ? (OR
>> >>>>>>>> am I missing something ?)
>> >>>>>>>>
>> >>>>>>> GFac and Internal components also use the AiravataAPI
>> >>>>>>> to populate the job data in the registry.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> Thanks Amila
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Regards,
>> >>>>>>>>>
>> >>>>>>>>> Saminda
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
>> >>>>>>>> raminderjsingh@gmail.com
>> >>>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Thanks Chathuri, I will test and let you know.
>> >>>>>>>>>>
>> >>>>>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri
>> >>>>>>>>>> Wimalasena wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi Raman,
>> >>>>>>>>>>>
>> >>>>>>>>>>> The method Saminda mentioned will work for your
>> >>>>>>>>>>> case. I tested with
>> >>>>>>>>> your
>> >>>>>>>>>>> test class. I fixed deserialization issues from
>> >>>>>>>>>>> REST service side.
>> >>>>>>>> You
>> >>>>>>>>>> will
>> >>>>>>>>>>> probably have to update both server and
>> >>>>>>>>>>> client.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Regards, Chathuri
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda
>> >>>>>>>>>>> Wijeratne <
>> >>>>>>>> samindaw@gmail.com
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> So far what you are saying is true. If the
>> >>>>>>>>>>>> client knows which node
>> >>>>>>>>>> failed
>> >>>>>>>>>>>> he/she can get the list of errors for that
>> >>>>>>>>>>>> node using the getExecutionError(...)
>> >>>>>>>>>>>> function. Use the "type" as ALL and leave the
>> >>>>>>>>>>>> gfacJobId as null. I couldn;t think of doing
>> >>>>>>>>>>>> this any other better
>> >>>>>>>> way
>> >>>>>>>>>> than
>> >>>>>>>>>>>> introducing a whole lot of functions which
>> >>>>>>>>>>>> could be more
>> >>>>>>>>>> annoying/confusing
>> >>>>>>>>>>>> to figure-out. I'm welcome for suggestions
>> >>>>>>>>>>>> for improvements.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Saminda
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder
>> >>>>>>>>>>>> Singh <
>> >>>>>>>>>> raminderjsingh@gmail.com
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks Chathuri, the
>> >>>>>>>>>>>>> WorkflowExecutionError works. Integrating
>> >>>>>>>>>>>>> to
>> >>>>>>>>>> Error
>> >>>>>>>>>>>>> API brought few questions. We have
>> >>>>>>>>>>>>> ErrorTypes as :
>> >>>>>>>>>>>> WorkflowExecutionError,
>> >>>>>>>>>>>>> ExperimentExecutionError,
>> >>>>>>>>>>>>> NodeExecutionError,
>> >>>>>>>> GFacJobExecutionError,
>> >>>>>>>>>>>>> ExecutionError . How do we want clients to
>> >>>>>>>>>>>>> know which error type to
>> >>>>>>>>>>>> query?
>> >>>>>>>>>>>>> I ran a job and it failed, now i want to
>> >>>>>>>>>>>>> get error. Error occurred
>> >>>>>>>> at
>> >>>>>>>>>>>> GFac
>> >>>>>>>>>>>>> provider level and client does not have
>> >>>>>>>>>>>>> any information. Do we
>> >>>>>>>> expect
>> >>>>>>>>>>>>> client to query all different types?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> I thing we can have is a method to get all
>> >>>>>>>>>>>>> error and return the
>> >>>>>>>> Error
>> >>>>>>>>>>>> type
>> >>>>>>>>>>>>> part of the object to help the client to
>> >>>>>>>>>>>>> classify the error
>> >>>>>>>> location.
>> >>>>>>>>>>>>> Other suggestions?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Raminder
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri
>> >>>>>>>>>>>>> Wimalasena wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Hi Raman,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> WorkflowExecutionError method not
>> >>>>>>>>>>>>>> returning any data issue should
>> >>>>>>>> be
>> >>>>>>>>>>>>> fixed
>> >>>>>>>>>>>>>> now. Can you get an update and check.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Regards, Chathuri
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM,
>> >>>>>>>>>>>>>> Raminder Singh
>> >>>>>>>>>>>>>> <ra...@gmail.com>wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thank, these are very useful for API
>> >>>>>>>>>>>>>>> users. Only thing i found
>> >>>>>>>>>>>> confusing
>> >>>>>>>>>>>>>>> for the user is API managers.
>> >>>>>>>>>>>>>>> ProvenanceManager is the one used
>> >>>>>>>> to
>> >>>>>>>>>>>> get
>> >>>>>>>>>>>>>>> experiment status. Users will get
>> >>>>>>>>>>>>>>> FINISHED or FAILED status from
>> >>>>>>>>> the
>> >>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>> object. On failure, i was trying to
>> >>>>>>>>>>>>>>> find a method to get the
>> >>>>>>>> error
>> >>>>>>>>>>>> data
>> >>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>> i was not able to find as those methods
>> >>>>>>>>>>>>>>> exist in
>> >>>>>>>> ExecutionManager.
>> >>>>>>>>>>>>>>> According to me, experiment status and
>> >>>>>>>>>>>>>>> in case of failures get
>> >>>>>>>>> error
>> >>>>>>>>>>>>> need
>> >>>>>>>>>>>>>>> to be part of one manager. May be a
>> >>>>>>>>>>>>>>> ExperimentManager? Getting
>> >>>>>>>>>> output
>> >>>>>>>>>>>>> data
>> >>>>>>>>>>>>>>> can be left as part of provenance
>> >>>>>>>>>>>>>>> manages as only few client may
>> >>>>>>>>> want
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>>>>> get the data in few cases. Getting
>> >>>>>>>>>>>>>>> error detail in case of error
>> >>>>>>>> is
>> >>>>>>>>>>>>> obvious
>> >>>>>>>>>>>>>>> step.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Other thing can you please explain the
>> >>>>>>>>>>>>>>> difference between
>> >>>>>>>>>>>>>>> WorkflowExecutionError,
>> >>>>>>>>>>>>>>> ExperimentExecutionError,
>> >>>>>>>>> NodeExecutionError,
>> >>>>>>>>>>>>>>> GFacExecutionError on a Wiki page for
>> >>>>>>>>>>>>>>> the users. Also add a
>> >>>>>>>> method
>> >>>>>>>>> to
>> >>>>>>>>>>>>> get
>> >>>>>>>>>>>>>>> JobID as that is required to get
>> >>>>>>>>>>>>>>> GFacExecutionError.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I tested the WorkflowExecutionError
>> >>>>>>>>>>>>>>> method and its not returning
>> >>>>>>>>> any
>> >>>>>>>>>>>>> data.
>> >>>>>>>>>>>>>>> Can you please test that as well?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> ExperimentData data =
>> >>>>>>>>>>>>>>>
>> >>>>>>>>
>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>> >>>>>>>>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >>>>>>>>
>> >
>> >>>>>>>>
>> List<WorkflowExecutionError> experimentExecutionErrors =
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >
>> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >>>>
>> >
>> >
>> experimentID);
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thanks Raminder
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda
>> >>>>>>>>>>>>>>> Wijeratne wrote:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Following updates will be present
>> >>>>>>>>>>>>>>>> Airavata API from 0.8 release.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 1. *Error message from the immediate
>> >>>>>>>>>>>>>>>> exception used as the error
>> >>>>>>>>>>>>>>> message
>> >>>>>>>>>>>>>>>> for the
>> >>>>>>>>>>>>>>>> AiravataAPInvocationException*.
>> >>>>>>>>>>>>>>>> Earlier it was just
>> >>>>>>>> "Error
>> >>>>>>>>>>>>>>>> invoking API". *This change was made
>> >>>>>>>>>>>>>>>> so that the users of the
>> >>>>>>>> API
>> >>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>> show the error from the exception
>> >>>>>>>>>>>>>>>> without having to dig in to
>> >>>>>>>> the
>> >>>>>>>>>>>>> inner
>> >>>>>>>>>>>>>>>> exception to get a better error
>> >>>>>>>>>>>>>>>> message.* 2. *Saving and retrieving
>> >>>>>>>>>>>>>>>> experiment execution error details*.
>> >>>>>>>>>>>> Earlier
>> >>>>>>>>>>>>>>>> we were not persisting any execution
>> >>>>>>>>>>>>>>>> error details of the
>> >>>>>>>>> workflow.
>> >>>>>>>>>>>>>>> Error
>> >>>>>>>>>>>>>>>> details are only available only if
>> >>>>>>>>>>>>>>>> the developers are monitoring
>> >>>>>>>>>>>>>>> experiment
>> >>>>>>>>>>>>>>>> at the time of error occurs. *This is
>> >>>>>>>>>>>>>>>> not feasible if users
>> >>>>>>>> would
>> >>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>> want to go through the error details
>> >>>>>>>>>>>>>>>> later without doing
>> >>>>>>>>> monitoring
>> >>>>>>>>>>>>>>> from
>> >>>>>>>>>>>>>>>> the beginning. Therefore we are now
>> >>>>>>>>>>>>>>>> persisting the error details
>> >>>>>>>>>>>> (not
>> >>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>> part of provenance data) of
>> >>>>>>>>>>>>>>>> executing workflows (i.e.
>> >>>>>>>>> experiments)*
>> >>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>> to the registry and also retrieving
>> >>>>>>>>>>>>>>>> from registry through
>> >>>>>>>> Airavata
>> >>>>>>>>>>>> API
>> >>>>>>>>>>>>>>>> (which calls the Registry API).
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> For now we have the functions for
>> >>>>>>>>>>>>>>>> saving and retrieving errors
>> >>>>>>>> in
>> >>>>>>>>>> the
>> >>>>>>>>>>>>>>>> ExecutionManager[1]. We need to
>> >>>>>>>>>>>>>>>> decide if this is the correct
>> >>>>>>>>> place
>> >>>>>>>>>>>> to
>> >>>>>>>>>>>>>>> put
>> >>>>>>>>>>>>>>>> these functions. Your thoughts in
>> >>>>>>>>>>>>>>>> this matter are most
>> >>>>>>>> welcome...
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks, Saminda
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> 1.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>
>> >
>> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>
>> >
>> >
>> >
>> >
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQEcBAEBAgAGBQJRuy8SAAoJEOEgD2XReDo53w0H/2wxRPUk2TjXqwRnCdVuSYQL
>> uAA6I/TCp8j1+uugNNcCKApi3npo8FTtZJDIWGDACzpzZg/zsTOp583DldzpXZXL
>> e6N2Bzb+dz5woWYG/i9/EBq585ErH/e9ieDgRP0MfrryjIoNj+AKcCtutIWmfjHt
>> 1E3MhpXO0yfYxiUdiYtH1nK/O5qJHhVc7foZj+Q5XiCza9Z8p3BPcKY9l5AUWMUE
>> N/4HZ5mat78bJriz9qYejscVDDadfagkYAYoTQ3srKlJdXBloeL4Yhw7kE4P5gU6
>> MVFqlRIC18Fj5Ugv0bj1rMUMw7vPWiQe7fTGG2hYyCGTfsu81r9V1/3FQtyRkY0=
>> =roFX
>> -----END PGP SIGNATURE-----
>>
>
>
>
> --
> System Analyst Programmer
> PTI Lab
> Indiana University
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Updates to Airavata API version 0.8
Posted by Lahiru Gunathilake <gl...@gmail.com>.
I got to know about this yesterday, sorry I couldn't create a jira yet.
Will do now !
Lahiru
On Fri, Jun 14, 2013 at 10:56 AM, Marlon Pierce <ma...@iu.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> OK--which Jira task? Did it get marked as a blocker for 0.8?
>
>
> Marlon
>
>
> On 6/14/13 10:54 AM, Lahiru Gunathilake wrote:
> > The one I am working is blocker !
> >
> > Lahiru
> >
> >
> > On Fri, Jun 14, 2013 at 10:52 AM, Marlon Pierce <ma...@iu.edu>
> > wrote:
> >
> > Are the issues blockers for 0.8 release?
> >
> >
> > Marlon
> >
> >
> > On 6/14/13 10:49 AM, Lahiru Gunathilake wrote:
> >>>> Hi Marlon,
> >>>>
> >>>> I have no update about the release, since I've been working
> >>>> with Gary in Ultrascan project. I found some issues to fix
> >>>> for ultrascan, now working on them, so we have to wait until
> >>>> Gary gives the green light to do the release.
> >>>>
> >>>> WDYT ?
> >>>>
> >>>> Regards Lahiru
> >>>>
> >>>>
> >>>> On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce
> >>>> <ma...@iu.edu> wrote:
> >>>>
> >>>> What's the status of the 0.8 release?
> >>>>
> >>>>
> >>>> Marlon
> >>>>
> >>>>
> >>>> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> >>>>>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
> >>>>>>> <th...@gmail.com>wrote:
> >>>>>>>
> >>>>>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
> >>>>>>>> <samindaw@gmail.com
> >>>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Latest update on AiravataAPI updates for 0.8
> >>>>>>>>> version
> >>>>>>>>>
> >>>>>>>>> 1. *Error message from the immediate exception used
> >>>>>>>>> as the error
> >>>>>>>> message
> >>>>>>>>> for the AiravataAPInvocationException*. Earlier it
> >>>>>>>>> was just "Error invoking API". *This change was
> >>>>>>>>> made so that the users of the API can show the
> >>>>>>>>> error from the exception without having to dig in
> >>>>>>>>> to the
> >>>>>>>> inner
> >>>>>>>>> exception to get a better error message.* 2.
> >>>>>>>>> *Saving and retrieving experiment execution error
> >>>>>>>>> details*. Earlier we were not persisting any
> >>>>>>>>> execution error details of the workflow. Error
> >>>>>>>>> details are only available only if the developers
> >>>>>>>>> are monitoring experiment at the time of error
> >>>>>>>>> occurs. *This is not feasible if users would only
> >>>>>>>>> want to go through the error details later without
> >>>>>>>>> doing monitoring
> >>>>>>>> from
> >>>>>>>>> the beginning. Therefore we are now persisting the
> >>>>>>>>> error details (not
> >>>>>>>> as
> >>>>>>>>> part of provenance data) of executing workflows
> >>>>>>>>> (i.e. experiments) in
> >>>>>>>> to
> >>>>>>>>> the registry and also retrieving from registry
> >>>>>>>>> through Airavata API (which calls the Registry
> >>>>>>>>> API). These functions are available in the
> >>>>>>>>> "ExecutionManager" of the AiravataAPI and
> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI. * 3.
> >>>>>>>>> Persisting & retrieving GFac job data. There are
> >>>>>>>>> many benefits of saving the GRAM data, EC2 data
> >>>>>>>>> etc. such as for debugging, analysis
> >>>>>>>> etc.
> >>>>>>>>> Currently these data are available only through
> >>>>>>>>> the message notifications. *It is not feasible for
> >>>>>>>>> the same reasons mentioned for API change for
> >>>>>>>>> saving error data. Hence the introduction of
> >>>>>>>>> functions in the API to save & retrieve GFac job
> >>>>>>>>> data.** These functions are available as
> >>>>>>>>> get/setApplicationJob***(...) functions in the
> >>>>>>>>> "ProvenanceManager" of the AiravataAPI and
> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI.*
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> May I ask why we need setApplicationJob****() methods
> >>>>>>>> in the API ? As far as I understood job information
> >>>>>>>> is populated by GFac and other internal components.
> >>>>>>>> So why do we need API methods to set those data ? (OR
> >>>>>>>> am I missing something ?)
> >>>>>>>>
> >>>>>>> GFac and Internal components also use the AiravataAPI
> >>>>>>> to populate the job data in the registry.
> >>>>>>>
> >>>>>>>
> >>>>>>>> Thanks Amila
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>>
> >>>>>>>>> Saminda
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> >>>>>>>> raminderjsingh@gmail.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks Chathuri, I will test and let you know.
> >>>>>>>>>>
> >>>>>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri
> >>>>>>>>>> Wimalasena wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Raman,
> >>>>>>>>>>>
> >>>>>>>>>>> The method Saminda mentioned will work for your
> >>>>>>>>>>> case. I tested with
> >>>>>>>>> your
> >>>>>>>>>>> test class. I fixed deserialization issues from
> >>>>>>>>>>> REST service side.
> >>>>>>>> You
> >>>>>>>>>> will
> >>>>>>>>>>> probably have to update both server and
> >>>>>>>>>>> client.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards, Chathuri
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda
> >>>>>>>>>>> Wijeratne <
> >>>>>>>> samindaw@gmail.com
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> So far what you are saying is true. If the
> >>>>>>>>>>>> client knows which node
> >>>>>>>>>> failed
> >>>>>>>>>>>> he/she can get the list of errors for that
> >>>>>>>>>>>> node using the getExecutionError(...)
> >>>>>>>>>>>> function. Use the "type" as ALL and leave the
> >>>>>>>>>>>> gfacJobId as null. I couldn;t think of doing
> >>>>>>>>>>>> this any other better
> >>>>>>>> way
> >>>>>>>>>> than
> >>>>>>>>>>>> introducing a whole lot of functions which
> >>>>>>>>>>>> could be more
> >>>>>>>>>> annoying/confusing
> >>>>>>>>>>>> to figure-out. I'm welcome for suggestions
> >>>>>>>>>>>> for improvements.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Saminda
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder
> >>>>>>>>>>>> Singh <
> >>>>>>>>>> raminderjsingh@gmail.com
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks Chathuri, the
> >>>>>>>>>>>>> WorkflowExecutionError works. Integrating
> >>>>>>>>>>>>> to
> >>>>>>>>>> Error
> >>>>>>>>>>>>> API brought few questions. We have
> >>>>>>>>>>>>> ErrorTypes as :
> >>>>>>>>>>>> WorkflowExecutionError,
> >>>>>>>>>>>>> ExperimentExecutionError,
> >>>>>>>>>>>>> NodeExecutionError,
> >>>>>>>> GFacJobExecutionError,
> >>>>>>>>>>>>> ExecutionError . How do we want clients to
> >>>>>>>>>>>>> know which error type to
> >>>>>>>>>>>> query?
> >>>>>>>>>>>>> I ran a job and it failed, now i want to
> >>>>>>>>>>>>> get error. Error occurred
> >>>>>>>> at
> >>>>>>>>>>>> GFac
> >>>>>>>>>>>>> provider level and client does not have
> >>>>>>>>>>>>> any information. Do we
> >>>>>>>> expect
> >>>>>>>>>>>>> client to query all different types?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I thing we can have is a method to get all
> >>>>>>>>>>>>> error and return the
> >>>>>>>> Error
> >>>>>>>>>>>> type
> >>>>>>>>>>>>> part of the object to help the client to
> >>>>>>>>>>>>> classify the error
> >>>>>>>> location.
> >>>>>>>>>>>>> Other suggestions?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Raminder
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri
> >>>>>>>>>>>>> Wimalasena wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Raman,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> WorkflowExecutionError method not
> >>>>>>>>>>>>>> returning any data issue should
> >>>>>>>> be
> >>>>>>>>>>>>> fixed
> >>>>>>>>>>>>>> now. Can you get an update and check.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regards, Chathuri
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM,
> >>>>>>>>>>>>>> Raminder Singh
> >>>>>>>>>>>>>> <ra...@gmail.com>wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thank, these are very useful for API
> >>>>>>>>>>>>>>> users. Only thing i found
> >>>>>>>>>>>> confusing
> >>>>>>>>>>>>>>> for the user is API managers.
> >>>>>>>>>>>>>>> ProvenanceManager is the one used
> >>>>>>>> to
> >>>>>>>>>>>> get
> >>>>>>>>>>>>>>> experiment status. Users will get
> >>>>>>>>>>>>>>> FINISHED or FAILED status from
> >>>>>>>>> the
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>> object. On failure, i was trying to
> >>>>>>>>>>>>>>> find a method to get the
> >>>>>>>> error
> >>>>>>>>>>>> data
> >>>>>>>>>>>>> but
> >>>>>>>>>>>>>>> i was not able to find as those methods
> >>>>>>>>>>>>>>> exist in
> >>>>>>>> ExecutionManager.
> >>>>>>>>>>>>>>> According to me, experiment status and
> >>>>>>>>>>>>>>> in case of failures get
> >>>>>>>>> error
> >>>>>>>>>>>>> need
> >>>>>>>>>>>>>>> to be part of one manager. May be a
> >>>>>>>>>>>>>>> ExperimentManager? Getting
> >>>>>>>>>> output
> >>>>>>>>>>>>> data
> >>>>>>>>>>>>>>> can be left as part of provenance
> >>>>>>>>>>>>>>> manages as only few client may
> >>>>>>>>> want
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> get the data in few cases. Getting
> >>>>>>>>>>>>>>> error detail in case of error
> >>>>>>>> is
> >>>>>>>>>>>>> obvious
> >>>>>>>>>>>>>>> step.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Other thing can you please explain the
> >>>>>>>>>>>>>>> difference between
> >>>>>>>>>>>>>>> WorkflowExecutionError,
> >>>>>>>>>>>>>>> ExperimentExecutionError,
> >>>>>>>>> NodeExecutionError,
> >>>>>>>>>>>>>>> GFacExecutionError on a Wiki page for
> >>>>>>>>>>>>>>> the users. Also add a
> >>>>>>>> method
> >>>>>>>>> to
> >>>>>>>>>>>>> get
> >>>>>>>>>>>>>>> JobID as that is required to get
> >>>>>>>>>>>>>>> GFacExecutionError.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I tested the WorkflowExecutionError
> >>>>>>>>>>>>>>> method and its not returning
> >>>>>>>>> any
> >>>>>>>>>>>>> data.
> >>>>>>>>>>>>>>> Can you please test that as well?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> ExperimentData data =
> >>>>>>>>>>>>>>>
> >>>>>>>>
> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >>>>>>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>>>>>>>
> >
> >>>>>>>>
> List<WorkflowExecutionError> experimentExecutionErrors =
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >>>>>>>>>>>>>>>
> >>>>>>>>
> >>>>
> >>>>
> >
> >
> experimentID);
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thanks Raminder
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda
> >>>>>>>>>>>>>>> Wijeratne wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Following updates will be present
> >>>>>>>>>>>>>>>> Airavata API from 0.8 release.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 1. *Error message from the immediate
> >>>>>>>>>>>>>>>> exception used as the error
> >>>>>>>>>>>>>>> message
> >>>>>>>>>>>>>>>> for the
> >>>>>>>>>>>>>>>> AiravataAPInvocationException*.
> >>>>>>>>>>>>>>>> Earlier it was just
> >>>>>>>> "Error
> >>>>>>>>>>>>>>>> invoking API". *This change was made
> >>>>>>>>>>>>>>>> so that the users of the
> >>>>>>>> API
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>>>> show the error from the exception
> >>>>>>>>>>>>>>>> without having to dig in to
> >>>>>>>> the
> >>>>>>>>>>>>> inner
> >>>>>>>>>>>>>>>> exception to get a better error
> >>>>>>>>>>>>>>>> message.* 2. *Saving and retrieving
> >>>>>>>>>>>>>>>> experiment execution error details*.
> >>>>>>>>>>>> Earlier
> >>>>>>>>>>>>>>>> we were not persisting any execution
> >>>>>>>>>>>>>>>> error details of the
> >>>>>>>>> workflow.
> >>>>>>>>>>>>>>> Error
> >>>>>>>>>>>>>>>> details are only available only if
> >>>>>>>>>>>>>>>> the developers are monitoring
> >>>>>>>>>>>>>>> experiment
> >>>>>>>>>>>>>>>> at the time of error occurs. *This is
> >>>>>>>>>>>>>>>> not feasible if users
> >>>>>>>> would
> >>>>>>>>>>>> only
> >>>>>>>>>>>>>>>> want to go through the error details
> >>>>>>>>>>>>>>>> later without doing
> >>>>>>>>> monitoring
> >>>>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>> the beginning. Therefore we are now
> >>>>>>>>>>>>>>>> persisting the error details
> >>>>>>>>>>>> (not
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> part of provenance data) of
> >>>>>>>>>>>>>>>> executing workflows (i.e.
> >>>>>>>>> experiments)*
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> to the registry and also retrieving
> >>>>>>>>>>>>>>>> from registry through
> >>>>>>>> Airavata
> >>>>>>>>>>>> API
> >>>>>>>>>>>>>>>> (which calls the Registry API).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> For now we have the functions for
> >>>>>>>>>>>>>>>> saving and retrieving errors
> >>>>>>>> in
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> ExecutionManager[1]. We need to
> >>>>>>>>>>>>>>>> decide if this is the correct
> >>>>>>>>> place
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> put
> >>>>>>>>>>>>>>>> these functions. Your thoughts in
> >>>>>>>>>>>>>>>> this matter are most
> >>>>>>>> welcome...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks, Saminda
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 1.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>
> >
> >
> >
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJRuy8SAAoJEOEgD2XReDo53w0H/2wxRPUk2TjXqwRnCdVuSYQL
> uAA6I/TCp8j1+uugNNcCKApi3npo8FTtZJDIWGDACzpzZg/zsTOp583DldzpXZXL
> e6N2Bzb+dz5woWYG/i9/EBq585ErH/e9ieDgRP0MfrryjIoNj+AKcCtutIWmfjHt
> 1E3MhpXO0yfYxiUdiYtH1nK/O5qJHhVc7foZj+Q5XiCza9Z8p3BPcKY9l5AUWMUE
> N/4HZ5mat78bJriz9qYejscVDDadfagkYAYoTQ3srKlJdXBloeL4Yhw7kE4P5gU6
> MVFqlRIC18Fj5Ugv0bj1rMUMw7vPWiQe7fTGG2hYyCGTfsu81r9V1/3FQtyRkY0=
> =roFX
> -----END PGP SIGNATURE-----
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Updates to Airavata API version 0.8
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
OK--which Jira task? Did it get marked as a blocker for 0.8?
Marlon
On 6/14/13 10:54 AM, Lahiru Gunathilake wrote:
> The one I am working is blocker !
>
> Lahiru
>
>
> On Fri, Jun 14, 2013 at 10:52 AM, Marlon Pierce <ma...@iu.edu>
> wrote:
>
> Are the issues blockers for 0.8 release?
>
>
> Marlon
>
>
> On 6/14/13 10:49 AM, Lahiru Gunathilake wrote:
>>>> Hi Marlon,
>>>>
>>>> I have no update about the release, since I've been working
>>>> with Gary in Ultrascan project. I found some issues to fix
>>>> for ultrascan, now working on them, so we have to wait until
>>>> Gary gives the green light to do the release.
>>>>
>>>> WDYT ?
>>>>
>>>> Regards Lahiru
>>>>
>>>>
>>>> On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce
>>>> <ma...@iu.edu> wrote:
>>>>
>>>> What's the status of the 0.8 release?
>>>>
>>>>
>>>> Marlon
>>>>
>>>>
>>>> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
>>>>>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
>>>>>>> <th...@gmail.com>wrote:
>>>>>>>
>>>>>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
>>>>>>>> <samindaw@gmail.com
>>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Latest update on AiravataAPI updates for 0.8
>>>>>>>>> version
>>>>>>>>>
>>>>>>>>> 1. *Error message from the immediate exception used
>>>>>>>>> as the error
>>>>>>>> message
>>>>>>>>> for the AiravataAPInvocationException*. Earlier it
>>>>>>>>> was just "Error invoking API". *This change was
>>>>>>>>> made so that the users of the API can show the
>>>>>>>>> error from the exception without having to dig in
>>>>>>>>> to the
>>>>>>>> inner
>>>>>>>>> exception to get a better error message.* 2.
>>>>>>>>> *Saving and retrieving experiment execution error
>>>>>>>>> details*. Earlier we were not persisting any
>>>>>>>>> execution error details of the workflow. Error
>>>>>>>>> details are only available only if the developers
>>>>>>>>> are monitoring experiment at the time of error
>>>>>>>>> occurs. *This is not feasible if users would only
>>>>>>>>> want to go through the error details later without
>>>>>>>>> doing monitoring
>>>>>>>> from
>>>>>>>>> the beginning. Therefore we are now persisting the
>>>>>>>>> error details (not
>>>>>>>> as
>>>>>>>>> part of provenance data) of executing workflows
>>>>>>>>> (i.e. experiments) in
>>>>>>>> to
>>>>>>>>> the registry and also retrieving from registry
>>>>>>>>> through Airavata API (which calls the Registry
>>>>>>>>> API). These functions are available in the
>>>>>>>>> "ExecutionManager" of the AiravataAPI and
>>>>>>>>> "ProvenanceRegistry" in the RegistryAPI. * 3.
>>>>>>>>> Persisting & retrieving GFac job data. There are
>>>>>>>>> many benefits of saving the GRAM data, EC2 data
>>>>>>>>> etc. such as for debugging, analysis
>>>>>>>> etc.
>>>>>>>>> Currently these data are available only through
>>>>>>>>> the message notifications. *It is not feasible for
>>>>>>>>> the same reasons mentioned for API change for
>>>>>>>>> saving error data. Hence the introduction of
>>>>>>>>> functions in the API to save & retrieve GFac job
>>>>>>>>> data.** These functions are available as
>>>>>>>>> get/setApplicationJob***(...) functions in the
>>>>>>>>> "ProvenanceManager" of the AiravataAPI and
>>>>>>>>> "ProvenanceRegistry" in the RegistryAPI.*
>>>>>>>>>
>>>>>>>>
>>>>>>>> May I ask why we need setApplicationJob****() methods
>>>>>>>> in the API ? As far as I understood job information
>>>>>>>> is populated by GFac and other internal components.
>>>>>>>> So why do we need API methods to set those data ? (OR
>>>>>>>> am I missing something ?)
>>>>>>>>
>>>>>>> GFac and Internal components also use the AiravataAPI
>>>>>>> to populate the job data in the registry.
>>>>>>>
>>>>>>>
>>>>>>>> Thanks Amila
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Saminda
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
>>>>>>>> raminderjsingh@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Chathuri, I will test and let you know.
>>>>>>>>>>
>>>>>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri
>>>>>>>>>> Wimalasena wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Raman,
>>>>>>>>>>>
>>>>>>>>>>> The method Saminda mentioned will work for your
>>>>>>>>>>> case. I tested with
>>>>>>>>> your
>>>>>>>>>>> test class. I fixed deserialization issues from
>>>>>>>>>>> REST service side.
>>>>>>>> You
>>>>>>>>>> will
>>>>>>>>>>> probably have to update both server and
>>>>>>>>>>> client.
>>>>>>>>>>>
>>>>>>>>>>> Regards, Chathuri
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda
>>>>>>>>>>> Wijeratne <
>>>>>>>> samindaw@gmail.com
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> So far what you are saying is true. If the
>>>>>>>>>>>> client knows which node
>>>>>>>>>> failed
>>>>>>>>>>>> he/she can get the list of errors for that
>>>>>>>>>>>> node using the getExecutionError(...)
>>>>>>>>>>>> function. Use the "type" as ALL and leave the
>>>>>>>>>>>> gfacJobId as null. I couldn;t think of doing
>>>>>>>>>>>> this any other better
>>>>>>>> way
>>>>>>>>>> than
>>>>>>>>>>>> introducing a whole lot of functions which
>>>>>>>>>>>> could be more
>>>>>>>>>> annoying/confusing
>>>>>>>>>>>> to figure-out. I'm welcome for suggestions
>>>>>>>>>>>> for improvements.
>>>>>>>>>>>>
>>>>>>>>>>>> Saminda
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder
>>>>>>>>>>>> Singh <
>>>>>>>>>> raminderjsingh@gmail.com
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks Chathuri, the
>>>>>>>>>>>>> WorkflowExecutionError works. Integrating
>>>>>>>>>>>>> to
>>>>>>>>>> Error
>>>>>>>>>>>>> API brought few questions. We have
>>>>>>>>>>>>> ErrorTypes as :
>>>>>>>>>>>> WorkflowExecutionError,
>>>>>>>>>>>>> ExperimentExecutionError,
>>>>>>>>>>>>> NodeExecutionError,
>>>>>>>> GFacJobExecutionError,
>>>>>>>>>>>>> ExecutionError . How do we want clients to
>>>>>>>>>>>>> know which error type to
>>>>>>>>>>>> query?
>>>>>>>>>>>>> I ran a job and it failed, now i want to
>>>>>>>>>>>>> get error. Error occurred
>>>>>>>> at
>>>>>>>>>>>> GFac
>>>>>>>>>>>>> provider level and client does not have
>>>>>>>>>>>>> any information. Do we
>>>>>>>> expect
>>>>>>>>>>>>> client to query all different types?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I thing we can have is a method to get all
>>>>>>>>>>>>> error and return the
>>>>>>>> Error
>>>>>>>>>>>> type
>>>>>>>>>>>>> part of the object to help the client to
>>>>>>>>>>>>> classify the error
>>>>>>>> location.
>>>>>>>>>>>>> Other suggestions?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Raminder
>>>>>>>>>>>>>
>>>>>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri
>>>>>>>>>>>>> Wimalasena wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Raman,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> WorkflowExecutionError method not
>>>>>>>>>>>>>> returning any data issue should
>>>>>>>> be
>>>>>>>>>>>>> fixed
>>>>>>>>>>>>>> now. Can you get an update and check.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards, Chathuri
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM,
>>>>>>>>>>>>>> Raminder Singh
>>>>>>>>>>>>>> <ra...@gmail.com>wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank, these are very useful for API
>>>>>>>>>>>>>>> users. Only thing i found
>>>>>>>>>>>> confusing
>>>>>>>>>>>>>>> for the user is API managers.
>>>>>>>>>>>>>>> ProvenanceManager is the one used
>>>>>>>> to
>>>>>>>>>>>> get
>>>>>>>>>>>>>>> experiment status. Users will get
>>>>>>>>>>>>>>> FINISHED or FAILED status from
>>>>>>>>> the
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>> object. On failure, i was trying to
>>>>>>>>>>>>>>> find a method to get the
>>>>>>>> error
>>>>>>>>>>>> data
>>>>>>>>>>>>> but
>>>>>>>>>>>>>>> i was not able to find as those methods
>>>>>>>>>>>>>>> exist in
>>>>>>>> ExecutionManager.
>>>>>>>>>>>>>>> According to me, experiment status and
>>>>>>>>>>>>>>> in case of failures get
>>>>>>>>> error
>>>>>>>>>>>>> need
>>>>>>>>>>>>>>> to be part of one manager. May be a
>>>>>>>>>>>>>>> ExperimentManager? Getting
>>>>>>>>>> output
>>>>>>>>>>>>> data
>>>>>>>>>>>>>>> can be left as part of provenance
>>>>>>>>>>>>>>> manages as only few client may
>>>>>>>>> want
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> get the data in few cases. Getting
>>>>>>>>>>>>>>> error detail in case of error
>>>>>>>> is
>>>>>>>>>>>>> obvious
>>>>>>>>>>>>>>> step.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Other thing can you please explain the
>>>>>>>>>>>>>>> difference between
>>>>>>>>>>>>>>> WorkflowExecutionError,
>>>>>>>>>>>>>>> ExperimentExecutionError,
>>>>>>>>> NodeExecutionError,
>>>>>>>>>>>>>>> GFacExecutionError on a Wiki page for
>>>>>>>>>>>>>>> the users. Also add a
>>>>>>>> method
>>>>>>>>> to
>>>>>>>>>>>>> get
>>>>>>>>>>>>>>> JobID as that is required to get
>>>>>>>>>>>>>>> GFacExecutionError.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I tested the WorkflowExecutionError
>>>>>>>>>>>>>>> method and its not returning
>>>>>>>>> any
>>>>>>>>>>>>> data.
>>>>>>>>>>>>>>> Can you please test that as well?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ExperimentData data =
>>>>>>>>>>>>>>>
>>>>>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>>>>>>>>>>>>>>>
>>>>>>>>
>>>>
>>>>>>>>
>
>>>>>>>>
List<WorkflowExecutionError> experimentExecutionErrors =
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>>>>>>>>>>>>>>>
>>>>>>>>
>>>>
>>>>
>
>
experimentID);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks Raminder
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda
>>>>>>>>>>>>>>> Wijeratne wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Following updates will be present
>>>>>>>>>>>>>>>> Airavata API from 0.8 release.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. *Error message from the immediate
>>>>>>>>>>>>>>>> exception used as the error
>>>>>>>>>>>>>>> message
>>>>>>>>>>>>>>>> for the
>>>>>>>>>>>>>>>> AiravataAPInvocationException*.
>>>>>>>>>>>>>>>> Earlier it was just
>>>>>>>> "Error
>>>>>>>>>>>>>>>> invoking API". *This change was made
>>>>>>>>>>>>>>>> so that the users of the
>>>>>>>> API
>>>>>>>>>>>> can
>>>>>>>>>>>>>>>> show the error from the exception
>>>>>>>>>>>>>>>> without having to dig in to
>>>>>>>> the
>>>>>>>>>>>>> inner
>>>>>>>>>>>>>>>> exception to get a better error
>>>>>>>>>>>>>>>> message.* 2. *Saving and retrieving
>>>>>>>>>>>>>>>> experiment execution error details*.
>>>>>>>>>>>> Earlier
>>>>>>>>>>>>>>>> we were not persisting any execution
>>>>>>>>>>>>>>>> error details of the
>>>>>>>>> workflow.
>>>>>>>>>>>>>>> Error
>>>>>>>>>>>>>>>> details are only available only if
>>>>>>>>>>>>>>>> the developers are monitoring
>>>>>>>>>>>>>>> experiment
>>>>>>>>>>>>>>>> at the time of error occurs. *This is
>>>>>>>>>>>>>>>> not feasible if users
>>>>>>>> would
>>>>>>>>>>>> only
>>>>>>>>>>>>>>>> want to go through the error details
>>>>>>>>>>>>>>>> later without doing
>>>>>>>>> monitoring
>>>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>> the beginning. Therefore we are now
>>>>>>>>>>>>>>>> persisting the error details
>>>>>>>>>>>> (not
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> part of provenance data) of
>>>>>>>>>>>>>>>> executing workflows (i.e.
>>>>>>>>> experiments)*
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> to the registry and also retrieving
>>>>>>>>>>>>>>>> from registry through
>>>>>>>> Airavata
>>>>>>>>>>>> API
>>>>>>>>>>>>>>>> (which calls the Registry API).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> For now we have the functions for
>>>>>>>>>>>>>>>> saving and retrieving errors
>>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>>>>>>> ExecutionManager[1]. We need to
>>>>>>>>>>>>>>>> decide if this is the correct
>>>>>>>>> place
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>> these functions. Your thoughts in
>>>>>>>>>>>>>>>> this matter are most
>>>>>>>> welcome...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks, Saminda
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJRuy8SAAoJEOEgD2XReDo53w0H/2wxRPUk2TjXqwRnCdVuSYQL
uAA6I/TCp8j1+uugNNcCKApi3npo8FTtZJDIWGDACzpzZg/zsTOp583DldzpXZXL
e6N2Bzb+dz5woWYG/i9/EBq585ErH/e9ieDgRP0MfrryjIoNj+AKcCtutIWmfjHt
1E3MhpXO0yfYxiUdiYtH1nK/O5qJHhVc7foZj+Q5XiCza9Z8p3BPcKY9l5AUWMUE
N/4HZ5mat78bJriz9qYejscVDDadfagkYAYoTQ3srKlJdXBloeL4Yhw7kE4P5gU6
MVFqlRIC18Fj5Ugv0bj1rMUMw7vPWiQe7fTGG2hYyCGTfsu81r9V1/3FQtyRkY0=
=roFX
-----END PGP SIGNATURE-----
Re: Updates to Airavata API version 0.8
Posted by Lahiru Gunathilake <gl...@gmail.com>.
The one I am working is blocker !
Lahiru
On Fri, Jun 14, 2013 at 10:52 AM, Marlon Pierce <ma...@iu.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Are the issues blockers for 0.8 release?
>
>
> Marlon
>
>
> On 6/14/13 10:49 AM, Lahiru Gunathilake wrote:
> > Hi Marlon,
> >
> > I have no update about the release, since I've been working with
> > Gary in Ultrascan project. I found some issues to fix for
> > ultrascan, now working on them, so we have to wait until Gary gives
> > the green light to do the release.
> >
> > WDYT ?
> >
> > Regards Lahiru
> >
> >
> > On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce <ma...@iu.edu>
> > wrote:
> >
> > What's the status of the 0.8 release?
> >
> >
> > Marlon
> >
> >
> > On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> >>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
> >>>> <th...@gmail.com>wrote:
> >>>>
> >>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
> >>>>> <samindaw@gmail.com
> >>>>>> wrote:
> >>>>>
> >>>>>> Latest update on AiravataAPI updates for 0.8 version
> >>>>>>
> >>>>>> 1. *Error message from the immediate exception used as
> >>>>>> the error
> >>>>> message
> >>>>>> for the AiravataAPInvocationException*. Earlier it was
> >>>>>> just "Error invoking API". *This change was made so that
> >>>>>> the users of the API can show the error from the
> >>>>>> exception without having to dig in to the
> >>>>> inner
> >>>>>> exception to get a better error message.* 2. *Saving and
> >>>>>> retrieving experiment execution error details*. Earlier
> >>>>>> we were not persisting any execution error details of the
> >>>>>> workflow. Error details are only available only if the
> >>>>>> developers are monitoring experiment at the time of error
> >>>>>> occurs. *This is not feasible if users would only want to
> >>>>>> go through the error details later without doing
> >>>>>> monitoring
> >>>>> from
> >>>>>> the beginning. Therefore we are now persisting the error
> >>>>>> details (not
> >>>>> as
> >>>>>> part of provenance data) of executing workflows (i.e.
> >>>>>> experiments) in
> >>>>> to
> >>>>>> the registry and also retrieving from registry through
> >>>>>> Airavata API (which calls the Registry API). These
> >>>>>> functions are available in the "ExecutionManager" of the
> >>>>>> AiravataAPI and "ProvenanceRegistry" in the RegistryAPI.
> >>>>>> * 3. Persisting & retrieving GFac job data. There are
> >>>>>> many benefits of saving the GRAM data, EC2 data etc. such
> >>>>>> as for debugging, analysis
> >>>>> etc.
> >>>>>> Currently these data are available only through the
> >>>>>> message notifications. *It is not feasible for the same
> >>>>>> reasons mentioned for API change for saving error data.
> >>>>>> Hence the introduction of functions in the API to save &
> >>>>>> retrieve GFac job data.** These functions are available
> >>>>>> as get/setApplicationJob***(...) functions in the
> >>>>>> "ProvenanceManager" of the AiravataAPI and
> >>>>>> "ProvenanceRegistry" in the RegistryAPI.*
> >>>>>>
> >>>>>
> >>>>> May I ask why we need setApplicationJob****() methods in
> >>>>> the API ? As far as I understood job information is
> >>>>> populated by GFac and other internal components. So why do
> >>>>> we need API methods to set those data ? (OR am I missing
> >>>>> something ?)
> >>>>>
> >>>> GFac and Internal components also use the AiravataAPI to
> >>>> populate the job data in the registry.
> >>>>
> >>>>
> >>>>> Thanks Amila
> >>>>>
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> Saminda
> >>>>>>
> >>>>>>
> >>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> >>>>> raminderjsingh@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Thanks Chathuri, I will test and let you know.
> >>>>>>>
> >>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Raman,
> >>>>>>>>
> >>>>>>>> The method Saminda mentioned will work for your case.
> >>>>>>>> I tested with
> >>>>>> your
> >>>>>>>> test class. I fixed deserialization issues from REST
> >>>>>>>> service side.
> >>>>> You
> >>>>>>> will
> >>>>>>>> probably have to update both server and client.
> >>>>>>>>
> >>>>>>>> Regards, Chathuri
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
> >>>>> samindaw@gmail.com
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> So far what you are saying is true. If the client
> >>>>>>>>> knows which node
> >>>>>>> failed
> >>>>>>>>> he/she can get the list of errors for that node
> >>>>>>>>> using the getExecutionError(...) function. Use the
> >>>>>>>>> "type" as ALL and leave the gfacJobId as null. I
> >>>>>>>>> couldn;t think of doing this any other better
> >>>>> way
> >>>>>>> than
> >>>>>>>>> introducing a whole lot of functions which could be
> >>>>>>>>> more
> >>>>>>> annoying/confusing
> >>>>>>>>> to figure-out. I'm welcome for suggestions for
> >>>>>>>>> improvements.
> >>>>>>>>>
> >>>>>>>>> Saminda
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> >>>>>>> raminderjsingh@gmail.com
> >>>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks Chathuri, the WorkflowExecutionError
> >>>>>>>>>> works. Integrating to
> >>>>>>> Error
> >>>>>>>>>> API brought few questions. We have ErrorTypes as
> >>>>>>>>>> :
> >>>>>>>>> WorkflowExecutionError,
> >>>>>>>>>> ExperimentExecutionError, NodeExecutionError,
> >>>>> GFacJobExecutionError,
> >>>>>>>>>> ExecutionError . How do we want clients to know
> >>>>>>>>>> which error type to
> >>>>>>>>> query?
> >>>>>>>>>> I ran a job and it failed, now i want to get
> >>>>>>>>>> error. Error occurred
> >>>>> at
> >>>>>>>>> GFac
> >>>>>>>>>> provider level and client does not have any
> >>>>>>>>>> information. Do we
> >>>>> expect
> >>>>>>>>>> client to query all different types?
> >>>>>>>>>>
> >>>>>>>>>> I thing we can have is a method to get all error
> >>>>>>>>>> and return the
> >>>>> Error
> >>>>>>>>> type
> >>>>>>>>>> part of the object to help the client to classify
> >>>>>>>>>> the error
> >>>>> location.
> >>>>>>>>>> Other suggestions?
> >>>>>>>>>>
> >>>>>>>>>> Raminder
> >>>>>>>>>>
> >>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri
> >>>>>>>>>> Wimalasena wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi Raman,
> >>>>>>>>>>>
> >>>>>>>>>>> WorkflowExecutionError method not returning any
> >>>>>>>>>>> data issue should
> >>>>> be
> >>>>>>>>>> fixed
> >>>>>>>>>>> now. Can you get an update and check.
> >>>>>>>>>>>
> >>>>>>>>>>> Regards, Chathuri
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder
> >>>>>>>>>>> Singh <ra...@gmail.com>wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Thank, these are very useful for API users.
> >>>>>>>>>>>> Only thing i found
> >>>>>>>>> confusing
> >>>>>>>>>>>> for the user is API managers.
> >>>>>>>>>>>> ProvenanceManager is the one used
> >>>>> to
> >>>>>>>>> get
> >>>>>>>>>>>> experiment status. Users will get FINISHED
> >>>>>>>>>>>> or FAILED status from
> >>>>>> the
> >>>>>>>>>> data
> >>>>>>>>>>>> object. On failure, i was trying to find a
> >>>>>>>>>>>> method to get the
> >>>>> error
> >>>>>>>>> data
> >>>>>>>>>> but
> >>>>>>>>>>>> i was not able to find as those methods exist
> >>>>>>>>>>>> in
> >>>>> ExecutionManager.
> >>>>>>>>>>>> According to me, experiment status and in
> >>>>>>>>>>>> case of failures get
> >>>>>> error
> >>>>>>>>>> need
> >>>>>>>>>>>> to be part of one manager. May be a
> >>>>>>>>>>>> ExperimentManager? Getting
> >>>>>>> output
> >>>>>>>>>> data
> >>>>>>>>>>>> can be left as part of provenance manages as
> >>>>>>>>>>>> only few client may
> >>>>>> want
> >>>>>>>>> to
> >>>>>>>>>>>> get the data in few cases. Getting error
> >>>>>>>>>>>> detail in case of error
> >>>>> is
> >>>>>>>>>> obvious
> >>>>>>>>>>>> step.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Other thing can you please explain the
> >>>>>>>>>>>> difference between WorkflowExecutionError,
> >>>>>>>>>>>> ExperimentExecutionError,
> >>>>>> NodeExecutionError,
> >>>>>>>>>>>> GFacExecutionError on a Wiki page for the
> >>>>>>>>>>>> users. Also add a
> >>>>> method
> >>>>>> to
> >>>>>>>>>> get
> >>>>>>>>>>>> JobID as that is required to get
> >>>>>>>>>>>> GFacExecutionError.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I tested the WorkflowExecutionError method
> >>>>>>>>>>>> and its not returning
> >>>>>> any
> >>>>>>>>>> data.
> >>>>>>>>>>>> Can you please test that as well?
> >>>>>>>>>>>>
> >>>>>>>>>>>> ExperimentData data =
> >>>>>>>>>>>>
> >>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >>>>>>>>>>>>
> >>>>>
> >
> >>>>>
> List<WorkflowExecutionError> experimentExecutionErrors =
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >>>>>>>>>>>>
> >>>>>
> >
> >
> experimentID);
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks Raminder
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda
> >>>>>>>>>>>> Wijeratne wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Following updates will be present Airavata
> >>>>>>>>>>>>> API from 0.8 release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1. *Error message from the immediate
> >>>>>>>>>>>>> exception used as the error
> >>>>>>>>>>>> message
> >>>>>>>>>>>>> for the AiravataAPInvocationException*.
> >>>>>>>>>>>>> Earlier it was just
> >>>>> "Error
> >>>>>>>>>>>>> invoking API". *This change was made so
> >>>>>>>>>>>>> that the users of the
> >>>>> API
> >>>>>>>>> can
> >>>>>>>>>>>>> show the error from the exception without
> >>>>>>>>>>>>> having to dig in to
> >>>>> the
> >>>>>>>>>> inner
> >>>>>>>>>>>>> exception to get a better error message.*
> >>>>>>>>>>>>> 2. *Saving and retrieving experiment
> >>>>>>>>>>>>> execution error details*.
> >>>>>>>>> Earlier
> >>>>>>>>>>>>> we were not persisting any execution error
> >>>>>>>>>>>>> details of the
> >>>>>> workflow.
> >>>>>>>>>>>> Error
> >>>>>>>>>>>>> details are only available only if the
> >>>>>>>>>>>>> developers are monitoring
> >>>>>>>>>>>> experiment
> >>>>>>>>>>>>> at the time of error occurs. *This is not
> >>>>>>>>>>>>> feasible if users
> >>>>> would
> >>>>>>>>> only
> >>>>>>>>>>>>> want to go through the error details later
> >>>>>>>>>>>>> without doing
> >>>>>> monitoring
> >>>>>>>>>>>> from
> >>>>>>>>>>>>> the beginning. Therefore we are now
> >>>>>>>>>>>>> persisting the error details
> >>>>>>>>> (not
> >>>>>>>>>>>> as
> >>>>>>>>>>>>> part of provenance data) of executing
> >>>>>>>>>>>>> workflows (i.e.
> >>>>>> experiments)*
> >>>>>>>>> in
> >>>>>>>>>>>>> to the registry and also retrieving from
> >>>>>>>>>>>>> registry through
> >>>>> Airavata
> >>>>>>>>> API
> >>>>>>>>>>>>> (which calls the Registry API).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> For now we have the functions for saving
> >>>>>>>>>>>>> and retrieving errors
> >>>>> in
> >>>>>>> the
> >>>>>>>>>>>>> ExecutionManager[1]. We need to decide if
> >>>>>>>>>>>>> this is the correct
> >>>>>> place
> >>>>>>>>> to
> >>>>>>>>>>>> put
> >>>>>>>>>>>>> these functions. Your thoughts in this
> >>>>>>>>>>>>> matter are most
> >>>>> welcome...
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks, Saminda
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 1.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >
> >
> >
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJRuy4oAAoJEOEgD2XReDo5s34H/1y35R/dwZMDhxZJOjR76wV9
> 441eAf5T/KK6Jh5ipGZVi2BaBqf3Cpudg0RvKdSj/eDwZy+d2pYyjCUFz6XVjC41
> 4expdAUykNcTKwYWEtKFbdDhdAJMeqk1UEGjw9j2BLDvpwBiym38L5WhmNMZi5pM
> RjFShClw3VK/Hdjkx/wTVQVmmMS8vWSUcLEppHzjoYBTLVSapNe2YYR0RwxwQ60i
> nQG+zxDbZqZGobFg+5cYt5xE585FalRHEc66b16Mbor7URzuhfMQuFJK1jWVl3JZ
> dMOyUQMP+JAkGhHfDty9XxeF46lxf6t0ond8y9k9fuHrdEtl1Ub2Sk9xkC17DWw=
> =HTFS
> -----END PGP SIGNATURE-----
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Updates to Airavata API version 0.8
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Are the issues blockers for 0.8 release?
Marlon
On 6/14/13 10:49 AM, Lahiru Gunathilake wrote:
> Hi Marlon,
>
> I have no update about the release, since I've been working with
> Gary in Ultrascan project. I found some issues to fix for
> ultrascan, now working on them, so we have to wait until Gary gives
> the green light to do the release.
>
> WDYT ?
>
> Regards Lahiru
>
>
> On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce <ma...@iu.edu>
> wrote:
>
> What's the status of the 0.8 release?
>
>
> Marlon
>
>
> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
>>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
>>>> <th...@gmail.com>wrote:
>>>>
>>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
>>>>> <samindaw@gmail.com
>>>>>> wrote:
>>>>>
>>>>>> Latest update on AiravataAPI updates for 0.8 version
>>>>>>
>>>>>> 1. *Error message from the immediate exception used as
>>>>>> the error
>>>>> message
>>>>>> for the AiravataAPInvocationException*. Earlier it was
>>>>>> just "Error invoking API". *This change was made so that
>>>>>> the users of the API can show the error from the
>>>>>> exception without having to dig in to the
>>>>> inner
>>>>>> exception to get a better error message.* 2. *Saving and
>>>>>> retrieving experiment execution error details*. Earlier
>>>>>> we were not persisting any execution error details of the
>>>>>> workflow. Error details are only available only if the
>>>>>> developers are monitoring experiment at the time of error
>>>>>> occurs. *This is not feasible if users would only want to
>>>>>> go through the error details later without doing
>>>>>> monitoring
>>>>> from
>>>>>> the beginning. Therefore we are now persisting the error
>>>>>> details (not
>>>>> as
>>>>>> part of provenance data) of executing workflows (i.e.
>>>>>> experiments) in
>>>>> to
>>>>>> the registry and also retrieving from registry through
>>>>>> Airavata API (which calls the Registry API). These
>>>>>> functions are available in the "ExecutionManager" of the
>>>>>> AiravataAPI and "ProvenanceRegistry" in the RegistryAPI.
>>>>>> * 3. Persisting & retrieving GFac job data. There are
>>>>>> many benefits of saving the GRAM data, EC2 data etc. such
>>>>>> as for debugging, analysis
>>>>> etc.
>>>>>> Currently these data are available only through the
>>>>>> message notifications. *It is not feasible for the same
>>>>>> reasons mentioned for API change for saving error data.
>>>>>> Hence the introduction of functions in the API to save &
>>>>>> retrieve GFac job data.** These functions are available
>>>>>> as get/setApplicationJob***(...) functions in the
>>>>>> "ProvenanceManager" of the AiravataAPI and
>>>>>> "ProvenanceRegistry" in the RegistryAPI.*
>>>>>>
>>>>>
>>>>> May I ask why we need setApplicationJob****() methods in
>>>>> the API ? As far as I understood job information is
>>>>> populated by GFac and other internal components. So why do
>>>>> we need API methods to set those data ? (OR am I missing
>>>>> something ?)
>>>>>
>>>> GFac and Internal components also use the AiravataAPI to
>>>> populate the job data in the registry.
>>>>
>>>>
>>>>> Thanks Amila
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Saminda
>>>>>>
>>>>>>
>>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
>>>>> raminderjsingh@gmail.com
>>>>>>> wrote:
>>>>>>
>>>>>>> Thanks Chathuri, I will test and let you know.
>>>>>>>
>>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Raman,
>>>>>>>>
>>>>>>>> The method Saminda mentioned will work for your case.
>>>>>>>> I tested with
>>>>>> your
>>>>>>>> test class. I fixed deserialization issues from REST
>>>>>>>> service side.
>>>>> You
>>>>>>> will
>>>>>>>> probably have to update both server and client.
>>>>>>>>
>>>>>>>> Regards, Chathuri
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
>>>>> samindaw@gmail.com
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> So far what you are saying is true. If the client
>>>>>>>>> knows which node
>>>>>>> failed
>>>>>>>>> he/she can get the list of errors for that node
>>>>>>>>> using the getExecutionError(...) function. Use the
>>>>>>>>> "type" as ALL and leave the gfacJobId as null. I
>>>>>>>>> couldn;t think of doing this any other better
>>>>> way
>>>>>>> than
>>>>>>>>> introducing a whole lot of functions which could be
>>>>>>>>> more
>>>>>>> annoying/confusing
>>>>>>>>> to figure-out. I'm welcome for suggestions for
>>>>>>>>> improvements.
>>>>>>>>>
>>>>>>>>> Saminda
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
>>>>>>> raminderjsingh@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Chathuri, the WorkflowExecutionError
>>>>>>>>>> works. Integrating to
>>>>>>> Error
>>>>>>>>>> API brought few questions. We have ErrorTypes as
>>>>>>>>>> :
>>>>>>>>> WorkflowExecutionError,
>>>>>>>>>> ExperimentExecutionError, NodeExecutionError,
>>>>> GFacJobExecutionError,
>>>>>>>>>> ExecutionError . How do we want clients to know
>>>>>>>>>> which error type to
>>>>>>>>> query?
>>>>>>>>>> I ran a job and it failed, now i want to get
>>>>>>>>>> error. Error occurred
>>>>> at
>>>>>>>>> GFac
>>>>>>>>>> provider level and client does not have any
>>>>>>>>>> information. Do we
>>>>> expect
>>>>>>>>>> client to query all different types?
>>>>>>>>>>
>>>>>>>>>> I thing we can have is a method to get all error
>>>>>>>>>> and return the
>>>>> Error
>>>>>>>>> type
>>>>>>>>>> part of the object to help the client to classify
>>>>>>>>>> the error
>>>>> location.
>>>>>>>>>> Other suggestions?
>>>>>>>>>>
>>>>>>>>>> Raminder
>>>>>>>>>>
>>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri
>>>>>>>>>> Wimalasena wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Raman,
>>>>>>>>>>>
>>>>>>>>>>> WorkflowExecutionError method not returning any
>>>>>>>>>>> data issue should
>>>>> be
>>>>>>>>>> fixed
>>>>>>>>>>> now. Can you get an update and check.
>>>>>>>>>>>
>>>>>>>>>>> Regards, Chathuri
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder
>>>>>>>>>>> Singh <ra...@gmail.com>wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thank, these are very useful for API users.
>>>>>>>>>>>> Only thing i found
>>>>>>>>> confusing
>>>>>>>>>>>> for the user is API managers.
>>>>>>>>>>>> ProvenanceManager is the one used
>>>>> to
>>>>>>>>> get
>>>>>>>>>>>> experiment status. Users will get FINISHED
>>>>>>>>>>>> or FAILED status from
>>>>>> the
>>>>>>>>>> data
>>>>>>>>>>>> object. On failure, i was trying to find a
>>>>>>>>>>>> method to get the
>>>>> error
>>>>>>>>> data
>>>>>>>>>> but
>>>>>>>>>>>> i was not able to find as those methods exist
>>>>>>>>>>>> in
>>>>> ExecutionManager.
>>>>>>>>>>>> According to me, experiment status and in
>>>>>>>>>>>> case of failures get
>>>>>> error
>>>>>>>>>> need
>>>>>>>>>>>> to be part of one manager. May be a
>>>>>>>>>>>> ExperimentManager? Getting
>>>>>>> output
>>>>>>>>>> data
>>>>>>>>>>>> can be left as part of provenance manages as
>>>>>>>>>>>> only few client may
>>>>>> want
>>>>>>>>> to
>>>>>>>>>>>> get the data in few cases. Getting error
>>>>>>>>>>>> detail in case of error
>>>>> is
>>>>>>>>>> obvious
>>>>>>>>>>>> step.
>>>>>>>>>>>>
>>>>>>>>>>>> Other thing can you please explain the
>>>>>>>>>>>> difference between WorkflowExecutionError,
>>>>>>>>>>>> ExperimentExecutionError,
>>>>>> NodeExecutionError,
>>>>>>>>>>>> GFacExecutionError on a Wiki page for the
>>>>>>>>>>>> users. Also add a
>>>>> method
>>>>>> to
>>>>>>>>>> get
>>>>>>>>>>>> JobID as that is required to get
>>>>>>>>>>>> GFacExecutionError.
>>>>>>>>>>>>
>>>>>>>>>>>> I tested the WorkflowExecutionError method
>>>>>>>>>>>> and its not returning
>>>>>> any
>>>>>>>>>> data.
>>>>>>>>>>>> Can you please test that as well?
>>>>>>>>>>>>
>>>>>>>>>>>> ExperimentData data =
>>>>>>>>>>>>
>>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>>>>>>>>>>>>
>>>>>
>
>>>>>
List<WorkflowExecutionError> experimentExecutionErrors =
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>>>>>>>>>>>>
>>>>>
>
>
experimentID);
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks Raminder
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda
>>>>>>>>>>>> Wijeratne wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Following updates will be present Airavata
>>>>>>>>>>>>> API from 0.8 release.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. *Error message from the immediate
>>>>>>>>>>>>> exception used as the error
>>>>>>>>>>>> message
>>>>>>>>>>>>> for the AiravataAPInvocationException*.
>>>>>>>>>>>>> Earlier it was just
>>>>> "Error
>>>>>>>>>>>>> invoking API". *This change was made so
>>>>>>>>>>>>> that the users of the
>>>>> API
>>>>>>>>> can
>>>>>>>>>>>>> show the error from the exception without
>>>>>>>>>>>>> having to dig in to
>>>>> the
>>>>>>>>>> inner
>>>>>>>>>>>>> exception to get a better error message.*
>>>>>>>>>>>>> 2. *Saving and retrieving experiment
>>>>>>>>>>>>> execution error details*.
>>>>>>>>> Earlier
>>>>>>>>>>>>> we were not persisting any execution error
>>>>>>>>>>>>> details of the
>>>>>> workflow.
>>>>>>>>>>>> Error
>>>>>>>>>>>>> details are only available only if the
>>>>>>>>>>>>> developers are monitoring
>>>>>>>>>>>> experiment
>>>>>>>>>>>>> at the time of error occurs. *This is not
>>>>>>>>>>>>> feasible if users
>>>>> would
>>>>>>>>> only
>>>>>>>>>>>>> want to go through the error details later
>>>>>>>>>>>>> without doing
>>>>>> monitoring
>>>>>>>>>>>> from
>>>>>>>>>>>>> the beginning. Therefore we are now
>>>>>>>>>>>>> persisting the error details
>>>>>>>>> (not
>>>>>>>>>>>> as
>>>>>>>>>>>>> part of provenance data) of executing
>>>>>>>>>>>>> workflows (i.e.
>>>>>> experiments)*
>>>>>>>>> in
>>>>>>>>>>>>> to the registry and also retrieving from
>>>>>>>>>>>>> registry through
>>>>> Airavata
>>>>>>>>> API
>>>>>>>>>>>>> (which calls the Registry API).
>>>>>>>>>>>>>
>>>>>>>>>>>>> For now we have the functions for saving
>>>>>>>>>>>>> and retrieving errors
>>>>> in
>>>>>>> the
>>>>>>>>>>>>> ExecutionManager[1]. We need to decide if
>>>>>>>>>>>>> this is the correct
>>>>>> place
>>>>>>>>> to
>>>>>>>>>>>> put
>>>>>>>>>>>>> these functions. Your thoughts in this
>>>>>>>>>>>>> matter are most
>>>>> welcome...
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks, Saminda
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>
>
>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJRuy4oAAoJEOEgD2XReDo5s34H/1y35R/dwZMDhxZJOjR76wV9
441eAf5T/KK6Jh5ipGZVi2BaBqf3Cpudg0RvKdSj/eDwZy+d2pYyjCUFz6XVjC41
4expdAUykNcTKwYWEtKFbdDhdAJMeqk1UEGjw9j2BLDvpwBiym38L5WhmNMZi5pM
RjFShClw3VK/Hdjkx/wTVQVmmMS8vWSUcLEppHzjoYBTLVSapNe2YYR0RwxwQ60i
nQG+zxDbZqZGobFg+5cYt5xE585FalRHEc66b16Mbor7URzuhfMQuFJK1jWVl3JZ
dMOyUQMP+JAkGhHfDty9XxeF46lxf6t0ond8y9k9fuHrdEtl1Ub2Sk9xkC17DWw=
=HTFS
-----END PGP SIGNATURE-----
Re: Updates to Airavata API version 0.8
Posted by Lahiru Gunathilake <gl...@gmail.com>.
Hi Marlon,
I have no update about the release, since I've been working with Gary in
Ultrascan project. I found some issues to fix for ultrascan, now working on
them, so we have to wait until Gary gives the green light to do the release.
WDYT ?
Regards
Lahiru
On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce <ma...@iu.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> What's the status of the 0.8 release?
>
>
> Marlon
>
>
> On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> > On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
> > <th...@gmail.com>wrote:
> >
> >> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
> >> <samindaw@gmail.com
> >>> wrote:
> >>
> >>> Latest update on AiravataAPI updates for 0.8 version
> >>>
> >>> 1. *Error message from the immediate exception used as the
> >>> error
> >> message
> >>> for the AiravataAPInvocationException*. Earlier it was just
> >>> "Error invoking API". *This change was made so that the users
> >>> of the API can show the error from the exception without having
> >>> to dig in to the
> >> inner
> >>> exception to get a better error message.* 2. *Saving and
> >>> retrieving experiment execution error details*. Earlier we were
> >>> not persisting any execution error details of the workflow.
> >>> Error details are only available only if the developers are
> >>> monitoring experiment at the time of error occurs. *This is not
> >>> feasible if users would only want to go through the error
> >>> details later without doing monitoring
> >> from
> >>> the beginning. Therefore we are now persisting the error
> >>> details (not
> >> as
> >>> part of provenance data) of executing workflows (i.e.
> >>> experiments) in
> >> to
> >>> the registry and also retrieving from registry through Airavata
> >>> API (which calls the Registry API). These functions are
> >>> available in the "ExecutionManager" of the AiravataAPI and
> >>> "ProvenanceRegistry" in the RegistryAPI. * 3. Persisting &
> >>> retrieving GFac job data. There are many benefits of saving the
> >>> GRAM data, EC2 data etc. such as for debugging, analysis
> >> etc.
> >>> Currently these data are available only through the message
> >>> notifications. *It is not feasible for the same reasons
> >>> mentioned for API change for saving error data. Hence the
> >>> introduction of functions in the API to save & retrieve GFac
> >>> job data.** These functions are available as
> >>> get/setApplicationJob***(...) functions in the
> >>> "ProvenanceManager" of the AiravataAPI and "ProvenanceRegistry"
> >>> in the RegistryAPI.*
> >>>
> >>
> >> May I ask why we need setApplicationJob****() methods in the API
> >> ? As far as I understood job information is populated by GFac and
> >> other internal components. So why do we need API methods to set
> >> those data ? (OR am I missing something ?)
> >>
> > GFac and Internal components also use the AiravataAPI to populate
> > the job data in the registry.
> >
> >
> >> Thanks Amila
> >>
> >>
> >>>
> >>>
> >>> Regards,
> >>>
> >>> Saminda
> >>>
> >>>
> >>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> >> raminderjsingh@gmail.com
> >>>> wrote:
> >>>
> >>>> Thanks Chathuri, I will test and let you know.
> >>>>
> >>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
> >>>> wrote:
> >>>>
> >>>>> Hi Raman,
> >>>>>
> >>>>> The method Saminda mentioned will work for your case. I
> >>>>> tested with
> >>> your
> >>>>> test class. I fixed deserialization issues from REST
> >>>>> service side.
> >> You
> >>>> will
> >>>>> probably have to update both server and client.
> >>>>>
> >>>>> Regards, Chathuri
> >>>>>
> >>>>>
> >>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
> >> samindaw@gmail.com
> >>>>> wrote:
> >>>>>
> >>>>>> So far what you are saying is true. If the client knows
> >>>>>> which node
> >>>> failed
> >>>>>> he/she can get the list of errors for that node using
> >>>>>> the getExecutionError(...) function. Use the "type" as
> >>>>>> ALL and leave the gfacJobId as null. I couldn;t think of
> >>>>>> doing this any other better
> >> way
> >>>> than
> >>>>>> introducing a whole lot of functions which could be more
> >>>> annoying/confusing
> >>>>>> to figure-out. I'm welcome for suggestions for
> >>>>>> improvements.
> >>>>>>
> >>>>>> Saminda
> >>>>>>
> >>>>>>
> >>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> >>>> raminderjsingh@gmail.com
> >>>>>>> wrote:
> >>>>>>
> >>>>>>> Thanks Chathuri, the WorkflowExecutionError works.
> >>>>>>> Integrating to
> >>>> Error
> >>>>>>> API brought few questions. We have ErrorTypes as :
> >>>>>> WorkflowExecutionError,
> >>>>>>> ExperimentExecutionError, NodeExecutionError,
> >> GFacJobExecutionError,
> >>>>>>> ExecutionError . How do we want clients to know which
> >>>>>>> error type to
> >>>>>> query?
> >>>>>>> I ran a job and it failed, now i want to get error.
> >>>>>>> Error occurred
> >> at
> >>>>>> GFac
> >>>>>>> provider level and client does not have any
> >>>>>>> information. Do we
> >> expect
> >>>>>>> client to query all different types?
> >>>>>>>
> >>>>>>> I thing we can have is a method to get all error and
> >>>>>>> return the
> >> Error
> >>>>>> type
> >>>>>>> part of the object to help the client to classify the
> >>>>>>> error
> >> location.
> >>>>>>> Other suggestions?
> >>>>>>>
> >>>>>>> Raminder
> >>>>>>>
> >>>>>>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Raman,
> >>>>>>>>
> >>>>>>>> WorkflowExecutionError method not returning any data
> >>>>>>>> issue should
> >> be
> >>>>>>> fixed
> >>>>>>>> now. Can you get an update and check.
> >>>>>>>>
> >>>>>>>> Regards, Chathuri
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> >>>>>>>> <ra...@gmail.com>wrote:
> >>>>>>>>
> >>>>>>>>> Thank, these are very useful for API users. Only
> >>>>>>>>> thing i found
> >>>>>> confusing
> >>>>>>>>> for the user is API managers. ProvenanceManager is
> >>>>>>>>> the one used
> >> to
> >>>>>> get
> >>>>>>>>> experiment status. Users will get FINISHED or
> >>>>>>>>> FAILED status from
> >>> the
> >>>>>>> data
> >>>>>>>>> object. On failure, i was trying to find a method
> >>>>>>>>> to get the
> >> error
> >>>>>> data
> >>>>>>> but
> >>>>>>>>> i was not able to find as those methods exist in
> >> ExecutionManager.
> >>>>>>>>> According to me, experiment status and in case of
> >>>>>>>>> failures get
> >>> error
> >>>>>>> need
> >>>>>>>>> to be part of one manager. May be a
> >>>>>>>>> ExperimentManager? Getting
> >>>> output
> >>>>>>> data
> >>>>>>>>> can be left as part of provenance manages as only
> >>>>>>>>> few client may
> >>> want
> >>>>>> to
> >>>>>>>>> get the data in few cases. Getting error detail in
> >>>>>>>>> case of error
> >> is
> >>>>>>> obvious
> >>>>>>>>> step.
> >>>>>>>>>
> >>>>>>>>> Other thing can you please explain the difference
> >>>>>>>>> between WorkflowExecutionError,
> >>>>>>>>> ExperimentExecutionError,
> >>> NodeExecutionError,
> >>>>>>>>> GFacExecutionError on a Wiki page for the users.
> >>>>>>>>> Also add a
> >> method
> >>> to
> >>>>>>> get
> >>>>>>>>> JobID as that is required to get
> >>>>>>>>> GFacExecutionError.
> >>>>>>>>>
> >>>>>>>>> I tested the WorkflowExecutionError method and its
> >>>>>>>>> not returning
> >>> any
> >>>>>>> data.
> >>>>>>>>> Can you please test that as well?
> >>>>>>>>>
> >>>>>>>>> ExperimentData data =
> >>>>>>>>>
> >> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >>>>>>>>>
> >>
> List<WorkflowExecutionError> experimentExecutionErrors =
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >>>>>>>>>
> >>
> experimentID);
> >>>>>>>>>
> >>>>>>>>> Thanks Raminder
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Following updates will be present Airavata API
> >>>>>>>>>> from 0.8 release.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> 1. *Error message from the immediate exception
> >>>>>>>>>> used as the error
> >>>>>>>>> message
> >>>>>>>>>> for the AiravataAPInvocationException*. Earlier
> >>>>>>>>>> it was just
> >> "Error
> >>>>>>>>>> invoking API". *This change was made so that the
> >>>>>>>>>> users of the
> >> API
> >>>>>> can
> >>>>>>>>>> show the error from the exception without having
> >>>>>>>>>> to dig in to
> >> the
> >>>>>>> inner
> >>>>>>>>>> exception to get a better error message.* 2.
> >>>>>>>>>> *Saving and retrieving experiment execution error
> >>>>>>>>>> details*.
> >>>>>> Earlier
> >>>>>>>>>> we were not persisting any execution error
> >>>>>>>>>> details of the
> >>> workflow.
> >>>>>>>>> Error
> >>>>>>>>>> details are only available only if the developers
> >>>>>>>>>> are monitoring
> >>>>>>>>> experiment
> >>>>>>>>>> at the time of error occurs. *This is not
> >>>>>>>>>> feasible if users
> >> would
> >>>>>> only
> >>>>>>>>>> want to go through the error details later
> >>>>>>>>>> without doing
> >>> monitoring
> >>>>>>>>> from
> >>>>>>>>>> the beginning. Therefore we are now persisting
> >>>>>>>>>> the error details
> >>>>>> (not
> >>>>>>>>> as
> >>>>>>>>>> part of provenance data) of executing workflows
> >>>>>>>>>> (i.e.
> >>> experiments)*
> >>>>>> in
> >>>>>>>>>> to the registry and also retrieving from registry
> >>>>>>>>>> through
> >> Airavata
> >>>>>> API
> >>>>>>>>>> (which calls the Registry API).
> >>>>>>>>>>
> >>>>>>>>>> For now we have the functions for saving and
> >>>>>>>>>> retrieving errors
> >> in
> >>>> the
> >>>>>>>>>> ExecutionManager[1]. We need to decide if this is
> >>>>>>>>>> the correct
> >>> place
> >>>>>> to
> >>>>>>>>> put
> >>>>>>>>>> these functions. Your thoughts in this matter are
> >>>>>>>>>> most
> >> welcome...
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Thanks, Saminda
> >>>>>>>>>>
> >>>>>>>>>> 1.
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
> >
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJRuyxqAAoJEOEgD2XReDo54nwIAKjDjVqPk6YldnJD+T91k2HU
> Yj/VcvH/XAxvoZOu9zGpcUFNuLPbSiJEJes9Tx3Wk63k8v+l4rxTTRB5rWihbyYn
> xPEKtSnD8sCAiuFXlRHWLygzE80OVhC9Q43xkrnjjubKaWKp3ZgU1uF1ZL6LiTZb
> R83+Yl2WCdAAckX1W+567APk202nw93GLKGjmRcuj2mBPKgWXzzD7AX3cQX6pt9e
> g93PKGWJWdk/kb6YlhjkEfKH2RxJS/AgZvUYSAmf1ELL+siGqxEEOg7bwWwpfLNd
> NsjUr/rN88EcGfUKxYb4fSh1zGceqp02rWG2TwufZEIVaF8WjztY6CsE2TgfS0M=
> =vpMl
> -----END PGP SIGNATURE-----
>
--
System Analyst Programmer
PTI Lab
Indiana University
Re: Updates to Airavata API version 0.8
Posted by Marlon Pierce <ma...@iu.edu>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
What's the status of the 0.8 release?
Marlon
On 5/31/13 11:53 AM, Saminda Wijeratne wrote:
> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
> <th...@gmail.com>wrote:
>
>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne
>> <samindaw@gmail.com
>>> wrote:
>>
>>> Latest update on AiravataAPI updates for 0.8 version
>>>
>>> 1. *Error message from the immediate exception used as the
>>> error
>> message
>>> for the AiravataAPInvocationException*. Earlier it was just
>>> "Error invoking API". *This change was made so that the users
>>> of the API can show the error from the exception without having
>>> to dig in to the
>> inner
>>> exception to get a better error message.* 2. *Saving and
>>> retrieving experiment execution error details*. Earlier we were
>>> not persisting any execution error details of the workflow.
>>> Error details are only available only if the developers are
>>> monitoring experiment at the time of error occurs. *This is not
>>> feasible if users would only want to go through the error
>>> details later without doing monitoring
>> from
>>> the beginning. Therefore we are now persisting the error
>>> details (not
>> as
>>> part of provenance data) of executing workflows (i.e.
>>> experiments) in
>> to
>>> the registry and also retrieving from registry through Airavata
>>> API (which calls the Registry API). These functions are
>>> available in the "ExecutionManager" of the AiravataAPI and
>>> "ProvenanceRegistry" in the RegistryAPI. * 3. Persisting &
>>> retrieving GFac job data. There are many benefits of saving the
>>> GRAM data, EC2 data etc. such as for debugging, analysis
>> etc.
>>> Currently these data are available only through the message
>>> notifications. *It is not feasible for the same reasons
>>> mentioned for API change for saving error data. Hence the
>>> introduction of functions in the API to save & retrieve GFac
>>> job data.** These functions are available as
>>> get/setApplicationJob***(...) functions in the
>>> "ProvenanceManager" of the AiravataAPI and "ProvenanceRegistry"
>>> in the RegistryAPI.*
>>>
>>
>> May I ask why we need setApplicationJob****() methods in the API
>> ? As far as I understood job information is populated by GFac and
>> other internal components. So why do we need API methods to set
>> those data ? (OR am I missing something ?)
>>
> GFac and Internal components also use the AiravataAPI to populate
> the job data in the registry.
>
>
>> Thanks Amila
>>
>>
>>>
>>>
>>> Regards,
>>>
>>> Saminda
>>>
>>>
>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
>> raminderjsingh@gmail.com
>>>> wrote:
>>>
>>>> Thanks Chathuri, I will test and let you know.
>>>>
>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri Wimalasena
>>>> wrote:
>>>>
>>>>> Hi Raman,
>>>>>
>>>>> The method Saminda mentioned will work for your case. I
>>>>> tested with
>>> your
>>>>> test class. I fixed deserialization issues from REST
>>>>> service side.
>> You
>>>> will
>>>>> probably have to update both server and client.
>>>>>
>>>>> Regards, Chathuri
>>>>>
>>>>>
>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
>> samindaw@gmail.com
>>>>> wrote:
>>>>>
>>>>>> So far what you are saying is true. If the client knows
>>>>>> which node
>>>> failed
>>>>>> he/she can get the list of errors for that node using
>>>>>> the getExecutionError(...) function. Use the "type" as
>>>>>> ALL and leave the gfacJobId as null. I couldn;t think of
>>>>>> doing this any other better
>> way
>>>> than
>>>>>> introducing a whole lot of functions which could be more
>>>> annoying/confusing
>>>>>> to figure-out. I'm welcome for suggestions for
>>>>>> improvements.
>>>>>>
>>>>>> Saminda
>>>>>>
>>>>>>
>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
>>>> raminderjsingh@gmail.com
>>>>>>> wrote:
>>>>>>
>>>>>>> Thanks Chathuri, the WorkflowExecutionError works.
>>>>>>> Integrating to
>>>> Error
>>>>>>> API brought few questions. We have ErrorTypes as :
>>>>>> WorkflowExecutionError,
>>>>>>> ExperimentExecutionError, NodeExecutionError,
>> GFacJobExecutionError,
>>>>>>> ExecutionError . How do we want clients to know which
>>>>>>> error type to
>>>>>> query?
>>>>>>> I ran a job and it failed, now i want to get error.
>>>>>>> Error occurred
>> at
>>>>>> GFac
>>>>>>> provider level and client does not have any
>>>>>>> information. Do we
>> expect
>>>>>>> client to query all different types?
>>>>>>>
>>>>>>> I thing we can have is a method to get all error and
>>>>>>> return the
>> Error
>>>>>> type
>>>>>>> part of the object to help the client to classify the
>>>>>>> error
>> location.
>>>>>>> Other suggestions?
>>>>>>>
>>>>>>> Raminder
>>>>>>>
>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Raman,
>>>>>>>>
>>>>>>>> WorkflowExecutionError method not returning any data
>>>>>>>> issue should
>> be
>>>>>>> fixed
>>>>>>>> now. Can you get an update and check.
>>>>>>>>
>>>>>>>> Regards, Chathuri
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
>>>>>>>> <ra...@gmail.com>wrote:
>>>>>>>>
>>>>>>>>> Thank, these are very useful for API users. Only
>>>>>>>>> thing i found
>>>>>> confusing
>>>>>>>>> for the user is API managers. ProvenanceManager is
>>>>>>>>> the one used
>> to
>>>>>> get
>>>>>>>>> experiment status. Users will get FINISHED or
>>>>>>>>> FAILED status from
>>> the
>>>>>>> data
>>>>>>>>> object. On failure, i was trying to find a method
>>>>>>>>> to get the
>> error
>>>>>> data
>>>>>>> but
>>>>>>>>> i was not able to find as those methods exist in
>> ExecutionManager.
>>>>>>>>> According to me, experiment status and in case of
>>>>>>>>> failures get
>>> error
>>>>>>> need
>>>>>>>>> to be part of one manager. May be a
>>>>>>>>> ExperimentManager? Getting
>>>> output
>>>>>>> data
>>>>>>>>> can be left as part of provenance manages as only
>>>>>>>>> few client may
>>> want
>>>>>> to
>>>>>>>>> get the data in few cases. Getting error detail in
>>>>>>>>> case of error
>> is
>>>>>>> obvious
>>>>>>>>> step.
>>>>>>>>>
>>>>>>>>> Other thing can you please explain the difference
>>>>>>>>> between WorkflowExecutionError,
>>>>>>>>> ExperimentExecutionError,
>>> NodeExecutionError,
>>>>>>>>> GFacExecutionError on a Wiki page for the users.
>>>>>>>>> Also add a
>> method
>>> to
>>>>>>> get
>>>>>>>>> JobID as that is required to get
>>>>>>>>> GFacExecutionError.
>>>>>>>>>
>>>>>>>>> I tested the WorkflowExecutionError method and its
>>>>>>>>> not returning
>>> any
>>>>>>> data.
>>>>>>>>> Can you please test that as well?
>>>>>>>>>
>>>>>>>>> ExperimentData data =
>>>>>>>>>
>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>>>>>>>>>
>>
List<WorkflowExecutionError> experimentExecutionErrors =
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>>>>>>>>>
>>
experimentID);
>>>>>>>>>
>>>>>>>>> Thanks Raminder
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Following updates will be present Airavata API
>>>>>>>>>> from 0.8 release.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 1. *Error message from the immediate exception
>>>>>>>>>> used as the error
>>>>>>>>> message
>>>>>>>>>> for the AiravataAPInvocationException*. Earlier
>>>>>>>>>> it was just
>> "Error
>>>>>>>>>> invoking API". *This change was made so that the
>>>>>>>>>> users of the
>> API
>>>>>> can
>>>>>>>>>> show the error from the exception without having
>>>>>>>>>> to dig in to
>> the
>>>>>>> inner
>>>>>>>>>> exception to get a better error message.* 2.
>>>>>>>>>> *Saving and retrieving experiment execution error
>>>>>>>>>> details*.
>>>>>> Earlier
>>>>>>>>>> we were not persisting any execution error
>>>>>>>>>> details of the
>>> workflow.
>>>>>>>>> Error
>>>>>>>>>> details are only available only if the developers
>>>>>>>>>> are monitoring
>>>>>>>>> experiment
>>>>>>>>>> at the time of error occurs. *This is not
>>>>>>>>>> feasible if users
>> would
>>>>>> only
>>>>>>>>>> want to go through the error details later
>>>>>>>>>> without doing
>>> monitoring
>>>>>>>>> from
>>>>>>>>>> the beginning. Therefore we are now persisting
>>>>>>>>>> the error details
>>>>>> (not
>>>>>>>>> as
>>>>>>>>>> part of provenance data) of executing workflows
>>>>>>>>>> (i.e.
>>> experiments)*
>>>>>> in
>>>>>>>>>> to the registry and also retrieving from registry
>>>>>>>>>> through
>> Airavata
>>>>>> API
>>>>>>>>>> (which calls the Registry API).
>>>>>>>>>>
>>>>>>>>>> For now we have the functions for saving and
>>>>>>>>>> retrieving errors
>> in
>>>> the
>>>>>>>>>> ExecutionManager[1]. We need to decide if this is
>>>>>>>>>> the correct
>>> place
>>>>>> to
>>>>>>>>> put
>>>>>>>>>> these functions. Your thoughts in this matter are
>>>>>>>>>> most
>> welcome...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks, Saminda
>>>>>>>>>>
>>>>>>>>>> 1.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>
>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.18 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQEcBAEBAgAGBQJRuyxqAAoJEOEgD2XReDo54nwIAKjDjVqPk6YldnJD+T91k2HU
Yj/VcvH/XAxvoZOu9zGpcUFNuLPbSiJEJes9Tx3Wk63k8v+l4rxTTRB5rWihbyYn
xPEKtSnD8sCAiuFXlRHWLygzE80OVhC9Q43xkrnjjubKaWKp3ZgU1uF1ZL6LiTZb
R83+Yl2WCdAAckX1W+567APk202nw93GLKGjmRcuj2mBPKgWXzzD7AX3cQX6pt9e
g93PKGWJWdk/kb6YlhjkEfKH2RxJS/AgZvUYSAmf1ELL+siGqxEEOg7bwWwpfLNd
NsjUr/rN88EcGfUKxYb4fSh1zGceqp02rWG2TwufZEIVaF8WjztY6CsE2TgfS0M=
=vpMl
-----END PGP SIGNATURE-----
Re: Updates to Airavata API version 0.8
Posted by Saminda Wijeratne <sa...@gmail.com>.
On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara
<th...@gmail.com>wrote:
> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne <samindaw@gmail.com
> >wrote:
>
> > Latest update on AiravataAPI updates for 0.8 version
> >
> > 1. *Error message from the immediate exception used as the error
> message
> > for the AiravataAPInvocationException*. Earlier it was just "Error
> > invoking API". *This change was made so that the users of the API can
> > show the error from the exception without having to dig in to the
> inner
> > exception to get a better error message.*
> > 2. *Saving and retrieving experiment execution error details*. Earlier
> > we were not persisting any execution error details of the workflow.
> > Error
> > details are only available only if the developers are monitoring
> > experiment
> > at the time of error occurs. *This is not feasible if users would only
> > want to go through the error details later without doing monitoring
> from
> > the beginning. Therefore we are now persisting the error details (not
> as
> > part of provenance data) of executing workflows (i.e. experiments) in
> to
> > the registry and also retrieving from registry through Airavata API
> > (which
> > calls the Registry API). These functions are available in the
> > "ExecutionManager" of the AiravataAPI and "ProvenanceRegistry" in the
> > RegistryAPI.
> > *
> > 3. Persisting & retrieving GFac job data. There are many benefits of
> > saving the GRAM data, EC2 data etc. such as for debugging, analysis
> etc.
> > Currently these data are available only through the message
> > notifications.
> > *It is not feasible for the same reasons mentioned for API change for
> > saving error data. Hence the introduction of functions in the API to
> > save &
> > retrieve GFac job data.** These functions are available as
> > get/setApplicationJob***(...) functions in the "ProvenanceManager" of
> > the
> > AiravataAPI and "ProvenanceRegistry" in the RegistryAPI.*
> >
>
> May I ask why we need setApplicationJob****() methods in the API ?
> As far as I understood job information is populated by GFac and other
> internal components. So why do we need API methods to set those data ? (OR
> am I missing something ?)
>
GFac and Internal components also use the AiravataAPI to populate the job
data in the registry.
> Thanks
> Amila
>
>
> >
> >
> > Regards,
> >
> > Saminda
> >
> >
> > On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <
> raminderjsingh@gmail.com
> > >wrote:
> >
> > > Thanks Chathuri, I will test and let you know.
> > >
> > > Raman
> > > On May 21, 2013, at 4:55 PM, Chathuri Wimalasena wrote:
> > >
> > > > Hi Raman,
> > > >
> > > > The method Saminda mentioned will work for your case. I tested with
> > your
> > > > test class. I fixed deserialization issues from REST service side.
> You
> > > will
> > > > probably have to update both server and client.
> > > >
> > > > Regards,
> > > > Chathuri
> > > >
> > > >
> > > > On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <
> samindaw@gmail.com
> > > >wrote:
> > > >
> > > >> So far what you are saying is true. If the client knows which node
> > > failed
> > > >> he/she can get the list of errors for that node using the
> > > >> getExecutionError(...) function. Use the "type" as ALL and leave the
> > > >> gfacJobId as null. I couldn;t think of doing this any other better
> way
> > > than
> > > >> introducing a whole lot of functions which could be more
> > > annoying/confusing
> > > >> to figure-out. I'm welcome for suggestions for improvements.
> > > >>
> > > >> Saminda
> > > >>
> > > >>
> > > >> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> > > raminderjsingh@gmail.com
> > > >>> wrote:
> > > >>
> > > >>> Thanks Chathuri, the WorkflowExecutionError works. Integrating to
> > > Error
> > > >>> API brought few questions. We have ErrorTypes as :
> > > >> WorkflowExecutionError,
> > > >>> ExperimentExecutionError, NodeExecutionError,
> GFacJobExecutionError,
> > > >>> ExecutionError . How do we want clients to know which error type to
> > > >> query?
> > > >>> I ran a job and it failed, now i want to get error. Error occurred
> at
> > > >> GFac
> > > >>> provider level and client does not have any information. Do we
> expect
> > > >>> client to query all different types?
> > > >>>
> > > >>> I thing we can have is a method to get all error and return the
> Error
> > > >> type
> > > >>> part of the object to help the client to classify the error
> location.
> > > >>> Other suggestions?
> > > >>>
> > > >>> Raminder
> > > >>>
> > > >>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
> > > >>>
> > > >>>> Hi Raman,
> > > >>>>
> > > >>>> WorkflowExecutionError method not returning any data issue should
> be
> > > >>> fixed
> > > >>>> now. Can you get an update and check.
> > > >>>>
> > > >>>> Regards,
> > > >>>> Chathuri
> > > >>>>
> > > >>>>
> > > >>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> > > >>>> <ra...@gmail.com>wrote:
> > > >>>>
> > > >>>>> Thank, these are very useful for API users. Only thing i found
> > > >> confusing
> > > >>>>> for the user is API managers. ProvenanceManager is the one used
> to
> > > >> get
> > > >>>>> experiment status. Users will get FINISHED or FAILED status from
> > the
> > > >>> data
> > > >>>>> object. On failure, i was trying to find a method to get the
> error
> > > >> data
> > > >>> but
> > > >>>>> i was not able to find as those methods exist in
> ExecutionManager.
> > > >>>>> According to me, experiment status and in case of failures get
> > error
> > > >>> need
> > > >>>>> to be part of one manager. May be a ExperimentManager? Getting
> > > output
> > > >>> data
> > > >>>>> can be left as part of provenance manages as only few client may
> > want
> > > >> to
> > > >>>>> get the data in few cases. Getting error detail in case of error
> is
> > > >>> obvious
> > > >>>>> step.
> > > >>>>>
> > > >>>>> Other thing can you please explain the difference between
> > > >>>>> WorkflowExecutionError, ExperimentExecutionError,
> > NodeExecutionError,
> > > >>>>> GFacExecutionError on a Wiki page for the users. Also add a
> method
> > to
> > > >>> get
> > > >>>>> JobID as that is required to get GFacExecutionError.
> > > >>>>>
> > > >>>>> I tested the WorkflowExecutionError method and its not returning
> > any
> > > >>> data.
> > > >>>>> Can you please test that as well?
> > > >>>>>
> > > >>>>> ExperimentData data =
> > > >>>>>
> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> > > >>>>> List<WorkflowExecutionError> experimentExecutionErrors =
> > > >>>>>
> > > >>>
> > > >>
> > >
> >
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> > > >>>>> experimentID);
> > > >>>>>
> > > >>>>> Thanks
> > > >>>>> Raminder
> > > >>>>>
> > > >>>>>
> > > >>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
> > > >>>>>
> > > >>>>>> Following updates will be present Airavata API from 0.8 release.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> 1. *Error message from the immediate exception used as the error
> > > >>>>> message
> > > >>>>>> for the AiravataAPInvocationException*. Earlier it was just
> "Error
> > > >>>>>> invoking API". *This change was made so that the users of the
> API
> > > >> can
> > > >>>>>> show the error from the exception without having to dig in to
> the
> > > >>> inner
> > > >>>>>> exception to get a better error message.*
> > > >>>>>> 2. *Saving and retrieving experiment execution error details*.
> > > >> Earlier
> > > >>>>>> we were not persisting any execution error details of the
> > workflow.
> > > >>>>> Error
> > > >>>>>> details are only available only if the developers are monitoring
> > > >>>>> experiment
> > > >>>>>> at the time of error occurs. *This is not feasible if users
> would
> > > >> only
> > > >>>>>> want to go through the error details later without doing
> > monitoring
> > > >>>>> from
> > > >>>>>> the beginning. Therefore we are now persisting the error details
> > > >> (not
> > > >>>>> as
> > > >>>>>> part of provenance data) of executing workflows (i.e.
> > experiments)*
> > > >> in
> > > >>>>>> to the registry and also retrieving from registry through
> Airavata
> > > >> API
> > > >>>>>> (which calls the Registry API).
> > > >>>>>>
> > > >>>>>> For now we have the functions for saving and retrieving errors
> in
> > > the
> > > >>>>>> ExecutionManager[1]. We need to decide if this is the correct
> > place
> > > >> to
> > > >>>>> put
> > > >>>>>> these functions. Your thoughts in this matter are most
> welcome...
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Thanks,
> > > >>>>>> Saminda
> > > >>>>>>
> > > >>>>>> 1.
> > > >>>>>>
> > > >>>>>
> > > >>>
> > > >>
> > >
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> > > >>>>>
> > > >>>>>
> > > >>>
> > > >>>
> > > >>
> > >
> > >
> >
>
Re: Updates to Airavata API version 0.8
Posted by Amila Jayasekara <th...@gmail.com>.
On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne <sa...@gmail.com>wrote:
> Latest update on AiravataAPI updates for 0.8 version
>
> 1. *Error message from the immediate exception used as the error message
> for the AiravataAPInvocationException*. Earlier it was just "Error
> invoking API". *This change was made so that the users of the API can
> show the error from the exception without having to dig in to the inner
> exception to get a better error message.*
> 2. *Saving and retrieving experiment execution error details*. Earlier
> we were not persisting any execution error details of the workflow.
> Error
> details are only available only if the developers are monitoring
> experiment
> at the time of error occurs. *This is not feasible if users would only
> want to go through the error details later without doing monitoring from
> the beginning. Therefore we are now persisting the error details (not as
> part of provenance data) of executing workflows (i.e. experiments) in to
> the registry and also retrieving from registry through Airavata API
> (which
> calls the Registry API). These functions are available in the
> "ExecutionManager" of the AiravataAPI and "ProvenanceRegistry" in the
> RegistryAPI.
> *
> 3. Persisting & retrieving GFac job data. There are many benefits of
> saving the GRAM data, EC2 data etc. such as for debugging, analysis etc.
> Currently these data are available only through the message
> notifications.
> *It is not feasible for the same reasons mentioned for API change for
> saving error data. Hence the introduction of functions in the API to
> save &
> retrieve GFac job data.** These functions are available as
> get/setApplicationJob***(...) functions in the "ProvenanceManager" of
> the
> AiravataAPI and "ProvenanceRegistry" in the RegistryAPI.*
>
May I ask why we need setApplicationJob****() methods in the API ?
As far as I understood job information is populated by GFac and other
internal components. So why do we need API methods to set those data ? (OR
am I missing something ?)
Thanks
Amila
>
>
> Regards,
>
> Saminda
>
>
> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <raminderjsingh@gmail.com
> >wrote:
>
> > Thanks Chathuri, I will test and let you know.
> >
> > Raman
> > On May 21, 2013, at 4:55 PM, Chathuri Wimalasena wrote:
> >
> > > Hi Raman,
> > >
> > > The method Saminda mentioned will work for your case. I tested with
> your
> > > test class. I fixed deserialization issues from REST service side. You
> > will
> > > probably have to update both server and client.
> > >
> > > Regards,
> > > Chathuri
> > >
> > >
> > > On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <samindaw@gmail.com
> > >wrote:
> > >
> > >> So far what you are saying is true. If the client knows which node
> > failed
> > >> he/she can get the list of errors for that node using the
> > >> getExecutionError(...) function. Use the "type" as ALL and leave the
> > >> gfacJobId as null. I couldn;t think of doing this any other better way
> > than
> > >> introducing a whole lot of functions which could be more
> > annoying/confusing
> > >> to figure-out. I'm welcome for suggestions for improvements.
> > >>
> > >> Saminda
> > >>
> > >>
> > >> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> > raminderjsingh@gmail.com
> > >>> wrote:
> > >>
> > >>> Thanks Chathuri, the WorkflowExecutionError works. Integrating to
> > Error
> > >>> API brought few questions. We have ErrorTypes as :
> > >> WorkflowExecutionError,
> > >>> ExperimentExecutionError, NodeExecutionError, GFacJobExecutionError,
> > >>> ExecutionError . How do we want clients to know which error type to
> > >> query?
> > >>> I ran a job and it failed, now i want to get error. Error occurred at
> > >> GFac
> > >>> provider level and client does not have any information. Do we expect
> > >>> client to query all different types?
> > >>>
> > >>> I thing we can have is a method to get all error and return the Error
> > >> type
> > >>> part of the object to help the client to classify the error location.
> > >>> Other suggestions?
> > >>>
> > >>> Raminder
> > >>>
> > >>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
> > >>>
> > >>>> Hi Raman,
> > >>>>
> > >>>> WorkflowExecutionError method not returning any data issue should be
> > >>> fixed
> > >>>> now. Can you get an update and check.
> > >>>>
> > >>>> Regards,
> > >>>> Chathuri
> > >>>>
> > >>>>
> > >>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> > >>>> <ra...@gmail.com>wrote:
> > >>>>
> > >>>>> Thank, these are very useful for API users. Only thing i found
> > >> confusing
> > >>>>> for the user is API managers. ProvenanceManager is the one used to
> > >> get
> > >>>>> experiment status. Users will get FINISHED or FAILED status from
> the
> > >>> data
> > >>>>> object. On failure, i was trying to find a method to get the error
> > >> data
> > >>> but
> > >>>>> i was not able to find as those methods exist in ExecutionManager.
> > >>>>> According to me, experiment status and in case of failures get
> error
> > >>> need
> > >>>>> to be part of one manager. May be a ExperimentManager? Getting
> > output
> > >>> data
> > >>>>> can be left as part of provenance manages as only few client may
> want
> > >> to
> > >>>>> get the data in few cases. Getting error detail in case of error is
> > >>> obvious
> > >>>>> step.
> > >>>>>
> > >>>>> Other thing can you please explain the difference between
> > >>>>> WorkflowExecutionError, ExperimentExecutionError,
> NodeExecutionError,
> > >>>>> GFacExecutionError on a Wiki page for the users. Also add a method
> to
> > >>> get
> > >>>>> JobID as that is required to get GFacExecutionError.
> > >>>>>
> > >>>>> I tested the WorkflowExecutionError method and its not returning
> any
> > >>> data.
> > >>>>> Can you please test that as well?
> > >>>>>
> > >>>>> ExperimentData data =
> > >>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> > >>>>> List<WorkflowExecutionError> experimentExecutionErrors =
> > >>>>>
> > >>>
> > >>
> >
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> > >>>>> experimentID);
> > >>>>>
> > >>>>> Thanks
> > >>>>> Raminder
> > >>>>>
> > >>>>>
> > >>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
> > >>>>>
> > >>>>>> Following updates will be present Airavata API from 0.8 release.
> > >>>>>>
> > >>>>>>
> > >>>>>> 1. *Error message from the immediate exception used as the error
> > >>>>> message
> > >>>>>> for the AiravataAPInvocationException*. Earlier it was just "Error
> > >>>>>> invoking API". *This change was made so that the users of the API
> > >> can
> > >>>>>> show the error from the exception without having to dig in to the
> > >>> inner
> > >>>>>> exception to get a better error message.*
> > >>>>>> 2. *Saving and retrieving experiment execution error details*.
> > >> Earlier
> > >>>>>> we were not persisting any execution error details of the
> workflow.
> > >>>>> Error
> > >>>>>> details are only available only if the developers are monitoring
> > >>>>> experiment
> > >>>>>> at the time of error occurs. *This is not feasible if users would
> > >> only
> > >>>>>> want to go through the error details later without doing
> monitoring
> > >>>>> from
> > >>>>>> the beginning. Therefore we are now persisting the error details
> > >> (not
> > >>>>> as
> > >>>>>> part of provenance data) of executing workflows (i.e.
> experiments)*
> > >> in
> > >>>>>> to the registry and also retrieving from registry through Airavata
> > >> API
> > >>>>>> (which calls the Registry API).
> > >>>>>>
> > >>>>>> For now we have the functions for saving and retrieving errors in
> > the
> > >>>>>> ExecutionManager[1]. We need to decide if this is the correct
> place
> > >> to
> > >>>>> put
> > >>>>>> these functions. Your thoughts in this matter are most welcome...
> > >>>>>>
> > >>>>>>
> > >>>>>> Thanks,
> > >>>>>> Saminda
> > >>>>>>
> > >>>>>> 1.
> > >>>>>>
> > >>>>>
> > >>>
> > >>
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> > >>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>
Re: Updates to Airavata API version 0.8
Posted by Saminda Wijeratne <sa...@gmail.com>.
Latest update on AiravataAPI updates for 0.8 version
1. *Error message from the immediate exception used as the error message
for the AiravataAPInvocationException*. Earlier it was just "Error
invoking API". *This change was made so that the users of the API can
show the error from the exception without having to dig in to the inner
exception to get a better error message.*
2. *Saving and retrieving experiment execution error details*. Earlier
we were not persisting any execution error details of the workflow. Error
details are only available only if the developers are monitoring experiment
at the time of error occurs. *This is not feasible if users would only
want to go through the error details later without doing monitoring from
the beginning. Therefore we are now persisting the error details (not as
part of provenance data) of executing workflows (i.e. experiments) in to
the registry and also retrieving from registry through Airavata API (which
calls the Registry API). These functions are available in the
"ExecutionManager" of the AiravataAPI and "ProvenanceRegistry" in the
RegistryAPI.
*
3. Persisting & retrieving GFac job data. There are many benefits of
saving the GRAM data, EC2 data etc. such as for debugging, analysis etc.
Currently these data are available only through the message notifications.
*It is not feasible for the same reasons mentioned for API change for
saving error data. Hence the introduction of functions in the API to save &
retrieve GFac job data.** These functions are available as
get/setApplicationJob***(...) functions in the "ProvenanceManager" of the
AiravataAPI and "ProvenanceRegistry" in the RegistryAPI.*
Regards,
Saminda
On Tue, May 21, 2013 at 4:59 PM, Raminder Singh <ra...@gmail.com>wrote:
> Thanks Chathuri, I will test and let you know.
>
> Raman
> On May 21, 2013, at 4:55 PM, Chathuri Wimalasena wrote:
>
> > Hi Raman,
> >
> > The method Saminda mentioned will work for your case. I tested with your
> > test class. I fixed deserialization issues from REST service side. You
> will
> > probably have to update both server and client.
> >
> > Regards,
> > Chathuri
> >
> >
> > On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <samindaw@gmail.com
> >wrote:
> >
> >> So far what you are saying is true. If the client knows which node
> failed
> >> he/she can get the list of errors for that node using the
> >> getExecutionError(...) function. Use the "type" as ALL and leave the
> >> gfacJobId as null. I couldn;t think of doing this any other better way
> than
> >> introducing a whole lot of functions which could be more
> annoying/confusing
> >> to figure-out. I'm welcome for suggestions for improvements.
> >>
> >> Saminda
> >>
> >>
> >> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <
> raminderjsingh@gmail.com
> >>> wrote:
> >>
> >>> Thanks Chathuri, the WorkflowExecutionError works. Integrating to
> Error
> >>> API brought few questions. We have ErrorTypes as :
> >> WorkflowExecutionError,
> >>> ExperimentExecutionError, NodeExecutionError, GFacJobExecutionError,
> >>> ExecutionError . How do we want clients to know which error type to
> >> query?
> >>> I ran a job and it failed, now i want to get error. Error occurred at
> >> GFac
> >>> provider level and client does not have any information. Do we expect
> >>> client to query all different types?
> >>>
> >>> I thing we can have is a method to get all error and return the Error
> >> type
> >>> part of the object to help the client to classify the error location.
> >>> Other suggestions?
> >>>
> >>> Raminder
> >>>
> >>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
> >>>
> >>>> Hi Raman,
> >>>>
> >>>> WorkflowExecutionError method not returning any data issue should be
> >>> fixed
> >>>> now. Can you get an update and check.
> >>>>
> >>>> Regards,
> >>>> Chathuri
> >>>>
> >>>>
> >>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> >>>> <ra...@gmail.com>wrote:
> >>>>
> >>>>> Thank, these are very useful for API users. Only thing i found
> >> confusing
> >>>>> for the user is API managers. ProvenanceManager is the one used to
> >> get
> >>>>> experiment status. Users will get FINISHED or FAILED status from the
> >>> data
> >>>>> object. On failure, i was trying to find a method to get the error
> >> data
> >>> but
> >>>>> i was not able to find as those methods exist in ExecutionManager.
> >>>>> According to me, experiment status and in case of failures get error
> >>> need
> >>>>> to be part of one manager. May be a ExperimentManager? Getting
> output
> >>> data
> >>>>> can be left as part of provenance manages as only few client may want
> >> to
> >>>>> get the data in few cases. Getting error detail in case of error is
> >>> obvious
> >>>>> step.
> >>>>>
> >>>>> Other thing can you please explain the difference between
> >>>>> WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError,
> >>>>> GFacExecutionError on a Wiki page for the users. Also add a method to
> >>> get
> >>>>> JobID as that is required to get GFacExecutionError.
> >>>>>
> >>>>> I tested the WorkflowExecutionError method and its not returning any
> >>> data.
> >>>>> Can you please test that as well?
> >>>>>
> >>>>> ExperimentData data =
> >>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >>>>> List<WorkflowExecutionError> experimentExecutionErrors =
> >>>>>
> >>>
> >>
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >>>>> experimentID);
> >>>>>
> >>>>> Thanks
> >>>>> Raminder
> >>>>>
> >>>>>
> >>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
> >>>>>
> >>>>>> Following updates will be present Airavata API from 0.8 release.
> >>>>>>
> >>>>>>
> >>>>>> 1. *Error message from the immediate exception used as the error
> >>>>> message
> >>>>>> for the AiravataAPInvocationException*. Earlier it was just "Error
> >>>>>> invoking API". *This change was made so that the users of the API
> >> can
> >>>>>> show the error from the exception without having to dig in to the
> >>> inner
> >>>>>> exception to get a better error message.*
> >>>>>> 2. *Saving and retrieving experiment execution error details*.
> >> Earlier
> >>>>>> we were not persisting any execution error details of the workflow.
> >>>>> Error
> >>>>>> details are only available only if the developers are monitoring
> >>>>> experiment
> >>>>>> at the time of error occurs. *This is not feasible if users would
> >> only
> >>>>>> want to go through the error details later without doing monitoring
> >>>>> from
> >>>>>> the beginning. Therefore we are now persisting the error details
> >> (not
> >>>>> as
> >>>>>> part of provenance data) of executing workflows (i.e. experiments)*
> >> in
> >>>>>> to the registry and also retrieving from registry through Airavata
> >> API
> >>>>>> (which calls the Registry API).
> >>>>>>
> >>>>>> For now we have the functions for saving and retrieving errors in
> the
> >>>>>> ExecutionManager[1]. We need to decide if this is the correct place
> >> to
> >>>>> put
> >>>>>> these functions. Your thoughts in this matter are most welcome...
> >>>>>>
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Saminda
> >>>>>>
> >>>>>> 1.
> >>>>>>
> >>>>>
> >>>
> >>
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>>>>
> >>>>>
> >>>
> >>>
> >>
>
>
Re: Updates to Airavata API version 0.8
Posted by Raminder Singh <ra...@gmail.com>.
Thanks Chathuri, I will test and let you know.
Raman
On May 21, 2013, at 4:55 PM, Chathuri Wimalasena wrote:
> Hi Raman,
>
> The method Saminda mentioned will work for your case. I tested with your
> test class. I fixed deserialization issues from REST service side. You will
> probably have to update both server and client.
>
> Regards,
> Chathuri
>
>
> On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <sa...@gmail.com>wrote:
>
>> So far what you are saying is true. If the client knows which node failed
>> he/she can get the list of errors for that node using the
>> getExecutionError(...) function. Use the "type" as ALL and leave the
>> gfacJobId as null. I couldn;t think of doing this any other better way than
>> introducing a whole lot of functions which could be more annoying/confusing
>> to figure-out. I'm welcome for suggestions for improvements.
>>
>> Saminda
>>
>>
>> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <raminderjsingh@gmail.com
>>> wrote:
>>
>>> Thanks Chathuri, the WorkflowExecutionError works. Integrating to Error
>>> API brought few questions. We have ErrorTypes as :
>> WorkflowExecutionError,
>>> ExperimentExecutionError, NodeExecutionError, GFacJobExecutionError,
>>> ExecutionError . How do we want clients to know which error type to
>> query?
>>> I ran a job and it failed, now i want to get error. Error occurred at
>> GFac
>>> provider level and client does not have any information. Do we expect
>>> client to query all different types?
>>>
>>> I thing we can have is a method to get all error and return the Error
>> type
>>> part of the object to help the client to classify the error location.
>>> Other suggestions?
>>>
>>> Raminder
>>>
>>> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
>>>
>>>> Hi Raman,
>>>>
>>>> WorkflowExecutionError method not returning any data issue should be
>>> fixed
>>>> now. Can you get an update and check.
>>>>
>>>> Regards,
>>>> Chathuri
>>>>
>>>>
>>>> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
>>>> <ra...@gmail.com>wrote:
>>>>
>>>>> Thank, these are very useful for API users. Only thing i found
>> confusing
>>>>> for the user is API managers. ProvenanceManager is the one used to
>> get
>>>>> experiment status. Users will get FINISHED or FAILED status from the
>>> data
>>>>> object. On failure, i was trying to find a method to get the error
>> data
>>> but
>>>>> i was not able to find as those methods exist in ExecutionManager.
>>>>> According to me, experiment status and in case of failures get error
>>> need
>>>>> to be part of one manager. May be a ExperimentManager? Getting output
>>> data
>>>>> can be left as part of provenance manages as only few client may want
>> to
>>>>> get the data in few cases. Getting error detail in case of error is
>>> obvious
>>>>> step.
>>>>>
>>>>> Other thing can you please explain the difference between
>>>>> WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError,
>>>>> GFacExecutionError on a Wiki page for the users. Also add a method to
>>> get
>>>>> JobID as that is required to get GFacExecutionError.
>>>>>
>>>>> I tested the WorkflowExecutionError method and its not returning any
>>> data.
>>>>> Can you please test that as well?
>>>>>
>>>>> ExperimentData data =
>>>>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>>>>> List<WorkflowExecutionError> experimentExecutionErrors =
>>>>>
>>>
>> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>>>>> experimentID);
>>>>>
>>>>> Thanks
>>>>> Raminder
>>>>>
>>>>>
>>>>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
>>>>>
>>>>>> Following updates will be present Airavata API from 0.8 release.
>>>>>>
>>>>>>
>>>>>> 1. *Error message from the immediate exception used as the error
>>>>> message
>>>>>> for the AiravataAPInvocationException*. Earlier it was just "Error
>>>>>> invoking API". *This change was made so that the users of the API
>> can
>>>>>> show the error from the exception without having to dig in to the
>>> inner
>>>>>> exception to get a better error message.*
>>>>>> 2. *Saving and retrieving experiment execution error details*.
>> Earlier
>>>>>> we were not persisting any execution error details of the workflow.
>>>>> Error
>>>>>> details are only available only if the developers are monitoring
>>>>> experiment
>>>>>> at the time of error occurs. *This is not feasible if users would
>> only
>>>>>> want to go through the error details later without doing monitoring
>>>>> from
>>>>>> the beginning. Therefore we are now persisting the error details
>> (not
>>>>> as
>>>>>> part of provenance data) of executing workflows (i.e. experiments)*
>> in
>>>>>> to the registry and also retrieving from registry through Airavata
>> API
>>>>>> (which calls the Registry API).
>>>>>>
>>>>>> For now we have the functions for saving and retrieving errors in the
>>>>>> ExecutionManager[1]. We need to decide if this is the correct place
>> to
>>>>> put
>>>>>> these functions. Your thoughts in this matter are most welcome...
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Saminda
>>>>>>
>>>>>> 1.
>>>>>>
>>>>>
>>>
>> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>>>>>
>>>>>
>>>
>>>
>>
Re: Updates to Airavata API version 0.8
Posted by Chathuri Wimalasena <ka...@gmail.com>.
Hi Raman,
The method Saminda mentioned will work for your case. I tested with your
test class. I fixed deserialization issues from REST service side. You will
probably have to update both server and client.
Regards,
Chathuri
On Tue, May 21, 2013 at 3:04 PM, Saminda Wijeratne <sa...@gmail.com>wrote:
> So far what you are saying is true. If the client knows which node failed
> he/she can get the list of errors for that node using the
> getExecutionError(...) function. Use the "type" as ALL and leave the
> gfacJobId as null. I couldn;t think of doing this any other better way than
> introducing a whole lot of functions which could be more annoying/confusing
> to figure-out. I'm welcome for suggestions for improvements.
>
> Saminda
>
>
> On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <raminderjsingh@gmail.com
> >wrote:
>
> > Thanks Chathuri, the WorkflowExecutionError works. Integrating to Error
> > API brought few questions. We have ErrorTypes as :
> WorkflowExecutionError,
> > ExperimentExecutionError, NodeExecutionError, GFacJobExecutionError,
> > ExecutionError . How do we want clients to know which error type to
> query?
> > I ran a job and it failed, now i want to get error. Error occurred at
> GFac
> > provider level and client does not have any information. Do we expect
> > client to query all different types?
> >
> > I thing we can have is a method to get all error and return the Error
> type
> > part of the object to help the client to classify the error location.
> > Other suggestions?
> >
> > Raminder
> >
> > On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
> >
> > > Hi Raman,
> > >
> > > WorkflowExecutionError method not returning any data issue should be
> > fixed
> > > now. Can you get an update and check.
> > >
> > > Regards,
> > > Chathuri
> > >
> > >
> > > On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> > > <ra...@gmail.com>wrote:
> > >
> > >> Thank, these are very useful for API users. Only thing i found
> confusing
> > >> for the user is API managers. ProvenanceManager is the one used to
> get
> > >> experiment status. Users will get FINISHED or FAILED status from the
> > data
> > >> object. On failure, i was trying to find a method to get the error
> data
> > but
> > >> i was not able to find as those methods exist in ExecutionManager.
> > >> According to me, experiment status and in case of failures get error
> > need
> > >> to be part of one manager. May be a ExperimentManager? Getting output
> > data
> > >> can be left as part of provenance manages as only few client may want
> to
> > >> get the data in few cases. Getting error detail in case of error is
> > obvious
> > >> step.
> > >>
> > >> Other thing can you please explain the difference between
> > >> WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError,
> > >> GFacExecutionError on a Wiki page for the users. Also add a method to
> > get
> > >> JobID as that is required to get GFacExecutionError.
> > >>
> > >> I tested the WorkflowExecutionError method and its not returning any
> > data.
> > >> Can you please test that as well?
> > >>
> > >> ExperimentData data =
> > >> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> > >> List<WorkflowExecutionError> experimentExecutionErrors =
> > >>
> >
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> > >> experimentID);
> > >>
> > >> Thanks
> > >> Raminder
> > >>
> > >>
> > >> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
> > >>
> > >>> Following updates will be present Airavata API from 0.8 release.
> > >>>
> > >>>
> > >>> 1. *Error message from the immediate exception used as the error
> > >> message
> > >>> for the AiravataAPInvocationException*. Earlier it was just "Error
> > >>> invoking API". *This change was made so that the users of the API
> can
> > >>> show the error from the exception without having to dig in to the
> > inner
> > >>> exception to get a better error message.*
> > >>> 2. *Saving and retrieving experiment execution error details*.
> Earlier
> > >>> we were not persisting any execution error details of the workflow.
> > >> Error
> > >>> details are only available only if the developers are monitoring
> > >> experiment
> > >>> at the time of error occurs. *This is not feasible if users would
> only
> > >>> want to go through the error details later without doing monitoring
> > >> from
> > >>> the beginning. Therefore we are now persisting the error details
> (not
> > >> as
> > >>> part of provenance data) of executing workflows (i.e. experiments)*
> in
> > >>> to the registry and also retrieving from registry through Airavata
> API
> > >>> (which calls the Registry API).
> > >>>
> > >>> For now we have the functions for saving and retrieving errors in the
> > >>> ExecutionManager[1]. We need to decide if this is the correct place
> to
> > >> put
> > >>> these functions. Your thoughts in this matter are most welcome...
> > >>>
> > >>>
> > >>> Thanks,
> > >>> Saminda
> > >>>
> > >>> 1.
> > >>>
> > >>
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> > >>
> > >>
> >
> >
>
Re: Updates to Airavata API version 0.8
Posted by Saminda Wijeratne <sa...@gmail.com>.
So far what you are saying is true. If the client knows which node failed
he/she can get the list of errors for that node using the
getExecutionError(...) function. Use the "type" as ALL and leave the
gfacJobId as null. I couldn;t think of doing this any other better way than
introducing a whole lot of functions which could be more annoying/confusing
to figure-out. I'm welcome for suggestions for improvements.
Saminda
On Tue, May 21, 2013 at 2:36 PM, Raminder Singh <ra...@gmail.com>wrote:
> Thanks Chathuri, the WorkflowExecutionError works. Integrating to Error
> API brought few questions. We have ErrorTypes as : WorkflowExecutionError,
> ExperimentExecutionError, NodeExecutionError, GFacJobExecutionError,
> ExecutionError . How do we want clients to know which error type to query?
> I ran a job and it failed, now i want to get error. Error occurred at GFac
> provider level and client does not have any information. Do we expect
> client to query all different types?
>
> I thing we can have is a method to get all error and return the Error type
> part of the object to help the client to classify the error location.
> Other suggestions?
>
> Raminder
>
> On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
>
> > Hi Raman,
> >
> > WorkflowExecutionError method not returning any data issue should be
> fixed
> > now. Can you get an update and check.
> >
> > Regards,
> > Chathuri
> >
> >
> > On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> > <ra...@gmail.com>wrote:
> >
> >> Thank, these are very useful for API users. Only thing i found confusing
> >> for the user is API managers. ProvenanceManager is the one used to get
> >> experiment status. Users will get FINISHED or FAILED status from the
> data
> >> object. On failure, i was trying to find a method to get the error data
> but
> >> i was not able to find as those methods exist in ExecutionManager.
> >> According to me, experiment status and in case of failures get error
> need
> >> to be part of one manager. May be a ExperimentManager? Getting output
> data
> >> can be left as part of provenance manages as only few client may want to
> >> get the data in few cases. Getting error detail in case of error is
> obvious
> >> step.
> >>
> >> Other thing can you please explain the difference between
> >> WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError,
> >> GFacExecutionError on a Wiki page for the users. Also add a method to
> get
> >> JobID as that is required to get GFacExecutionError.
> >>
> >> I tested the WorkflowExecutionError method and its not returning any
> data.
> >> Can you please test that as well?
> >>
> >> ExperimentData data =
> >> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> >> List<WorkflowExecutionError> experimentExecutionErrors =
> >>
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> >> experimentID);
> >>
> >> Thanks
> >> Raminder
> >>
> >>
> >> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
> >>
> >>> Following updates will be present Airavata API from 0.8 release.
> >>>
> >>>
> >>> 1. *Error message from the immediate exception used as the error
> >> message
> >>> for the AiravataAPInvocationException*. Earlier it was just "Error
> >>> invoking API". *This change was made so that the users of the API can
> >>> show the error from the exception without having to dig in to the
> inner
> >>> exception to get a better error message.*
> >>> 2. *Saving and retrieving experiment execution error details*. Earlier
> >>> we were not persisting any execution error details of the workflow.
> >> Error
> >>> details are only available only if the developers are monitoring
> >> experiment
> >>> at the time of error occurs. *This is not feasible if users would only
> >>> want to go through the error details later without doing monitoring
> >> from
> >>> the beginning. Therefore we are now persisting the error details (not
> >> as
> >>> part of provenance data) of executing workflows (i.e. experiments)* in
> >>> to the registry and also retrieving from registry through Airavata API
> >>> (which calls the Registry API).
> >>>
> >>> For now we have the functions for saving and retrieving errors in the
> >>> ExecutionManager[1]. We need to decide if this is the correct place to
> >> put
> >>> these functions. Your thoughts in this matter are most welcome...
> >>>
> >>>
> >>> Thanks,
> >>> Saminda
> >>>
> >>> 1.
> >>>
> >>
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
> >>
> >>
>
>
Re: Updates to Airavata API version 0.8
Posted by Raminder Singh <ra...@gmail.com>.
Thanks Chathuri, the WorkflowExecutionError works. Integrating to Error API brought few questions. We have ErrorTypes as : WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError, GFacJobExecutionError, ExecutionError . How do we want clients to know which error type to query? I ran a job and it failed, now i want to get error. Error occurred at GFac provider level and client does not have any information. Do we expect client to query all different types?
I thing we can have is a method to get all error and return the Error type part of the object to help the client to classify the error location. Other suggestions?
Raminder
On May 21, 2013, at 10:57 AM, Chathuri Wimalasena wrote:
> Hi Raman,
>
> WorkflowExecutionError method not returning any data issue should be fixed
> now. Can you get an update and check.
>
> Regards,
> Chathuri
>
>
> On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
> <ra...@gmail.com>wrote:
>
>> Thank, these are very useful for API users. Only thing i found confusing
>> for the user is API managers. ProvenanceManager is the one used to get
>> experiment status. Users will get FINISHED or FAILED status from the data
>> object. On failure, i was trying to find a method to get the error data but
>> i was not able to find as those methods exist in ExecutionManager.
>> According to me, experiment status and in case of failures get error need
>> to be part of one manager. May be a ExperimentManager? Getting output data
>> can be left as part of provenance manages as only few client may want to
>> get the data in few cases. Getting error detail in case of error is obvious
>> step.
>>
>> Other thing can you please explain the difference between
>> WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError,
>> GFacExecutionError on a Wiki page for the users. Also add a method to get
>> JobID as that is required to get GFacExecutionError.
>>
>> I tested the WorkflowExecutionError method and its not returning any data.
>> Can you please test that as well?
>>
>> ExperimentData data =
>> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
>> List<WorkflowExecutionError> experimentExecutionErrors =
>> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
>> experimentID);
>>
>> Thanks
>> Raminder
>>
>>
>> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
>>
>>> Following updates will be present Airavata API from 0.8 release.
>>>
>>>
>>> 1. *Error message from the immediate exception used as the error
>> message
>>> for the AiravataAPInvocationException*. Earlier it was just "Error
>>> invoking API". *This change was made so that the users of the API can
>>> show the error from the exception without having to dig in to the inner
>>> exception to get a better error message.*
>>> 2. *Saving and retrieving experiment execution error details*. Earlier
>>> we were not persisting any execution error details of the workflow.
>> Error
>>> details are only available only if the developers are monitoring
>> experiment
>>> at the time of error occurs. *This is not feasible if users would only
>>> want to go through the error details later without doing monitoring
>> from
>>> the beginning. Therefore we are now persisting the error details (not
>> as
>>> part of provenance data) of executing workflows (i.e. experiments)* in
>>> to the registry and also retrieving from registry through Airavata API
>>> (which calls the Registry API).
>>>
>>> For now we have the functions for saving and retrieving errors in the
>>> ExecutionManager[1]. We need to decide if this is the correct place to
>> put
>>> these functions. Your thoughts in this matter are most welcome...
>>>
>>>
>>> Thanks,
>>> Saminda
>>>
>>> 1.
>>>
>> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>>
>>
Re: Updates to Airavata API version 0.8
Posted by Chathuri Wimalasena <ka...@gmail.com>.
Hi Raman,
WorkflowExecutionError method not returning any data issue should be fixed
now. Can you get an update and check.
Regards,
Chathuri
On Tue, May 21, 2013 at 10:02 AM, Raminder Singh
<ra...@gmail.com>wrote:
> Thank, these are very useful for API users. Only thing i found confusing
> for the user is API managers. ProvenanceManager is the one used to get
> experiment status. Users will get FINISHED or FAILED status from the data
> object. On failure, i was trying to find a method to get the error data but
> i was not able to find as those methods exist in ExecutionManager.
> According to me, experiment status and in case of failures get error need
> to be part of one manager. May be a ExperimentManager? Getting output data
> can be left as part of provenance manages as only few client may want to
> get the data in few cases. Getting error detail in case of error is obvious
> step.
>
> Other thing can you please explain the difference between
> WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError,
> GFacExecutionError on a Wiki page for the users. Also add a method to get
> JobID as that is required to get GFacExecutionError.
>
> I tested the WorkflowExecutionError method and its not returning any data.
> Can you please test that as well?
>
> ExperimentData data =
> airavataAPI.getProvenanceManager().getExperimentData(experimentID);
> List<WorkflowExecutionError> experimentExecutionErrors =
> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID,
> experimentID);
>
> Thanks
> Raminder
>
>
> On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
>
> > Following updates will be present Airavata API from 0.8 release.
> >
> >
> > 1. *Error message from the immediate exception used as the error
> message
> > for the AiravataAPInvocationException*. Earlier it was just "Error
> > invoking API". *This change was made so that the users of the API can
> > show the error from the exception without having to dig in to the inner
> > exception to get a better error message.*
> > 2. *Saving and retrieving experiment execution error details*. Earlier
> > we were not persisting any execution error details of the workflow.
> Error
> > details are only available only if the developers are monitoring
> experiment
> > at the time of error occurs. *This is not feasible if users would only
> > want to go through the error details later without doing monitoring
> from
> > the beginning. Therefore we are now persisting the error details (not
> as
> > part of provenance data) of executing workflows (i.e. experiments)* in
> > to the registry and also retrieving from registry through Airavata API
> > (which calls the Registry API).
> >
> > For now we have the functions for saving and retrieving errors in the
> > ExecutionManager[1]. We need to decide if this is the correct place to
> put
> > these functions. Your thoughts in this matter are most welcome...
> >
> >
> > Thanks,
> > Saminda
> >
> > 1.
> >
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java
>
>
Re: Updates to Airavata API version 0.8
Posted by Raminder Singh <ra...@gmail.com>.
Thank, these are very useful for API users. Only thing i found confusing for the user is API managers. ProvenanceManager is the one used to get experiment status. Users will get FINISHED or FAILED status from the data object. On failure, i was trying to find a method to get the error data but i was not able to find as those methods exist in ExecutionManager. According to me, experiment status and in case of failures get error need to be part of one manager. May be a ExperimentManager? Getting output data can be left as part of provenance manages as only few client may want to get the data in few cases. Getting error detail in case of error is obvious step.
Other thing can you please explain the difference between WorkflowExecutionError, ExperimentExecutionError, NodeExecutionError, GFacExecutionError on a Wiki page for the users. Also add a method to get JobID as that is required to get GFacExecutionError.
I tested the WorkflowExecutionError method and its not returning any data. Can you please test that as well?
ExperimentData data = airavataAPI.getProvenanceManager().getExperimentData(experimentID);
List<WorkflowExecutionError> experimentExecutionErrors = airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID, experimentID);
Thanks
Raminder
On May 20, 2013, at 5:05 PM, Saminda Wijeratne wrote:
> Following updates will be present Airavata API from 0.8 release.
>
>
> 1. *Error message from the immediate exception used as the error message
> for the AiravataAPInvocationException*. Earlier it was just "Error
> invoking API". *This change was made so that the users of the API can
> show the error from the exception without having to dig in to the inner
> exception to get a better error message.*
> 2. *Saving and retrieving experiment execution error details*. Earlier
> we were not persisting any execution error details of the workflow. Error
> details are only available only if the developers are monitoring experiment
> at the time of error occurs. *This is not feasible if users would only
> want to go through the error details later without doing monitoring from
> the beginning. Therefore we are now persisting the error details (not as
> part of provenance data) of executing workflows (i.e. experiments)* in
> to the registry and also retrieving from registry through Airavata API
> (which calls the Registry API).
>
> For now we have the functions for saving and retrieving errors in the
> ExecutionManager[1]. We need to decide if this is the correct place to put
> these functions. Your thoughts in this matter are most welcome...
>
>
> Thanks,
> Saminda
>
> 1.
> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java