You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Mallder, Valerie" <Va...@jhuapl.edu> on 2017/03/06 14:50:44 UTC
RE: Merge Avro RPC
Hi Lewis,
I hope you are doing well. I have been silent on oodt email for a while. But now I need to do more pipeline development work and I'm looking for resources to help me come up to speed with some of the changes that have been made over the last few months. Is the link below the correct place to look for the XMRPC to AvroRPC upgrade information? It doesn't really say anything, so if there is a different place I should be looking please let me know.
Thanks,
Val
-----Original Message-----
From: lewis john mcgibbney [mailto:lewismc@apache.org]
Sent: Friday, October 07, 2016 2:57 PM
To: dev@oodt.apache.org
Subject: Re: Merge Avro RPC
Hi Val,
I'm working on this at the link below and will post an update here once complete https://cwiki.apache.org/confluence/display/OODT/OODT+XMLRPC+--%3E+AvroRPC+Upgrade
Thanks
On Thu, Oct 6, 2016 at 8:02 AM, lewis john mcgibbney <le...@apache.org>
wrote:
> Hi Val,
> Yes when Radu and I worked on this code a year or two ago now we were
> very much aware of the upgrade issue.
> I am preparing myself to put a wiki page together which will fully
> document 1) why we have put time and effort into the XMLRP --> AvroRPC
> shift, 2) Which services are affected, and 3) How you can upgrade.
> On another note, I am also going to put together a wiki page for the
> various configuration files and what configuration options you need to use.
> It would be less than wise for us to leave people like yourself (and a
> significant amount of others I would imagine) in the dark about all of
> this so be rest assured that we will not be mandating an upgrade
> without the relevant support for your existing pipelines and the path to upgrade.
> Actually, in an ideal world we would probably get a blog post together
> for this at some stage and post to the foundation blog.
> Lewis
>
>
> On Thu, Oct 6, 2016 at 7:19 AM, Mallder, Valerie <
> Valerie.Mallder@jhuapl.edu> wrote:
>
>> Hi Guys,
>>
>> So what happens now for the folks like me who are using each of the
>> following:
>>
>> XmlRpcFileManager
>> XmlRpcFileManagerClient
>> XmlRpcResourceManager
>> XmlRpcResourceManagerClient
>> XmlRpcBatchStub
>> XmlRpcWorkflowManager
>> XmlRpcWorkflowManagerClient
>>
>> I have two science data pipelines, and working on a third, that use
>> these objects. Does this mean that all of my pipelines will have to
>> be frozen to the current version? Is there an easy way to change over to the Avro-RPC?
>> Will someone document all the steps you need to take to change over
>> to using it?
>>
>> Thanks,
>> Val
>>
>>
>>
>>
>> -----Original Message-----
>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>> Sent: Wednesday, October 05, 2016 5:35 PM
>> To: dev@oodt.apache.org
>> Subject: Re: Merge Avro RPC
>>
>> +1 to removing XML-RPC period, and just defaulting to Avro-RPC.
>>
>>
>>
>> —
>> Chris Mattmann
>> chris.mattmann@gmail.com
>>
>> On 10/5/16, 2:22 PM, "Radu Manole" <ma...@gmail.com> wrote:
>>
>> Hello, I I revisited every module where avrorpc was implemented
>> and where
>> xmlrpc is,
>> and marked every class where avro rpc is not implemented. What kind of
>> transition to avro-rpc should be implemented? The easiest way is
>> to remove
>> xml-rpc and give up on compatibility with old version. Or make
>> the client
>> compatible to connect to both xml-rpc or avro-rpc servers, for the
>> transition.
>>
>> 2016-09-09 19:23 GMT+03:00 lewis john mcgibbney <le...@apache.org>:
>>
>> > So what's the plan then? Where is work going into? Did we
>> define a roadmap?
>> > Maybe we are best working off of 2.x branch then.
>> >
>> > On Wednesday, September 7, 2016, Tom Barber <
>> tom.barber@meteorite.bi>
>> > wrote:
>> >
>> > > Be warned folks, don't forget we've merged into mainline and
>> have a
>> > > development branch for 2.0!
>> > >
>> > > On Wed, Sep 7, 2016 at 7:38 PM, lewis john mcgibbney <
>> lewismc@apache.org
>> > > <javascript:;>>
>> > > wrote:
>> > >
>> > > > Hi Radu,
>> > > > Yes this is great.
>> > > > I would suggest that you submit a patch to '@Deprecate' all
>> of the
>> > XMLRPC
>> > > > code for this current development drive.
>> > > > I also suggest that we make the AvroRPC material default
>> from now on
>> > and
>> > > > get more people using it.
>> > > > We can also make upgrades to Avro 1.8.X
>> > > > Would be great if others can chime in here to discuss the
>> path for a
>> > > > complete replacement of the XMLRPC logic with AvroRPC for
>> the next
>> > > release
>> > > > or two.
>> > > > Thanks
>> > > > Lewis
>> > > >
>> > > > P.S. @Radu, in the meantime please log an issue for
>> @Deprecat'ing all
>> > of
>> > > > the XMLRPC stuff. This should be clearly stated and present
>> within
>> > > > generated Javadoc such that people know to use the AvroRPC
>> > > implementation.
>> > > > Thanks
>> > > >
>> > > > On Mon, Sep 5, 2016 at 6:17 AM, Radu Manole <
>> manole.v.radu@gmail.com
>> > > <javascript:;>>
>> > > > wrote:
>> > > >
>> > > > > Hello, great to hear that avro rpc support is still
>> wanted, if there
>> > is
>> > > > > anything else to be done, I have a lot of free time now.
>> > > > >
>> > > > > 2016-07-06 17:09 GMT+03:00 Mattmann, Chris A (3980) <
>> > > > > chris.a.mattmann@jpl.nasa.gov <javascript:;>>:
>> > > > >
>> > > > > > +1 awesome
>> > > > > >
>> > > > > > ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++++++++
>> > > > > > Chris Mattmann, Ph.D.
>> > > > > > Chief Architect
>> > > > > > Instrument Software and Science Data Systems Section (398)
>> > > > > > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > > > > > Office: 168-519, Mailstop: 168-527
>> > > > > > Email: chris.a.mattmann@nasa.gov <javascript:;>
>> > > > > > WWW: http://sunset.usc.edu/~mattmann/
>> > > > > > ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++++++++
>> > > > > > Director, Information Retrieval and Data Science Group
>> (IRDS)
>> > > > > > Adjunct Associate Professor, Computer Science Department
>> > > > > > University of Southern California, Los Angeles, CA 90089 USA
>> > > > > > WWW: http://irds.usc.edu/
>> > > > > > ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++++++++
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On 7/6/16, 7:07 AM, "Tom Barber" <tom.barber@meteorite.bi
>> > > <javascript:;>> wrote:
>> > > > > >
>> > > > > > >Absolutely, all we've done is merge master into 2.x,
>> the plan was
>> > to
>> > > > > keep
>> > > > > > >that in sync, rename it or whatever and let people who
>> care play
>> > > > around.
>> > > > > > We
>> > > > > > >also then planned to merge it into master but leave
>> XML-RPC as the
>> > > > > default
>> > > > > > >for a release or two marked depreciated, as extra
>> security in case
>> > > it
>> > > > > > >affects anything else. Then at some point just switch
>> the defaults
>> > > in
>> > > > > the
>> > > > > > >configs and remove the XML-RPC crud.
>> > > > > > >
>> > > > > > >On Wed, Jul 6, 2016 at 3:04 PM, Mattmann, Chris A (3980) <
>> > > > > > >chris.a.mattmann@jpl.nasa.gov <javascript:;>> wrote:
>> > > > > > >
>> > > > > > >> Great work.
>> > > > > > >>
>> > > > > > >> Can we agree to create a 2.x branch and work forward
>> from that
>> > > > > > >> with Avro-RPC? 1.x can remain XML-RPC for a few
>> versions while
>> > > > > > >> I fix all lingering crap I found on the BigTranslate end?
>> > > > > > >>
>> > > > > > >> Cheers,
>> > > > > > >> Chris
>> > > > > > >>
>> > > > > > >> ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++
>> > > ++++++
>> > > > > > >> Chris Mattmann, Ph.D.
>> > > > > > >> Chief Architect
>> > > > > > >> Instrument Software and Science Data Systems Section
>> (398)
>> > > > > > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > > > > > >> Office: 168-519, Mailstop: 168-527
>> > > > > > >> Email: chris.a.mattmann@nasa.gov <javascript:;>
>> > > > > > >> WWW: http://sunset.usc.edu/~mattmann/
>> > > > > > >> ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++
>> > > ++++++
>> > > > > > >> Director, Information Retrieval and Data Science
>> Group
>> (IRDS)
>> > > > > > >> Adjunct Associate Professor, Computer Science Department
>> > > > > > >> University of Southern California, Los Angeles, CA
>> 90089 USA
>> > > > > > >> WWW: http://irds.usc.edu/
>> > > > > > >> ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++
>> > > ++++++
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >> On 7/6/16, 7:01 AM, "Tom Barber" <
>> tom.barber@meteorite.bi
>> > > <javascript:;>> wrote:
>> > > > > > >>
>> > > > > > >> >I'm making progress here, I updated a few libs
>> which fixed a
>> > > couple
>> > > > > of
>> > > > > > the
>> > > > > > >> >test failures I was seeing locally. Timezones are
>> once again
>> > > > kicking
>> > > > > me
>> > > > > > >> >whilst I'm down, for some reason OODT fails badly
>> with people
>> > in
>> > > > > > >> >GMT/BST.... I'll figure it out though.
>> > > > > > >> >
>> > > > > > >> >On Fri, Jul 1, 2016 at 12:36 AM, Lewis John Mcgibbney <
>> > > > > > >> >lewis.mcgibbney@gmail.com <javascript:;>> wrote:
>> > > > > > >> >
>> > > > > > >> >> Sound ttyl
>> > > > > > >> >> On Jun 30, 2016 4:32 PM, "Tom Barber" <
>> > tom.barber@meteorite.bi
>> > > <javascript:;>>
>> > > > > > wrote:
>> > > > > > >> >>
>> > > > > > >> >> > Not really, tomoz?
>> > > > > > >> >> >
>> > > > > > >> >> > On Fri, Jul 1, 2016 at 12:24 AM, Lewis John
>> Mcgibbney <
>> > > > > > >> >> > lewis.mcgibbney@gmail.com <javascript:;>> wrote:
>> > > > > > >> >> >
>> > > > > > >> >> > > Want to do a hangout right now?
>> > > > > > >> >> > >
>> > > > > > >> >> > > On Thu, Jun 30, 2016 at 4:17 PM, Tom Barber <
>> > > > > > >> tom.barber@meteorite.bi <javascript:;>>
>> > > > > > >> >> > > wrote:
>> > > > > > >> >> > >
>> > > > > > >> >> > > > Hey Lewis
>> > > > > > >> >> > > >
>> > > > > > >> >> > > > I have no idea which branch is current, I
>> merged master
>> > > > into
>> > > > > > >> avrorpc
>> > > > > > >> >> > and
>> > > > > > >> >> > > it
>> > > > > > >> >> > > > merged okay and built so I pushed it back.
>> > > > > > >> >> > > >
>> > > > > > >> >> > > > Is that the correct branch to be looking at
>> for Avro
>> > > > Support?
>> > > > > > Lets
>> > > > > > >> >> > figure
>> > > > > > >> >> > > > it all out and get it merged into Master.
>> > > > > > >> >> > > >
>> > > > > > >> >> > > > Tom
>> > > > > > >> >> > > >
>> > > > > > >> >> > >
>> > > > > > >> >> > >
>> > > > > > >> >> > >
>> > > > > > >> >> > > --
>> > > > > > >> >> > > *Lewis*
>> > > > > > >> >> > >
>> > > > > > >> >> >
>> > > > > > >> >>
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > http://home.apache.org/~lewismc/
>> > > > @hectorMcSpector
>> > > > http://www.linkedin.com/in/lmcgibbney
>> > > >
>> > >
>> >
>> >
>> > --
>> > http://home.apache.org/~lewismc/
>> > @hectorMcSpector
>> > http://www.linkedin.com/in/lmcgibbney
>> >
>>
>>
>>
>>
>
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>
--
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney
RE: How to convert XMLRPC to AvroRPC??
Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.
Hi Lewis, oh no worries. I am trying to catch up on email too. I see you've sent a couple of other emails I will try to answer as well. Thanks for your attention to this. I know how it feels to be pulled in multiple directions at work and be pressed for time. Same happens with me all the time.
Val
Sent with BlackBerry Work (www.blackberry.com)
From: lewis john mcgibbney <le...@apache.org>>
Date: Tuesday, Mar 07, 2017, 8:06 PM
To: dev@oodt.apache.org <de...@oodt.apache.org>>
Subject: Re: How to convert XMLRPC to AvroRPC??
Hi Val,
I was going through my email, sorry for late reponse.
On Tue, Mar 7, 2017 at 4:40 PM, Mallder, Valerie <Valerie.Mallder@jhuapl.edu
> wrote:
> Hi All,
>
> Is there any documention on how to convert a XMLRPC-based OODT Pipeline to
> an AvroRPC-based OODT Pipeline? (See email thread below).
>
Right now no. I actually created a wiki page at
https://cwiki.apache.org/confluence/display/OODT/OODT+XMLRPC+--%3E+AvroRPC+Upgrade
I'll have a go at augmenting it maybe tonight and will write back here. I
am using a new RADiX deployment for a new project and this is as good a
time as any to augment the documentation.
>
> I am beginning work on a 3rd pipeline and I need to decide if I should
> abandon OODT altogether or upgrade from version 1.1 to the latest version.
> I definitely need some documention on how to do this conversion.
> Unfortunately, my scientists and project manager don't have the funds for
> me to spend a lot of time fumbling through it and figurring everything out
> on my own. :(
>
>
Yes I get it. I'll try to help you out as much as I can. Apologies, my
previous project got derailed and I didn't get around to augmenting the
documentation.
Lewis
--
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney
Re: How to convert XMLRPC to AvroRPC??
Posted by lewis john mcgibbney <le...@apache.org>.
Hi Val,
I was going through my email, sorry for late reponse.
On Tue, Mar 7, 2017 at 4:40 PM, Mallder, Valerie <Valerie.Mallder@jhuapl.edu
> wrote:
> Hi All,
>
> Is there any documention on how to convert a XMLRPC-based OODT Pipeline to
> an AvroRPC-based OODT Pipeline? (See email thread below).
>
Right now no. I actually created a wiki page at
https://cwiki.apache.org/confluence/display/OODT/OODT+XMLRPC+--%3E+AvroRPC+Upgrade
I'll have a go at augmenting it maybe tonight and will write back here. I
am using a new RADiX deployment for a new project and this is as good a
time as any to augment the documentation.
>
> I am beginning work on a 3rd pipeline and I need to decide if I should
> abandon OODT altogether or upgrade from version 1.1 to the latest version.
> I definitely need some documention on how to do this conversion.
> Unfortunately, my scientists and project manager don't have the funds for
> me to spend a lot of time fumbling through it and figurring everything out
> on my own. :(
>
>
Yes I get it. I'll try to help you out as much as I can. Apologies, my
previous project got derailed and I didn't get around to augmenting the
documentation.
Lewis
--
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney
RE: How to convert XMLRPC to AvroRPC??
Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.
Hi Lewis
No problem here either. I haven't worked with anything new with OODT since before the switch over to git so I don't totally understand the magnitude of the task you got yourself into, but I can imagine it requires tremendous attention to detail to do it correctly. So take your time.
I need to spend some familiarizing myself with all that has changed since the last time i checked out anything from the repository.
I am currently working with two non-radix imstantiations of OODT. One uses workflow manager only, and the other uses workflow manager, filemgr, and resource manager, and we tried to use the opsui web application but I haven't had time to do any customizations on it so it's not really that helpful to us at the moment. I really could get away with using just workflow manager with that one too, but we are interested we're interested in the opsui so I am running with resource manage because it was required in order to use opsui.
For my new pipeline, I am going to be interested to know if any new development work has been done on opsui in the newest version of OODT (we are still interested in building a dashboard) and to see how hard or easy it will be to switch over to the AvroRPC environment. I'm hoping to make a decision soon on how I want to proceed with the new pipeline which is for the EIS imstrument on the MMS mission.
Thanks,
Val
Sent with BlackBerry Work (www.blackberry.com)
From: lewis john mcgibbney <le...@apache.org>>
Date: Friday, Mar 10, 2017, 2:59 PM
To: dev@oodt.apache.org <de...@oodt.apache.org>>
Subject: Re: How to convert XMLRPC to AvroRPC??
Hi Val,
Please see my other thread on this topic. I'm going to work on tests again
tonight and hopefully stabilize them.
Once that's done, then I'll come back here with the documentation.
Lewis
On Tue, Mar 7, 2017 at 4:40 PM, Mallder, Valerie <Valerie.Mallder@jhuapl.edu
> wrote:
> Hi All,
>
> Is there any documention on how to convert a XMLRPC-based OODT Pipeline to
> an AvroRPC-based OODT Pipeline? (See email thread below).
>
> I am beginning work on a 3rd pipeline and I need to decide if I should
> abandon OODT altogether or upgrade from version 1.1 to the latest version.
> I definitely need some documention on how to do this conversion.
> Unfortunately, my scientists and project manager don't have the funds for
> me to spend a lot of time fumbling through it and figurring everything out
> on my own. :(
>
> Thanks,
> Val
>
>
>
> Sent with BlackBerry Work (www.blackberry.com<http://www.blackberry.com>)
>
> From: Mallder, Valerie <Valerie.Mallder@jhuapl.edu<mailto:
> Valerie.Mallder@jhuapl.edu>>
> Date: Monday, Mar 06, 2017, 9:50 AM
> To: dev@oodt.apache.org <de...@oodt.apache.org>>
> Subject: RE: Merge Avro RPC
>
> Hi Lewis,
>
> I hope you are doing well. I have been silent on oodt email for a while.
> But now I need to do more pipeline development work and I'm looking for
> resources to help me come up to speed with some of the changes that have
> been made over the last few months. Is the link below the correct place to
> look for the XMRPC to AvroRPC upgrade information? It doesn't really say
> anything, so if there is a different place I should be looking please let
> me know.
>
> Thanks,
> Val
>
>
> -----Original Message-----
> From: lewis john mcgibbney [mailto:lewismc@apache.org]
> Sent: Friday, October 07, 2016 2:57 PM
> To: dev@oodt.apache.org
> Subject: Re: Merge Avro RPC
>
> Hi Val,
> I'm working on this at the link below and will post an update here once
> complete https://cwiki.apache.org/confluence/display/OODT/OODT+
> XMLRPC+--%3E+AvroRPC+Upgrade
> Thanks
>
>
> On Thu, Oct 6, 2016 at 8:02 AM, lewis john mcgibbney <le...@apache.org>
> wrote:
>
> > Hi Val,
> > Yes when Radu and I worked on this code a year or two ago now we were
> > very much aware of the upgrade issue.
> > I am preparing myself to put a wiki page together which will fully
> > document 1) why we have put time and effort into the XMLRP --> AvroRPC
> > shift, 2) Which services are affected, and 3) How you can upgrade.
> > On another note, I am also going to put together a wiki page for the
> > various configuration files and what configuration options you need to
> use.
> > It would be less than wise for us to leave people like yourself (and a
> > significant amount of others I would imagine) in the dark about all of
> > this so be rest assured that we will not be mandating an upgrade
> > without the relevant support for your existing pipelines and the path to
> upgrade.
> > Actually, in an ideal world we would probably get a blog post together
> > for this at some stage and post to the foundation blog.
> > Lewis
> >
> >
> > On Thu, Oct 6, 2016 at 7:19 AM, Mallder, Valerie <
> > Valerie.Mallder@jhuapl.edu> wrote:
> >
> >> Hi Guys,
> >>
> >> So what happens now for the folks like me who are using each of the
> >> following:
> >>
> >> XmlRpcFileManager
> >> XmlRpcFileManagerClient
> >> XmlRpcResourceManager
> >> XmlRpcResourceManagerClient
> >> XmlRpcBatchStub
> >> XmlRpcWorkflowManager
> >> XmlRpcWorkflowManagerClient
> >>
> >> I have two science data pipelines, and working on a third, that use
> >> these objects. Does this mean that all of my pipelines will have to
> >> be frozen to the current version? Is there an easy way to change over
> to the Avro-RPC?
> >> Will someone document all the steps you need to take to change over
> >> to using it?
> >>
> >> Thanks,
> >> Val
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
> >> Sent: Wednesday, October 05, 2016 5:35 PM
> >> To: dev@oodt.apache.org
> >> Subject: Re: Merge Avro RPC
> >>
> >> +1 to removing XML-RPC period, and just defaulting to Avro-RPC.
> >>
> >>
> >>
> >> —
> >> Chris Mattmann
> >> chris.mattmann@gmail.com
> >>
> >> On 10/5/16, 2:22 PM, "Radu Manole" <ma...@gmail.com> wrote:
> >>
> >> Hello, I I revisited every module where avrorpc was implemented
> >> and where
> >> xmlrpc is,
> >> and marked every class where avro rpc is not implemented. What kind
> of
> >> transition to avro-rpc should be implemented? The easiest way is
> >> to remove
> >> xml-rpc and give up on compatibility with old version. Or make
> >> the client
> >> compatible to connect to both xml-rpc or avro-rpc servers, for the
> >> transition.
> >>
> >> 2016-09-09 19:23 GMT+03:00 lewis john mcgibbney <lewismc@apache.org
> >:
> >>
> >> > So what's the plan then? Where is work going into? Did we
> >> define a roadmap?
> >> > Maybe we are best working off of 2.x branch then.
> >> >
> >> > On Wednesday, September 7, 2016, Tom Barber <
> >> tom.barber@meteorite.bi>
> >> > wrote:
> >> >
> >> > > Be warned folks, don't forget we've merged into mainline and
> >> have a
> >> > > development branch for 2.0!
> >> > >
> >> > > On Wed, Sep 7, 2016 at 7:38 PM, lewis john mcgibbney <
> >> lewismc@apache.org
> >> > > <javascript:;>>
> >> > > wrote:
> >> > >
> >> > > > Hi Radu,
> >> > > > Yes this is great.
> >> > > > I would suggest that you submit a patch to '@Deprecate' all
> >> of the
> >> > XMLRPC
> >> > > > code for this current development drive.
> >> > > > I also suggest that we make the AvroRPC material default
> >> from now on
> >> > and
> >> > > > get more people using it.
> >> > > > We can also make upgrades to Avro 1.8.X
> >> > > > Would be great if others can chime in here to discuss the
> >> path for a
> >> > > > complete replacement of the XMLRPC logic with AvroRPC for
> >> the next
> >> > > release
> >> > > > or two.
> >> > > > Thanks
> >> > > > Lewis
> >> > > >
> >> > > > P.S. @Radu, in the meantime please log an issue for
> >> @Deprecat'ing all
> >> > of
> >> > > > the XMLRPC stuff. This should be clearly stated and present
> >> within
> >> > > > generated Javadoc such that people know to use the AvroRPC
> >> > > implementation.
> >> > > > Thanks
> >> > > >
> >> > > > On Mon, Sep 5, 2016 at 6:17 AM, Radu Manole <
> >> manole.v.radu@gmail.com
> >> > > <javascript:;>>
> >> > > > wrote:
> >> > > >
> >> > > > > Hello, great to hear that avro rpc support is still
> >> wanted, if there
> >> > is
> >> > > > > anything else to be done, I have a lot of free time now.
> >> > > > >
> >> > > > > 2016-07-06 17:09 GMT+03:00 Mattmann, Chris A (3980) <
> >> > > > > chris.a.mattmann@jpl.nasa.gov <javascript:;>>:
> >> > > > >
> >> > > > > > +1 awesome
> >> > > > > >
> >> > > > > > ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++++++++
> >> > > > > > Chris Mattmann, Ph.D.
> >> > > > > > Chief Architect
> >> > > > > > Instrument Software and Science Data Systems Section (398)
> >> > > > > > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> > > > > > Office: 168-519, Mailstop: 168-527
> >> > > > > > Email: chris.a.mattmann@nasa.gov <javascript:;>
> >> > > > > > WWW: http://sunset.usc.edu/~mattmann/
> >> > > > > > ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++++++++
> >> > > > > > Director, Information Retrieval and Data Science Group
> >> (IRDS)
> >> > > > > > Adjunct Associate Professor, Computer Science Department
> >> > > > > > University of Southern California, Los Angeles, CA 90089
> USA
> >> > > > > > WWW: http://irds.usc.edu/
> >> > > > > > ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++++++++
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On 7/6/16, 7:07 AM, "Tom Barber" <tom.barber@meteorite.bi
> >> > > <javascript:;>> wrote:
> >> > > > > >
> >> > > > > > >Absolutely, all we've done is merge master into 2.x,
> >> the plan was
> >> > to
> >> > > > > keep
> >> > > > > > >that in sync, rename it or whatever and let people who
> >> care play
> >> > > > around.
> >> > > > > > We
> >> > > > > > >also then planned to merge it into master but leave
> >> XML-RPC as the
> >> > > > > default
> >> > > > > > >for a release or two marked depreciated, as extra
> >> security in case
> >> > > it
> >> > > > > > >affects anything else. Then at some point just switch
> >> the defaults
> >> > > in
> >> > > > > the
> >> > > > > > >configs and remove the XML-RPC crud.
> >> > > > > > >
> >> > > > > > >On Wed, Jul 6, 2016 at 3:04 PM, Mattmann, Chris A (3980)
> <
> >> > > > > > >chris.a.mattmann@jpl.nasa.gov <javascript:;>> wrote:
> >> > > > > > >
> >> > > > > > >> Great work.
> >> > > > > > >>
> >> > > > > > >> Can we agree to create a 2.x branch and work forward
> >> from that
> >> > > > > > >> with Avro-RPC? 1.x can remain XML-RPC for a few
> >> versions while
> >> > > > > > >> I fix all lingering crap I found on the BigTranslate
> end?
> >> > > > > > >>
> >> > > > > > >> Cheers,
> >> > > > > > >> Chris
> >> > > > > > >>
> >> > > > > > >> ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++
> >> > > ++++++
> >> > > > > > >> Chris Mattmann, Ph.D.
> >> > > > > > >> Chief Architect
> >> > > > > > >> Instrument Software and Science Data Systems Section
> >> (398)
> >> > > > > > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> > > > > > >> Office: 168-519, Mailstop: 168-527
> >> > > > > > >> Email: chris.a.mattmann@nasa.gov <javascript:;>
> >> > > > > > >> WWW: http://sunset.usc.edu/~mattmann/
> >> > > > > > >> ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++
> >> > > ++++++
> >> > > > > > >> Director, Information Retrieval and Data Science
> >> Group
> >> (IRDS)
> >> > > > > > >> Adjunct Associate Professor, Computer Science
> Department
> >> > > > > > >> University of Southern California, Los Angeles, CA
> >> 90089 USA
> >> > > > > > >> WWW: http://irds.usc.edu/
> >> > > > > > >> ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++
> >> > > ++++++
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >> On 7/6/16, 7:01 AM, "Tom Barber" <
> >> tom.barber@meteorite.bi
> >> > > <javascript:;>> wrote:
> >> > > > > > >>
> >> > > > > > >> >I'm making progress here, I updated a few libs
> >> which fixed a
> >> > > couple
> >> > > > > of
> >> > > > > > the
> >> > > > > > >> >test failures I was seeing locally. Timezones are
> >> once again
> >> > > > kicking
> >> > > > > me
> >> > > > > > >> >whilst I'm down, for some reason OODT fails badly
> >> with people
> >> > in
> >> > > > > > >> >GMT/BST.... I'll figure it out though.
> >> > > > > > >> >
> >> > > > > > >> >On Fri, Jul 1, 2016 at 12:36 AM, Lewis John Mcgibbney
> <
> >> > > > > > >> >lewis.mcgibbney@gmail.com <javascript:;>> wrote:
> >> > > > > > >> >
> >> > > > > > >> >> Sound ttyl
> >> > > > > > >> >> On Jun 30, 2016 4:32 PM, "Tom Barber" <
> >> > tom.barber@meteorite.bi
> >> > > <javascript:;>>
> >> > > > > > wrote:
> >> > > > > > >> >>
> >> > > > > > >> >> > Not really, tomoz?
> >> > > > > > >> >> >
> >> > > > > > >> >> > On Fri, Jul 1, 2016 at 12:24 AM, Lewis John
> >> Mcgibbney <
> >> > > > > > >> >> > lewis.mcgibbney@gmail.com <javascript:;>> wrote:
> >> > > > > > >> >> >
> >> > > > > > >> >> > > Want to do a hangout right now?
> >> > > > > > >> >> > >
> >> > > > > > >> >> > > On Thu, Jun 30, 2016 at 4:17 PM, Tom Barber <
> >> > > > > > >> tom.barber@meteorite.bi <javascript:;>>
> >> > > > > > >> >> > > wrote:
> >> > > > > > >> >> > >
> >> > > > > > >> >> > > > Hey Lewis
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > > > I have no idea which branch is current, I
> >> merged master
> >> > > > into
> >> > > > > > >> avrorpc
> >> > > > > > >> >> > and
> >> > > > > > >> >> > > it
> >> > > > > > >> >> > > > merged okay and built so I pushed it back.
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > > > Is that the correct branch to be looking at
> >> for Avro
> >> > > > Support?
> >> > > > > > Lets
> >> > > > > > >> >> > figure
> >> > > > > > >> >> > > > it all out and get it merged into Master.
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > > > Tom
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > >
> >> > > > > > >> >> > >
> >> > > > > > >> >> > >
> >> > > > > > >> >> > > --
> >> > > > > > >> >> > > *Lewis*
> >> > > > > > >> >> > >
> >> > > > > > >> >> >
> >> > > > > > >> >>
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > http://home.apache.org/~lewismc/
> >> > > > @hectorMcSpector
> >> > > > http://www.linkedin.com/in/lmcgibbney
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > http://home.apache.org/~lewismc/
> >> > @hectorMcSpector
> >> > http://www.linkedin.com/in/lmcgibbney
> >> >
> >>
> >>
> >>
> >>
> >
> >
> > --
> > http://home.apache.org/~lewismc/
> > @hectorMcSpector
> > http://www.linkedin.com/in/lmcgibbney
> >
>
>
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>
--
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney
Re: How to convert XMLRPC to AvroRPC??
Posted by lewis john mcgibbney <le...@apache.org>.
Hi Val,
Please see my other thread on this topic. I'm going to work on tests again
tonight and hopefully stabilize them.
Once that's done, then I'll come back here with the documentation.
Lewis
On Tue, Mar 7, 2017 at 4:40 PM, Mallder, Valerie <Valerie.Mallder@jhuapl.edu
> wrote:
> Hi All,
>
> Is there any documention on how to convert a XMLRPC-based OODT Pipeline to
> an AvroRPC-based OODT Pipeline? (See email thread below).
>
> I am beginning work on a 3rd pipeline and I need to decide if I should
> abandon OODT altogether or upgrade from version 1.1 to the latest version.
> I definitely need some documention on how to do this conversion.
> Unfortunately, my scientists and project manager don't have the funds for
> me to spend a lot of time fumbling through it and figurring everything out
> on my own. :(
>
> Thanks,
> Val
>
>
>
> Sent with BlackBerry Work (www.blackberry.com)
>
> From: Mallder, Valerie <Valerie.Mallder@jhuapl.edu<mailto:
> Valerie.Mallder@jhuapl.edu>>
> Date: Monday, Mar 06, 2017, 9:50 AM
> To: dev@oodt.apache.org <de...@oodt.apache.org>>
> Subject: RE: Merge Avro RPC
>
> Hi Lewis,
>
> I hope you are doing well. I have been silent on oodt email for a while.
> But now I need to do more pipeline development work and I'm looking for
> resources to help me come up to speed with some of the changes that have
> been made over the last few months. Is the link below the correct place to
> look for the XMRPC to AvroRPC upgrade information? It doesn't really say
> anything, so if there is a different place I should be looking please let
> me know.
>
> Thanks,
> Val
>
>
> -----Original Message-----
> From: lewis john mcgibbney [mailto:lewismc@apache.org]
> Sent: Friday, October 07, 2016 2:57 PM
> To: dev@oodt.apache.org
> Subject: Re: Merge Avro RPC
>
> Hi Val,
> I'm working on this at the link below and will post an update here once
> complete https://cwiki.apache.org/confluence/display/OODT/OODT+
> XMLRPC+--%3E+AvroRPC+Upgrade
> Thanks
>
>
> On Thu, Oct 6, 2016 at 8:02 AM, lewis john mcgibbney <le...@apache.org>
> wrote:
>
> > Hi Val,
> > Yes when Radu and I worked on this code a year or two ago now we were
> > very much aware of the upgrade issue.
> > I am preparing myself to put a wiki page together which will fully
> > document 1) why we have put time and effort into the XMLRP --> AvroRPC
> > shift, 2) Which services are affected, and 3) How you can upgrade.
> > On another note, I am also going to put together a wiki page for the
> > various configuration files and what configuration options you need to
> use.
> > It would be less than wise for us to leave people like yourself (and a
> > significant amount of others I would imagine) in the dark about all of
> > this so be rest assured that we will not be mandating an upgrade
> > without the relevant support for your existing pipelines and the path to
> upgrade.
> > Actually, in an ideal world we would probably get a blog post together
> > for this at some stage and post to the foundation blog.
> > Lewis
> >
> >
> > On Thu, Oct 6, 2016 at 7:19 AM, Mallder, Valerie <
> > Valerie.Mallder@jhuapl.edu> wrote:
> >
> >> Hi Guys,
> >>
> >> So what happens now for the folks like me who are using each of the
> >> following:
> >>
> >> XmlRpcFileManager
> >> XmlRpcFileManagerClient
> >> XmlRpcResourceManager
> >> XmlRpcResourceManagerClient
> >> XmlRpcBatchStub
> >> XmlRpcWorkflowManager
> >> XmlRpcWorkflowManagerClient
> >>
> >> I have two science data pipelines, and working on a third, that use
> >> these objects. Does this mean that all of my pipelines will have to
> >> be frozen to the current version? Is there an easy way to change over
> to the Avro-RPC?
> >> Will someone document all the steps you need to take to change over
> >> to using it?
> >>
> >> Thanks,
> >> Val
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
> >> Sent: Wednesday, October 05, 2016 5:35 PM
> >> To: dev@oodt.apache.org
> >> Subject: Re: Merge Avro RPC
> >>
> >> +1 to removing XML-RPC period, and just defaulting to Avro-RPC.
> >>
> >>
> >>
> >> —
> >> Chris Mattmann
> >> chris.mattmann@gmail.com
> >>
> >> On 10/5/16, 2:22 PM, "Radu Manole" <ma...@gmail.com> wrote:
> >>
> >> Hello, I I revisited every module where avrorpc was implemented
> >> and where
> >> xmlrpc is,
> >> and marked every class where avro rpc is not implemented. What kind
> of
> >> transition to avro-rpc should be implemented? The easiest way is
> >> to remove
> >> xml-rpc and give up on compatibility with old version. Or make
> >> the client
> >> compatible to connect to both xml-rpc or avro-rpc servers, for the
> >> transition.
> >>
> >> 2016-09-09 19:23 GMT+03:00 lewis john mcgibbney <lewismc@apache.org
> >:
> >>
> >> > So what's the plan then? Where is work going into? Did we
> >> define a roadmap?
> >> > Maybe we are best working off of 2.x branch then.
> >> >
> >> > On Wednesday, September 7, 2016, Tom Barber <
> >> tom.barber@meteorite.bi>
> >> > wrote:
> >> >
> >> > > Be warned folks, don't forget we've merged into mainline and
> >> have a
> >> > > development branch for 2.0!
> >> > >
> >> > > On Wed, Sep 7, 2016 at 7:38 PM, lewis john mcgibbney <
> >> lewismc@apache.org
> >> > > <javascript:;>>
> >> > > wrote:
> >> > >
> >> > > > Hi Radu,
> >> > > > Yes this is great.
> >> > > > I would suggest that you submit a patch to '@Deprecate' all
> >> of the
> >> > XMLRPC
> >> > > > code for this current development drive.
> >> > > > I also suggest that we make the AvroRPC material default
> >> from now on
> >> > and
> >> > > > get more people using it.
> >> > > > We can also make upgrades to Avro 1.8.X
> >> > > > Would be great if others can chime in here to discuss the
> >> path for a
> >> > > > complete replacement of the XMLRPC logic with AvroRPC for
> >> the next
> >> > > release
> >> > > > or two.
> >> > > > Thanks
> >> > > > Lewis
> >> > > >
> >> > > > P.S. @Radu, in the meantime please log an issue for
> >> @Deprecat'ing all
> >> > of
> >> > > > the XMLRPC stuff. This should be clearly stated and present
> >> within
> >> > > > generated Javadoc such that people know to use the AvroRPC
> >> > > implementation.
> >> > > > Thanks
> >> > > >
> >> > > > On Mon, Sep 5, 2016 at 6:17 AM, Radu Manole <
> >> manole.v.radu@gmail.com
> >> > > <javascript:;>>
> >> > > > wrote:
> >> > > >
> >> > > > > Hello, great to hear that avro rpc support is still
> >> wanted, if there
> >> > is
> >> > > > > anything else to be done, I have a lot of free time now.
> >> > > > >
> >> > > > > 2016-07-06 17:09 GMT+03:00 Mattmann, Chris A (3980) <
> >> > > > > chris.a.mattmann@jpl.nasa.gov <javascript:;>>:
> >> > > > >
> >> > > > > > +1 awesome
> >> > > > > >
> >> > > > > > ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++++++++
> >> > > > > > Chris Mattmann, Ph.D.
> >> > > > > > Chief Architect
> >> > > > > > Instrument Software and Science Data Systems Section (398)
> >> > > > > > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> > > > > > Office: 168-519, Mailstop: 168-527
> >> > > > > > Email: chris.a.mattmann@nasa.gov <javascript:;>
> >> > > > > > WWW: http://sunset.usc.edu/~mattmann/
> >> > > > > > ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++++++++
> >> > > > > > Director, Information Retrieval and Data Science Group
> >> (IRDS)
> >> > > > > > Adjunct Associate Professor, Computer Science Department
> >> > > > > > University of Southern California, Los Angeles, CA 90089
> USA
> >> > > > > > WWW: http://irds.usc.edu/
> >> > > > > > ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++++++++
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On 7/6/16, 7:07 AM, "Tom Barber" <tom.barber@meteorite.bi
> >> > > <javascript:;>> wrote:
> >> > > > > >
> >> > > > > > >Absolutely, all we've done is merge master into 2.x,
> >> the plan was
> >> > to
> >> > > > > keep
> >> > > > > > >that in sync, rename it or whatever and let people who
> >> care play
> >> > > > around.
> >> > > > > > We
> >> > > > > > >also then planned to merge it into master but leave
> >> XML-RPC as the
> >> > > > > default
> >> > > > > > >for a release or two marked depreciated, as extra
> >> security in case
> >> > > it
> >> > > > > > >affects anything else. Then at some point just switch
> >> the defaults
> >> > > in
> >> > > > > the
> >> > > > > > >configs and remove the XML-RPC crud.
> >> > > > > > >
> >> > > > > > >On Wed, Jul 6, 2016 at 3:04 PM, Mattmann, Chris A (3980)
> <
> >> > > > > > >chris.a.mattmann@jpl.nasa.gov <javascript:;>> wrote:
> >> > > > > > >
> >> > > > > > >> Great work.
> >> > > > > > >>
> >> > > > > > >> Can we agree to create a 2.x branch and work forward
> >> from that
> >> > > > > > >> with Avro-RPC? 1.x can remain XML-RPC for a few
> >> versions while
> >> > > > > > >> I fix all lingering crap I found on the BigTranslate
> end?
> >> > > > > > >>
> >> > > > > > >> Cheers,
> >> > > > > > >> Chris
> >> > > > > > >>
> >> > > > > > >> ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++
> >> > > ++++++
> >> > > > > > >> Chris Mattmann, Ph.D.
> >> > > > > > >> Chief Architect
> >> > > > > > >> Instrument Software and Science Data Systems Section
> >> (398)
> >> > > > > > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> > > > > > >> Office: 168-519, Mailstop: 168-527
> >> > > > > > >> Email: chris.a.mattmann@nasa.gov <javascript:;>
> >> > > > > > >> WWW: http://sunset.usc.edu/~mattmann/
> >> > > > > > >> ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++
> >> > > ++++++
> >> > > > > > >> Director, Information Retrieval and Data Science
> >> Group
> >> (IRDS)
> >> > > > > > >> Adjunct Associate Professor, Computer Science
> Department
> >> > > > > > >> University of Southern California, Los Angeles, CA
> >> 90089 USA
> >> > > > > > >> WWW: http://irds.usc.edu/
> >> > > > > > >> ++++++++++++++++++++++++++++++
> >> ++++++++++++++++++++++++++++++
> >> > > ++++++
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >> On 7/6/16, 7:01 AM, "Tom Barber" <
> >> tom.barber@meteorite.bi
> >> > > <javascript:;>> wrote:
> >> > > > > > >>
> >> > > > > > >> >I'm making progress here, I updated a few libs
> >> which fixed a
> >> > > couple
> >> > > > > of
> >> > > > > > the
> >> > > > > > >> >test failures I was seeing locally. Timezones are
> >> once again
> >> > > > kicking
> >> > > > > me
> >> > > > > > >> >whilst I'm down, for some reason OODT fails badly
> >> with people
> >> > in
> >> > > > > > >> >GMT/BST.... I'll figure it out though.
> >> > > > > > >> >
> >> > > > > > >> >On Fri, Jul 1, 2016 at 12:36 AM, Lewis John Mcgibbney
> <
> >> > > > > > >> >lewis.mcgibbney@gmail.com <javascript:;>> wrote:
> >> > > > > > >> >
> >> > > > > > >> >> Sound ttyl
> >> > > > > > >> >> On Jun 30, 2016 4:32 PM, "Tom Barber" <
> >> > tom.barber@meteorite.bi
> >> > > <javascript:;>>
> >> > > > > > wrote:
> >> > > > > > >> >>
> >> > > > > > >> >> > Not really, tomoz?
> >> > > > > > >> >> >
> >> > > > > > >> >> > On Fri, Jul 1, 2016 at 12:24 AM, Lewis John
> >> Mcgibbney <
> >> > > > > > >> >> > lewis.mcgibbney@gmail.com <javascript:;>> wrote:
> >> > > > > > >> >> >
> >> > > > > > >> >> > > Want to do a hangout right now?
> >> > > > > > >> >> > >
> >> > > > > > >> >> > > On Thu, Jun 30, 2016 at 4:17 PM, Tom Barber <
> >> > > > > > >> tom.barber@meteorite.bi <javascript:;>>
> >> > > > > > >> >> > > wrote:
> >> > > > > > >> >> > >
> >> > > > > > >> >> > > > Hey Lewis
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > > > I have no idea which branch is current, I
> >> merged master
> >> > > > into
> >> > > > > > >> avrorpc
> >> > > > > > >> >> > and
> >> > > > > > >> >> > > it
> >> > > > > > >> >> > > > merged okay and built so I pushed it back.
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > > > Is that the correct branch to be looking at
> >> for Avro
> >> > > > Support?
> >> > > > > > Lets
> >> > > > > > >> >> > figure
> >> > > > > > >> >> > > > it all out and get it merged into Master.
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > > > Tom
> >> > > > > > >> >> > > >
> >> > > > > > >> >> > >
> >> > > > > > >> >> > >
> >> > > > > > >> >> > >
> >> > > > > > >> >> > > --
> >> > > > > > >> >> > > *Lewis*
> >> > > > > > >> >> > >
> >> > > > > > >> >> >
> >> > > > > > >> >>
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > http://home.apache.org/~lewismc/
> >> > > > @hectorMcSpector
> >> > > > http://www.linkedin.com/in/lmcgibbney
> >> > > >
> >> > >
> >> >
> >> >
> >> > --
> >> > http://home.apache.org/~lewismc/
> >> > @hectorMcSpector
> >> > http://www.linkedin.com/in/lmcgibbney
> >> >
> >>
> >>
> >>
> >>
> >
> >
> > --
> > http://home.apache.org/~lewismc/
> > @hectorMcSpector
> > http://www.linkedin.com/in/lmcgibbney
> >
>
>
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>
--
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney
How to convert XMLRPC to AvroRPC??
Posted by "Mallder, Valerie" <Va...@jhuapl.edu>.
Hi All,
Is there any documention on how to convert a XMLRPC-based OODT Pipeline to an AvroRPC-based OODT Pipeline? (See email thread below).
I am beginning work on a 3rd pipeline and I need to decide if I should abandon OODT altogether or upgrade from version 1.1 to the latest version. I definitely need some documention on how to do this conversion. Unfortunately, my scientists and project manager don't have the funds for me to spend a lot of time fumbling through it and figurring everything out on my own. :(
Thanks,
Val
Sent with BlackBerry Work (www.blackberry.com)
From: Mallder, Valerie <Va...@jhuapl.edu>>
Date: Monday, Mar 06, 2017, 9:50 AM
To: dev@oodt.apache.org <de...@oodt.apache.org>>
Subject: RE: Merge Avro RPC
Hi Lewis,
I hope you are doing well. I have been silent on oodt email for a while. But now I need to do more pipeline development work and I'm looking for resources to help me come up to speed with some of the changes that have been made over the last few months. Is the link below the correct place to look for the XMRPC to AvroRPC upgrade information? It doesn't really say anything, so if there is a different place I should be looking please let me know.
Thanks,
Val
-----Original Message-----
From: lewis john mcgibbney [mailto:lewismc@apache.org]
Sent: Friday, October 07, 2016 2:57 PM
To: dev@oodt.apache.org
Subject: Re: Merge Avro RPC
Hi Val,
I'm working on this at the link below and will post an update here once complete https://cwiki.apache.org/confluence/display/OODT/OODT+XMLRPC+--%3E+AvroRPC+Upgrade
Thanks
On Thu, Oct 6, 2016 at 8:02 AM, lewis john mcgibbney <le...@apache.org>
wrote:
> Hi Val,
> Yes when Radu and I worked on this code a year or two ago now we were
> very much aware of the upgrade issue.
> I am preparing myself to put a wiki page together which will fully
> document 1) why we have put time and effort into the XMLRP --> AvroRPC
> shift, 2) Which services are affected, and 3) How you can upgrade.
> On another note, I am also going to put together a wiki page for the
> various configuration files and what configuration options you need to use.
> It would be less than wise for us to leave people like yourself (and a
> significant amount of others I would imagine) in the dark about all of
> this so be rest assured that we will not be mandating an upgrade
> without the relevant support for your existing pipelines and the path to upgrade.
> Actually, in an ideal world we would probably get a blog post together
> for this at some stage and post to the foundation blog.
> Lewis
>
>
> On Thu, Oct 6, 2016 at 7:19 AM, Mallder, Valerie <
> Valerie.Mallder@jhuapl.edu> wrote:
>
>> Hi Guys,
>>
>> So what happens now for the folks like me who are using each of the
>> following:
>>
>> XmlRpcFileManager
>> XmlRpcFileManagerClient
>> XmlRpcResourceManager
>> XmlRpcResourceManagerClient
>> XmlRpcBatchStub
>> XmlRpcWorkflowManager
>> XmlRpcWorkflowManagerClient
>>
>> I have two science data pipelines, and working on a third, that use
>> these objects. Does this mean that all of my pipelines will have to
>> be frozen to the current version? Is there an easy way to change over to the Avro-RPC?
>> Will someone document all the steps you need to take to change over
>> to using it?
>>
>> Thanks,
>> Val
>>
>>
>>
>>
>> -----Original Message-----
>> From: Chris Mattmann [mailto:chris.mattmann@gmail.com]
>> Sent: Wednesday, October 05, 2016 5:35 PM
>> To: dev@oodt.apache.org
>> Subject: Re: Merge Avro RPC
>>
>> +1 to removing XML-RPC period, and just defaulting to Avro-RPC.
>>
>>
>>
>> —
>> Chris Mattmann
>> chris.mattmann@gmail.com
>>
>> On 10/5/16, 2:22 PM, "Radu Manole" <ma...@gmail.com> wrote:
>>
>> Hello, I I revisited every module where avrorpc was implemented
>> and where
>> xmlrpc is,
>> and marked every class where avro rpc is not implemented. What kind of
>> transition to avro-rpc should be implemented? The easiest way is
>> to remove
>> xml-rpc and give up on compatibility with old version. Or make
>> the client
>> compatible to connect to both xml-rpc or avro-rpc servers, for the
>> transition.
>>
>> 2016-09-09 19:23 GMT+03:00 lewis john mcgibbney <le...@apache.org>:
>>
>> > So what's the plan then? Where is work going into? Did we
>> define a roadmap?
>> > Maybe we are best working off of 2.x branch then.
>> >
>> > On Wednesday, September 7, 2016, Tom Barber <
>> tom.barber@meteorite.bi>
>> > wrote:
>> >
>> > > Be warned folks, don't forget we've merged into mainline and
>> have a
>> > > development branch for 2.0!
>> > >
>> > > On Wed, Sep 7, 2016 at 7:38 PM, lewis john mcgibbney <
>> lewismc@apache.org
>> > > <javascript:;>>
>> > > wrote:
>> > >
>> > > > Hi Radu,
>> > > > Yes this is great.
>> > > > I would suggest that you submit a patch to '@Deprecate' all
>> of the
>> > XMLRPC
>> > > > code for this current development drive.
>> > > > I also suggest that we make the AvroRPC material default
>> from now on
>> > and
>> > > > get more people using it.
>> > > > We can also make upgrades to Avro 1.8.X
>> > > > Would be great if others can chime in here to discuss the
>> path for a
>> > > > complete replacement of the XMLRPC logic with AvroRPC for
>> the next
>> > > release
>> > > > or two.
>> > > > Thanks
>> > > > Lewis
>> > > >
>> > > > P.S. @Radu, in the meantime please log an issue for
>> @Deprecat'ing all
>> > of
>> > > > the XMLRPC stuff. This should be clearly stated and present
>> within
>> > > > generated Javadoc such that people know to use the AvroRPC
>> > > implementation.
>> > > > Thanks
>> > > >
>> > > > On Mon, Sep 5, 2016 at 6:17 AM, Radu Manole <
>> manole.v.radu@gmail.com
>> > > <javascript:;>>
>> > > > wrote:
>> > > >
>> > > > > Hello, great to hear that avro rpc support is still
>> wanted, if there
>> > is
>> > > > > anything else to be done, I have a lot of free time now.
>> > > > >
>> > > > > 2016-07-06 17:09 GMT+03:00 Mattmann, Chris A (3980) <
>> > > > > chris.a.mattmann@jpl.nasa.gov <javascript:;>>:
>> > > > >
>> > > > > > +1 awesome
>> > > > > >
>> > > > > > ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++++++++
>> > > > > > Chris Mattmann, Ph.D.
>> > > > > > Chief Architect
>> > > > > > Instrument Software and Science Data Systems Section (398)
>> > > > > > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > > > > > Office: 168-519, Mailstop: 168-527
>> > > > > > Email: chris.a.mattmann@nasa.gov <javascript:;>
>> > > > > > WWW: http://sunset.usc.edu/~mattmann/
>> > > > > > ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++++++++
>> > > > > > Director, Information Retrieval and Data Science Group
>> (IRDS)
>> > > > > > Adjunct Associate Professor, Computer Science Department
>> > > > > > University of Southern California, Los Angeles, CA 90089 USA
>> > > > > > WWW: http://irds.usc.edu/
>> > > > > > ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++++++++
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On 7/6/16, 7:07 AM, "Tom Barber" <tom.barber@meteorite.bi
>> > > <javascript:;>> wrote:
>> > > > > >
>> > > > > > >Absolutely, all we've done is merge master into 2.x,
>> the plan was
>> > to
>> > > > > keep
>> > > > > > >that in sync, rename it or whatever and let people who
>> care play
>> > > > around.
>> > > > > > We
>> > > > > > >also then planned to merge it into master but leave
>> XML-RPC as the
>> > > > > default
>> > > > > > >for a release or two marked depreciated, as extra
>> security in case
>> > > it
>> > > > > > >affects anything else. Then at some point just switch
>> the defaults
>> > > in
>> > > > > the
>> > > > > > >configs and remove the XML-RPC crud.
>> > > > > > >
>> > > > > > >On Wed, Jul 6, 2016 at 3:04 PM, Mattmann, Chris A (3980) <
>> > > > > > >chris.a.mattmann@jpl.nasa.gov <javascript:;>> wrote:
>> > > > > > >
>> > > > > > >> Great work.
>> > > > > > >>
>> > > > > > >> Can we agree to create a 2.x branch and work forward
>> from that
>> > > > > > >> with Avro-RPC? 1.x can remain XML-RPC for a few
>> versions while
>> > > > > > >> I fix all lingering crap I found on the BigTranslate end?
>> > > > > > >>
>> > > > > > >> Cheers,
>> > > > > > >> Chris
>> > > > > > >>
>> > > > > > >> ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++
>> > > ++++++
>> > > > > > >> Chris Mattmann, Ph.D.
>> > > > > > >> Chief Architect
>> > > > > > >> Instrument Software and Science Data Systems Section
>> (398)
>> > > > > > >> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > > > > > >> Office: 168-519, Mailstop: 168-527
>> > > > > > >> Email: chris.a.mattmann@nasa.gov <javascript:;>
>> > > > > > >> WWW: http://sunset.usc.edu/~mattmann/
>> > > > > > >> ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++
>> > > ++++++
>> > > > > > >> Director, Information Retrieval and Data Science
>> Group
>> (IRDS)
>> > > > > > >> Adjunct Associate Professor, Computer Science Department
>> > > > > > >> University of Southern California, Los Angeles, CA
>> 90089 USA
>> > > > > > >> WWW: http://irds.usc.edu/
>> > > > > > >> ++++++++++++++++++++++++++++++
>> ++++++++++++++++++++++++++++++
>> > > ++++++
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >> On 7/6/16, 7:01 AM, "Tom Barber" <
>> tom.barber@meteorite.bi
>> > > <javascript:;>> wrote:
>> > > > > > >>
>> > > > > > >> >I'm making progress here, I updated a few libs
>> which fixed a
>> > > couple
>> > > > > of
>> > > > > > the
>> > > > > > >> >test failures I was seeing locally. Timezones are
>> once again
>> > > > kicking
>> > > > > me
>> > > > > > >> >whilst I'm down, for some reason OODT fails badly
>> with people
>> > in
>> > > > > > >> >GMT/BST.... I'll figure it out though.
>> > > > > > >> >
>> > > > > > >> >On Fri, Jul 1, 2016 at 12:36 AM, Lewis John Mcgibbney <
>> > > > > > >> >lewis.mcgibbney@gmail.com <javascript:;>> wrote:
>> > > > > > >> >
>> > > > > > >> >> Sound ttyl
>> > > > > > >> >> On Jun 30, 2016 4:32 PM, "Tom Barber" <
>> > tom.barber@meteorite.bi
>> > > <javascript:;>>
>> > > > > > wrote:
>> > > > > > >> >>
>> > > > > > >> >> > Not really, tomoz?
>> > > > > > >> >> >
>> > > > > > >> >> > On Fri, Jul 1, 2016 at 12:24 AM, Lewis John
>> Mcgibbney <
>> > > > > > >> >> > lewis.mcgibbney@gmail.com <javascript:;>> wrote:
>> > > > > > >> >> >
>> > > > > > >> >> > > Want to do a hangout right now?
>> > > > > > >> >> > >
>> > > > > > >> >> > > On Thu, Jun 30, 2016 at 4:17 PM, Tom Barber <
>> > > > > > >> tom.barber@meteorite.bi <javascript:;>>
>> > > > > > >> >> > > wrote:
>> > > > > > >> >> > >
>> > > > > > >> >> > > > Hey Lewis
>> > > > > > >> >> > > >
>> > > > > > >> >> > > > I have no idea which branch is current, I
>> merged master
>> > > > into
>> > > > > > >> avrorpc
>> > > > > > >> >> > and
>> > > > > > >> >> > > it
>> > > > > > >> >> > > > merged okay and built so I pushed it back.
>> > > > > > >> >> > > >
>> > > > > > >> >> > > > Is that the correct branch to be looking at
>> for Avro
>> > > > Support?
>> > > > > > Lets
>> > > > > > >> >> > figure
>> > > > > > >> >> > > > it all out and get it merged into Master.
>> > > > > > >> >> > > >
>> > > > > > >> >> > > > Tom
>> > > > > > >> >> > > >
>> > > > > > >> >> > >
>> > > > > > >> >> > >
>> > > > > > >> >> > >
>> > > > > > >> >> > > --
>> > > > > > >> >> > > *Lewis*
>> > > > > > >> >> > >
>> > > > > > >> >> >
>> > > > > > >> >>
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > http://home.apache.org/~lewismc/
>> > > > @hectorMcSpector
>> > > > http://www.linkedin.com/in/lmcgibbney
>> > > >
>> > >
>> >
>> >
>> > --
>> > http://home.apache.org/~lewismc/
>> > @hectorMcSpector
>> > http://www.linkedin.com/in/lmcgibbney
>> >
>>
>>
>>
>>
>
>
> --
> http://home.apache.org/~lewismc/
> @hectorMcSpector
> http://www.linkedin.com/in/lmcgibbney
>
--
http://home.apache.org/~lewismc/
@hectorMcSpector
http://www.linkedin.com/in/lmcgibbney