You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by Tom Barber <to...@meteorite.bi> on 2016/07/06 15:51:28 UTC
Upgrading Lucene and Solr
Just throwing this one out there as we're making a bunch of breaking
changes in 2.0.
Is it worth upgrading Lucene and Solr versions within OODT?
My feeling is that they are both pretty old and whilst we're making life
hard for everyone we might as well go the whole hog and drag them into the
21st century, but I'm interested in others opinions.
Chris you have a much better understand about those things than myself.
Tom
Re: Upgrading Lucene and Solr
Posted by Chris Mattmann <ch...@gmail.com>.
I’ll give it a row
—
Chris Mattmann
chris.mattmann@gmail.com
On 7/15/16, 6:22 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>None of that, lets develop on a separate branch and make sure it works eh?!
>;)
>
>Merged.
>
>Tom
>
>On Fri, Jul 15, 2016 at 2:05 AM, Chris Mattmann <ch...@gmail.com>
>wrote:
>
>> can you merge to master?
>>
>> —
>> Chris Mattmann
>> chris.mattmann@gmail.com
>>
>>
>>
>>
>>
>>
>>
>> On 7/14/16, 4:22 PM, "Lewis John Mcgibbney" <le...@gmail.com>
>> wrote:
>>
>> >Dyno mate
>> >
>> >On Mon, Jul 11, 2016 at 12:02 PM, Tom Barber <to...@meteorite.bi>
>> >wrote:
>> >
>> >> [INFO] Installing /root/oodt/pom.xml to
>> >>
>> >>
>> /root/.m2/repository/org/apache/oodt/oodt/1.1-SNAPSHOT/oodt-1.1-SNAPSHOT.pom
>> >>
>> >> [INFO]
>> >> ------------------------------------------------------------------------
>> >> [INFO] Reactor Summary:
>> >> [INFO]
>> >> [INFO] OODT Core .......................................... SUCCESS [
>> >> 1.290 s]
>> >> [INFO] Common Utilities ................................... SUCCESS [
>> >> 15.731 s]
>> >> [INFO] CAS Command Line Interface ......................... SUCCESS [
>> >> 4.385 s]
>> >> [INFO] Process Control System Input Data Package .......... SUCCESS [
>> >> 1.186 s]
>> >> [INFO] Catalog and Archive Service Generic Multi-valued Metadata
>> Container
>> >> SUCCESS [ 2.357 s]
>> >> [INFO] CAS Protocol ....................................... SUCCESS [
>> >> 5.509 s]
>> >> [INFO] CAS Protocol FTP Implementation .................... SUCCESS [
>> >> 2.993 s]
>> >> [INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
>> >> 10.829 s]
>> >> [INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
>> >> 3.236 s]
>> >> [INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
>> >> 2.445 s]
>> >> [INFO] Query Expression ................................... SUCCESS [
>> >> 2.873 s]
>> >> [INFO] OODT Single Sign On Security Package ............... SUCCESS [
>> >> 1.121 s]
>> >> [INFO] Catalog and Archive File Management Component ...... SUCCESS
>> [03:36
>> >> min]
>> >> [INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
>> >> 9.659 s]
>> >> [INFO] Catalog and Archive Resource Management Component .. SUCCESS [
>> >> 14.399 s]
>> >> [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
>> >> 34.506 s]
>> >> [INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
>> >> 15.162 s]
>> >> [INFO] CAS Curation Web Services .......................... SUCCESS [
>> >> 10.538 s]
>> >> [INFO] Process Control System Core Package ................ SUCCESS [
>> >> 9.441 s]
>> >> [INFO] OODT Wicket Web Components ......................... SUCCESS [
>> >> 2.837 s]
>> >> [INFO] CAS Curation Interface ............................. SUCCESS [
>> >> 15.108 s]
>> >> [INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
>> >> 10.969 s]
>> >> [INFO] CAS Installer Maven Mojo ........................... SUCCESS [
>> >> 1.550 s]
>> >> [INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
>> >> 0.738 s]
>> >> [INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
>> >> 0.490 s]
>> >> [INFO] OODT :: Archetypes ................................. SUCCESS [
>> >> 0.014 s]
>> >> [INFO] CAS Push-Pull-Framework ............................ SUCCESS [
>> >> 10.270 s]
>> >> [INFO] Product Service .................................... SUCCESS [
>> >> 1.391 s]
>> >> [INFO] Profile Service .................................... SUCCESS [
>> >> 3.924 s]
>> >> [INFO] OODT Web Grid ...................................... SUCCESS [
>> >> 8.825 s]
>> >> [INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
>> >> 5.605 s]
>> >> [INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
>> >> 14.057 s]
>> >> [INFO] CAS File Manager Browser Web App ................... SUCCESS [
>> >> 12.046 s]
>> >> [INFO] CAS Product Server Web Application ................. SUCCESS [
>> >> 12.723 s]
>> >> [INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
>> >> 4.691 s]
>> >> [INFO] Catalog and Archive File Management Browser ........ SUCCESS [
>> >> 5.015 s]
>> >> [INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
>> >> 7.072 s]
>> >> [INFO] Process Control System Operator Interface Webapp ... SUCCESS [
>> >> 10.877 s]
>> >> [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
>> >> 5.047 s]
>> >> [INFO] Apache OODT ........................................ SUCCESS [
>> >> 0.221 s]
>> >> [INFO]
>> >> ------------------------------------------------------------------------
>> >> [INFO] BUILD SUCCESS
>> >> [INFO]
>> >> ------------------------------------------------------------------------
>> >> [INFO] Total time: 08:18 min
>> >> [INFO] Finished at: 2016-07-11T19:00:29+00:00
>> >> [INFO] Final Memory: 136M/2984M
>> >> [INFO]
>> >> ------------------------------------------------------------------------
>> >> root@ubuntu:~/oodt# git status
>> >> On branch 1.x-lucene
>> >> Your branch is up-to-date with 'origin/1.x-lucene'.
>> >>
>> >>
>> >> You should be good to go.
>> >>
>> >> On Mon, Jul 11, 2016 at 4:19 PM, Mattmann, Chris A (3980) <
>> >> chris.a.mattmann@jpl.nasa.gov> wrote:
>> >>
>> >> > I have a beautiful use case for BigTranslate that I can test this
>> >> > in pretty immediately - so commit to 1.x soon and I’ll be your
>> >> > guinea pig.
>> >> >
>> >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >> > Chris Mattmann, Ph.D.
>> >> > Chief Architect
>> >> > Instrument Software and Science Data Systems Section (398)
>> >> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> >> > Office: 168-519, Mailstop: 168-527
>> >> > Email: chris.a.mattmann@nasa.gov
>> >> > WWW: http://sunset.usc.edu/~mattmann/
>> >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >> > Director, Information Retrieval and Data Science Group (IRDS)
>> >> > Adjunct Associate Professor, Computer Science Department
>> >> > University of Southern California, Los Angeles, CA 90089 USA
>> >> > WWW: http://irds.usc.edu/
>> >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On 7/11/16, 11:17 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>> >> >
>> >> > >INFO] Reactor Summary:
>> >> > >[INFO]
>> >> > >[INFO] OODT Core .......................................... SUCCESS [
>> >> > >10.222 s]
>> >> > >[INFO] Common Utilities ................................... SUCCESS [
>> >> > >17.715 s]
>> >> > >[INFO] CAS Command Line Interface ......................... SUCCESS [
>> >> > > 9.428 s]
>> >> > >[INFO] Process Control System Input Data Package .......... SUCCESS [
>> >> > > 1.564 s]
>> >> > >[INFO] Catalog and Archive Service Generic Multi-valued Metadata
>> >> Container
>> >> > >SUCCESS [ 3.094 s]
>> >> > >[INFO] CAS Protocol ....................................... SUCCESS [
>> >> > > 9.456 s]
>> >> > >[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
>> >> > > 4.550 s]
>> >> > >[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
>> >> > >16.927 s]
>> >> > >[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
>> >> > > 4.857 s]
>> >> > >[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
>> >> > > 4.020 s]
>> >> > >[INFO] Query Expression ................................... SUCCESS [
>> >> > > 3.140 s]
>> >> > >[INFO] OODT Single Sign On Security Package ............... SUCCESS [
>> >> > > 2.458 s]
>> >> > >[INFO] Catalog and Archive File Management Component ...... SUCCESS
>> >> [04:32
>> >> > >min]
>> >> > >[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
>> >> > >15.820 s]
>> >> > >[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
>> >> > >23.145 s]
>> >> > >[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
>> >> > >59.957 s]
>> >> > >[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
>> >> > >22.978 s]
>> >> > >[INFO] CAS Curation Web Services .......................... SUCCESS [
>> >> > > 9.949 s]
>> >> > >[INFO] Process Control System Core Package ................ SUCCESS [
>> >> > >13.890 s]
>> >> > >[INFO] OODT Wicket Web Components ......................... SUCCESS [
>> >> > > 4.355 s]
>> >> > >[INFO] CAS Curation Interface ............................. SUCCESS [
>> >> > > 9.994 s]
>> >> > >[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
>> >> > >18.437 s]
>> >> > >[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
>> >> > > 6.322 s]
>> >> > >[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
>> >> > > 9.322 s]
>> >> > >[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
>> >> > > 1.049 s]
>> >> > >[INFO] OODT :: Archetypes ................................. SUCCESS [
>> >> > > 0.014 s]
>> >> > >[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
>> >> > >15.797 s]
>> >> > >[INFO] Product Service .................................... SUCCESS [
>> >> > > 1.704 s]
>> >> > >[INFO] Profile Service .................................... SUCCESS [
>> >> > > 7.741 s]
>> >> > >[INFO] OODT Web Grid ...................................... SUCCESS [
>> >> > >19.080 s]
>> >> > >[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS
>> [
>> >> > >12.202 s]
>> >> > >[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
>> >> > >21.353 s]
>> >> > >[INFO] CAS File Manager Browser Web App ................... SUCCESS [
>> >> > > 9.061 s]
>> >> > >[INFO] CAS Product Server Web Application ................. SUCCESS [
>> >> > >12.943 s]
>> >> > >[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
>> >> > > 3.712 s]
>> >> > >[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
>> >> > > 8.218 s]
>> >> > >[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
>> >> > >10.785 s]
>> >> > >[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
>> >> > >13.757 s]
>> >> > >[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
>> >> > > 7.015 s]
>> >> > >[INFO] Apache OODT ........................................ SUCCESS [
>> >> > > 0.020 s]
>> >> > >[INFO]
>> >> >
>> >------------------------------------------------------------------------
>> >> > >[INFO] BUILD SUCCESS
>> >> > >[INFO]
>> >> >
>> >------------------------------------------------------------------------
>> >> > >[INFO] Total time: 11:40 min
>> >> > >[INFO] Finished at: 2016-07-11T15:11:00+00:00
>> >> > >[INFO] Final Memory: 167M/2913M
>> >> > >[INFO]
>> >> >
>> >------------------------------------------------------------------------
>> >> > >root@ubuntu:~/oodt#
>> >> > >
>> >> > >
>> >> > >Okay I finished up the workflow changes last night and all the tests
>> >> pass
>> >> > >on 2.x so I'll lift those commits tonight and patch the 1.x branch.
>> >> > >
>> >> > >Just because the tests pass, don't expect it to be perfect, I don't
>> know
>> >> > >lucene I just manipulated what existed to work with the new API,
>> someone
>> >> > >who knows what they are doing needs to take a look, specifically at
>> the
>> >> > >todoc() methods in Filemanager and Workflow.
>> >> > >
>> >> > >Tom
>> >> > >
>> >> > >On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
>> >> > >chris.a.mattmann@jpl.nasa.gov> wrote:
>> >> > >
>> >> > >> Yeah we can back port the Lucene part to 1.x
>> >> > >>
>> >> > >> Sent from my iPhone
>> >> > >>
>> >> > >> > On Jul 8, 2016, at 11:15 PM, Tom Barber <tom.barber@meteorite.bi
>> >
>> >> > wrote:
>> >> > >> >
>> >> > >> > yeah there's no reason we can't diff the branch and merge it into
>> >> 1.0
>> >> > as
>> >> > >> > well
>> >> > >> >> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org>
>> wrote:
>> >> > >> >>
>> >> > >> >> awesome!!!
>> >> > >> >>
>> >> > >> >> Please backport to 1.x :)
>> >> > >> >>
>> >> > >> >>
>> >> > >> >>
>> >> > >> >>
>> >> > >> >>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi>
>> wrote:
>> >> > >> >>>
>> >> > >> >>> There you go, running commentary over. The filemanager test
>> suite
>> >> > now
>> >> > >> >>> passes on Lucene 6, I'll fix up the rest next week.
>> >> > >> >>>
>> >> > >> >>> Tom
>> >> > >> >>>
>> >> > >> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <
>> >> > tom.barber@meteorite.bi>
>> >> > >> >>> wrote:
>> >> > >> >>>
>> >> > >> >>>> Better than I thought actually:
>> >> > >> >>>>
>> >> > >> >>>> Tests in error:
>> >> > >> >>>>
>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
>> >> > >> >>>> CmdLineAction ...
>> >> > >> >>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
>> >> > >> >>>> org.apache.oodt.cas.cli.excep...
>> >> > >> >>>>
>> >> > >> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
>> >> > >> >>>>
>> >> > >> >>>> The failure is actually one Avro. So only 2 failing on lucene
>> 6
>> >> in
>> >> > the
>> >> > >> >> FM.
>> >> > >> >>>>
>> >> > >> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <
>> >> > tom.barber@meteorite.bi>
>> >> > >> >>>> wrote:
>> >> > >> >>>>
>> >> > >> >>>>> Just FYI:
>> >> > >> >>>>>
>> >> > >> >>>>> Doesn't work yet, but I've pushed a lucene branch which is
>> based
>> >> > on
>> >> > >> 2.0
>> >> > >> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager
>> has
>> >> > been
>> >> > >> 75%
>> >> > >> >>>>> (guess) converted and tests are half passing.
>> >> > >> >>>>>
>> >> > >> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
>> >> > >> >>>>> lewis.mcgibbney@gmail.com> wrote:
>> >> > >> >>>>>
>> >> > >> >>>>>> ACK
>> >> > >> >>>>>>
>> >> > >> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <
>> >> > tom.barber@meteorite.bi
>> >> > >> >
>> >> > >> >>>>>> wrote:
>> >> > >> >>>>>>
>> >> > >> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch
>> >> 2.0-dev
>> >> > or
>> >> > >> >>>>>> similar
>> >> > >> >>>>>>> then branch the branch to uprade solr and lucene outside of
>> >> that
>> >> > >> >> scope
>> >> > >> >>>>>> so
>> >> > >> >>>>>>> we don't destroy the semi working avro upgrades ;)
>> >> > >> >>>>>>>
>> >> > >> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
>> >> > >> tom.barber@meteorite.bi
>> >> > >> >>>
>> >> > >> >>>>>>> wrote:
>> >> > >> >>>>>>>
>> >> > >> >>>>>>>> Coolio!
>> >> > >> >>>>>>>>
>> >> > >> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
>> >> > >> >> mattmann@apache.org>
>> >> > >> >>>>>>>> wrote:
>> >> > >> >>>>>>>>
>> >> > >> >>>>>>>>> Yes I was just telling Paul R. the other day we need to
>> do
>> >> > >> >>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
>> >> > >> >>>>>>>>> issues with Big Translate and DRAT using the old Lucene
>> >> > >> >>>>>>>>> library. Let’s get it fixed in 2.x.
>> >> > >> >>>>>>>>>
>> >> > >> >>>>>>>>> Cheers,
>> >> > >> >>>>>>>>> Chris
>> >> > >> >>>>>>>>>
>> >> > >> >>>>>>>>>
>> >> > >> >>>>>>>>>
>> >> > >> >>>>>>>>>
>> >> > >> >>>>>>>>>
>> >> > >> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <
>> tom.barber@meteorite.bi>
>> >> > >> >> wrote:
>> >> > >> >>>>>>>>>
>> >> > >> >>>>>>>>>> Just throwing this one out there as we're making a
>> bunch of
>> >> > >> >>>>>> breaking
>> >> > >> >>>>>>>>>> changes in 2.0.
>> >> > >> >>>>>>>>>>
>> >> > >> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions within
>> OODT?
>> >> > >> >>>>>>>>>>
>> >> > >> >>>>>>>>>> My feeling is that they are both pretty old and whilst
>> >> we're
>> >> > >> >> making
>> >> > >> >>>>>>> life
>> >> > >> >>>>>>>>>> hard for everyone we might as well go the whole hog and
>> >> drag
>> >> > >> >> them
>> >> > >> >>>>>> into
>> >> > >> >>>>>>>>> the
>> >> > >> >>>>>>>>>> 21st century, but I'm interested in others opinions.
>> >> > >> >>>>>>>>>>
>> >> > >> >>>>>>>>>> Chris you have a much better understand about those
>> things
>> >> > than
>> >> > >> >>>>>> myself.
>> >> > >> >>>>>>>>>>
>> >> > >> >>>>>>>>>> Tom
>> >> > >> >>>>>>
>> >> > >> >>>>>>
>> >> > >> >>>>>>
>> >> > >> >>>>>> --
>> >> > >> >>>>>> *Lewis*
>> >> > >> >>
>> >> > >> >>
>> >> > >>
>> >> >
>> >>
>> >
>> >
>> >
>> >--
>> >*Lewis*
>>
>>
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
None of that, lets develop on a separate branch and make sure it works eh?!
;)
Merged.
Tom
On Fri, Jul 15, 2016 at 2:05 AM, Chris Mattmann <ch...@gmail.com>
wrote:
> can you merge to master?
>
> —
> Chris Mattmann
> chris.mattmann@gmail.com
>
>
>
>
>
>
>
> On 7/14/16, 4:22 PM, "Lewis John Mcgibbney" <le...@gmail.com>
> wrote:
>
> >Dyno mate
> >
> >On Mon, Jul 11, 2016 at 12:02 PM, Tom Barber <to...@meteorite.bi>
> >wrote:
> >
> >> [INFO] Installing /root/oodt/pom.xml to
> >>
> >>
> /root/.m2/repository/org/apache/oodt/oodt/1.1-SNAPSHOT/oodt-1.1-SNAPSHOT.pom
> >>
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] Reactor Summary:
> >> [INFO]
> >> [INFO] OODT Core .......................................... SUCCESS [
> >> 1.290 s]
> >> [INFO] Common Utilities ................................... SUCCESS [
> >> 15.731 s]
> >> [INFO] CAS Command Line Interface ......................... SUCCESS [
> >> 4.385 s]
> >> [INFO] Process Control System Input Data Package .......... SUCCESS [
> >> 1.186 s]
> >> [INFO] Catalog and Archive Service Generic Multi-valued Metadata
> Container
> >> SUCCESS [ 2.357 s]
> >> [INFO] CAS Protocol ....................................... SUCCESS [
> >> 5.509 s]
> >> [INFO] CAS Protocol FTP Implementation .................... SUCCESS [
> >> 2.993 s]
> >> [INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
> >> 10.829 s]
> >> [INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
> >> 3.236 s]
> >> [INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
> >> 2.445 s]
> >> [INFO] Query Expression ................................... SUCCESS [
> >> 2.873 s]
> >> [INFO] OODT Single Sign On Security Package ............... SUCCESS [
> >> 1.121 s]
> >> [INFO] Catalog and Archive File Management Component ...... SUCCESS
> [03:36
> >> min]
> >> [INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
> >> 9.659 s]
> >> [INFO] Catalog and Archive Resource Management Component .. SUCCESS [
> >> 14.399 s]
> >> [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
> >> 34.506 s]
> >> [INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
> >> 15.162 s]
> >> [INFO] CAS Curation Web Services .......................... SUCCESS [
> >> 10.538 s]
> >> [INFO] Process Control System Core Package ................ SUCCESS [
> >> 9.441 s]
> >> [INFO] OODT Wicket Web Components ......................... SUCCESS [
> >> 2.837 s]
> >> [INFO] CAS Curation Interface ............................. SUCCESS [
> >> 15.108 s]
> >> [INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
> >> 10.969 s]
> >> [INFO] CAS Installer Maven Mojo ........................... SUCCESS [
> >> 1.550 s]
> >> [INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
> >> 0.738 s]
> >> [INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
> >> 0.490 s]
> >> [INFO] OODT :: Archetypes ................................. SUCCESS [
> >> 0.014 s]
> >> [INFO] CAS Push-Pull-Framework ............................ SUCCESS [
> >> 10.270 s]
> >> [INFO] Product Service .................................... SUCCESS [
> >> 1.391 s]
> >> [INFO] Profile Service .................................... SUCCESS [
> >> 3.924 s]
> >> [INFO] OODT Web Grid ...................................... SUCCESS [
> >> 8.825 s]
> >> [INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
> >> 5.605 s]
> >> [INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
> >> 14.057 s]
> >> [INFO] CAS File Manager Browser Web App ................... SUCCESS [
> >> 12.046 s]
> >> [INFO] CAS Product Server Web Application ................. SUCCESS [
> >> 12.723 s]
> >> [INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
> >> 4.691 s]
> >> [INFO] Catalog and Archive File Management Browser ........ SUCCESS [
> >> 5.015 s]
> >> [INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
> >> 7.072 s]
> >> [INFO] Process Control System Operator Interface Webapp ... SUCCESS [
> >> 10.877 s]
> >> [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
> >> 5.047 s]
> >> [INFO] Apache OODT ........................................ SUCCESS [
> >> 0.221 s]
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] BUILD SUCCESS
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] Total time: 08:18 min
> >> [INFO] Finished at: 2016-07-11T19:00:29+00:00
> >> [INFO] Final Memory: 136M/2984M
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> root@ubuntu:~/oodt# git status
> >> On branch 1.x-lucene
> >> Your branch is up-to-date with 'origin/1.x-lucene'.
> >>
> >>
> >> You should be good to go.
> >>
> >> On Mon, Jul 11, 2016 at 4:19 PM, Mattmann, Chris A (3980) <
> >> chris.a.mattmann@jpl.nasa.gov> wrote:
> >>
> >> > I have a beautiful use case for BigTranslate that I can test this
> >> > in pretty immediately - so commit to 1.x soon and I’ll be your
> >> > guinea pig.
> >> >
> >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> > Chris Mattmann, Ph.D.
> >> > Chief Architect
> >> > Instrument Software and Science Data Systems Section (398)
> >> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> >> > Office: 168-519, Mailstop: 168-527
> >> > Email: chris.a.mattmann@nasa.gov
> >> > WWW: http://sunset.usc.edu/~mattmann/
> >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> > Director, Information Retrieval and Data Science Group (IRDS)
> >> > Adjunct Associate Professor, Computer Science Department
> >> > University of Southern California, Los Angeles, CA 90089 USA
> >> > WWW: http://irds.usc.edu/
> >> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On 7/11/16, 11:17 AM, "Tom Barber" <to...@meteorite.bi> wrote:
> >> >
> >> > >INFO] Reactor Summary:
> >> > >[INFO]
> >> > >[INFO] OODT Core .......................................... SUCCESS [
> >> > >10.222 s]
> >> > >[INFO] Common Utilities ................................... SUCCESS [
> >> > >17.715 s]
> >> > >[INFO] CAS Command Line Interface ......................... SUCCESS [
> >> > > 9.428 s]
> >> > >[INFO] Process Control System Input Data Package .......... SUCCESS [
> >> > > 1.564 s]
> >> > >[INFO] Catalog and Archive Service Generic Multi-valued Metadata
> >> Container
> >> > >SUCCESS [ 3.094 s]
> >> > >[INFO] CAS Protocol ....................................... SUCCESS [
> >> > > 9.456 s]
> >> > >[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
> >> > > 4.550 s]
> >> > >[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
> >> > >16.927 s]
> >> > >[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
> >> > > 4.857 s]
> >> > >[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
> >> > > 4.020 s]
> >> > >[INFO] Query Expression ................................... SUCCESS [
> >> > > 3.140 s]
> >> > >[INFO] OODT Single Sign On Security Package ............... SUCCESS [
> >> > > 2.458 s]
> >> > >[INFO] Catalog and Archive File Management Component ...... SUCCESS
> >> [04:32
> >> > >min]
> >> > >[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
> >> > >15.820 s]
> >> > >[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
> >> > >23.145 s]
> >> > >[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
> >> > >59.957 s]
> >> > >[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
> >> > >22.978 s]
> >> > >[INFO] CAS Curation Web Services .......................... SUCCESS [
> >> > > 9.949 s]
> >> > >[INFO] Process Control System Core Package ................ SUCCESS [
> >> > >13.890 s]
> >> > >[INFO] OODT Wicket Web Components ......................... SUCCESS [
> >> > > 4.355 s]
> >> > >[INFO] CAS Curation Interface ............................. SUCCESS [
> >> > > 9.994 s]
> >> > >[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
> >> > >18.437 s]
> >> > >[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
> >> > > 6.322 s]
> >> > >[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
> >> > > 9.322 s]
> >> > >[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
> >> > > 1.049 s]
> >> > >[INFO] OODT :: Archetypes ................................. SUCCESS [
> >> > > 0.014 s]
> >> > >[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
> >> > >15.797 s]
> >> > >[INFO] Product Service .................................... SUCCESS [
> >> > > 1.704 s]
> >> > >[INFO] Profile Service .................................... SUCCESS [
> >> > > 7.741 s]
> >> > >[INFO] OODT Web Grid ...................................... SUCCESS [
> >> > >19.080 s]
> >> > >[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS
> [
> >> > >12.202 s]
> >> > >[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
> >> > >21.353 s]
> >> > >[INFO] CAS File Manager Browser Web App ................... SUCCESS [
> >> > > 9.061 s]
> >> > >[INFO] CAS Product Server Web Application ................. SUCCESS [
> >> > >12.943 s]
> >> > >[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
> >> > > 3.712 s]
> >> > >[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
> >> > > 8.218 s]
> >> > >[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
> >> > >10.785 s]
> >> > >[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
> >> > >13.757 s]
> >> > >[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
> >> > > 7.015 s]
> >> > >[INFO] Apache OODT ........................................ SUCCESS [
> >> > > 0.020 s]
> >> > >[INFO]
> >> >
> >------------------------------------------------------------------------
> >> > >[INFO] BUILD SUCCESS
> >> > >[INFO]
> >> >
> >------------------------------------------------------------------------
> >> > >[INFO] Total time: 11:40 min
> >> > >[INFO] Finished at: 2016-07-11T15:11:00+00:00
> >> > >[INFO] Final Memory: 167M/2913M
> >> > >[INFO]
> >> >
> >------------------------------------------------------------------------
> >> > >root@ubuntu:~/oodt#
> >> > >
> >> > >
> >> > >Okay I finished up the workflow changes last night and all the tests
> >> pass
> >> > >on 2.x so I'll lift those commits tonight and patch the 1.x branch.
> >> > >
> >> > >Just because the tests pass, don't expect it to be perfect, I don't
> know
> >> > >lucene I just manipulated what existed to work with the new API,
> someone
> >> > >who knows what they are doing needs to take a look, specifically at
> the
> >> > >todoc() methods in Filemanager and Workflow.
> >> > >
> >> > >Tom
> >> > >
> >> > >On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
> >> > >chris.a.mattmann@jpl.nasa.gov> wrote:
> >> > >
> >> > >> Yeah we can back port the Lucene part to 1.x
> >> > >>
> >> > >> Sent from my iPhone
> >> > >>
> >> > >> > On Jul 8, 2016, at 11:15 PM, Tom Barber <tom.barber@meteorite.bi
> >
> >> > wrote:
> >> > >> >
> >> > >> > yeah there's no reason we can't diff the branch and merge it into
> >> 1.0
> >> > as
> >> > >> > well
> >> > >> >> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org>
> wrote:
> >> > >> >>
> >> > >> >> awesome!!!
> >> > >> >>
> >> > >> >> Please backport to 1.x :)
> >> > >> >>
> >> > >> >>
> >> > >> >>
> >> > >> >>
> >> > >> >>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi>
> wrote:
> >> > >> >>>
> >> > >> >>> There you go, running commentary over. The filemanager test
> suite
> >> > now
> >> > >> >>> passes on Lucene 6, I'll fix up the rest next week.
> >> > >> >>>
> >> > >> >>> Tom
> >> > >> >>>
> >> > >> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <
> >> > tom.barber@meteorite.bi>
> >> > >> >>> wrote:
> >> > >> >>>
> >> > >> >>>> Better than I thought actually:
> >> > >> >>>>
> >> > >> >>>> Tests in error:
> >> > >> >>>>
> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
> >> > >> >>>> CmdLineAction ...
> >> > >> >>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
> >> > >> >>>> org.apache.oodt.cas.cli.excep...
> >> > >> >>>>
> >> > >> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
> >> > >> >>>>
> >> > >> >>>> The failure is actually one Avro. So only 2 failing on lucene
> 6
> >> in
> >> > the
> >> > >> >> FM.
> >> > >> >>>>
> >> > >> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <
> >> > tom.barber@meteorite.bi>
> >> > >> >>>> wrote:
> >> > >> >>>>
> >> > >> >>>>> Just FYI:
> >> > >> >>>>>
> >> > >> >>>>> Doesn't work yet, but I've pushed a lucene branch which is
> based
> >> > on
> >> > >> 2.0
> >> > >> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager
> has
> >> > been
> >> > >> 75%
> >> > >> >>>>> (guess) converted and tests are half passing.
> >> > >> >>>>>
> >> > >> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
> >> > >> >>>>> lewis.mcgibbney@gmail.com> wrote:
> >> > >> >>>>>
> >> > >> >>>>>> ACK
> >> > >> >>>>>>
> >> > >> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <
> >> > tom.barber@meteorite.bi
> >> > >> >
> >> > >> >>>>>> wrote:
> >> > >> >>>>>>
> >> > >> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch
> >> 2.0-dev
> >> > or
> >> > >> >>>>>> similar
> >> > >> >>>>>>> then branch the branch to uprade solr and lucene outside of
> >> that
> >> > >> >> scope
> >> > >> >>>>>> so
> >> > >> >>>>>>> we don't destroy the semi working avro upgrades ;)
> >> > >> >>>>>>>
> >> > >> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
> >> > >> tom.barber@meteorite.bi
> >> > >> >>>
> >> > >> >>>>>>> wrote:
> >> > >> >>>>>>>
> >> > >> >>>>>>>> Coolio!
> >> > >> >>>>>>>>
> >> > >> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
> >> > >> >> mattmann@apache.org>
> >> > >> >>>>>>>> wrote:
> >> > >> >>>>>>>>
> >> > >> >>>>>>>>> Yes I was just telling Paul R. the other day we need to
> do
> >> > >> >>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
> >> > >> >>>>>>>>> issues with Big Translate and DRAT using the old Lucene
> >> > >> >>>>>>>>> library. Let’s get it fixed in 2.x.
> >> > >> >>>>>>>>>
> >> > >> >>>>>>>>> Cheers,
> >> > >> >>>>>>>>> Chris
> >> > >> >>>>>>>>>
> >> > >> >>>>>>>>>
> >> > >> >>>>>>>>>
> >> > >> >>>>>>>>>
> >> > >> >>>>>>>>>
> >> > >> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <
> tom.barber@meteorite.bi>
> >> > >> >> wrote:
> >> > >> >>>>>>>>>
> >> > >> >>>>>>>>>> Just throwing this one out there as we're making a
> bunch of
> >> > >> >>>>>> breaking
> >> > >> >>>>>>>>>> changes in 2.0.
> >> > >> >>>>>>>>>>
> >> > >> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions within
> OODT?
> >> > >> >>>>>>>>>>
> >> > >> >>>>>>>>>> My feeling is that they are both pretty old and whilst
> >> we're
> >> > >> >> making
> >> > >> >>>>>>> life
> >> > >> >>>>>>>>>> hard for everyone we might as well go the whole hog and
> >> drag
> >> > >> >> them
> >> > >> >>>>>> into
> >> > >> >>>>>>>>> the
> >> > >> >>>>>>>>>> 21st century, but I'm interested in others opinions.
> >> > >> >>>>>>>>>>
> >> > >> >>>>>>>>>> Chris you have a much better understand about those
> things
> >> > than
> >> > >> >>>>>> myself.
> >> > >> >>>>>>>>>>
> >> > >> >>>>>>>>>> Tom
> >> > >> >>>>>>
> >> > >> >>>>>>
> >> > >> >>>>>>
> >> > >> >>>>>> --
> >> > >> >>>>>> *Lewis*
> >> > >> >>
> >> > >> >>
> >> > >>
> >> >
> >>
> >
> >
> >
> >--
> >*Lewis*
>
>
Re: Upgrading Lucene and Solr
Posted by Chris Mattmann <ch...@gmail.com>.
can you merge to master?
—
Chris Mattmann
chris.mattmann@gmail.com
On 7/14/16, 4:22 PM, "Lewis John Mcgibbney" <le...@gmail.com> wrote:
>Dyno mate
>
>On Mon, Jul 11, 2016 at 12:02 PM, Tom Barber <to...@meteorite.bi>
>wrote:
>
>> [INFO] Installing /root/oodt/pom.xml to
>>
>> /root/.m2/repository/org/apache/oodt/oodt/1.1-SNAPSHOT/oodt-1.1-SNAPSHOT.pom
>>
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] OODT Core .......................................... SUCCESS [
>> 1.290 s]
>> [INFO] Common Utilities ................................... SUCCESS [
>> 15.731 s]
>> [INFO] CAS Command Line Interface ......................... SUCCESS [
>> 4.385 s]
>> [INFO] Process Control System Input Data Package .......... SUCCESS [
>> 1.186 s]
>> [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container
>> SUCCESS [ 2.357 s]
>> [INFO] CAS Protocol ....................................... SUCCESS [
>> 5.509 s]
>> [INFO] CAS Protocol FTP Implementation .................... SUCCESS [
>> 2.993 s]
>> [INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
>> 10.829 s]
>> [INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
>> 3.236 s]
>> [INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
>> 2.445 s]
>> [INFO] Query Expression ................................... SUCCESS [
>> 2.873 s]
>> [INFO] OODT Single Sign On Security Package ............... SUCCESS [
>> 1.121 s]
>> [INFO] Catalog and Archive File Management Component ...... SUCCESS [03:36
>> min]
>> [INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
>> 9.659 s]
>> [INFO] Catalog and Archive Resource Management Component .. SUCCESS [
>> 14.399 s]
>> [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
>> 34.506 s]
>> [INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
>> 15.162 s]
>> [INFO] CAS Curation Web Services .......................... SUCCESS [
>> 10.538 s]
>> [INFO] Process Control System Core Package ................ SUCCESS [
>> 9.441 s]
>> [INFO] OODT Wicket Web Components ......................... SUCCESS [
>> 2.837 s]
>> [INFO] CAS Curation Interface ............................. SUCCESS [
>> 15.108 s]
>> [INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
>> 10.969 s]
>> [INFO] CAS Installer Maven Mojo ........................... SUCCESS [
>> 1.550 s]
>> [INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
>> 0.738 s]
>> [INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
>> 0.490 s]
>> [INFO] OODT :: Archetypes ................................. SUCCESS [
>> 0.014 s]
>> [INFO] CAS Push-Pull-Framework ............................ SUCCESS [
>> 10.270 s]
>> [INFO] Product Service .................................... SUCCESS [
>> 1.391 s]
>> [INFO] Profile Service .................................... SUCCESS [
>> 3.924 s]
>> [INFO] OODT Web Grid ...................................... SUCCESS [
>> 8.825 s]
>> [INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
>> 5.605 s]
>> [INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
>> 14.057 s]
>> [INFO] CAS File Manager Browser Web App ................... SUCCESS [
>> 12.046 s]
>> [INFO] CAS Product Server Web Application ................. SUCCESS [
>> 12.723 s]
>> [INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
>> 4.691 s]
>> [INFO] Catalog and Archive File Management Browser ........ SUCCESS [
>> 5.015 s]
>> [INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
>> 7.072 s]
>> [INFO] Process Control System Operator Interface Webapp ... SUCCESS [
>> 10.877 s]
>> [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
>> 5.047 s]
>> [INFO] Apache OODT ........................................ SUCCESS [
>> 0.221 s]
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD SUCCESS
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 08:18 min
>> [INFO] Finished at: 2016-07-11T19:00:29+00:00
>> [INFO] Final Memory: 136M/2984M
>> [INFO]
>> ------------------------------------------------------------------------
>> root@ubuntu:~/oodt# git status
>> On branch 1.x-lucene
>> Your branch is up-to-date with 'origin/1.x-lucene'.
>>
>>
>> You should be good to go.
>>
>> On Mon, Jul 11, 2016 at 4:19 PM, Mattmann, Chris A (3980) <
>> chris.a.mattmann@jpl.nasa.gov> wrote:
>>
>> > I have a beautiful use case for BigTranslate that I can test this
>> > in pretty immediately - so commit to 1.x soon and I’ll be your
>> > guinea pig.
>> >
>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> > Chris Mattmann, Ph.D.
>> > Chief Architect
>> > Instrument Software and Science Data Systems Section (398)
>> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > Office: 168-519, Mailstop: 168-527
>> > Email: chris.a.mattmann@nasa.gov
>> > WWW: http://sunset.usc.edu/~mattmann/
>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> > Director, Information Retrieval and Data Science Group (IRDS)
>> > Adjunct Associate Professor, Computer Science Department
>> > University of Southern California, Los Angeles, CA 90089 USA
>> > WWW: http://irds.usc.edu/
>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On 7/11/16, 11:17 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>> >
>> > >INFO] Reactor Summary:
>> > >[INFO]
>> > >[INFO] OODT Core .......................................... SUCCESS [
>> > >10.222 s]
>> > >[INFO] Common Utilities ................................... SUCCESS [
>> > >17.715 s]
>> > >[INFO] CAS Command Line Interface ......................... SUCCESS [
>> > > 9.428 s]
>> > >[INFO] Process Control System Input Data Package .......... SUCCESS [
>> > > 1.564 s]
>> > >[INFO] Catalog and Archive Service Generic Multi-valued Metadata
>> Container
>> > >SUCCESS [ 3.094 s]
>> > >[INFO] CAS Protocol ....................................... SUCCESS [
>> > > 9.456 s]
>> > >[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
>> > > 4.550 s]
>> > >[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
>> > >16.927 s]
>> > >[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
>> > > 4.857 s]
>> > >[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
>> > > 4.020 s]
>> > >[INFO] Query Expression ................................... SUCCESS [
>> > > 3.140 s]
>> > >[INFO] OODT Single Sign On Security Package ............... SUCCESS [
>> > > 2.458 s]
>> > >[INFO] Catalog and Archive File Management Component ...... SUCCESS
>> [04:32
>> > >min]
>> > >[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
>> > >15.820 s]
>> > >[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
>> > >23.145 s]
>> > >[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
>> > >59.957 s]
>> > >[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
>> > >22.978 s]
>> > >[INFO] CAS Curation Web Services .......................... SUCCESS [
>> > > 9.949 s]
>> > >[INFO] Process Control System Core Package ................ SUCCESS [
>> > >13.890 s]
>> > >[INFO] OODT Wicket Web Components ......................... SUCCESS [
>> > > 4.355 s]
>> > >[INFO] CAS Curation Interface ............................. SUCCESS [
>> > > 9.994 s]
>> > >[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
>> > >18.437 s]
>> > >[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
>> > > 6.322 s]
>> > >[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
>> > > 9.322 s]
>> > >[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
>> > > 1.049 s]
>> > >[INFO] OODT :: Archetypes ................................. SUCCESS [
>> > > 0.014 s]
>> > >[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
>> > >15.797 s]
>> > >[INFO] Product Service .................................... SUCCESS [
>> > > 1.704 s]
>> > >[INFO] Profile Service .................................... SUCCESS [
>> > > 7.741 s]
>> > >[INFO] OODT Web Grid ...................................... SUCCESS [
>> > >19.080 s]
>> > >[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
>> > >12.202 s]
>> > >[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
>> > >21.353 s]
>> > >[INFO] CAS File Manager Browser Web App ................... SUCCESS [
>> > > 9.061 s]
>> > >[INFO] CAS Product Server Web Application ................. SUCCESS [
>> > >12.943 s]
>> > >[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
>> > > 3.712 s]
>> > >[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
>> > > 8.218 s]
>> > >[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
>> > >10.785 s]
>> > >[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
>> > >13.757 s]
>> > >[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
>> > > 7.015 s]
>> > >[INFO] Apache OODT ........................................ SUCCESS [
>> > > 0.020 s]
>> > >[INFO]
>> > >------------------------------------------------------------------------
>> > >[INFO] BUILD SUCCESS
>> > >[INFO]
>> > >------------------------------------------------------------------------
>> > >[INFO] Total time: 11:40 min
>> > >[INFO] Finished at: 2016-07-11T15:11:00+00:00
>> > >[INFO] Final Memory: 167M/2913M
>> > >[INFO]
>> > >------------------------------------------------------------------------
>> > >root@ubuntu:~/oodt#
>> > >
>> > >
>> > >Okay I finished up the workflow changes last night and all the tests
>> pass
>> > >on 2.x so I'll lift those commits tonight and patch the 1.x branch.
>> > >
>> > >Just because the tests pass, don't expect it to be perfect, I don't know
>> > >lucene I just manipulated what existed to work with the new API, someone
>> > >who knows what they are doing needs to take a look, specifically at the
>> > >todoc() methods in Filemanager and Workflow.
>> > >
>> > >Tom
>> > >
>> > >On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
>> > >chris.a.mattmann@jpl.nasa.gov> wrote:
>> > >
>> > >> Yeah we can back port the Lucene part to 1.x
>> > >>
>> > >> Sent from my iPhone
>> > >>
>> > >> > On Jul 8, 2016, at 11:15 PM, Tom Barber <to...@meteorite.bi>
>> > wrote:
>> > >> >
>> > >> > yeah there's no reason we can't diff the branch and merge it into
>> 1.0
>> > as
>> > >> > well
>> > >> >> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org> wrote:
>> > >> >>
>> > >> >> awesome!!!
>> > >> >>
>> > >> >> Please backport to 1.x :)
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >>
>> > >> >>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
>> > >> >>>
>> > >> >>> There you go, running commentary over. The filemanager test suite
>> > now
>> > >> >>> passes on Lucene 6, I'll fix up the rest next week.
>> > >> >>>
>> > >> >>> Tom
>> > >> >>>
>> > >> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <
>> > tom.barber@meteorite.bi>
>> > >> >>> wrote:
>> > >> >>>
>> > >> >>>> Better than I thought actually:
>> > >> >>>>
>> > >> >>>> Tests in error:
>> > >> >>>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
>> > >> >>>> CmdLineAction ...
>> > >> >>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
>> > >> >>>> org.apache.oodt.cas.cli.excep...
>> > >> >>>>
>> > >> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
>> > >> >>>>
>> > >> >>>> The failure is actually one Avro. So only 2 failing on lucene 6
>> in
>> > the
>> > >> >> FM.
>> > >> >>>>
>> > >> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <
>> > tom.barber@meteorite.bi>
>> > >> >>>> wrote:
>> > >> >>>>
>> > >> >>>>> Just FYI:
>> > >> >>>>>
>> > >> >>>>> Doesn't work yet, but I've pushed a lucene branch which is based
>> > on
>> > >> 2.0
>> > >> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has
>> > been
>> > >> 75%
>> > >> >>>>> (guess) converted and tests are half passing.
>> > >> >>>>>
>> > >> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
>> > >> >>>>> lewis.mcgibbney@gmail.com> wrote:
>> > >> >>>>>
>> > >> >>>>>> ACK
>> > >> >>>>>>
>> > >> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <
>> > tom.barber@meteorite.bi
>> > >> >
>> > >> >>>>>> wrote:
>> > >> >>>>>>
>> > >> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch
>> 2.0-dev
>> > or
>> > >> >>>>>> similar
>> > >> >>>>>>> then branch the branch to uprade solr and lucene outside of
>> that
>> > >> >> scope
>> > >> >>>>>> so
>> > >> >>>>>>> we don't destroy the semi working avro upgrades ;)
>> > >> >>>>>>>
>> > >> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
>> > >> tom.barber@meteorite.bi
>> > >> >>>
>> > >> >>>>>>> wrote:
>> > >> >>>>>>>
>> > >> >>>>>>>> Coolio!
>> > >> >>>>>>>>
>> > >> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
>> > >> >> mattmann@apache.org>
>> > >> >>>>>>>> wrote:
>> > >> >>>>>>>>
>> > >> >>>>>>>>> Yes I was just telling Paul R. the other day we need to do
>> > >> >>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
>> > >> >>>>>>>>> issues with Big Translate and DRAT using the old Lucene
>> > >> >>>>>>>>> library. Let’s get it fixed in 2.x.
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> Cheers,
>> > >> >>>>>>>>> Chris
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>
>> > >> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi>
>> > >> >> wrote:
>> > >> >>>>>>>>>
>> > >> >>>>>>>>>> Just throwing this one out there as we're making a bunch of
>> > >> >>>>>> breaking
>> > >> >>>>>>>>>> changes in 2.0.
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions within OODT?
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>> My feeling is that they are both pretty old and whilst
>> we're
>> > >> >> making
>> > >> >>>>>>> life
>> > >> >>>>>>>>>> hard for everyone we might as well go the whole hog and
>> drag
>> > >> >> them
>> > >> >>>>>> into
>> > >> >>>>>>>>> the
>> > >> >>>>>>>>>> 21st century, but I'm interested in others opinions.
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>> Chris you have a much better understand about those things
>> > than
>> > >> >>>>>> myself.
>> > >> >>>>>>>>>>
>> > >> >>>>>>>>>> Tom
>> > >> >>>>>>
>> > >> >>>>>>
>> > >> >>>>>>
>> > >> >>>>>> --
>> > >> >>>>>> *Lewis*
>> > >> >>
>> > >> >>
>> > >>
>> >
>>
>
>
>
>--
>*Lewis*
Re: Upgrading Lucene and Solr
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Dyno mate
On Mon, Jul 11, 2016 at 12:02 PM, Tom Barber <to...@meteorite.bi>
wrote:
> [INFO] Installing /root/oodt/pom.xml to
>
> /root/.m2/repository/org/apache/oodt/oodt/1.1-SNAPSHOT/oodt-1.1-SNAPSHOT.pom
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] OODT Core .......................................... SUCCESS [
> 1.290 s]
> [INFO] Common Utilities ................................... SUCCESS [
> 15.731 s]
> [INFO] CAS Command Line Interface ......................... SUCCESS [
> 4.385 s]
> [INFO] Process Control System Input Data Package .......... SUCCESS [
> 1.186 s]
> [INFO] Catalog and Archive Service Generic Multi-valued Metadata Container
> SUCCESS [ 2.357 s]
> [INFO] CAS Protocol ....................................... SUCCESS [
> 5.509 s]
> [INFO] CAS Protocol FTP Implementation .................... SUCCESS [
> 2.993 s]
> [INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
> 10.829 s]
> [INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
> 3.236 s]
> [INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
> 2.445 s]
> [INFO] Query Expression ................................... SUCCESS [
> 2.873 s]
> [INFO] OODT Single Sign On Security Package ............... SUCCESS [
> 1.121 s]
> [INFO] Catalog and Archive File Management Component ...... SUCCESS [03:36
> min]
> [INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
> 9.659 s]
> [INFO] Catalog and Archive Resource Management Component .. SUCCESS [
> 14.399 s]
> [INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
> 34.506 s]
> [INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
> 15.162 s]
> [INFO] CAS Curation Web Services .......................... SUCCESS [
> 10.538 s]
> [INFO] Process Control System Core Package ................ SUCCESS [
> 9.441 s]
> [INFO] OODT Wicket Web Components ......................... SUCCESS [
> 2.837 s]
> [INFO] CAS Curation Interface ............................. SUCCESS [
> 15.108 s]
> [INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
> 10.969 s]
> [INFO] CAS Installer Maven Mojo ........................... SUCCESS [
> 1.550 s]
> [INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
> 0.738 s]
> [INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
> 0.490 s]
> [INFO] OODT :: Archetypes ................................. SUCCESS [
> 0.014 s]
> [INFO] CAS Push-Pull-Framework ............................ SUCCESS [
> 10.270 s]
> [INFO] Product Service .................................... SUCCESS [
> 1.391 s]
> [INFO] Profile Service .................................... SUCCESS [
> 3.924 s]
> [INFO] OODT Web Grid ...................................... SUCCESS [
> 8.825 s]
> [INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
> 5.605 s]
> [INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
> 14.057 s]
> [INFO] CAS File Manager Browser Web App ................... SUCCESS [
> 12.046 s]
> [INFO] CAS Product Server Web Application ................. SUCCESS [
> 12.723 s]
> [INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
> 4.691 s]
> [INFO] Catalog and Archive File Management Browser ........ SUCCESS [
> 5.015 s]
> [INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
> 7.072 s]
> [INFO] Process Control System Operator Interface Webapp ... SUCCESS [
> 10.877 s]
> [INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
> 5.047 s]
> [INFO] Apache OODT ........................................ SUCCESS [
> 0.221 s]
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 08:18 min
> [INFO] Finished at: 2016-07-11T19:00:29+00:00
> [INFO] Final Memory: 136M/2984M
> [INFO]
> ------------------------------------------------------------------------
> root@ubuntu:~/oodt# git status
> On branch 1.x-lucene
> Your branch is up-to-date with 'origin/1.x-lucene'.
>
>
> You should be good to go.
>
> On Mon, Jul 11, 2016 at 4:19 PM, Mattmann, Chris A (3980) <
> chris.a.mattmann@jpl.nasa.gov> wrote:
>
> > I have a beautiful use case for BigTranslate that I can test this
> > in pretty immediately - so commit to 1.x soon and I’ll be your
> > guinea pig.
> >
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Chris Mattmann, Ph.D.
> > Chief Architect
> > Instrument Software and Science Data Systems Section (398)
> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> > Office: 168-519, Mailstop: 168-527
> > Email: chris.a.mattmann@nasa.gov
> > WWW: http://sunset.usc.edu/~mattmann/
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Director, Information Retrieval and Data Science Group (IRDS)
> > Adjunct Associate Professor, Computer Science Department
> > University of Southern California, Los Angeles, CA 90089 USA
> > WWW: http://irds.usc.edu/
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > On 7/11/16, 11:17 AM, "Tom Barber" <to...@meteorite.bi> wrote:
> >
> > >INFO] Reactor Summary:
> > >[INFO]
> > >[INFO] OODT Core .......................................... SUCCESS [
> > >10.222 s]
> > >[INFO] Common Utilities ................................... SUCCESS [
> > >17.715 s]
> > >[INFO] CAS Command Line Interface ......................... SUCCESS [
> > > 9.428 s]
> > >[INFO] Process Control System Input Data Package .......... SUCCESS [
> > > 1.564 s]
> > >[INFO] Catalog and Archive Service Generic Multi-valued Metadata
> Container
> > >SUCCESS [ 3.094 s]
> > >[INFO] CAS Protocol ....................................... SUCCESS [
> > > 9.456 s]
> > >[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
> > > 4.550 s]
> > >[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
> > >16.927 s]
> > >[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
> > > 4.857 s]
> > >[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
> > > 4.020 s]
> > >[INFO] Query Expression ................................... SUCCESS [
> > > 3.140 s]
> > >[INFO] OODT Single Sign On Security Package ............... SUCCESS [
> > > 2.458 s]
> > >[INFO] Catalog and Archive File Management Component ...... SUCCESS
> [04:32
> > >min]
> > >[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
> > >15.820 s]
> > >[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
> > >23.145 s]
> > >[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
> > >59.957 s]
> > >[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
> > >22.978 s]
> > >[INFO] CAS Curation Web Services .......................... SUCCESS [
> > > 9.949 s]
> > >[INFO] Process Control System Core Package ................ SUCCESS [
> > >13.890 s]
> > >[INFO] OODT Wicket Web Components ......................... SUCCESS [
> > > 4.355 s]
> > >[INFO] CAS Curation Interface ............................. SUCCESS [
> > > 9.994 s]
> > >[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
> > >18.437 s]
> > >[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
> > > 6.322 s]
> > >[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
> > > 9.322 s]
> > >[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
> > > 1.049 s]
> > >[INFO] OODT :: Archetypes ................................. SUCCESS [
> > > 0.014 s]
> > >[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
> > >15.797 s]
> > >[INFO] Product Service .................................... SUCCESS [
> > > 1.704 s]
> > >[INFO] Profile Service .................................... SUCCESS [
> > > 7.741 s]
> > >[INFO] OODT Web Grid ...................................... SUCCESS [
> > >19.080 s]
> > >[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
> > >12.202 s]
> > >[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
> > >21.353 s]
> > >[INFO] CAS File Manager Browser Web App ................... SUCCESS [
> > > 9.061 s]
> > >[INFO] CAS Product Server Web Application ................. SUCCESS [
> > >12.943 s]
> > >[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
> > > 3.712 s]
> > >[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
> > > 8.218 s]
> > >[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
> > >10.785 s]
> > >[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
> > >13.757 s]
> > >[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
> > > 7.015 s]
> > >[INFO] Apache OODT ........................................ SUCCESS [
> > > 0.020 s]
> > >[INFO]
> > >------------------------------------------------------------------------
> > >[INFO] BUILD SUCCESS
> > >[INFO]
> > >------------------------------------------------------------------------
> > >[INFO] Total time: 11:40 min
> > >[INFO] Finished at: 2016-07-11T15:11:00+00:00
> > >[INFO] Final Memory: 167M/2913M
> > >[INFO]
> > >------------------------------------------------------------------------
> > >root@ubuntu:~/oodt#
> > >
> > >
> > >Okay I finished up the workflow changes last night and all the tests
> pass
> > >on 2.x so I'll lift those commits tonight and patch the 1.x branch.
> > >
> > >Just because the tests pass, don't expect it to be perfect, I don't know
> > >lucene I just manipulated what existed to work with the new API, someone
> > >who knows what they are doing needs to take a look, specifically at the
> > >todoc() methods in Filemanager and Workflow.
> > >
> > >Tom
> > >
> > >On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
> > >chris.a.mattmann@jpl.nasa.gov> wrote:
> > >
> > >> Yeah we can back port the Lucene part to 1.x
> > >>
> > >> Sent from my iPhone
> > >>
> > >> > On Jul 8, 2016, at 11:15 PM, Tom Barber <to...@meteorite.bi>
> > wrote:
> > >> >
> > >> > yeah there's no reason we can't diff the branch and merge it into
> 1.0
> > as
> > >> > well
> > >> >> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org> wrote:
> > >> >>
> > >> >> awesome!!!
> > >> >>
> > >> >> Please backport to 1.x :)
> > >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
> > >> >>>
> > >> >>> There you go, running commentary over. The filemanager test suite
> > now
> > >> >>> passes on Lucene 6, I'll fix up the rest next week.
> > >> >>>
> > >> >>> Tom
> > >> >>>
> > >> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <
> > tom.barber@meteorite.bi>
> > >> >>> wrote:
> > >> >>>
> > >> >>>> Better than I thought actually:
> > >> >>>>
> > >> >>>> Tests in error:
> > >> >>>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
> > >> >>>> CmdLineAction ...
> > >> >>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
> > >> >>>> org.apache.oodt.cas.cli.excep...
> > >> >>>>
> > >> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
> > >> >>>>
> > >> >>>> The failure is actually one Avro. So only 2 failing on lucene 6
> in
> > the
> > >> >> FM.
> > >> >>>>
> > >> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <
> > tom.barber@meteorite.bi>
> > >> >>>> wrote:
> > >> >>>>
> > >> >>>>> Just FYI:
> > >> >>>>>
> > >> >>>>> Doesn't work yet, but I've pushed a lucene branch which is based
> > on
> > >> 2.0
> > >> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has
> > been
> > >> 75%
> > >> >>>>> (guess) converted and tests are half passing.
> > >> >>>>>
> > >> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
> > >> >>>>> lewis.mcgibbney@gmail.com> wrote:
> > >> >>>>>
> > >> >>>>>> ACK
> > >> >>>>>>
> > >> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <
> > tom.barber@meteorite.bi
> > >> >
> > >> >>>>>> wrote:
> > >> >>>>>>
> > >> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch
> 2.0-dev
> > or
> > >> >>>>>> similar
> > >> >>>>>>> then branch the branch to uprade solr and lucene outside of
> that
> > >> >> scope
> > >> >>>>>> so
> > >> >>>>>>> we don't destroy the semi working avro upgrades ;)
> > >> >>>>>>>
> > >> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
> > >> tom.barber@meteorite.bi
> > >> >>>
> > >> >>>>>>> wrote:
> > >> >>>>>>>
> > >> >>>>>>>> Coolio!
> > >> >>>>>>>>
> > >> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
> > >> >> mattmann@apache.org>
> > >> >>>>>>>> wrote:
> > >> >>>>>>>>
> > >> >>>>>>>>> Yes I was just telling Paul R. the other day we need to do
> > >> >>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
> > >> >>>>>>>>> issues with Big Translate and DRAT using the old Lucene
> > >> >>>>>>>>> library. Let’s get it fixed in 2.x.
> > >> >>>>>>>>>
> > >> >>>>>>>>> Cheers,
> > >> >>>>>>>>> Chris
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>>
> > >> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi>
> > >> >> wrote:
> > >> >>>>>>>>>
> > >> >>>>>>>>>> Just throwing this one out there as we're making a bunch of
> > >> >>>>>> breaking
> > >> >>>>>>>>>> changes in 2.0.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions within OODT?
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> My feeling is that they are both pretty old and whilst
> we're
> > >> >> making
> > >> >>>>>>> life
> > >> >>>>>>>>>> hard for everyone we might as well go the whole hog and
> drag
> > >> >> them
> > >> >>>>>> into
> > >> >>>>>>>>> the
> > >> >>>>>>>>>> 21st century, but I'm interested in others opinions.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Chris you have a much better understand about those things
> > than
> > >> >>>>>> myself.
> > >> >>>>>>>>>>
> > >> >>>>>>>>>> Tom
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>>
> > >> >>>>>> --
> > >> >>>>>> *Lewis*
> > >> >>
> > >> >>
> > >>
> >
>
--
*Lewis*
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
[INFO] Installing /root/oodt/pom.xml to
/root/.m2/repository/org/apache/oodt/oodt/1.1-SNAPSHOT/oodt-1.1-SNAPSHOT.pom
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] OODT Core .......................................... SUCCESS [
1.290 s]
[INFO] Common Utilities ................................... SUCCESS [
15.731 s]
[INFO] CAS Command Line Interface ......................... SUCCESS [
4.385 s]
[INFO] Process Control System Input Data Package .......... SUCCESS [
1.186 s]
[INFO] Catalog and Archive Service Generic Multi-valued Metadata Container
SUCCESS [ 2.357 s]
[INFO] CAS Protocol ....................................... SUCCESS [
5.509 s]
[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
2.993 s]
[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
10.829 s]
[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
3.236 s]
[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
2.445 s]
[INFO] Query Expression ................................... SUCCESS [
2.873 s]
[INFO] OODT Single Sign On Security Package ............... SUCCESS [
1.121 s]
[INFO] Catalog and Archive File Management Component ...... SUCCESS [03:36
min]
[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
9.659 s]
[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
14.399 s]
[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
34.506 s]
[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
15.162 s]
[INFO] CAS Curation Web Services .......................... SUCCESS [
10.538 s]
[INFO] Process Control System Core Package ................ SUCCESS [
9.441 s]
[INFO] OODT Wicket Web Components ......................... SUCCESS [
2.837 s]
[INFO] CAS Curation Interface ............................. SUCCESS [
15.108 s]
[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
10.969 s]
[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
1.550 s]
[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
0.738 s]
[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
0.490 s]
[INFO] OODT :: Archetypes ................................. SUCCESS [
0.014 s]
[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
10.270 s]
[INFO] Product Service .................................... SUCCESS [
1.391 s]
[INFO] Profile Service .................................... SUCCESS [
3.924 s]
[INFO] OODT Web Grid ...................................... SUCCESS [
8.825 s]
[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
5.605 s]
[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
14.057 s]
[INFO] CAS File Manager Browser Web App ................... SUCCESS [
12.046 s]
[INFO] CAS Product Server Web Application ................. SUCCESS [
12.723 s]
[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
4.691 s]
[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
5.015 s]
[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
7.072 s]
[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
10.877 s]
[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
5.047 s]
[INFO] Apache OODT ........................................ SUCCESS [
0.221 s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 08:18 min
[INFO] Finished at: 2016-07-11T19:00:29+00:00
[INFO] Final Memory: 136M/2984M
[INFO]
------------------------------------------------------------------------
root@ubuntu:~/oodt# git status
On branch 1.x-lucene
Your branch is up-to-date with 'origin/1.x-lucene'.
You should be good to go.
On Mon, Jul 11, 2016 at 4:19 PM, Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> wrote:
> I have a beautiful use case for BigTranslate that I can test this
> in pretty immediately - so commit to 1.x soon and I’ll be your
> guinea pig.
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Chief Architect
> Instrument Software and Science Data Systems Section (398)
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 168-519, Mailstop: 168-527
> Email: chris.a.mattmann@nasa.gov
> WWW: http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Director, Information Retrieval and Data Science Group (IRDS)
> Adjunct Associate Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> WWW: http://irds.usc.edu/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
>
>
>
>
>
>
>
> On 7/11/16, 11:17 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>
> >INFO] Reactor Summary:
> >[INFO]
> >[INFO] OODT Core .......................................... SUCCESS [
> >10.222 s]
> >[INFO] Common Utilities ................................... SUCCESS [
> >17.715 s]
> >[INFO] CAS Command Line Interface ......................... SUCCESS [
> > 9.428 s]
> >[INFO] Process Control System Input Data Package .......... SUCCESS [
> > 1.564 s]
> >[INFO] Catalog and Archive Service Generic Multi-valued Metadata Container
> >SUCCESS [ 3.094 s]
> >[INFO] CAS Protocol ....................................... SUCCESS [
> > 9.456 s]
> >[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
> > 4.550 s]
> >[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
> >16.927 s]
> >[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
> > 4.857 s]
> >[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
> > 4.020 s]
> >[INFO] Query Expression ................................... SUCCESS [
> > 3.140 s]
> >[INFO] OODT Single Sign On Security Package ............... SUCCESS [
> > 2.458 s]
> >[INFO] Catalog and Archive File Management Component ...... SUCCESS [04:32
> >min]
> >[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
> >15.820 s]
> >[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
> >23.145 s]
> >[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
> >59.957 s]
> >[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
> >22.978 s]
> >[INFO] CAS Curation Web Services .......................... SUCCESS [
> > 9.949 s]
> >[INFO] Process Control System Core Package ................ SUCCESS [
> >13.890 s]
> >[INFO] OODT Wicket Web Components ......................... SUCCESS [
> > 4.355 s]
> >[INFO] CAS Curation Interface ............................. SUCCESS [
> > 9.994 s]
> >[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
> >18.437 s]
> >[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
> > 6.322 s]
> >[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
> > 9.322 s]
> >[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
> > 1.049 s]
> >[INFO] OODT :: Archetypes ................................. SUCCESS [
> > 0.014 s]
> >[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
> >15.797 s]
> >[INFO] Product Service .................................... SUCCESS [
> > 1.704 s]
> >[INFO] Profile Service .................................... SUCCESS [
> > 7.741 s]
> >[INFO] OODT Web Grid ...................................... SUCCESS [
> >19.080 s]
> >[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
> >12.202 s]
> >[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
> >21.353 s]
> >[INFO] CAS File Manager Browser Web App ................... SUCCESS [
> > 9.061 s]
> >[INFO] CAS Product Server Web Application ................. SUCCESS [
> >12.943 s]
> >[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
> > 3.712 s]
> >[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
> > 8.218 s]
> >[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
> >10.785 s]
> >[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
> >13.757 s]
> >[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
> > 7.015 s]
> >[INFO] Apache OODT ........................................ SUCCESS [
> > 0.020 s]
> >[INFO]
> >------------------------------------------------------------------------
> >[INFO] BUILD SUCCESS
> >[INFO]
> >------------------------------------------------------------------------
> >[INFO] Total time: 11:40 min
> >[INFO] Finished at: 2016-07-11T15:11:00+00:00
> >[INFO] Final Memory: 167M/2913M
> >[INFO]
> >------------------------------------------------------------------------
> >root@ubuntu:~/oodt#
> >
> >
> >Okay I finished up the workflow changes last night and all the tests pass
> >on 2.x so I'll lift those commits tonight and patch the 1.x branch.
> >
> >Just because the tests pass, don't expect it to be perfect, I don't know
> >lucene I just manipulated what existed to work with the new API, someone
> >who knows what they are doing needs to take a look, specifically at the
> >todoc() methods in Filemanager and Workflow.
> >
> >Tom
> >
> >On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
> >chris.a.mattmann@jpl.nasa.gov> wrote:
> >
> >> Yeah we can back port the Lucene part to 1.x
> >>
> >> Sent from my iPhone
> >>
> >> > On Jul 8, 2016, at 11:15 PM, Tom Barber <to...@meteorite.bi>
> wrote:
> >> >
> >> > yeah there's no reason we can't diff the branch and merge it into 1.0
> as
> >> > well
> >> >> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org> wrote:
> >> >>
> >> >> awesome!!!
> >> >>
> >> >> Please backport to 1.x :)
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
> >> >>>
> >> >>> There you go, running commentary over. The filemanager test suite
> now
> >> >>> passes on Lucene 6, I'll fix up the rest next week.
> >> >>>
> >> >>> Tom
> >> >>>
> >> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <
> tom.barber@meteorite.bi>
> >> >>> wrote:
> >> >>>
> >> >>>> Better than I thought actually:
> >> >>>>
> >> >>>> Tests in error:
> >> >>>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
> >> >>>> CmdLineAction ...
> >> >>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
> >> >>>> org.apache.oodt.cas.cli.excep...
> >> >>>>
> >> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
> >> >>>>
> >> >>>> The failure is actually one Avro. So only 2 failing on lucene 6 in
> the
> >> >> FM.
> >> >>>>
> >> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <
> tom.barber@meteorite.bi>
> >> >>>> wrote:
> >> >>>>
> >> >>>>> Just FYI:
> >> >>>>>
> >> >>>>> Doesn't work yet, but I've pushed a lucene branch which is based
> on
> >> 2.0
> >> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has
> been
> >> 75%
> >> >>>>> (guess) converted and tests are half passing.
> >> >>>>>
> >> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
> >> >>>>> lewis.mcgibbney@gmail.com> wrote:
> >> >>>>>
> >> >>>>>> ACK
> >> >>>>>>
> >> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <
> tom.barber@meteorite.bi
> >> >
> >> >>>>>> wrote:
> >> >>>>>>
> >> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch 2.0-dev
> or
> >> >>>>>> similar
> >> >>>>>>> then branch the branch to uprade solr and lucene outside of that
> >> >> scope
> >> >>>>>> so
> >> >>>>>>> we don't destroy the semi working avro upgrades ;)
> >> >>>>>>>
> >> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
> >> tom.barber@meteorite.bi
> >> >>>
> >> >>>>>>> wrote:
> >> >>>>>>>
> >> >>>>>>>> Coolio!
> >> >>>>>>>>
> >> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
> >> >> mattmann@apache.org>
> >> >>>>>>>> wrote:
> >> >>>>>>>>
> >> >>>>>>>>> Yes I was just telling Paul R. the other day we need to do
> >> >>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
> >> >>>>>>>>> issues with Big Translate and DRAT using the old Lucene
> >> >>>>>>>>> library. Let’s get it fixed in 2.x.
> >> >>>>>>>>>
> >> >>>>>>>>> Cheers,
> >> >>>>>>>>> Chris
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi>
> >> >> wrote:
> >> >>>>>>>>>
> >> >>>>>>>>>> Just throwing this one out there as we're making a bunch of
> >> >>>>>> breaking
> >> >>>>>>>>>> changes in 2.0.
> >> >>>>>>>>>>
> >> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions within OODT?
> >> >>>>>>>>>>
> >> >>>>>>>>>> My feeling is that they are both pretty old and whilst we're
> >> >> making
> >> >>>>>>> life
> >> >>>>>>>>>> hard for everyone we might as well go the whole hog and drag
> >> >> them
> >> >>>>>> into
> >> >>>>>>>>> the
> >> >>>>>>>>>> 21st century, but I'm interested in others opinions.
> >> >>>>>>>>>>
> >> >>>>>>>>>> Chris you have a much better understand about those things
> than
> >> >>>>>> myself.
> >> >>>>>>>>>>
> >> >>>>>>>>>> Tom
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>> --
> >> >>>>>> *Lewis*
> >> >>
> >> >>
> >>
>
Re: Upgrading Lucene and Solr
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
I have a beautiful use case for BigTranslate that I can test this
in pretty immediately - so commit to 1.x soon and I’ll be your
guinea pig.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Director, Information Retrieval and Data Science Group (IRDS)
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
WWW: http://irds.usc.edu/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
On 7/11/16, 11:17 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>INFO] Reactor Summary:
>[INFO]
>[INFO] OODT Core .......................................... SUCCESS [
>10.222 s]
>[INFO] Common Utilities ................................... SUCCESS [
>17.715 s]
>[INFO] CAS Command Line Interface ......................... SUCCESS [
> 9.428 s]
>[INFO] Process Control System Input Data Package .......... SUCCESS [
> 1.564 s]
>[INFO] Catalog and Archive Service Generic Multi-valued Metadata Container
>SUCCESS [ 3.094 s]
>[INFO] CAS Protocol ....................................... SUCCESS [
> 9.456 s]
>[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
> 4.550 s]
>[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
>16.927 s]
>[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
> 4.857 s]
>[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
> 4.020 s]
>[INFO] Query Expression ................................... SUCCESS [
> 3.140 s]
>[INFO] OODT Single Sign On Security Package ............... SUCCESS [
> 2.458 s]
>[INFO] Catalog and Archive File Management Component ...... SUCCESS [04:32
>min]
>[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
>15.820 s]
>[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
>23.145 s]
>[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
>59.957 s]
>[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
>22.978 s]
>[INFO] CAS Curation Web Services .......................... SUCCESS [
> 9.949 s]
>[INFO] Process Control System Core Package ................ SUCCESS [
>13.890 s]
>[INFO] OODT Wicket Web Components ......................... SUCCESS [
> 4.355 s]
>[INFO] CAS Curation Interface ............................. SUCCESS [
> 9.994 s]
>[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
>18.437 s]
>[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
> 6.322 s]
>[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
> 9.322 s]
>[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
> 1.049 s]
>[INFO] OODT :: Archetypes ................................. SUCCESS [
> 0.014 s]
>[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
>15.797 s]
>[INFO] Product Service .................................... SUCCESS [
> 1.704 s]
>[INFO] Profile Service .................................... SUCCESS [
> 7.741 s]
>[INFO] OODT Web Grid ...................................... SUCCESS [
>19.080 s]
>[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
>12.202 s]
>[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
>21.353 s]
>[INFO] CAS File Manager Browser Web App ................... SUCCESS [
> 9.061 s]
>[INFO] CAS Product Server Web Application ................. SUCCESS [
>12.943 s]
>[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
> 3.712 s]
>[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
> 8.218 s]
>[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
>10.785 s]
>[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
>13.757 s]
>[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
> 7.015 s]
>[INFO] Apache OODT ........................................ SUCCESS [
> 0.020 s]
>[INFO]
>------------------------------------------------------------------------
>[INFO] BUILD SUCCESS
>[INFO]
>------------------------------------------------------------------------
>[INFO] Total time: 11:40 min
>[INFO] Finished at: 2016-07-11T15:11:00+00:00
>[INFO] Final Memory: 167M/2913M
>[INFO]
>------------------------------------------------------------------------
>root@ubuntu:~/oodt#
>
>
>Okay I finished up the workflow changes last night and all the tests pass
>on 2.x so I'll lift those commits tonight and patch the 1.x branch.
>
>Just because the tests pass, don't expect it to be perfect, I don't know
>lucene I just manipulated what existed to work with the new API, someone
>who knows what they are doing needs to take a look, specifically at the
>todoc() methods in Filemanager and Workflow.
>
>Tom
>
>On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
>chris.a.mattmann@jpl.nasa.gov> wrote:
>
>> Yeah we can back port the Lucene part to 1.x
>>
>> Sent from my iPhone
>>
>> > On Jul 8, 2016, at 11:15 PM, Tom Barber <to...@meteorite.bi> wrote:
>> >
>> > yeah there's no reason we can't diff the branch and merge it into 1.0 as
>> > well
>> >> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org> wrote:
>> >>
>> >> awesome!!!
>> >>
>> >> Please backport to 1.x :)
>> >>
>> >>
>> >>
>> >>
>> >>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
>> >>>
>> >>> There you go, running commentary over. The filemanager test suite now
>> >>> passes on Lucene 6, I'll fix up the rest next week.
>> >>>
>> >>> Tom
>> >>>
>> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <to...@meteorite.bi>
>> >>> wrote:
>> >>>
>> >>>> Better than I thought actually:
>> >>>>
>> >>>> Tests in error:
>> >>>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
>> >>>> CmdLineAction ...
>> >>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
>> >>>> org.apache.oodt.cas.cli.excep...
>> >>>>
>> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
>> >>>>
>> >>>> The failure is actually one Avro. So only 2 failing on lucene 6 in the
>> >> FM.
>> >>>>
>> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <to...@meteorite.bi>
>> >>>> wrote:
>> >>>>
>> >>>>> Just FYI:
>> >>>>>
>> >>>>> Doesn't work yet, but I've pushed a lucene branch which is based on
>> 2.0
>> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been
>> 75%
>> >>>>> (guess) converted and tests are half passing.
>> >>>>>
>> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
>> >>>>> lewis.mcgibbney@gmail.com> wrote:
>> >>>>>
>> >>>>>> ACK
>> >>>>>>
>> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <tom.barber@meteorite.bi
>> >
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
>> >>>>>> similar
>> >>>>>>> then branch the branch to uprade solr and lucene outside of that
>> >> scope
>> >>>>>> so
>> >>>>>>> we don't destroy the semi working avro upgrades ;)
>> >>>>>>>
>> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
>> tom.barber@meteorite.bi
>> >>>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Coolio!
>> >>>>>>>>
>> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
>> >> mattmann@apache.org>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Yes I was just telling Paul R. the other day we need to do
>> >>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
>> >>>>>>>>> issues with Big Translate and DRAT using the old Lucene
>> >>>>>>>>> library. Let’s get it fixed in 2.x.
>> >>>>>>>>>
>> >>>>>>>>> Cheers,
>> >>>>>>>>> Chris
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi>
>> >> wrote:
>> >>>>>>>>>
>> >>>>>>>>>> Just throwing this one out there as we're making a bunch of
>> >>>>>> breaking
>> >>>>>>>>>> changes in 2.0.
>> >>>>>>>>>>
>> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions within OODT?
>> >>>>>>>>>>
>> >>>>>>>>>> My feeling is that they are both pretty old and whilst we're
>> >> making
>> >>>>>>> life
>> >>>>>>>>>> hard for everyone we might as well go the whole hog and drag
>> >> them
>> >>>>>> into
>> >>>>>>>>> the
>> >>>>>>>>>> 21st century, but I'm interested in others opinions.
>> >>>>>>>>>>
>> >>>>>>>>>> Chris you have a much better understand about those things than
>> >>>>>> myself.
>> >>>>>>>>>>
>> >>>>>>>>>> Tom
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> *Lewis*
>> >>
>> >>
>>
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
INFO] Reactor Summary:
[INFO]
[INFO] OODT Core .......................................... SUCCESS [
10.222 s]
[INFO] Common Utilities ................................... SUCCESS [
17.715 s]
[INFO] CAS Command Line Interface ......................... SUCCESS [
9.428 s]
[INFO] Process Control System Input Data Package .......... SUCCESS [
1.564 s]
[INFO] Catalog and Archive Service Generic Multi-valued Metadata Container
SUCCESS [ 3.094 s]
[INFO] CAS Protocol ....................................... SUCCESS [
9.456 s]
[INFO] CAS Protocol FTP Implementation .................... SUCCESS [
4.550 s]
[INFO] CAS Protocol HTTP Implementation ................... SUCCESS [
16.927 s]
[INFO] CAS Protocol IMAPS Implementation .................. SUCCESS [
4.857 s]
[INFO] CAS Protocol SFTP Implementation ................... SUCCESS [
4.020 s]
[INFO] Query Expression ................................... SUCCESS [
3.140 s]
[INFO] OODT Single Sign On Security Package ............... SUCCESS [
2.458 s]
[INFO] Catalog and Archive File Management Component ...... SUCCESS [04:32
min]
[INFO] OODT CAS Virtual Catalog and Integration Service. .. SUCCESS [
15.820 s]
[INFO] Catalog and Archive Resource Management Component .. SUCCESS [
23.145 s]
[INFO] Catalog and Archive Workflow Management Component .. SUCCESS [
59.957 s]
[INFO] Catalog and Archive Crawling Framework ............. SUCCESS [
22.978 s]
[INFO] CAS Curation Web Services .......................... SUCCESS [
9.949 s]
[INFO] Process Control System Core Package ................ SUCCESS [
13.890 s]
[INFO] OODT Wicket Web Components ......................... SUCCESS [
4.355 s]
[INFO] CAS Curation Interface ............................. SUCCESS [
9.994 s]
[INFO] CAS PGE Adaptor Framework .......................... SUCCESS [
18.437 s]
[INFO] CAS Installer Maven Mojo ........................... SUCCESS [
6.322 s]
[INFO] OODT :: Archetypes :: OpsUI ........................ SUCCESS [
9.322 s]
[INFO] OODT :: Archetypes :: RADiX ........................ SUCCESS [
1.049 s]
[INFO] OODT :: Archetypes ................................. SUCCESS [
0.014 s]
[INFO] CAS Push-Pull-Framework ............................ SUCCESS [
15.797 s]
[INFO] Product Service .................................... SUCCESS [
1.704 s]
[INFO] Profile Service .................................... SUCCESS [
7.741 s]
[INFO] OODT Web Grid ...................................... SUCCESS [
19.080 s]
[INFO] XML-configured, DBMS-based Product and Profile Server SUCCESS [
12.202 s]
[INFO] Apache OODT Configurable OPeNDAP Profile Server .... SUCCESS [
21.353 s]
[INFO] CAS File Manager Browser Web App ................... SUCCESS [
9.061 s]
[INFO] CAS Product Server Web Application ................. SUCCESS [
12.943 s]
[INFO] CAS Workflow Manager Monitor Web App ............... SUCCESS [
3.712 s]
[INFO] Catalog and Archive File Management Browser ........ SUCCESS [
8.218 s]
[INFO] Catalog and Archive Workflow Management GUI Editor . SUCCESS [
10.785 s]
[INFO] Process Control System Operator Interface Webapp ... SUCCESS [
13.757 s]
[INFO] OODT Process Control System JAX-RS service layer ... SUCCESS [
7.015 s]
[INFO] Apache OODT ........................................ SUCCESS [
0.020 s]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 11:40 min
[INFO] Finished at: 2016-07-11T15:11:00+00:00
[INFO] Final Memory: 167M/2913M
[INFO]
------------------------------------------------------------------------
root@ubuntu:~/oodt#
Okay I finished up the workflow changes last night and all the tests pass
on 2.x so I'll lift those commits tonight and patch the 1.x branch.
Just because the tests pass, don't expect it to be perfect, I don't know
lucene I just manipulated what existed to work with the new API, someone
who knows what they are doing needs to take a look, specifically at the
todoc() methods in Filemanager and Workflow.
Tom
On Sat, Jul 9, 2016 at 7:18 AM, Mattmann, Chris A (3980) <
chris.a.mattmann@jpl.nasa.gov> wrote:
> Yeah we can back port the Lucene part to 1.x
>
> Sent from my iPhone
>
> > On Jul 8, 2016, at 11:15 PM, Tom Barber <to...@meteorite.bi> wrote:
> >
> > yeah there's no reason we can't diff the branch and merge it into 1.0 as
> > well
> >> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org> wrote:
> >>
> >> awesome!!!
> >>
> >> Please backport to 1.x :)
> >>
> >>
> >>
> >>
> >>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
> >>>
> >>> There you go, running commentary over. The filemanager test suite now
> >>> passes on Lucene 6, I'll fix up the rest next week.
> >>>
> >>> Tom
> >>>
> >>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <to...@meteorite.bi>
> >>> wrote:
> >>>
> >>>> Better than I thought actually:
> >>>>
> >>>> Tests in error:
> >>>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
> >>>> CmdLineAction ...
> >>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
> >>>> org.apache.oodt.cas.cli.excep...
> >>>>
> >>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
> >>>>
> >>>> The failure is actually one Avro. So only 2 failing on lucene 6 in the
> >> FM.
> >>>>
> >>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <to...@meteorite.bi>
> >>>> wrote:
> >>>>
> >>>>> Just FYI:
> >>>>>
> >>>>> Doesn't work yet, but I've pushed a lucene branch which is based on
> 2.0
> >>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been
> 75%
> >>>>> (guess) converted and tests are half passing.
> >>>>>
> >>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
> >>>>> lewis.mcgibbney@gmail.com> wrote:
> >>>>>
> >>>>>> ACK
> >>>>>>
> >>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <tom.barber@meteorite.bi
> >
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
> >>>>>> similar
> >>>>>>> then branch the branch to uprade solr and lucene outside of that
> >> scope
> >>>>>> so
> >>>>>>> we don't destroy the semi working avro upgrades ;)
> >>>>>>>
> >>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <
> tom.barber@meteorite.bi
> >>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Coolio!
> >>>>>>>>
> >>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
> >> mattmann@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Yes I was just telling Paul R. the other day we need to do
> >>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
> >>>>>>>>> issues with Big Translate and DRAT using the old Lucene
> >>>>>>>>> library. Let’s get it fixed in 2.x.
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Chris
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi>
> >> wrote:
> >>>>>>>>>
> >>>>>>>>>> Just throwing this one out there as we're making a bunch of
> >>>>>> breaking
> >>>>>>>>>> changes in 2.0.
> >>>>>>>>>>
> >>>>>>>>>> Is it worth upgrading Lucene and Solr versions within OODT?
> >>>>>>>>>>
> >>>>>>>>>> My feeling is that they are both pretty old and whilst we're
> >> making
> >>>>>>> life
> >>>>>>>>>> hard for everyone we might as well go the whole hog and drag
> >> them
> >>>>>> into
> >>>>>>>>> the
> >>>>>>>>>> 21st century, but I'm interested in others opinions.
> >>>>>>>>>>
> >>>>>>>>>> Chris you have a much better understand about those things than
> >>>>>> myself.
> >>>>>>>>>>
> >>>>>>>>>> Tom
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> *Lewis*
> >>
> >>
>
Re: Upgrading Lucene and Solr
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Yeah we can back port the Lucene part to 1.x
Sent from my iPhone
> On Jul 8, 2016, at 11:15 PM, Tom Barber <to...@meteorite.bi> wrote:
>
> yeah there's no reason we can't diff the branch and merge it into 1.0 as
> well
>> On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org> wrote:
>>
>> awesome!!!
>>
>> Please backport to 1.x :)
>>
>>
>>
>>
>>> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
>>>
>>> There you go, running commentary over. The filemanager test suite now
>>> passes on Lucene 6, I'll fix up the rest next week.
>>>
>>> Tom
>>>
>>>> On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <to...@meteorite.bi>
>>> wrote:
>>>
>>>> Better than I thought actually:
>>>>
>>>> Tests in error:
>>>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
>>>> CmdLineAction ...
>>>> TestFileManagerCli.testLuceneQuery:350 » Runtime
>>>> org.apache.oodt.cas.cli.excep...
>>>>
>>>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
>>>>
>>>> The failure is actually one Avro. So only 2 failing on lucene 6 in the
>> FM.
>>>>
>>>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <to...@meteorite.bi>
>>>> wrote:
>>>>
>>>>> Just FYI:
>>>>>
>>>>> Doesn't work yet, but I've pushed a lucene branch which is based on 2.0
>>>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been 75%
>>>>> (guess) converted and tests are half passing.
>>>>>
>>>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
>>>>> lewis.mcgibbney@gmail.com> wrote:
>>>>>
>>>>>> ACK
>>>>>>
>>>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <to...@meteorite.bi>
>>>>>> wrote:
>>>>>>
>>>>>>> Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
>>>>>> similar
>>>>>>> then branch the branch to uprade solr and lucene outside of that
>> scope
>>>>>> so
>>>>>>> we don't destroy the semi working avro upgrades ;)
>>>>>>>
>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <tom.barber@meteorite.bi
>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Coolio!
>>>>>>>>
>>>>>>>> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
>> mattmann@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yes I was just telling Paul R. the other day we need to do
>>>>>>>>> this, or we’re DOA. I’ve first hand seen the scalability
>>>>>>>>> issues with Big Translate and DRAT using the old Lucene
>>>>>>>>> library. Let’s get it fixed in 2.x.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi>
>> wrote:
>>>>>>>>>
>>>>>>>>>> Just throwing this one out there as we're making a bunch of
>>>>>> breaking
>>>>>>>>>> changes in 2.0.
>>>>>>>>>>
>>>>>>>>>> Is it worth upgrading Lucene and Solr versions within OODT?
>>>>>>>>>>
>>>>>>>>>> My feeling is that they are both pretty old and whilst we're
>> making
>>>>>>> life
>>>>>>>>>> hard for everyone we might as well go the whole hog and drag
>> them
>>>>>> into
>>>>>>>>> the
>>>>>>>>>> 21st century, but I'm interested in others opinions.
>>>>>>>>>>
>>>>>>>>>> Chris you have a much better understand about those things than
>>>>>> myself.
>>>>>>>>>>
>>>>>>>>>> Tom
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> *Lewis*
>>
>>
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
yeah there's no reason we can't diff the branch and merge it into 1.0 as
well
On 9 Jul 2016 00:26, "Chris Mattmann" <ma...@apache.org> wrote:
> awesome!!!
>
> Please backport to 1.x :)
>
>
>
>
> On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
>
> >There you go, running commentary over. The filemanager test suite now
> >passes on Lucene 6, I'll fix up the rest next week.
> >
> >Tom
> >
> >On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <to...@meteorite.bi>
> wrote:
> >
> >> Better than I thought actually:
> >>
> >> Tests in error:
> >> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
> >> CmdLineAction ...
> >> TestFileManagerCli.testLuceneQuery:350 » Runtime
> >> org.apache.oodt.cas.cli.excep...
> >>
> >> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
> >>
> >> The failure is actually one Avro. So only 2 failing on lucene 6 in the
> FM.
> >>
> >> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <to...@meteorite.bi>
> >> wrote:
> >>
> >>> Just FYI:
> >>>
> >>> Doesn't work yet, but I've pushed a lucene branch which is based on 2.0
> >>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been 75%
> >>> (guess) converted and tests are half passing.
> >>>
> >>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
> >>> lewis.mcgibbney@gmail.com> wrote:
> >>>
> >>>> ACK
> >>>>
> >>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <to...@meteorite.bi>
> >>>> wrote:
> >>>>
> >>>> > Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
> >>>> similar
> >>>> > then branch the branch to uprade solr and lucene outside of that
> scope
> >>>> so
> >>>> > we don't destroy the semi working avro upgrades ;)
> >>>> >
> >>>> > On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <tom.barber@meteorite.bi
> >
> >>>> > wrote:
> >>>> >
> >>>> > > Coolio!
> >>>> > >
> >>>> > > On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <
> mattmann@apache.org>
> >>>> > > wrote:
> >>>> > >
> >>>> > >> Yes I was just telling Paul R. the other day we need to do
> >>>> > >> this, or we’re DOA. I’ve first hand seen the scalability
> >>>> > >> issues with Big Translate and DRAT using the old Lucene
> >>>> > >> library. Let’s get it fixed in 2.x.
> >>>> > >>
> >>>> > >> Cheers,
> >>>> > >> Chris
> >>>> > >>
> >>>> > >>
> >>>> > >>
> >>>> > >>
> >>>> > >>
> >>>> > >> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi>
> wrote:
> >>>> > >>
> >>>> > >> >Just throwing this one out there as we're making a bunch of
> >>>> breaking
> >>>> > >> >changes in 2.0.
> >>>> > >> >
> >>>> > >> >Is it worth upgrading Lucene and Solr versions within OODT?
> >>>> > >> >
> >>>> > >> >My feeling is that they are both pretty old and whilst we're
> making
> >>>> > life
> >>>> > >> >hard for everyone we might as well go the whole hog and drag
> them
> >>>> into
> >>>> > >> the
> >>>> > >> >21st century, but I'm interested in others opinions.
> >>>> > >> >
> >>>> > >> >Chris you have a much better understand about those things than
> >>>> myself.
> >>>> > >> >
> >>>> > >> >Tom
> >>>> > >>
> >>>> > >>
> >>>> > >
> >>>> >
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> *Lewis*
> >>>>
> >>>
> >>>
> >>
>
>
Re: Upgrading Lucene and Solr
Posted by Chris Mattmann <ma...@apache.org>.
awesome!!!
Please backport to 1.x :)
On 7/8/16, 3:01 PM, "Tom Barber" <to...@meteorite.bi> wrote:
>There you go, running commentary over. The filemanager test suite now
>passes on Lucene 6, I'll fix up the rest next week.
>
>Tom
>
>On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <to...@meteorite.bi> wrote:
>
>> Better than I thought actually:
>>
>> Tests in error:
>> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
>> CmdLineAction ...
>> TestFileManagerCli.testLuceneQuery:350 » Runtime
>> org.apache.oodt.cas.cli.excep...
>>
>> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
>>
>> The failure is actually one Avro. So only 2 failing on lucene 6 in the FM.
>>
>> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <to...@meteorite.bi>
>> wrote:
>>
>>> Just FYI:
>>>
>>> Doesn't work yet, but I've pushed a lucene branch which is based on 2.0
>>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been 75%
>>> (guess) converted and tests are half passing.
>>>
>>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
>>> lewis.mcgibbney@gmail.com> wrote:
>>>
>>>> ACK
>>>>
>>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <to...@meteorite.bi>
>>>> wrote:
>>>>
>>>> > Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
>>>> similar
>>>> > then branch the branch to uprade solr and lucene outside of that scope
>>>> so
>>>> > we don't destroy the semi working avro upgrades ;)
>>>> >
>>>> > On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <to...@meteorite.bi>
>>>> > wrote:
>>>> >
>>>> > > Coolio!
>>>> > >
>>>> > > On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <ma...@apache.org>
>>>> > > wrote:
>>>> > >
>>>> > >> Yes I was just telling Paul R. the other day we need to do
>>>> > >> this, or we’re DOA. I’ve first hand seen the scalability
>>>> > >> issues with Big Translate and DRAT using the old Lucene
>>>> > >> library. Let’s get it fixed in 2.x.
>>>> > >>
>>>> > >> Cheers,
>>>> > >> Chris
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>>>> > >>
>>>> > >> >Just throwing this one out there as we're making a bunch of
>>>> breaking
>>>> > >> >changes in 2.0.
>>>> > >> >
>>>> > >> >Is it worth upgrading Lucene and Solr versions within OODT?
>>>> > >> >
>>>> > >> >My feeling is that they are both pretty old and whilst we're making
>>>> > life
>>>> > >> >hard for everyone we might as well go the whole hog and drag them
>>>> into
>>>> > >> the
>>>> > >> >21st century, but I'm interested in others opinions.
>>>> > >> >
>>>> > >> >Chris you have a much better understand about those things than
>>>> myself.
>>>> > >> >
>>>> > >> >Tom
>>>> > >>
>>>> > >>
>>>> > >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>
>>>
>>
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
There you go, running commentary over. The filemanager test suite now
passes on Lucene 6, I'll fix up the rest next week.
Tom
On Fri, Jul 8, 2016 at 10:31 PM, Tom Barber <to...@meteorite.bi> wrote:
> Better than I thought actually:
>
> Tests in error:
> TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
> CmdLineAction ...
> TestFileManagerCli.testLuceneQuery:350 » Runtime
> org.apache.oodt.cas.cli.excep...
>
> Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
>
> The failure is actually one Avro. So only 2 failing on lucene 6 in the FM.
>
> On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <to...@meteorite.bi>
> wrote:
>
>> Just FYI:
>>
>> Doesn't work yet, but I've pushed a lucene branch which is based on 2.0
>> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been 75%
>> (guess) converted and tests are half passing.
>>
>> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
>> lewis.mcgibbney@gmail.com> wrote:
>>
>>> ACK
>>>
>>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <to...@meteorite.bi>
>>> wrote:
>>>
>>> > Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
>>> similar
>>> > then branch the branch to uprade solr and lucene outside of that scope
>>> so
>>> > we don't destroy the semi working avro upgrades ;)
>>> >
>>> > On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <to...@meteorite.bi>
>>> > wrote:
>>> >
>>> > > Coolio!
>>> > >
>>> > > On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <ma...@apache.org>
>>> > > wrote:
>>> > >
>>> > >> Yes I was just telling Paul R. the other day we need to do
>>> > >> this, or we’re DOA. I’ve first hand seen the scalability
>>> > >> issues with Big Translate and DRAT using the old Lucene
>>> > >> library. Let’s get it fixed in 2.x.
>>> > >>
>>> > >> Cheers,
>>> > >> Chris
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >>
>>> > >> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>>> > >>
>>> > >> >Just throwing this one out there as we're making a bunch of
>>> breaking
>>> > >> >changes in 2.0.
>>> > >> >
>>> > >> >Is it worth upgrading Lucene and Solr versions within OODT?
>>> > >> >
>>> > >> >My feeling is that they are both pretty old and whilst we're making
>>> > life
>>> > >> >hard for everyone we might as well go the whole hog and drag them
>>> into
>>> > >> the
>>> > >> >21st century, but I'm interested in others opinions.
>>> > >> >
>>> > >> >Chris you have a much better understand about those things than
>>> myself.
>>> > >> >
>>> > >> >Tom
>>> > >>
>>> > >>
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> *Lewis*
>>>
>>
>>
>
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
Better than I thought actually:
Tests in error:
TestLuceneQueryCliAction.testClientTransTrueAndFlatProduct:78 »
CmdLineAction ...
TestFileManagerCli.testLuceneQuery:350 » Runtime
org.apache.oodt.cas.cli.excep...
Tests run: 221, Failures: 1, Errors: 2, Skipped: 0
The failure is actually one Avro. So only 2 failing on lucene 6 in the FM.
On Fri, Jul 8, 2016 at 9:48 PM, Tom Barber <to...@meteorite.bi> wrote:
> Just FYI:
>
> Doesn't work yet, but I've pushed a lucene branch which is based on 2.0
> and upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been 75%
> (guess) converted and tests are half passing.
>
> On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> ACK
>>
>> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <to...@meteorite.bi>
>> wrote:
>>
>> > Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
>> similar
>> > then branch the branch to uprade solr and lucene outside of that scope
>> so
>> > we don't destroy the semi working avro upgrades ;)
>> >
>> > On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <to...@meteorite.bi>
>> > wrote:
>> >
>> > > Coolio!
>> > >
>> > > On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <ma...@apache.org>
>> > > wrote:
>> > >
>> > >> Yes I was just telling Paul R. the other day we need to do
>> > >> this, or we’re DOA. I’ve first hand seen the scalability
>> > >> issues with Big Translate and DRAT using the old Lucene
>> > >> library. Let’s get it fixed in 2.x.
>> > >>
>> > >> Cheers,
>> > >> Chris
>> > >>
>> > >>
>> > >>
>> > >>
>> > >>
>> > >> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>> > >>
>> > >> >Just throwing this one out there as we're making a bunch of breaking
>> > >> >changes in 2.0.
>> > >> >
>> > >> >Is it worth upgrading Lucene and Solr versions within OODT?
>> > >> >
>> > >> >My feeling is that they are both pretty old and whilst we're making
>> > life
>> > >> >hard for everyone we might as well go the whole hog and drag them
>> into
>> > >> the
>> > >> >21st century, but I'm interested in others opinions.
>> > >> >
>> > >> >Chris you have a much better understand about those things than
>> myself.
>> > >> >
>> > >> >Tom
>> > >>
>> > >>
>> > >
>> >
>>
>>
>>
>> --
>> *Lewis*
>>
>
>
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
Just FYI:
Doesn't work yet, but I've pushed a lucene branch which is based on 2.0 and
upgrades lucene to 6.1.0. LuceneCatalog in filemanager has been 75% (guess)
converted and tests are half passing.
On Wed, Jul 6, 2016 at 8:30 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> ACK
>
> On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <to...@meteorite.bi>
> wrote:
>
> > Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or
> similar
> > then branch the branch to uprade solr and lucene outside of that scope so
> > we don't destroy the semi working avro upgrades ;)
> >
> > On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <to...@meteorite.bi>
> > wrote:
> >
> > > Coolio!
> > >
> > > On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <ma...@apache.org>
> > > wrote:
> > >
> > >> Yes I was just telling Paul R. the other day we need to do
> > >> this, or we’re DOA. I’ve first hand seen the scalability
> > >> issues with Big Translate and DRAT using the old Lucene
> > >> library. Let’s get it fixed in 2.x.
> > >>
> > >> Cheers,
> > >> Chris
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
> > >>
> > >> >Just throwing this one out there as we're making a bunch of breaking
> > >> >changes in 2.0.
> > >> >
> > >> >Is it worth upgrading Lucene and Solr versions within OODT?
> > >> >
> > >> >My feeling is that they are both pretty old and whilst we're making
> > life
> > >> >hard for everyone we might as well go the whole hog and drag them
> into
> > >> the
> > >> >21st century, but I'm interested in others opinions.
> > >> >
> > >> >Chris you have a much better understand about those things than
> myself.
> > >> >
> > >> >Tom
> > >>
> > >>
> > >
> >
>
>
>
> --
> *Lewis*
>
Re: Upgrading Lucene and Solr
Posted by Lewis John Mcgibbney <le...@gmail.com>.
ACK
On Wed, Jul 6, 2016 at 8:55 AM, Tom Barber <to...@meteorite.bi> wrote:
> Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or similar
> then branch the branch to uprade solr and lucene outside of that scope so
> we don't destroy the semi working avro upgrades ;)
>
> On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <to...@meteorite.bi>
> wrote:
>
> > Coolio!
> >
> > On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <ma...@apache.org>
> > wrote:
> >
> >> Yes I was just telling Paul R. the other day we need to do
> >> this, or we’re DOA. I’ve first hand seen the scalability
> >> issues with Big Translate and DRAT using the old Lucene
> >> library. Let’s get it fixed in 2.x.
> >>
> >> Cheers,
> >> Chris
> >>
> >>
> >>
> >>
> >>
> >> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
> >>
> >> >Just throwing this one out there as we're making a bunch of breaking
> >> >changes in 2.0.
> >> >
> >> >Is it worth upgrading Lucene and Solr versions within OODT?
> >> >
> >> >My feeling is that they are both pretty old and whilst we're making
> life
> >> >hard for everyone we might as well go the whole hog and drag them into
> >> the
> >> >21st century, but I'm interested in others opinions.
> >> >
> >> >Chris you have a much better understand about those things than myself.
> >> >
> >> >Tom
> >>
> >>
> >
>
--
*Lewis*
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
Once avro_rpc is stable, I suggest we rename the branch 2.0-dev or similar
then branch the branch to uprade solr and lucene outside of that scope so
we don't destroy the semi working avro upgrades ;)
On Wed, Jul 6, 2016 at 4:54 PM, Tom Barber <to...@meteorite.bi> wrote:
> Coolio!
>
> On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <ma...@apache.org>
> wrote:
>
>> Yes I was just telling Paul R. the other day we need to do
>> this, or we’re DOA. I’ve first hand seen the scalability
>> issues with Big Translate and DRAT using the old Lucene
>> library. Let’s get it fixed in 2.x.
>>
>> Cheers,
>> Chris
>>
>>
>>
>>
>>
>> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>>
>> >Just throwing this one out there as we're making a bunch of breaking
>> >changes in 2.0.
>> >
>> >Is it worth upgrading Lucene and Solr versions within OODT?
>> >
>> >My feeling is that they are both pretty old and whilst we're making life
>> >hard for everyone we might as well go the whole hog and drag them into
>> the
>> >21st century, but I'm interested in others opinions.
>> >
>> >Chris you have a much better understand about those things than myself.
>> >
>> >Tom
>>
>>
>
Re: Upgrading Lucene and Solr
Posted by Tom Barber <to...@meteorite.bi>.
Coolio!
On Wed, Jul 6, 2016 at 4:54 PM, Chris Mattmann <ma...@apache.org> wrote:
> Yes I was just telling Paul R. the other day we need to do
> this, or we’re DOA. I’ve first hand seen the scalability
> issues with Big Translate and DRAT using the old Lucene
> library. Let’s get it fixed in 2.x.
>
> Cheers,
> Chris
>
>
>
>
>
> On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>
> >Just throwing this one out there as we're making a bunch of breaking
> >changes in 2.0.
> >
> >Is it worth upgrading Lucene and Solr versions within OODT?
> >
> >My feeling is that they are both pretty old and whilst we're making life
> >hard for everyone we might as well go the whole hog and drag them into the
> >21st century, but I'm interested in others opinions.
> >
> >Chris you have a much better understand about those things than myself.
> >
> >Tom
>
>
Re: Upgrading Lucene and Solr
Posted by Chris Mattmann <ma...@apache.org>.
Yes I was just telling Paul R. the other day we need to do
this, or we’re DOA. I’ve first hand seen the scalability
issues with Big Translate and DRAT using the old Lucene
library. Let’s get it fixed in 2.x.
Cheers,
Chris
On 7/6/16, 8:51 AM, "Tom Barber" <to...@meteorite.bi> wrote:
>Just throwing this one out there as we're making a bunch of breaking
>changes in 2.0.
>
>Is it worth upgrading Lucene and Solr versions within OODT?
>
>My feeling is that they are both pretty old and whilst we're making life
>hard for everyone we might as well go the whole hog and drag them into the
>21st century, but I'm interested in others opinions.
>
>Chris you have a much better understand about those things than myself.
>
>Tom