You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flume.apache.org by "Wahrmann, Helmut" <he...@rsa.com> on 2018/02/12 08:54:35 UTC

Merge of patch in Flume-3021?

Hi,

now that the blocker for FLUME-3021 is removed by committing FLUME-2957, can we get the patch from 3021 merged to trunk?

Thanks,

Helmut

Re: Merge of patch in Flume-3021?

Posted by Mike Percy <mp...@apache.org>.
Hi Helmut, yes I started investigating but I haven't gotten to the bottom
of the issue yet. I went out of town on holiday but I'll be back next week
so I will take another look soon.

Mike

On Tue, Apr 17, 2018 at 6:52 PM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> Did you have a chance to look into this?
>
> thanks,
>
> Helmut
>
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Montag, 26. März 2018 21:46
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> I haven't figured this out yet but I'll look into it this week.
>
> Mike
>
> On Mon, Mar 19, 2018 at 2:41 AM, Wahrmann, Helmut <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Mike,
> >
> > With the help of Ferenc, I got rid of the initial errors.
> >
> > Only one is remaining now:
> >
> > [INFO] -------------------------------------------------------
> > [INFO]  T E S T S
> > [INFO] -------------------------------------------------------
> > [INFO] Running
> > org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> > [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 1.126 s - in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> > [INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
> > [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 1.125 s - in org.apache.flume.sink.solr.morphline.TestBlobHandler
> > [INFO] Running org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 7.724 s - in
> > org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
> > [INFO] Running
> > org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> > 1.797 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > TestMorphlineSolrSink
> > [ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> > Time
> > elapsed: 1.797 s  <<< ERROR!
> > java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]
> >
> > [INFO] Running
> > org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> > [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> > 0.391 s - in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> > [INFO]
> > [INFO] Results:
> > [INFO]
> > [ERROR] Errors:
> > [ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588
> > ╗ IllformedLocale
> > [INFO]
> > [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO]
> > [INFO]
> > ----------------------------------------------------------------------
> > --
> > [INFO] BUILD FAILURE
> >
> >
> > The LuceneTestCase is extended by SolrTestCaseJ4.
> > No idea, what I could do against this.
> >
> > best regards,
> >
> > Helmut
> >
> > -----Original Message-----
> > From: Mike Percy [mailto:mpercy@apache.org]
> > Sent: Montag, 19. März 2018 05:12
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > Nice! Thanks Ferenc. Answered better than I could have done. :)
> >
> > Sorry, I meant to send this last week but I just found it in my drafts.
> >
> > Helmut, please let us know if you need more help with this.
> >
> > Mike
> >
> > On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fs...@cloudera.com>
> wrote:
> >
> > > the createJetty method of a Test class became final in the new
> > > versions of solr.
> > > the kite sdk test-jar has to be removed because it depends on a
> > > different incompatible version of solr
> > >
> > >     <dependency>
> > >       <groupId>org.kitesdk</groupId>
> > >       <artifactId>kite-morphlines-solr-core</artifactId>
> > >       <version>${kite.version}</version>
> > >       <type>test-jar</type>
> > >       <scope>test</scope>
> > >     </dependency>
> > >
> > > TestEnvironment.java has to be removed as well because it depends on
> > > the incompatible dependency
> > >
> > > then we need this class:
> > > https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> > > /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> > > morphline/solr/TestEmbeddedSolrServer.java
> > > I believe it is ok to have a copy of this because it is part of the
> > > incompatible test dependency we just removed
> > >
> > > then we need a newer version of commons-compress:
> > >     <commons-compress.version>1.10</commons-compress.version>
> > >
> > > from here you can fix the actual solr related test errors :)
> > >
> > >
> > >
> > > On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut
> > > <helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Ferenc,
> > > >
> > > > Thanks for offering help.
> > > > In agreement with Mike I want to have support for Solr 7.2.1 in
> > > > the morphline solr sink, so that we can easily upgrade the
> > > > Elasticsearch
> > > sink.
> > > >
> > > > My updates are here: https://github.com/hwahrmann/
> > > > flume/tree/Upgrade_Morphline_Sink
> > > >
> > > > I changed the solr version to 7.2.1 and was able to compile the
> > > > sink withput any problems.
> > > > I can also compile the tests, but when running, I get multiple
> > > > errors
> > > like
> > > > this:
> > > >
> > > > [INFO] Running org.apache.flume.sink.solr.morphline.
> > > > TestMorphlineInterceptor
> > > > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time
> > elapsed:
> > > > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > > > TestMorphlineInterceptor
> > > > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s
> > > > <<< ERROR!
> > > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > > tractSolrMorphlineZkTest
> > > > overrides final method createJetty.(Ljava/io/File;
> > > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > > JettySolrRunner;
> > > >         at
> > > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > > r.
> > > > build(TestMorphlineInterceptor.java:151)
> > > >         at
> > > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > > r.
> > > > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java
> > > > :1
> > > > 39)
> > > >
> > > > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s
> > > > <<< ERROR!
> > > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > > tractSolrMorphlineZkTest
> > > > overrides final method createJetty.(Ljava/io/File;
> > > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > > JettySolrRunner;
> > > >         at
> > > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > > r.
> > > > build(TestMorphlineInterceptor.java:151)
> > > >         at
> > > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > > r.
> > > > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83
> > > > )
> > > >
> > > >
> > > > The all have problems with createJetty. So it seems that I maybe
> > > > need a different version of jetty or something like that.
> > > > And for that I have too less knowledge about maven.
> > > >
> > > > thx,
> > > >
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > > > Sent: Dienstag, 6. März 2018 11:04
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > Hi Helmut,
> > > >
> > > > let me know what can I help You with.
> > > > share your current code on a github fork and describe the issue. I
> > > > will see what can we do to solve it.
> > > >
> > > >
> > > > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> > > helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > I am stuck with Solr.
> > > > > The morphline-solr sink compiles without any problems, but I am
> > > > > struggling with the tests.
> > > > > Seems I need to exclude some stuff from maven, but my knowledge
> > > > > about maven is not good enough to figure out what to do.
> > > > > Anyone able to help?
> > > > >
> > > > > regards,
> > > > >
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > Hi Helmut,
> > > > > As long as the integration tests still pass and the packaging
> > > > > issues are not exacerbated, I don't see why we couldn't merge an
> > > > > upgrade patch, barring any serious concerns with the patch.
> > > > >
> > > > > Mike
> > > > >
> > > > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut
> > > > > <helmut.wahrmann@rsa.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mike,
> > > > > >
> > > > > > I won't have a problem upgrading the Solr sink to the latest
> > version.
> > > > > > I am missing test environment however.
> > > > > > So while it may build correctly and all integration tests
> > > > > > work, I have no real environment to test with.
> > > > > >
> > > > > > best regards,
> > > > > > Helmut
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > > > To: dev@flume.apache.org
> > > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > > >
> > > > > > OK. In the pull request, it would be nice if whoever submits
> > > > > > or merges it mentions all of the contributors to the patch in
> > > > > > the commit
> > > > message.
> > > > > >
> > > > > > I asked Wolfgang H. about the SolrServer thing and this is
> > > > > > what he told
> > > > > me:
> > > > > >
> > > > > > Hi Mike, the class has been renamed to "SolrClient" (which
> > > > > > unfortunately
> > > > > > > breaks compat). It's just a class rename. The functionality
> > > > > > > is the same as before. It was called SolrServer in Solr4
> > > > > > > because it was a client proxy that sends RPCs to a Solr
> > > > > > > server, but calling it SolrClient is more straightforward to
> > > > > > > understand, hence the community decided to rename the class.
> > > > > > > It's possible to spawn an embedded Solr server, for example
> > > > > > > for testing purposes, via class EmbeddedSolrServer (a class
> > > > > > > that retains the same name in Solr7 and Solr4), which
> > > > > > > extends the
> > > > SolrClient class.
> > > > > >
> > > > > >
> > > > > > Hope this helps,
> > > > > > Mike
> > > > > >
> > > > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut
> > > > > > <helmut.wahrmann@rsa.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Mike,
> > > > > > >
> > > > > > > Thanks for the response. Would be cool if we get that sorted
> out.
> > > > > > >
> > > > > > > I've asked Yonghao Zou to submit the Pull Request, since he
> > > > > > > did most of the work and should get the credit.
> > > > > > > He'll  do so after the Chinese New Year's Eve.
> > > > > > >
> > > > > > > I will then issue a Pull request for the new ES Rest client,
> > > > > > > which is dependent on the above work.
> > > > > > >
> > > > > > > best regards,
> > > > > > > Helmut
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > > > To: dev@flume.apache.org
> > > > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > > > >
> > > > > > > Hi Helmut,
> > > > > > > I see that I neglected to follow up on the other thread on
> > > > > > > this topic after your reply about SolrServer missing from
> > > > > > > the
> > solrj jar.
> > > > > > > Let me ask around w/ some folks I know that work on Solr and
> > > > > > > see if there is any way to retain the SolrServer for our
> > > > > > > tests after upgrading to the
> > > > > > new version.
> > > > > > >
> > > > > > > Thank you very much for working on upgrading Solr. Would you
> > > > > > > mind submitting a pull request with your (apparently
> > > > > > > work-in-progress) patch to upgrade both Solr and ES?
> > > > > > >
> > > > > > > To reply to your email in this thread, the JAR packaging
> > > > > > > situation is largely the same after merging FLUME-2957 so
> > > > > > > unfortunately most of what I noted in my reply in the other
> > > > > > > thread ( https://s.apache.org/GqcX ) still holds.
> > > > > > >
> > > > > > > I hope that we can upgrade the Solr dependencies as part of
> > > > > > > the same commit as the ES dependencies to avoid worrying
> > > > > > > about which lucene jar is first in the classpath, and ensure
> > > > > > > we are not adding any additional dependency conflicts to mvn
> > dependency:tree.
> > > > > > >
> > > > > > > Regards,
> > > > > > > Mike
> > > > > > >
> > > > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> > > > > > > helmut.wahrmann@rsa.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > now that the blocker for FLUME-3021 is removed by
> > > > > > > > committing FLUME-2957, can we get the patch from 3021 merged
> to trunk?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Helmut
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

RE: Merge of patch in Flume-3021?

Posted by "Wahrmann, Helmut" <he...@rsa.com>.
Hi Mike,

Did you have a chance to look into this?

thanks,

Helmut
 

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org] 
Sent: Montag, 26. März 2018 21:46
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

I haven't figured this out yet but I'll look into it this week.

Mike

On Mon, Mar 19, 2018 at 2:41 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> With the help of Ferenc, I got rid of the initial errors.
>
> Only one is remaining now:
>
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running 
> org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.126 s - in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.125 s - in org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Running org.apache.flume.sink.solr.morphline.
> TestMorphlineInterceptor
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 7.724 s - in 
> org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
> [INFO] Running 
> org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 1.797 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> TestMorphlineSolrSink
> [ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink  
> Time
> elapsed: 1.797 s  <<< ERROR!
> java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]
>
> [INFO] Running 
> org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 0.391 s - in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588
> ╗ IllformedLocale
> [INFO]
> [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO] 
> [INFO]
> ----------------------------------------------------------------------
> --
> [INFO] BUILD FAILURE
>
>
> The LuceneTestCase is extended by SolrTestCaseJ4.
> No idea, what I could do against this.
>
> best regards,
>
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Montag, 19. März 2018 05:12
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Nice! Thanks Ferenc. Answered better than I could have done. :)
>
> Sorry, I meant to send this last week but I just found it in my drafts.
>
> Helmut, please let us know if you need more help with this.
>
> Mike
>
> On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fs...@cloudera.com> wrote:
>
> > the createJetty method of a Test class became final in the new 
> > versions of solr.
> > the kite sdk test-jar has to be removed because it depends on a 
> > different incompatible version of solr
> >
> >     <dependency>
> >       <groupId>org.kitesdk</groupId>
> >       <artifactId>kite-morphlines-solr-core</artifactId>
> >       <version>${kite.version}</version>
> >       <type>test-jar</type>
> >       <scope>test</scope>
> >     </dependency>
> >
> > TestEnvironment.java has to be removed as well because it depends on 
> > the incompatible dependency
> >
> > then we need this class:
> > https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> > /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> > morphline/solr/TestEmbeddedSolrServer.java
> > I believe it is ok to have a copy of this because it is part of the 
> > incompatible test dependency we just removed
> >
> > then we need a newer version of commons-compress:
> >     <commons-compress.version>1.10</commons-compress.version>
> >
> > from here you can fix the actual solr related test errors :)
> >
> >
> >
> > On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut 
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Ferenc,
> > >
> > > Thanks for offering help.
> > > In agreement with Mike I want to have support for Solr 7.2.1 in 
> > > the morphline solr sink, so that we can easily upgrade the 
> > > Elasticsearch
> > sink.
> > >
> > > My updates are here: https://github.com/hwahrmann/ 
> > > flume/tree/Upgrade_Morphline_Sink
> > >
> > > I changed the solr version to 7.2.1 and was able to compile the 
> > > sink withput any problems.
> > > I can also compile the tests, but when running, I get multiple 
> > > errors
> > like
> > > this:
> > >
> > > [INFO] Running org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time
> elapsed:
> > > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s  
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java
> > > :1
> > > 39)
> > >
> > > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s  
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83
> > > )
> > >
> > >
> > > The all have problems with createJetty. So it seems that I maybe 
> > > need a different version of jetty or something like that.
> > > And for that I have too less knowledge about maven.
> > >
> > > thx,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > > Sent: Dienstag, 6. März 2018 11:04
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > >
> > > let me know what can I help You with.
> > > share your current code on a github fork and describe the issue. I 
> > > will see what can we do to solve it.
> > >
> > >
> > > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> > helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I am stuck with Solr.
> > > > The morphline-solr sink compiles without any problems, but I am 
> > > > struggling with the tests.
> > > > Seems I need to exclude some stuff from maven, but my knowledge 
> > > > about maven is not good enough to figure out what to do.
> > > > Anyone able to help?
> > > >
> > > > regards,
> > > >
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > Hi Helmut,
> > > > As long as the integration tests still pass and the packaging 
> > > > issues are not exacerbated, I don't see why we couldn't merge an 
> > > > upgrade patch, barring any serious concerns with the patch.
> > > >
> > > > Mike
> > > >
> > > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut 
> > > > <helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > I won't have a problem upgrading the Solr sink to the latest
> version.
> > > > > I am missing test environment however.
> > > > > So while it may build correctly and all integration tests 
> > > > > work, I have no real environment to test with.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > OK. In the pull request, it would be nice if whoever submits 
> > > > > or merges it mentions all of the contributors to the patch in 
> > > > > the commit
> > > message.
> > > > >
> > > > > I asked Wolfgang H. about the SolrServer thing and this is 
> > > > > what he told
> > > > me:
> > > > >
> > > > > Hi Mike, the class has been renamed to "SolrClient" (which 
> > > > > unfortunately
> > > > > > breaks compat). It's just a class rename. The functionality 
> > > > > > is the same as before. It was called SolrServer in Solr4 
> > > > > > because it was a client proxy that sends RPCs to a Solr 
> > > > > > server, but calling it SolrClient is more straightforward to 
> > > > > > understand, hence the community decided to rename the class.
> > > > > > It's possible to spawn an embedded Solr server, for example 
> > > > > > for testing purposes, via class EmbeddedSolrServer (a class 
> > > > > > that retains the same name in Solr7 and Solr4), which 
> > > > > > extends the
> > > SolrClient class.
> > > > >
> > > > >
> > > > > Hope this helps,
> > > > > Mike
> > > > >
> > > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut 
> > > > > <helmut.wahrmann@rsa.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mike,
> > > > > >
> > > > > > Thanks for the response. Would be cool if we get that sorted out.
> > > > > >
> > > > > > I've asked Yonghao Zou to submit the Pull Request, since he 
> > > > > > did most of the work and should get the credit.
> > > > > > He'll  do so after the Chinese New Year's Eve.
> > > > > >
> > > > > > I will then issue a Pull request for the new ES Rest client, 
> > > > > > which is dependent on the above work.
> > > > > >
> > > > > > best regards,
> > > > > > Helmut
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > > To: dev@flume.apache.org
> > > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > > >
> > > > > > Hi Helmut,
> > > > > > I see that I neglected to follow up on the other thread on 
> > > > > > this topic after your reply about SolrServer missing from 
> > > > > > the
> solrj jar.
> > > > > > Let me ask around w/ some folks I know that work on Solr and 
> > > > > > see if there is any way to retain the SolrServer for our 
> > > > > > tests after upgrading to the
> > > > > new version.
> > > > > >
> > > > > > Thank you very much for working on upgrading Solr. Would you 
> > > > > > mind submitting a pull request with your (apparently
> > > > > > work-in-progress) patch to upgrade both Solr and ES?
> > > > > >
> > > > > > To reply to your email in this thread, the JAR packaging 
> > > > > > situation is largely the same after merging FLUME-2957 so 
> > > > > > unfortunately most of what I noted in my reply in the other 
> > > > > > thread ( https://s.apache.org/GqcX ) still holds.
> > > > > >
> > > > > > I hope that we can upgrade the Solr dependencies as part of 
> > > > > > the same commit as the ES dependencies to avoid worrying 
> > > > > > about which lucene jar is first in the classpath, and ensure 
> > > > > > we are not adding any additional dependency conflicts to mvn
> dependency:tree.
> > > > > >
> > > > > > Regards,
> > > > > > Mike
> > > > > >
> > > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> > > > > > helmut.wahrmann@rsa.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > now that the blocker for FLUME-3021 is removed by 
> > > > > > > committing FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Helmut
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

答复: Merge of patch in Flume-3021?

Posted by 邹浩子 <yo...@gmail.com>.
Should we try to merge this with shade plugin or some other ways which can ignore conflicts about dependencies?

发送自 Windows 10 版邮件应用

发件人: Wahrmann, Helmut
发送时间: 2018年8月29日 17:30
收件人: dev@flume.apache.org
主题: RE: Merge of patch in Flume-3021?

Hi Mike,

Did you have a chance to look at that?


best regards,
Helmut

-----Original Message-----
From: Wahrmann, Helmut 
Sent: Dienstag, 17. April 2018 18:53
To: 'dev@flume.apache.org'
Subject: RE: Merge of patch in Flume-3021?

Hi Mike,

Did you have a chance to look into this?

thanks,

Helmut
 

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org]
Sent: Montag, 26. März 2018 21:46
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

I haven't figured this out yet but I'll look into it this week.

Mike

On Mon, Mar 19, 2018 at 2:41 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> With the help of Ferenc, I got rid of the initial errors.
>
> Only one is remaining now:
>
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.126 s - in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.125 s - in org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Running org.apache.flume.sink.solr.morphline.
> TestMorphlineInterceptor
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 7.724 s - in
> org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 1.797 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> TestMorphlineSolrSink
> [ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> Time
> elapsed: 1.797 s  <<< ERROR!
> java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]
>
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 0.391 s - in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588
> ╗ IllformedLocale
> [INFO]
> [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO] 
> [INFO]
> ----------------------------------------------------------------------
> --
> [INFO] BUILD FAILURE
>
>
> The LuceneTestCase is extended by SolrTestCaseJ4.
> No idea, what I could do against this.
>
> best regards,
>
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Montag, 19. März 2018 05:12
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Nice! Thanks Ferenc. Answered better than I could have done. :)
>
> Sorry, I meant to send this last week but I just found it in my drafts.
>
> Helmut, please let us know if you need more help with this.
>
> Mike
>
> On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fs...@cloudera.com> wrote:
>
> > the createJetty method of a Test class became final in the new 
> > versions of solr.
> > the kite sdk test-jar has to be removed because it depends on a 
> > different incompatible version of solr
> >
> >     <dependency>
> >       <groupId>org.kitesdk</groupId>
> >       <artifactId>kite-morphlines-solr-core</artifactId>
> >       <version>${kite.version}</version>
> >       <type>test-jar</type>
> >       <scope>test</scope>
> >     </dependency>
> >
> > TestEnvironment.java has to be removed as well because it depends on 
> > the incompatible dependency
> >
> > then we need this class:
> > https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> > /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> > morphline/solr/TestEmbeddedSolrServer.java
> > I believe it is ok to have a copy of this because it is part of the 
> > incompatible test dependency we just removed
> >
> > then we need a newer version of commons-compress:
> >     <commons-compress.version>1.10</commons-compress.version>
> >
> > from here you can fix the actual solr related test errors :)
> >
> >
> >
> > On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut 
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Ferenc,
> > >
> > > Thanks for offering help.
> > > In agreement with Mike I want to have support for Solr 7.2.1 in 
> > > the morphline solr sink, so that we can easily upgrade the 
> > > Elasticsearch
> > sink.
> > >
> > > My updates are here: https://github.com/hwahrmann/ 
> > > flume/tree/Upgrade_Morphline_Sink
> > >
> > > I changed the solr version to 7.2.1 and was able to compile the 
> > > sink withput any problems.
> > > I can also compile the tests, but when running, I get multiple 
> > > errors
> > like
> > > this:
> > >
> > > [INFO] Running org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time
> elapsed:
> > > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s 
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java
> > > :1
> > > 39)
> > >
> > > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s 
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83
> > > )
> > >
> > >
> > > The all have problems with createJetty. So it seems that I maybe 
> > > need a different version of jetty or something like that.
> > > And for that I have too less knowledge about maven.
> > >
> > > thx,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > > Sent: Dienstag, 6. März 2018 11:04
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > >
> > > let me know what can I help You with.
> > > share your current code on a github fork and describe the issue. I 
> > > will see what can we do to solve it.
> > >
> > >
> > > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> > helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I am stuck with Solr.
> > > > The morphline-solr sink compiles without any problems, but I am 
> > > > struggling with the tests.
> > > > Seems I need to exclude some stuff from maven, but my knowledge 
> > > > about maven is not good enough to figure out what to do.
> > > > Anyone able to help?
> > > >
> > > > regards,
> > > >
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > Hi Helmut,
> > > > As long as the integration tests still pass and the packaging 
> > > > issues are not exacerbated, I don't see why we couldn't merge an 
> > > > upgrade patch, barring any serious concerns with the patch.
> > > >
> > > > Mike
> > > >
> > > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut 
> > > > <helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > I won't have a problem upgrading the Solr sink to the latest
> version.
> > > > > I am missing test environment however.
> > > > > So while it may build correctly and all integration tests 
> > > > > work, I have no real environment to test with.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > OK. In the pull request, it would be nice if whoever submits 
> > > > > or merges it mentions all of the contributors to the patch in 
> > > > > the commit
> > > message.
> > > > >
> > > > > I asked Wolfgang H. about the SolrServer thing and this is 
> > > > > what he told
> > > > me:
> > > > >
> > > > > Hi Mike, the class has been renamed to "SolrClient" (which 
> > > > > unfortunately
> > > > > > breaks compat). It's just a class rename. The functionality 
> > > > > > is the same as before. It was called SolrServer in Solr4 
> > > > > > because it was a client proxy that sends RPCs to a Solr 
> > > > > > server, but calling it SolrClient is more straightforward to 
> > > > > > understand, hence the community decided to rename the class.
> > > > > > It's possible to spawn an embedded Solr server, for example 
> > > > > > for testing purposes, via class EmbeddedSolrServer (a class 
> > > > > > that retains the same name in Solr7 and Solr4), which 
> > > > > > extends the
> > > SolrClient class.
> > > > >
> > > > >
> > > > > Hope this helps,
> > > > > Mike
> > > > >
> > > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut 
> > > > > <helmut.wahrmann@rsa.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mike,
> > > > > >
> > > > > > Thanks for the response. Would be cool if we get that sorted out.
> > > > > >
> > > > > > I've asked Yonghao Zou to submit the Pull Request, since he 
> > > > > > did most of the work and should get the credit.
> > > > > > He'll  do so after the Chinese New Year's Eve.
> > > > > >
> > > > > > I will then issue a Pull request for the new ES Rest client, 
> > > > > > which is dependent on the above work.
> > > > > >
> > > > > > best regards,
> > > > > > Helmut
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > > To: dev@flume.apache.org
> > > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > > >
> > > > > > Hi Helmut,
> > > > > > I see that I neglected to follow up on the other thread on 
> > > > > > this topic after your reply about SolrServer missing from 
> > > > > > the
> solrj jar.
> > > > > > Let me ask around w/ some folks I know that work on Solr and 
> > > > > > see if there is any way to retain the SolrServer for our 
> > > > > > tests after upgrading to the
> > > > > new version.
> > > > > >
> > > > > > Thank you very much for working on upgrading Solr. Would you 
> > > > > > mind submitting a pull request with your (apparently
> > > > > > work-in-progress) patch to upgrade both Solr and ES?
> > > > > >
> > > > > > To reply to your email in this thread, the JAR packaging 
> > > > > > situation is largely the same after merging FLUME-2957 so 
> > > > > > unfortunately most of what I noted in my reply in the other 
> > > > > > thread ( https://s.apache.org/GqcX ) still holds.
> > > > > >
> > > > > > I hope that we can upgrade the Solr dependencies as part of 
> > > > > > the same commit as the ES dependencies to avoid worrying 
> > > > > > about which lucene jar is first in the classpath, and ensure 
> > > > > > we are not adding any additional dependency conflicts to mvn
> dependency:tree.
> > > > > >
> > > > > > Regards,
> > > > > > Mike
> > > > > >
> > > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> > > > > > helmut.wahrmann@rsa.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > now that the blocker for FLUME-3021 is removed by 
> > > > > > > committing FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Helmut
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

RE: Merge of patch in Flume-3021?

Posted by "Wahrmann, Helmut" <he...@rsa.com>.
Hi Mike,

Did you have a chance to look at that?


best regards,
Helmut

-----Original Message-----
From: Wahrmann, Helmut 
Sent: Dienstag, 17. April 2018 18:53
To: 'dev@flume.apache.org'
Subject: RE: Merge of patch in Flume-3021?

Hi Mike,

Did you have a chance to look into this?

thanks,

Helmut
 

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org]
Sent: Montag, 26. März 2018 21:46
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

I haven't figured this out yet but I'll look into it this week.

Mike

On Mon, Mar 19, 2018 at 2:41 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> With the help of Ferenc, I got rid of the initial errors.
>
> Only one is remaining now:
>
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.126 s - in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.125 s - in org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Running org.apache.flume.sink.solr.morphline.
> TestMorphlineInterceptor
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 7.724 s - in
> org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 1.797 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> TestMorphlineSolrSink
> [ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> Time
> elapsed: 1.797 s  <<< ERROR!
> java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]
>
> [INFO] Running
> org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 0.391 s - in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588
> ╗ IllformedLocale
> [INFO]
> [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO] 
> [INFO]
> ----------------------------------------------------------------------
> --
> [INFO] BUILD FAILURE
>
>
> The LuceneTestCase is extended by SolrTestCaseJ4.
> No idea, what I could do against this.
>
> best regards,
>
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Montag, 19. März 2018 05:12
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Nice! Thanks Ferenc. Answered better than I could have done. :)
>
> Sorry, I meant to send this last week but I just found it in my drafts.
>
> Helmut, please let us know if you need more help with this.
>
> Mike
>
> On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fs...@cloudera.com> wrote:
>
> > the createJetty method of a Test class became final in the new 
> > versions of solr.
> > the kite sdk test-jar has to be removed because it depends on a 
> > different incompatible version of solr
> >
> >     <dependency>
> >       <groupId>org.kitesdk</groupId>
> >       <artifactId>kite-morphlines-solr-core</artifactId>
> >       <version>${kite.version}</version>
> >       <type>test-jar</type>
> >       <scope>test</scope>
> >     </dependency>
> >
> > TestEnvironment.java has to be removed as well because it depends on 
> > the incompatible dependency
> >
> > then we need this class:
> > https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> > /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> > morphline/solr/TestEmbeddedSolrServer.java
> > I believe it is ok to have a copy of this because it is part of the 
> > incompatible test dependency we just removed
> >
> > then we need a newer version of commons-compress:
> >     <commons-compress.version>1.10</commons-compress.version>
> >
> > from here you can fix the actual solr related test errors :)
> >
> >
> >
> > On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut 
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Ferenc,
> > >
> > > Thanks for offering help.
> > > In agreement with Mike I want to have support for Solr 7.2.1 in 
> > > the morphline solr sink, so that we can easily upgrade the 
> > > Elasticsearch
> > sink.
> > >
> > > My updates are here: https://github.com/hwahrmann/ 
> > > flume/tree/Upgrade_Morphline_Sink
> > >
> > > I changed the solr version to 7.2.1 and was able to compile the 
> > > sink withput any problems.
> > > I can also compile the tests, but when running, I get multiple 
> > > errors
> > like
> > > this:
> > >
> > > [INFO] Running org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time
> elapsed:
> > > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s 
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java
> > > :1
> > > 39)
> > >
> > > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s 
> > > <<< ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File; 
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83
> > > )
> > >
> > >
> > > The all have problems with createJetty. So it seems that I maybe 
> > > need a different version of jetty or something like that.
> > > And for that I have too less knowledge about maven.
> > >
> > > thx,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > > Sent: Dienstag, 6. März 2018 11:04
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > >
> > > let me know what can I help You with.
> > > share your current code on a github fork and describe the issue. I 
> > > will see what can we do to solve it.
> > >
> > >
> > > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> > helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I am stuck with Solr.
> > > > The morphline-solr sink compiles without any problems, but I am 
> > > > struggling with the tests.
> > > > Seems I need to exclude some stuff from maven, but my knowledge 
> > > > about maven is not good enough to figure out what to do.
> > > > Anyone able to help?
> > > >
> > > > regards,
> > > >
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > Hi Helmut,
> > > > As long as the integration tests still pass and the packaging 
> > > > issues are not exacerbated, I don't see why we couldn't merge an 
> > > > upgrade patch, barring any serious concerns with the patch.
> > > >
> > > > Mike
> > > >
> > > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut 
> > > > <helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > I won't have a problem upgrading the Solr sink to the latest
> version.
> > > > > I am missing test environment however.
> > > > > So while it may build correctly and all integration tests 
> > > > > work, I have no real environment to test with.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > OK. In the pull request, it would be nice if whoever submits 
> > > > > or merges it mentions all of the contributors to the patch in 
> > > > > the commit
> > > message.
> > > > >
> > > > > I asked Wolfgang H. about the SolrServer thing and this is 
> > > > > what he told
> > > > me:
> > > > >
> > > > > Hi Mike, the class has been renamed to "SolrClient" (which 
> > > > > unfortunately
> > > > > > breaks compat). It's just a class rename. The functionality 
> > > > > > is the same as before. It was called SolrServer in Solr4 
> > > > > > because it was a client proxy that sends RPCs to a Solr 
> > > > > > server, but calling it SolrClient is more straightforward to 
> > > > > > understand, hence the community decided to rename the class.
> > > > > > It's possible to spawn an embedded Solr server, for example 
> > > > > > for testing purposes, via class EmbeddedSolrServer (a class 
> > > > > > that retains the same name in Solr7 and Solr4), which 
> > > > > > extends the
> > > SolrClient class.
> > > > >
> > > > >
> > > > > Hope this helps,
> > > > > Mike
> > > > >
> > > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut 
> > > > > <helmut.wahrmann@rsa.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mike,
> > > > > >
> > > > > > Thanks for the response. Would be cool if we get that sorted out.
> > > > > >
> > > > > > I've asked Yonghao Zou to submit the Pull Request, since he 
> > > > > > did most of the work and should get the credit.
> > > > > > He'll  do so after the Chinese New Year's Eve.
> > > > > >
> > > > > > I will then issue a Pull request for the new ES Rest client, 
> > > > > > which is dependent on the above work.
> > > > > >
> > > > > > best regards,
> > > > > > Helmut
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > > To: dev@flume.apache.org
> > > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > > >
> > > > > > Hi Helmut,
> > > > > > I see that I neglected to follow up on the other thread on 
> > > > > > this topic after your reply about SolrServer missing from 
> > > > > > the
> solrj jar.
> > > > > > Let me ask around w/ some folks I know that work on Solr and 
> > > > > > see if there is any way to retain the SolrServer for our 
> > > > > > tests after upgrading to the
> > > > > new version.
> > > > > >
> > > > > > Thank you very much for working on upgrading Solr. Would you 
> > > > > > mind submitting a pull request with your (apparently
> > > > > > work-in-progress) patch to upgrade both Solr and ES?
> > > > > >
> > > > > > To reply to your email in this thread, the JAR packaging 
> > > > > > situation is largely the same after merging FLUME-2957 so 
> > > > > > unfortunately most of what I noted in my reply in the other 
> > > > > > thread ( https://s.apache.org/GqcX ) still holds.
> > > > > >
> > > > > > I hope that we can upgrade the Solr dependencies as part of 
> > > > > > the same commit as the ES dependencies to avoid worrying 
> > > > > > about which lucene jar is first in the classpath, and ensure 
> > > > > > we are not adding any additional dependency conflicts to mvn
> dependency:tree.
> > > > > >
> > > > > > Regards,
> > > > > > Mike
> > > > > >
> > > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> > > > > > helmut.wahrmann@rsa.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > now that the blocker for FLUME-3021 is removed by 
> > > > > > > committing FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Helmut
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Merge of patch in Flume-3021?

Posted by Mike Percy <mp...@apache.org>.
I haven't figured this out yet but I'll look into it this week.

Mike

On Mon, Mar 19, 2018 at 2:41 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> With the help of Ferenc, I got rid of the initial errors.
>
> Only one is remaining now:
>
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.126 s - in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
> [INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 1.125 s - in org.apache.flume.sink.solr.morphline.TestBlobHandler
> [INFO] Running org.apache.flume.sink.solr.morphline.
> TestMorphlineInterceptor
> [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 7.724 s - in org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
> [INFO] Running org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 1.797 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> TestMorphlineSolrSink
> [ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink  Time
> elapsed: 1.797 s  <<< ERROR!
> java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]
>
> [INFO] Running org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 0.391 s - in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Errors:
> [ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588
> ╗ IllformedLocale
> [INFO]
> [ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO] [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
>
>
> The LuceneTestCase is extended by SolrTestCaseJ4.
> No idea, what I could do against this.
>
> best regards,
>
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Montag, 19. März 2018 05:12
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Nice! Thanks Ferenc. Answered better than I could have done. :)
>
> Sorry, I meant to send this last week but I just found it in my drafts.
>
> Helmut, please let us know if you need more help with this.
>
> Mike
>
> On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fs...@cloudera.com> wrote:
>
> > the createJetty method of a Test class became final in the new
> > versions of solr.
> > the kite sdk test-jar has to be removed because it depends on a
> > different incompatible version of solr
> >
> >     <dependency>
> >       <groupId>org.kitesdk</groupId>
> >       <artifactId>kite-morphlines-solr-core</artifactId>
> >       <version>${kite.version}</version>
> >       <type>test-jar</type>
> >       <scope>test</scope>
> >     </dependency>
> >
> > TestEnvironment.java has to be removed as well because it depends on
> > the incompatible dependency
> >
> > then we need this class:
> > https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> > /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> > morphline/solr/TestEmbeddedSolrServer.java
> > I believe it is ok to have a copy of this because it is part of the
> > incompatible test dependency we just removed
> >
> > then we need a newer version of commons-compress:
> >     <commons-compress.version>1.10</commons-compress.version>
> >
> > from here you can fix the actual solr related test errors :)
> >
> >
> >
> > On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Ferenc,
> > >
> > > Thanks for offering help.
> > > In agreement with Mike I want to have support for Solr 7.2.1 in the
> > > morphline solr sink, so that we can easily upgrade the Elasticsearch
> > sink.
> > >
> > > My updates are here: https://github.com/hwahrmann/
> > > flume/tree/Upgrade_Morphline_Sink
> > >
> > > I changed the solr version to 7.2.1 and was able to compile the sink
> > > withput any problems.
> > > I can also compile the tests, but when running, I get multiple
> > > errors
> > like
> > > this:
> > >
> > > [INFO] Running org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time
> elapsed:
> > > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > > TestMorphlineInterceptor
> > > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s  <<<
> > > ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File;
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java:1
> > > 39)
> > >
> > > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s  <<<
> > > ERROR!
> > > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> > tractSolrMorphlineZkTest
> > > overrides final method createJetty.(Ljava/io/File;
> > > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> > JettySolrRunner;
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > build(TestMorphlineInterceptor.java:151)
> > >         at
> > > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> > r.
> > > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83)
> > >
> > >
> > > The all have problems with createJetty. So it seems that I maybe
> > > need a different version of jetty or something like that.
> > > And for that I have too less knowledge about maven.
> > >
> > > thx,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > > Sent: Dienstag, 6. März 2018 11:04
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > >
> > > let me know what can I help You with.
> > > share your current code on a github fork and describe the issue. I
> > > will see what can we do to solve it.
> > >
> > >
> > > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> > helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I am stuck with Solr.
> > > > The morphline-solr sink compiles without any problems, but I am
> > > > struggling with the tests.
> > > > Seems I need to exclude some stuff from maven, but my knowledge
> > > > about maven is not good enough to figure out what to do.
> > > > Anyone able to help?
> > > >
> > > > regards,
> > > >
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > Hi Helmut,
> > > > As long as the integration tests still pass and the packaging
> > > > issues are not exacerbated, I don't see why we couldn't merge an
> > > > upgrade patch, barring any serious concerns with the patch.
> > > >
> > > > Mike
> > > >
> > > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut
> > > > <helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > I won't have a problem upgrading the Solr sink to the latest
> version.
> > > > > I am missing test environment however.
> > > > > So while it may build correctly and all integration tests work,
> > > > > I have no real environment to test with.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > OK. In the pull request, it would be nice if whoever submits or
> > > > > merges it mentions all of the contributors to the patch in the
> > > > > commit
> > > message.
> > > > >
> > > > > I asked Wolfgang H. about the SolrServer thing and this is what
> > > > > he told
> > > > me:
> > > > >
> > > > > Hi Mike, the class has been renamed to "SolrClient" (which
> > > > > unfortunately
> > > > > > breaks compat). It's just a class rename. The functionality is
> > > > > > the same as before. It was called SolrServer in Solr4 because
> > > > > > it was a client proxy that sends RPCs to a Solr server, but
> > > > > > calling it SolrClient is more straightforward to understand,
> > > > > > hence the community decided to rename the class.
> > > > > > It's possible to spawn an embedded Solr server, for example
> > > > > > for testing purposes, via class EmbeddedSolrServer (a class
> > > > > > that retains the same name in Solr7 and Solr4), which extends
> > > > > > the
> > > SolrClient class.
> > > > >
> > > > >
> > > > > Hope this helps,
> > > > > Mike
> > > > >
> > > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut
> > > > > <helmut.wahrmann@rsa.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Mike,
> > > > > >
> > > > > > Thanks for the response. Would be cool if we get that sorted out.
> > > > > >
> > > > > > I've asked Yonghao Zou to submit the Pull Request, since he
> > > > > > did most of the work and should get the credit.
> > > > > > He'll  do so after the Chinese New Year's Eve.
> > > > > >
> > > > > > I will then issue a Pull request for the new ES Rest client,
> > > > > > which is dependent on the above work.
> > > > > >
> > > > > > best regards,
> > > > > > Helmut
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > > To: dev@flume.apache.org
> > > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > > >
> > > > > > Hi Helmut,
> > > > > > I see that I neglected to follow up on the other thread on
> > > > > > this topic after your reply about SolrServer missing from the
> solrj jar.
> > > > > > Let me ask around w/ some folks I know that work on Solr and
> > > > > > see if there is any way to retain the SolrServer for our tests
> > > > > > after upgrading to the
> > > > > new version.
> > > > > >
> > > > > > Thank you very much for working on upgrading Solr. Would you
> > > > > > mind submitting a pull request with your (apparently
> > > > > > work-in-progress) patch to upgrade both Solr and ES?
> > > > > >
> > > > > > To reply to your email in this thread, the JAR packaging
> > > > > > situation is largely the same after merging FLUME-2957 so
> > > > > > unfortunately most of what I noted in my reply in the other
> > > > > > thread ( https://s.apache.org/GqcX ) still holds.
> > > > > >
> > > > > > I hope that we can upgrade the Solr dependencies as part of
> > > > > > the same commit as the ES dependencies to avoid worrying about
> > > > > > which lucene jar is first in the classpath, and ensure we are
> > > > > > not adding any additional dependency conflicts to mvn
> dependency:tree.
> > > > > >
> > > > > > Regards,
> > > > > > Mike
> > > > > >
> > > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> > > > > > helmut.wahrmann@rsa.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > now that the blocker for FLUME-3021 is removed by committing
> > > > > > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Helmut
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

RE: Merge of patch in Flume-3021?

Posted by "Wahrmann, Helmut" <he...@rsa.com>.
Hi Mike,

With the help of Ferenc, I got rid of the initial errors.

Only one is remaining now:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.flume.sink.solr.morphline.TestBlobDeserializer
[INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.126 s - in org.apache.flume.sink.solr.morphline.TestBlobDeserializer
[INFO] Running org.apache.flume.sink.solr.morphline.TestBlobHandler
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.125 s - in org.apache.flume.sink.solr.morphline.TestBlobHandler
[INFO] Running org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.724 s - in org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
[INFO] Running org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.797 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink
[ERROR] org.apache.flume.sink.solr.morphline.TestMorphlineSolrSink  Time elapsed: 1.797 s  <<< ERROR!
java.util.IllformedLocaleException: Invalid subtag: en_us [at index 0]

[INFO] Running org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.391 s - in org.apache.flume.sink.solr.morphline.TestUUIDInterceptor
[INFO]
[INFO] Results:
[INFO]
[ERROR] Errors:
[ERROR]   TestMorphlineSolrSink>LuceneTestCase.localeForLanguageTag:1588 ╗ IllformedLocale
[INFO]
[ERROR] Tests run: 17, Failures: 0, Errors: 1, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE


The LuceneTestCase is extended by SolrTestCaseJ4.
No idea, what I could do against this.

best regards,

Helmut

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org] 
Sent: Montag, 19. März 2018 05:12
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

Nice! Thanks Ferenc. Answered better than I could have done. :)

Sorry, I meant to send this last week but I just found it in my drafts.

Helmut, please let us know if you need more help with this.

Mike

On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fs...@cloudera.com> wrote:

> the createJetty method of a Test class became final in the new 
> versions of solr.
> the kite sdk test-jar has to be removed because it depends on a 
> different incompatible version of solr
>
>     <dependency>
>       <groupId>org.kitesdk</groupId>
>       <artifactId>kite-morphlines-solr-core</artifactId>
>       <version>${kite.version}</version>
>       <type>test-jar</type>
>       <scope>test</scope>
>     </dependency>
>
> TestEnvironment.java has to be removed as well because it depends on 
> the incompatible dependency
>
> then we need this class:
> https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> morphline/solr/TestEmbeddedSolrServer.java
> I believe it is ok to have a copy of this because it is part of the 
> incompatible test dependency we just removed
>
> then we need a newer version of commons-compress:
>     <commons-compress.version>1.10</commons-compress.version>
>
> from here you can fix the actual solr related test errors :)
>
>
>
> On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut 
> <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Ferenc,
> >
> > Thanks for offering help.
> > In agreement with Mike I want to have support for Solr 7.2.1 in the 
> > morphline solr sink, so that we can easily upgrade the Elasticsearch
> sink.
> >
> > My updates are here: https://github.com/hwahrmann/ 
> > flume/tree/Upgrade_Morphline_Sink
> >
> > I changed the solr version to 7.2.1 and was able to compile the sink 
> > withput any problems.
> > I can also compile the tests, but when running, I get multiple 
> > errors
> like
> > this:
> >
> > [INFO] Running org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time elapsed:
> > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s  <<< 
> > ERROR!
> > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> tractSolrMorphlineZkTest
> > overrides final method createJetty.(Ljava/io/File; 
> > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> JettySolrRunner;
> >         at 
> > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > build(TestMorphlineInterceptor.java:151)
> >         at 
> > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java:1
> > 39)
> >
> > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s  <<< 
> > ERROR!
> > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> tractSolrMorphlineZkTest
> > overrides final method createJetty.(Ljava/io/File; 
> > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> JettySolrRunner;
> >         at 
> > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > build(TestMorphlineInterceptor.java:151)
> >         at 
> > org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83)
> >
> >
> > The all have problems with createJetty. So it seems that I maybe 
> > need a different version of jetty or something like that.
> > And for that I have too less knowledge about maven.
> >
> > thx,
> >
> > Helmut
> >
> > -----Original Message-----
> > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > Sent: Dienstag, 6. März 2018 11:04
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > Hi Helmut,
> >
> > let me know what can I help You with.
> > share your current code on a github fork and describe the issue. I 
> > will see what can we do to solve it.
> >
> >
> > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Mike,
> > >
> > > I am stuck with Solr.
> > > The morphline-solr sink compiles without any problems, but I am 
> > > struggling with the tests.
> > > Seems I need to exclude some stuff from maven, but my knowledge 
> > > about maven is not good enough to figure out what to do.
> > > Anyone able to help?
> > >
> > > regards,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Mike Percy [mailto:mpercy@apache.org]
> > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > > As long as the integration tests still pass and the packaging 
> > > issues are not exacerbated, I don't see why we couldn't merge an 
> > > upgrade patch, barring any serious concerns with the patch.
> > >
> > > Mike
> > >
> > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut 
> > > <helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I won't have a problem upgrading the Solr sink to the latest version.
> > > > I am missing test environment however.
> > > > So while it may build correctly and all integration tests work, 
> > > > I have no real environment to test with.
> > > >
> > > > best regards,
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > OK. In the pull request, it would be nice if whoever submits or 
> > > > merges it mentions all of the contributors to the patch in the 
> > > > commit
> > message.
> > > >
> > > > I asked Wolfgang H. about the SolrServer thing and this is what 
> > > > he told
> > > me:
> > > >
> > > > Hi Mike, the class has been renamed to "SolrClient" (which 
> > > > unfortunately
> > > > > breaks compat). It's just a class rename. The functionality is 
> > > > > the same as before. It was called SolrServer in Solr4 because 
> > > > > it was a client proxy that sends RPCs to a Solr server, but 
> > > > > calling it SolrClient is more straightforward to understand, 
> > > > > hence the community decided to rename the class.
> > > > > It's possible to spawn an embedded Solr server, for example 
> > > > > for testing purposes, via class EmbeddedSolrServer (a class 
> > > > > that retains the same name in Solr7 and Solr4), which extends 
> > > > > the
> > SolrClient class.
> > > >
> > > >
> > > > Hope this helps,
> > > > Mike
> > > >
> > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut 
> > > > <helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > Thanks for the response. Would be cool if we get that sorted out.
> > > > >
> > > > > I've asked Yonghao Zou to submit the Pull Request, since he 
> > > > > did most of the work and should get the credit.
> > > > > He'll  do so after the Chinese New Year's Eve.
> > > > >
> > > > > I will then issue a Pull request for the new ES Rest client, 
> > > > > which is dependent on the above work.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > Hi Helmut,
> > > > > I see that I neglected to follow up on the other thread on 
> > > > > this topic after your reply about SolrServer missing from the solrj jar.
> > > > > Let me ask around w/ some folks I know that work on Solr and 
> > > > > see if there is any way to retain the SolrServer for our tests 
> > > > > after upgrading to the
> > > > new version.
> > > > >
> > > > > Thank you very much for working on upgrading Solr. Would you 
> > > > > mind submitting a pull request with your (apparently 
> > > > > work-in-progress) patch to upgrade both Solr and ES?
> > > > >
> > > > > To reply to your email in this thread, the JAR packaging 
> > > > > situation is largely the same after merging FLUME-2957 so 
> > > > > unfortunately most of what I noted in my reply in the other 
> > > > > thread ( https://s.apache.org/GqcX ) still holds.
> > > > >
> > > > > I hope that we can upgrade the Solr dependencies as part of 
> > > > > the same commit as the ES dependencies to avoid worrying about 
> > > > > which lucene jar is first in the classpath, and ensure we are 
> > > > > not adding any additional dependency conflicts to mvn dependency:tree.
> > > > >
> > > > > Regards,
> > > > > Mike
> > > > >
> > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> > > > > helmut.wahrmann@rsa.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > now that the blocker for FLUME-3021 is removed by committing 
> > > > > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Helmut
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Merge of patch in Flume-3021?

Posted by Mike Percy <mp...@apache.org>.
Nice! Thanks Ferenc. Answered better than I could have done. :)

Sorry, I meant to send this last week but I just found it in my drafts.

Helmut, please let us know if you need more help with this.

Mike

On Tue, Mar 6, 2018 at 7:25 AM, Ferenc Szabo <fs...@cloudera.com> wrote:

> the createJetty method of a Test class became final in the new versions of
> solr.
> the kite sdk test-jar has to be removed because it depends on a different
> incompatible version of solr
>
>     <dependency>
>       <groupId>org.kitesdk</groupId>
>       <artifactId>kite-morphlines-solr-core</artifactId>
>       <version>${kite.version}</version>
>       <type>test-jar</type>
>       <scope>test</scope>
>     </dependency>
>
> TestEnvironment.java has to be removed as well because it depends on the
> incompatible dependency
>
> then we need this class:
> https://github.com/kite-sdk/kite/blob/master/kite-morphlines
> /kite-morphlines-solr-core/src/test/java/org/kitesdk/
> morphline/solr/TestEmbeddedSolrServer.java
> I believe it is ok to have a copy of this because it is part of the
> incompatible test dependency we just removed
>
> then we need a newer version of commons-compress:
>     <commons-compress.version>1.10</commons-compress.version>
>
> from here you can fix the actual solr related test errors :)
>
>
>
> On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Ferenc,
> >
> > Thanks for offering help.
> > In agreement with Mike I want to have support for Solr 7.2.1 in the
> > morphline solr sink, so that we can easily upgrade the Elasticsearch
> sink.
> >
> > My updates are here: https://github.com/hwahrmann/
> > flume/tree/Upgrade_Morphline_Sink
> >
> > I changed the solr version to 7.2.1 and was able to compile the sink
> > withput any problems.
> > I can also compile the tests, but when running, I get multiple errors
> like
> > this:
> >
> > [INFO] Running org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time elapsed:
> > 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> > TestMorphlineInterceptor
> > [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s  <<<
> > ERROR!
> > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> tractSolrMorphlineZkTest
> > overrides final method createJetty.(Ljava/io/File;
> > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> JettySolrRunner;
> >         at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > build(TestMorphlineInterceptor.java:151)
> >         at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java:139)
> >
> > [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> > solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s  <<<
> > ERROR!
> > java.lang.VerifyError: class org.kitesdk.morphline.solr.Abs
> tractSolrMorphlineZkTest
> > overrides final method createJetty.(Ljava/io/File;
> > Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> > Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/
> JettySolrRunner;
> >         at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > build(TestMorphlineInterceptor.java:151)
> >         at org.apache.flume.sink.solr.morphline.TestMorphlineIntercepto
> r.
> > testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83)
> >
> >
> > The all have problems with createJetty. So it seems that I maybe need a
> > different version of jetty or something like that.
> > And for that I have too less knowledge about maven.
> >
> > thx,
> >
> > Helmut
> >
> > -----Original Message-----
> > From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> > Sent: Dienstag, 6. März 2018 11:04
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > Hi Helmut,
> >
> > let me know what can I help You with.
> > share your current code on a github fork and describe the issue. I will
> > see what can we do to solve it.
> >
> >
> > On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <
> helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Mike,
> > >
> > > I am stuck with Solr.
> > > The morphline-solr sink compiles without any problems, but I am
> > > struggling with the tests.
> > > Seems I need to exclude some stuff from maven, but my knowledge about
> > > maven is not good enough to figure out what to do.
> > > Anyone able to help?
> > >
> > > regards,
> > >
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Mike Percy [mailto:mpercy@apache.org]
> > > Sent: Mittwoch, 14. Februar 2018 19:19
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > > As long as the integration tests still pass and the packaging issues
> > > are not exacerbated, I don't see why we couldn't merge an upgrade
> > > patch, barring any serious concerns with the patch.
> > >
> > > Mike
> > >
> > > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut
> > > <helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > I won't have a problem upgrading the Solr sink to the latest version.
> > > > I am missing test environment however.
> > > > So while it may build correctly and all integration tests work, I
> > > > have no real environment to test with.
> > > >
> > > > best regards,
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > OK. In the pull request, it would be nice if whoever submits or
> > > > merges it mentions all of the contributors to the patch in the commit
> > message.
> > > >
> > > > I asked Wolfgang H. about the SolrServer thing and this is what he
> > > > told
> > > me:
> > > >
> > > > Hi Mike, the class has been renamed to "SolrClient" (which
> > > > unfortunately
> > > > > breaks compat). It's just a class rename. The functionality is the
> > > > > same as before. It was called SolrServer in Solr4 because it was a
> > > > > client proxy that sends RPCs to a Solr server, but calling it
> > > > > SolrClient is more straightforward to understand, hence the
> > > > > community decided to rename the class.
> > > > > It's possible to spawn an embedded Solr server, for example for
> > > > > testing purposes, via class EmbeddedSolrServer (a class that
> > > > > retains the same name in Solr7 and Solr4), which extends the
> > SolrClient class.
> > > >
> > > >
> > > > Hope this helps,
> > > > Mike
> > > >
> > > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut
> > > > <helmut.wahrmann@rsa.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hi Mike,
> > > > >
> > > > > Thanks for the response. Would be cool if we get that sorted out.
> > > > >
> > > > > I've asked Yonghao Zou to submit the Pull Request, since he did
> > > > > most of the work and should get the credit.
> > > > > He'll  do so after the Chinese New Year's Eve.
> > > > >
> > > > > I will then issue a Pull request for the new ES Rest client, which
> > > > > is dependent on the above work.
> > > > >
> > > > > best regards,
> > > > > Helmut
> > > > >
> > > > > -----Original Message-----
> > > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > > To: dev@flume.apache.org
> > > > > Subject: Re: Merge of patch in Flume-3021?
> > > > >
> > > > > Hi Helmut,
> > > > > I see that I neglected to follow up on the other thread on this
> > > > > topic after your reply about SolrServer missing from the solrj jar.
> > > > > Let me ask around w/ some folks I know that work on Solr and see
> > > > > if there is any way to retain the SolrServer for our tests after
> > > > > upgrading to the
> > > > new version.
> > > > >
> > > > > Thank you very much for working on upgrading Solr. Would you mind
> > > > > submitting a pull request with your (apparently work-in-progress)
> > > > > patch to upgrade both Solr and ES?
> > > > >
> > > > > To reply to your email in this thread, the JAR packaging situation
> > > > > is largely the same after merging FLUME-2957 so unfortunately most
> > > > > of what I noted in my reply in the other thread (
> > > > > https://s.apache.org/GqcX ) still holds.
> > > > >
> > > > > I hope that we can upgrade the Solr dependencies as part of the
> > > > > same commit as the ES dependencies to avoid worrying about which
> > > > > lucene jar is first in the classpath, and ensure we are not adding
> > > > > any additional dependency conflicts to mvn dependency:tree.
> > > > >
> > > > > Regards,
> > > > > Mike
> > > > >
> > > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> > > > > helmut.wahrmann@rsa.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > now that the blocker for FLUME-3021 is removed by committing
> > > > > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Helmut
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Merge of patch in Flume-3021?

Posted by Ferenc Szabo <fs...@cloudera.com>.
the createJetty method of a Test class became final in the new versions of
solr.
the kite sdk test-jar has to be removed because it depends on a different
incompatible version of solr

    <dependency>
      <groupId>org.kitesdk</groupId>
      <artifactId>kite-morphlines-solr-core</artifactId>
      <version>${kite.version}</version>
      <type>test-jar</type>
      <scope>test</scope>
    </dependency>

TestEnvironment.java has to be removed as well because it depends on the
incompatible dependency

then we need this class:
https://github.com/kite-sdk/kite/blob/master/kite-morphlines/kite-morphlines-solr-core/src/test/java/org/kitesdk/morphline/solr/TestEmbeddedSolrServer.java
I believe it is ok to have a copy of this because it is part of the
incompatible test dependency we just removed

then we need a newer version of commons-compress:
    <commons-compress.version>1.10</commons-compress.version>

from here you can fix the actual solr related test errors :)



On Tue, Mar 6, 2018 at 11:56 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Ferenc,
>
> Thanks for offering help.
> In agreement with Mike I want to have support for Solr 7.2.1 in the
> morphline solr sink, so that we can easily upgrade the Elasticsearch sink.
>
> My updates are here: https://github.com/hwahrmann/
> flume/tree/Upgrade_Morphline_Sink
>
> I changed the solr version to 7.2.1 and was able to compile the sink
> withput any problems.
> I can also compile the tests, but when running, I get multiple errors like
> this:
>
> [INFO] Running org.apache.flume.sink.solr.morphline.
> TestMorphlineInterceptor
> [ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time elapsed:
> 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.
> TestMorphlineInterceptor
> [ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.
> solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s  <<<
> ERROR!
> java.lang.VerifyError: class org.kitesdk.morphline.solr.AbstractSolrMorphlineZkTest
> overrides final method createJetty.(Ljava/io/File;
> Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/JettySolrRunner;
>         at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.
> build(TestMorphlineInterceptor.java:151)
>         at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.
> testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java:139)
>
> [ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.
> solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s  <<<
> ERROR!
> java.lang.VerifyError: class org.kitesdk.morphline.solr.AbstractSolrMorphlineZkTest
> overrides final method createJetty.(Ljava/io/File;
> Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;
> Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/JettySolrRunner;
>         at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.
> build(TestMorphlineInterceptor.java:151)
>         at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.
> testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83)
>
>
> The all have problems with createJetty. So it seems that I maybe need a
> different version of jetty or something like that.
> And for that I have too less knowledge about maven.
>
> thx,
>
> Helmut
>
> -----Original Message-----
> From: Ferenc Szabo [mailto:fszabo@cloudera.com]
> Sent: Dienstag, 6. März 2018 11:04
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Hi Helmut,
>
> let me know what can I help You with.
> share your current code on a github fork and describe the issue. I will
> see what can we do to solve it.
>
>
> On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Mike,
> >
> > I am stuck with Solr.
> > The morphline-solr sink compiles without any problems, but I am
> > struggling with the tests.
> > Seems I need to exclude some stuff from maven, but my knowledge about
> > maven is not good enough to figure out what to do.
> > Anyone able to help?
> >
> > regards,
> >
> > Helmut
> >
> > -----Original Message-----
> > From: Mike Percy [mailto:mpercy@apache.org]
> > Sent: Mittwoch, 14. Februar 2018 19:19
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > Hi Helmut,
> > As long as the integration tests still pass and the packaging issues
> > are not exacerbated, I don't see why we couldn't merge an upgrade
> > patch, barring any serious concerns with the patch.
> >
> > Mike
> >
> > On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Mike,
> > >
> > > I won't have a problem upgrading the Solr sink to the latest version.
> > > I am missing test environment however.
> > > So while it may build correctly and all integration tests work, I
> > > have no real environment to test with.
> > >
> > > best regards,
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Mike Percy [mailto:mpercy@apache.org]
> > > Sent: Mittwoch, 14. Februar 2018 00:38
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > OK. In the pull request, it would be nice if whoever submits or
> > > merges it mentions all of the contributors to the patch in the commit
> message.
> > >
> > > I asked Wolfgang H. about the SolrServer thing and this is what he
> > > told
> > me:
> > >
> > > Hi Mike, the class has been renamed to "SolrClient" (which
> > > unfortunately
> > > > breaks compat). It's just a class rename. The functionality is the
> > > > same as before. It was called SolrServer in Solr4 because it was a
> > > > client proxy that sends RPCs to a Solr server, but calling it
> > > > SolrClient is more straightforward to understand, hence the
> > > > community decided to rename the class.
> > > > It's possible to spawn an embedded Solr server, for example for
> > > > testing purposes, via class EmbeddedSolrServer (a class that
> > > > retains the same name in Solr7 and Solr4), which extends the
> SolrClient class.
> > >
> > >
> > > Hope this helps,
> > > Mike
> > >
> > > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut
> > > <helmut.wahrmann@rsa.com
> > > >
> > > wrote:
> > >
> > > > Hi Mike,
> > > >
> > > > Thanks for the response. Would be cool if we get that sorted out.
> > > >
> > > > I've asked Yonghao Zou to submit the Pull Request, since he did
> > > > most of the work and should get the credit.
> > > > He'll  do so after the Chinese New Year's Eve.
> > > >
> > > > I will then issue a Pull request for the new ES Rest client, which
> > > > is dependent on the above work.
> > > >
> > > > best regards,
> > > > Helmut
> > > >
> > > > -----Original Message-----
> > > > From: Mike Percy [mailto:mpercy@apache.org]
> > > > Sent: Dienstag, 13. Februar 2018 04:30
> > > > To: dev@flume.apache.org
> > > > Subject: Re: Merge of patch in Flume-3021?
> > > >
> > > > Hi Helmut,
> > > > I see that I neglected to follow up on the other thread on this
> > > > topic after your reply about SolrServer missing from the solrj jar.
> > > > Let me ask around w/ some folks I know that work on Solr and see
> > > > if there is any way to retain the SolrServer for our tests after
> > > > upgrading to the
> > > new version.
> > > >
> > > > Thank you very much for working on upgrading Solr. Would you mind
> > > > submitting a pull request with your (apparently work-in-progress)
> > > > patch to upgrade both Solr and ES?
> > > >
> > > > To reply to your email in this thread, the JAR packaging situation
> > > > is largely the same after merging FLUME-2957 so unfortunately most
> > > > of what I noted in my reply in the other thread (
> > > > https://s.apache.org/GqcX ) still holds.
> > > >
> > > > I hope that we can upgrade the Solr dependencies as part of the
> > > > same commit as the ES dependencies to avoid worrying about which
> > > > lucene jar is first in the classpath, and ensure we are not adding
> > > > any additional dependency conflicts to mvn dependency:tree.
> > > >
> > > > Regards,
> > > > Mike
> > > >
> > > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> > > > helmut.wahrmann@rsa.com>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > now that the blocker for FLUME-3021 is removed by committing
> > > > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Helmut
> > > > >
> > > >
> > >
> >
>

RE: Merge of patch in Flume-3021?

Posted by "Wahrmann, Helmut" <he...@rsa.com>.
Hi Ferenc,

Thanks for offering help.
In agreement with Mike I want to have support for Solr 7.2.1 in the morphline solr sink, so that we can easily upgrade the Elasticsearch sink.

My updates are here: https://github.com/hwahrmann/flume/tree/Upgrade_Morphline_Sink

I changed the solr version to 7.2.1 and was able to compile the sink withput any problems.
I can also compile the tests, but when running, I get multiple errors like this:

[INFO] Running org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
[ERROR] Tests run: 66, Failures: 0, Errors: 66, Skipped: 0, Time elapsed: 24.438 s <<< FAILURE! - in org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor
[ERROR] testIfDetectMimeTypeRouteToNorthPole(org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor)  Time elapsed: 1.985 s  <<< ERROR!
java.lang.VerifyError: class org.kitesdk.morphline.solr.AbstractSolrMorphlineZkTest overrides final method createJetty.(Ljava/io/File;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/JettySolrRunner;
        at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.build(TestMorphlineInterceptor.java:151)
        at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.testIfDetectMimeTypeRouteToNorthPole(TestMorphlineInterceptor.java:139)

[ERROR] testGrokIfNotMatchDropEventRetain(org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor)  Time elapsed: 0.363 s  <<< ERROR!
java.lang.VerifyError: class org.kitesdk.morphline.solr.AbstractSolrMorphlineZkTest overrides final method createJetty.(Ljava/io/File;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lorg/apache/solr/client/solrj/embedded/JettySolrRunner;
        at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.build(TestMorphlineInterceptor.java:151)
        at org.apache.flume.sink.solr.morphline.TestMorphlineInterceptor.testGrokIfNotMatchDropEventRetain(TestMorphlineInterceptor.java:83)


The all have problems with createJetty. So it seems that I maybe need a different version of jetty or something like that.
And for that I have too less knowledge about maven.

thx,

Helmut

-----Original Message-----
From: Ferenc Szabo [mailto:fszabo@cloudera.com] 
Sent: Dienstag, 6. März 2018 11:04
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

Hi Helmut,

let me know what can I help You with.
share your current code on a github fork and describe the issue. I will see what can we do to solve it.


On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> I am stuck with Solr.
> The morphline-solr sink compiles without any problems, but I am 
> struggling with the tests.
> Seems I need to exclude some stuff from maven, but my knowledge about 
> maven is not good enough to figure out what to do.
> Anyone able to help?
>
> regards,
>
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Mittwoch, 14. Februar 2018 19:19
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Hi Helmut,
> As long as the integration tests still pass and the packaging issues 
> are not exacerbated, I don't see why we couldn't merge an upgrade 
> patch, barring any serious concerns with the patch.
>
> Mike
>
> On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut 
> <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Mike,
> >
> > I won't have a problem upgrading the Solr sink to the latest version.
> > I am missing test environment however.
> > So while it may build correctly and all integration tests work, I 
> > have no real environment to test with.
> >
> > best regards,
> > Helmut
> >
> > -----Original Message-----
> > From: Mike Percy [mailto:mpercy@apache.org]
> > Sent: Mittwoch, 14. Februar 2018 00:38
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > OK. In the pull request, it would be nice if whoever submits or 
> > merges it mentions all of the contributors to the patch in the commit message.
> >
> > I asked Wolfgang H. about the SolrServer thing and this is what he 
> > told
> me:
> >
> > Hi Mike, the class has been renamed to "SolrClient" (which 
> > unfortunately
> > > breaks compat). It's just a class rename. The functionality is the 
> > > same as before. It was called SolrServer in Solr4 because it was a 
> > > client proxy that sends RPCs to a Solr server, but calling it 
> > > SolrClient is more straightforward to understand, hence the 
> > > community decided to rename the class.
> > > It's possible to spawn an embedded Solr server, for example for 
> > > testing purposes, via class EmbeddedSolrServer (a class that 
> > > retains the same name in Solr7 and Solr4), which extends the SolrClient class.
> >
> >
> > Hope this helps,
> > Mike
> >
> > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut 
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Mike,
> > >
> > > Thanks for the response. Would be cool if we get that sorted out.
> > >
> > > I've asked Yonghao Zou to submit the Pull Request, since he did 
> > > most of the work and should get the credit.
> > > He'll  do so after the Chinese New Year's Eve.
> > >
> > > I will then issue a Pull request for the new ES Rest client, which 
> > > is dependent on the above work.
> > >
> > > best regards,
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Mike Percy [mailto:mpercy@apache.org]
> > > Sent: Dienstag, 13. Februar 2018 04:30
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > > I see that I neglected to follow up on the other thread on this 
> > > topic after your reply about SolrServer missing from the solrj jar.
> > > Let me ask around w/ some folks I know that work on Solr and see 
> > > if there is any way to retain the SolrServer for our tests after 
> > > upgrading to the
> > new version.
> > >
> > > Thank you very much for working on upgrading Solr. Would you mind 
> > > submitting a pull request with your (apparently work-in-progress) 
> > > patch to upgrade both Solr and ES?
> > >
> > > To reply to your email in this thread, the JAR packaging situation 
> > > is largely the same after merging FLUME-2957 so unfortunately most 
> > > of what I noted in my reply in the other thread ( 
> > > https://s.apache.org/GqcX ) still holds.
> > >
> > > I hope that we can upgrade the Solr dependencies as part of the 
> > > same commit as the ES dependencies to avoid worrying about which 
> > > lucene jar is first in the classpath, and ensure we are not adding 
> > > any additional dependency conflicts to mvn dependency:tree.
> > >
> > > Regards,
> > > Mike
> > >
> > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> > > helmut.wahrmann@rsa.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > now that the blocker for FLUME-3021 is removed by committing 
> > > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > >
> > > > Thanks,
> > > >
> > > > Helmut
> > > >
> > >
> >
>

Re: Merge of patch in Flume-3021?

Posted by Ferenc Szabo <fs...@cloudera.com>.
Hi Helmut,

let me know what can I help You with.
share your current code on a github fork and describe the issue. I will see
what can we do to solve it.


On Tue, Mar 6, 2018 at 10:50 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> I am stuck with Solr.
> The morphline-solr sink compiles without any problems, but I am struggling
> with the tests.
> Seems I need to exclude some stuff from maven, but my knowledge about
> maven is not good enough to figure out what to do.
> Anyone able to help?
>
> regards,
>
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Mittwoch, 14. Februar 2018 19:19
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Hi Helmut,
> As long as the integration tests still pass and the packaging issues are
> not exacerbated, I don't see why we couldn't merge an upgrade patch,
> barring any serious concerns with the patch.
>
> Mike
>
> On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Mike,
> >
> > I won't have a problem upgrading the Solr sink to the latest version.
> > I am missing test environment however.
> > So while it may build correctly and all integration tests work, I have
> > no real environment to test with.
> >
> > best regards,
> > Helmut
> >
> > -----Original Message-----
> > From: Mike Percy [mailto:mpercy@apache.org]
> > Sent: Mittwoch, 14. Februar 2018 00:38
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > OK. In the pull request, it would be nice if whoever submits or merges
> > it mentions all of the contributors to the patch in the commit message.
> >
> > I asked Wolfgang H. about the SolrServer thing and this is what he told
> me:
> >
> > Hi Mike, the class has been renamed to "SolrClient" (which
> > unfortunately
> > > breaks compat). It's just a class rename. The functionality is the
> > > same as before. It was called SolrServer in Solr4 because it was a
> > > client proxy that sends RPCs to a Solr server, but calling it
> > > SolrClient is more straightforward to understand, hence the
> > > community decided to rename the class.
> > > It's possible to spawn an embedded Solr server, for example for
> > > testing purposes, via class EmbeddedSolrServer (a class that retains
> > > the same name in Solr7 and Solr4), which extends the SolrClient class.
> >
> >
> > Hope this helps,
> > Mike
> >
> > On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut
> > <helmut.wahrmann@rsa.com
> > >
> > wrote:
> >
> > > Hi Mike,
> > >
> > > Thanks for the response. Would be cool if we get that sorted out.
> > >
> > > I've asked Yonghao Zou to submit the Pull Request, since he did most
> > > of the work and should get the credit.
> > > He'll  do so after the Chinese New Year's Eve.
> > >
> > > I will then issue a Pull request for the new ES Rest client, which
> > > is dependent on the above work.
> > >
> > > best regards,
> > > Helmut
> > >
> > > -----Original Message-----
> > > From: Mike Percy [mailto:mpercy@apache.org]
> > > Sent: Dienstag, 13. Februar 2018 04:30
> > > To: dev@flume.apache.org
> > > Subject: Re: Merge of patch in Flume-3021?
> > >
> > > Hi Helmut,
> > > I see that I neglected to follow up on the other thread on this
> > > topic after your reply about SolrServer missing from the solrj jar.
> > > Let me ask around w/ some folks I know that work on Solr and see if
> > > there is any way to retain the SolrServer for our tests after
> > > upgrading to the
> > new version.
> > >
> > > Thank you very much for working on upgrading Solr. Would you mind
> > > submitting a pull request with your (apparently work-in-progress)
> > > patch to upgrade both Solr and ES?
> > >
> > > To reply to your email in this thread, the JAR packaging situation
> > > is largely the same after merging FLUME-2957 so unfortunately most
> > > of what I noted in my reply in the other thread (
> > > https://s.apache.org/GqcX ) still holds.
> > >
> > > I hope that we can upgrade the Solr dependencies as part of the same
> > > commit as the ES dependencies to avoid worrying about which lucene
> > > jar is first in the classpath, and ensure we are not adding any
> > > additional dependency conflicts to mvn dependency:tree.
> > >
> > > Regards,
> > > Mike
> > >
> > > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> > > helmut.wahrmann@rsa.com>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > now that the blocker for FLUME-3021 is removed by committing
> > > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > > >
> > > > Thanks,
> > > >
> > > > Helmut
> > > >
> > >
> >
>

RE: Merge of patch in Flume-3021?

Posted by "Wahrmann, Helmut" <he...@rsa.com>.
Hi Mike,

I am stuck with Solr.
The morphline-solr sink compiles without any problems, but I am struggling with the tests.
Seems I need to exclude some stuff from maven, but my knowledge about maven is not good enough to figure out what to do.
Anyone able to help?

regards,

Helmut

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org] 
Sent: Mittwoch, 14. Februar 2018 19:19
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

Hi Helmut,
As long as the integration tests still pass and the packaging issues are not exacerbated, I don't see why we couldn't merge an upgrade patch, barring any serious concerns with the patch.

Mike

On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> I won't have a problem upgrading the Solr sink to the latest version.
> I am missing test environment however.
> So while it may build correctly and all integration tests work, I have 
> no real environment to test with.
>
> best regards,
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Mittwoch, 14. Februar 2018 00:38
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> OK. In the pull request, it would be nice if whoever submits or merges 
> it mentions all of the contributors to the patch in the commit message.
>
> I asked Wolfgang H. about the SolrServer thing and this is what he told me:
>
> Hi Mike, the class has been renamed to "SolrClient" (which 
> unfortunately
> > breaks compat). It's just a class rename. The functionality is the 
> > same as before. It was called SolrServer in Solr4 because it was a 
> > client proxy that sends RPCs to a Solr server, but calling it 
> > SolrClient is more straightforward to understand, hence the 
> > community decided to rename the class.
> > It's possible to spawn an embedded Solr server, for example for 
> > testing purposes, via class EmbeddedSolrServer (a class that retains 
> > the same name in Solr7 and Solr4), which extends the SolrClient class.
>
>
> Hope this helps,
> Mike
>
> On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut 
> <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Mike,
> >
> > Thanks for the response. Would be cool if we get that sorted out.
> >
> > I've asked Yonghao Zou to submit the Pull Request, since he did most 
> > of the work and should get the credit.
> > He'll  do so after the Chinese New Year's Eve.
> >
> > I will then issue a Pull request for the new ES Rest client, which 
> > is dependent on the above work.
> >
> > best regards,
> > Helmut
> >
> > -----Original Message-----
> > From: Mike Percy [mailto:mpercy@apache.org]
> > Sent: Dienstag, 13. Februar 2018 04:30
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > Hi Helmut,
> > I see that I neglected to follow up on the other thread on this 
> > topic after your reply about SolrServer missing from the solrj jar. 
> > Let me ask around w/ some folks I know that work on Solr and see if 
> > there is any way to retain the SolrServer for our tests after 
> > upgrading to the
> new version.
> >
> > Thank you very much for working on upgrading Solr. Would you mind 
> > submitting a pull request with your (apparently work-in-progress) 
> > patch to upgrade both Solr and ES?
> >
> > To reply to your email in this thread, the JAR packaging situation 
> > is largely the same after merging FLUME-2957 so unfortunately most 
> > of what I noted in my reply in the other thread ( 
> > https://s.apache.org/GqcX ) still holds.
> >
> > I hope that we can upgrade the Solr dependencies as part of the same 
> > commit as the ES dependencies to avoid worrying about which lucene 
> > jar is first in the classpath, and ensure we are not adding any 
> > additional dependency conflicts to mvn dependency:tree.
> >
> > Regards,
> > Mike
> >
> > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> > helmut.wahrmann@rsa.com>
> > wrote:
> >
> > > Hi,
> > >
> > > now that the blocker for FLUME-3021 is removed by committing 
> > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > >
> > > Thanks,
> > >
> > > Helmut
> > >
> >
>

Re: Merge of patch in Flume-3021?

Posted by Mike Percy <mp...@apache.org>.
Hi Helmut,
As long as the integration tests still pass and the packaging issues are
not exacerbated, I don't see why we couldn't merge an upgrade patch,
barring any serious concerns with the patch.

Mike

On Wed, Feb 14, 2018 at 1:47 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> I won't have a problem upgrading the Solr sink to the latest version.
> I am missing test environment however.
> So while it may build correctly and all integration tests work, I have no
> real environment to test with.
>
> best regards,
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Mittwoch, 14. Februar 2018 00:38
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> OK. In the pull request, it would be nice if whoever submits or merges it
> mentions all of the contributors to the patch in the commit message.
>
> I asked Wolfgang H. about the SolrServer thing and this is what he told me:
>
> Hi Mike, the class has been renamed to "SolrClient" (which unfortunately
> > breaks compat). It's just a class rename. The functionality is the
> > same as before. It was called SolrServer in Solr4 because it was a
> > client proxy that sends RPCs to a Solr server, but calling it
> > SolrClient is more straightforward to understand, hence the community
> > decided to rename the class.
> > It's possible to spawn an embedded Solr server, for example for
> > testing purposes, via class EmbeddedSolrServer (a class that retains
> > the same name in Solr7 and Solr4), which extends the SolrClient class.
>
>
> Hope this helps,
> Mike
>
> On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut <helmut.wahrmann@rsa.com
> >
> wrote:
>
> > Hi Mike,
> >
> > Thanks for the response. Would be cool if we get that sorted out.
> >
> > I've asked Yonghao Zou to submit the Pull Request, since he did most
> > of the work and should get the credit.
> > He'll  do so after the Chinese New Year's Eve.
> >
> > I will then issue a Pull request for the new ES Rest client, which is
> > dependent on the above work.
> >
> > best regards,
> > Helmut
> >
> > -----Original Message-----
> > From: Mike Percy [mailto:mpercy@apache.org]
> > Sent: Dienstag, 13. Februar 2018 04:30
> > To: dev@flume.apache.org
> > Subject: Re: Merge of patch in Flume-3021?
> >
> > Hi Helmut,
> > I see that I neglected to follow up on the other thread on this topic
> > after your reply about SolrServer missing from the solrj jar. Let me
> > ask around w/ some folks I know that work on Solr and see if there is
> > any way to retain the SolrServer for our tests after upgrading to the
> new version.
> >
> > Thank you very much for working on upgrading Solr. Would you mind
> > submitting a pull request with your (apparently work-in-progress)
> > patch to upgrade both Solr and ES?
> >
> > To reply to your email in this thread, the JAR packaging situation is
> > largely the same after merging FLUME-2957 so unfortunately most of
> > what I noted in my reply in the other thread (
> > https://s.apache.org/GqcX ) still holds.
> >
> > I hope that we can upgrade the Solr dependencies as part of the same
> > commit as the ES dependencies to avoid worrying about which lucene jar
> > is first in the classpath, and ensure we are not adding any additional
> > dependency conflicts to mvn dependency:tree.
> >
> > Regards,
> > Mike
> >
> > On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> > helmut.wahrmann@rsa.com>
> > wrote:
> >
> > > Hi,
> > >
> > > now that the blocker for FLUME-3021 is removed by committing
> > > FLUME-2957, can we get the patch from 3021 merged to trunk?
> > >
> > > Thanks,
> > >
> > > Helmut
> > >
> >
>

RE: Merge of patch in Flume-3021?

Posted by "Wahrmann, Helmut" <he...@rsa.com>.
Hi Mike,

I won't have a problem upgrading the Solr sink to the latest version.
I am missing test environment however.
So while it may build correctly and all integration tests work, I have no real environment to test with.

best regards,
Helmut

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org] 
Sent: Mittwoch, 14. Februar 2018 00:38
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

OK. In the pull request, it would be nice if whoever submits or merges it mentions all of the contributors to the patch in the commit message.

I asked Wolfgang H. about the SolrServer thing and this is what he told me:

Hi Mike, the class has been renamed to "SolrClient" (which unfortunately
> breaks compat). It's just a class rename. The functionality is the 
> same as before. It was called SolrServer in Solr4 because it was a 
> client proxy that sends RPCs to a Solr server, but calling it 
> SolrClient is more straightforward to understand, hence the community 
> decided to rename the class.
> It's possible to spawn an embedded Solr server, for example for 
> testing purposes, via class EmbeddedSolrServer (a class that retains 
> the same name in Solr7 and Solr4), which extends the SolrClient class.


Hope this helps,
Mike

On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> Thanks for the response. Would be cool if we get that sorted out.
>
> I've asked Yonghao Zou to submit the Pull Request, since he did most 
> of the work and should get the credit.
> He'll  do so after the Chinese New Year's Eve.
>
> I will then issue a Pull request for the new ES Rest client, which is 
> dependent on the above work.
>
> best regards,
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Dienstag, 13. Februar 2018 04:30
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Hi Helmut,
> I see that I neglected to follow up on the other thread on this topic 
> after your reply about SolrServer missing from the solrj jar. Let me 
> ask around w/ some folks I know that work on Solr and see if there is 
> any way to retain the SolrServer for our tests after upgrading to the new version.
>
> Thank you very much for working on upgrading Solr. Would you mind 
> submitting a pull request with your (apparently work-in-progress) 
> patch to upgrade both Solr and ES?
>
> To reply to your email in this thread, the JAR packaging situation is 
> largely the same after merging FLUME-2957 so unfortunately most of 
> what I noted in my reply in the other thread ( 
> https://s.apache.org/GqcX ) still holds.
>
> I hope that we can upgrade the Solr dependencies as part of the same 
> commit as the ES dependencies to avoid worrying about which lucene jar 
> is first in the classpath, and ensure we are not adding any additional 
> dependency conflicts to mvn dependency:tree.
>
> Regards,
> Mike
>
> On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut < 
> helmut.wahrmann@rsa.com>
> wrote:
>
> > Hi,
> >
> > now that the blocker for FLUME-3021 is removed by committing 
> > FLUME-2957, can we get the patch from 3021 merged to trunk?
> >
> > Thanks,
> >
> > Helmut
> >
>

Re: Merge of patch in Flume-3021?

Posted by Mike Percy <mp...@apache.org>.
OK. In the pull request, it would be nice if whoever submits or merges it
mentions all of the contributors to the patch in the commit message.

I asked Wolfgang H. about the SolrServer thing and this is what he told me:

Hi Mike, the class has been renamed to "SolrClient" (which unfortunately
> breaks compat). It's just a class rename. The functionality is the same as
> before. It was called SolrServer in Solr4 because it was a client proxy
> that sends RPCs to a Solr server, but calling it SolrClient is more
> straightforward to understand, hence the community decided to rename the
> class.
> It's possible to spawn an embedded Solr server, for example for testing
> purposes, via class EmbeddedSolrServer (a class that retains the same name
> in Solr7 and Solr4), which extends the SolrClient class.


Hope this helps,
Mike

On Tue, Feb 13, 2018 at 4:41 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi Mike,
>
> Thanks for the response. Would be cool if we get that sorted out.
>
> I've asked Yonghao Zou to submit the Pull Request, since he did most of
> the work and should get the credit.
> He'll  do so after the Chinese New Year's Eve.
>
> I will then issue a Pull request for the new ES Rest client, which is
> dependent on the above work.
>
> best regards,
> Helmut
>
> -----Original Message-----
> From: Mike Percy [mailto:mpercy@apache.org]
> Sent: Dienstag, 13. Februar 2018 04:30
> To: dev@flume.apache.org
> Subject: Re: Merge of patch in Flume-3021?
>
> Hi Helmut,
> I see that I neglected to follow up on the other thread on this topic
> after your reply about SolrServer missing from the solrj jar. Let me ask
> around w/ some folks I know that work on Solr and see if there is any way
> to retain the SolrServer for our tests after upgrading to the new version.
>
> Thank you very much for working on upgrading Solr. Would you mind
> submitting a pull request with your (apparently work-in-progress) patch to
> upgrade both Solr and ES?
>
> To reply to your email in this thread, the JAR packaging situation is
> largely the same after merging FLUME-2957 so unfortunately most of what I
> noted in my reply in the other thread ( https://s.apache.org/GqcX ) still
> holds.
>
> I hope that we can upgrade the Solr dependencies as part of the same
> commit as the ES dependencies to avoid worrying about which lucene jar is
> first in the classpath, and ensure we are not adding any additional
> dependency conflicts to mvn dependency:tree.
>
> Regards,
> Mike
>
> On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <
> helmut.wahrmann@rsa.com>
> wrote:
>
> > Hi,
> >
> > now that the blocker for FLUME-3021 is removed by committing
> > FLUME-2957, can we get the patch from 3021 merged to trunk?
> >
> > Thanks,
> >
> > Helmut
> >
>

RE: Merge of patch in Flume-3021?

Posted by "Wahrmann, Helmut" <he...@rsa.com>.
Hi Mike,

Thanks for the response. Would be cool if we get that sorted out.

I've asked Yonghao Zou to submit the Pull Request, since he did most of the work and should get the credit.
He'll  do so after the Chinese New Year's Eve.

I will then issue a Pull request for the new ES Rest client, which is dependent on the above work.

best regards,
Helmut

-----Original Message-----
From: Mike Percy [mailto:mpercy@apache.org] 
Sent: Dienstag, 13. Februar 2018 04:30
To: dev@flume.apache.org
Subject: Re: Merge of patch in Flume-3021?

Hi Helmut,
I see that I neglected to follow up on the other thread on this topic after your reply about SolrServer missing from the solrj jar. Let me ask around w/ some folks I know that work on Solr and see if there is any way to retain the SolrServer for our tests after upgrading to the new version.

Thank you very much for working on upgrading Solr. Would you mind submitting a pull request with your (apparently work-in-progress) patch to upgrade both Solr and ES?

To reply to your email in this thread, the JAR packaging situation is largely the same after merging FLUME-2957 so unfortunately most of what I noted in my reply in the other thread ( https://s.apache.org/GqcX ) still holds.

I hope that we can upgrade the Solr dependencies as part of the same commit as the ES dependencies to avoid worrying about which lucene jar is first in the classpath, and ensure we are not adding any additional dependency conflicts to mvn dependency:tree.

Regards,
Mike

On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi,
>
> now that the blocker for FLUME-3021 is removed by committing 
> FLUME-2957, can we get the patch from 3021 merged to trunk?
>
> Thanks,
>
> Helmut
>

Re: Merge of patch in Flume-3021?

Posted by Mike Percy <mp...@apache.org>.
Hi Helmut,
I see that I neglected to follow up on the other thread on this topic after
your reply about SolrServer missing from the solrj jar. Let me ask around
w/ some folks I know that work on Solr and see if there is any way to
retain the SolrServer for our tests after upgrading to the new version.

Thank you very much for working on upgrading Solr. Would you mind
submitting a pull request with your (apparently work-in-progress) patch to
upgrade both Solr and ES?

To reply to your email in this thread, the JAR packaging situation is
largely the same after merging FLUME-2957 so unfortunately most of what I
noted in my reply in the other thread ( https://s.apache.org/GqcX ) still
holds.

I hope that we can upgrade the Solr dependencies as part of the same commit
as the ES dependencies to avoid worrying about which lucene jar is first in
the classpath, and ensure we are not adding any additional dependency
conflicts to mvn dependency:tree.

Regards,
Mike

On Mon, Feb 12, 2018 at 12:54 AM, Wahrmann, Helmut <he...@rsa.com>
wrote:

> Hi,
>
> now that the blocker for FLUME-3021 is removed by committing FLUME-2957,
> can we get the patch from 3021 merged to trunk?
>
> Thanks,
>
> Helmut
>