You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@metron.apache.org by Tyler Moore <tm...@goflyball.com> on 2017/01/05 22:17:05 UTC
Kafka error when upgrading to Metron 0.3.0
Hey all,
Wondering if there is a solution to the "Offset lags for kafka not
supported for older versions. Please update kafka spout to latest version."
error seen when upgrading to metron 0.3.0?
I know it is due to kafka-storm dependency that needs updated, but what is
the best way to go about that? Is it as simple at changing the version in
the metron pom.xml and provisioning?
Regards,
Tyler Moore
Software Engineer
Phone: 248-909-2769
Email: moore.tyler@goflyball.com
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Tyler Moore <tm...@goflyball.com>.
I ran the same cmd you posted Michael, straight from the readme.
Otto I think that is exactly the issue, I had about 50 tabs open and 2
IDE's along with 3 other server instances running.
So looks like it was just a resource issue do you mind if I add in a small
note on that in the readme, in case other people run into the same issue? I
could put in a ticket for it if you like
Regards,
Tyler Moore
Software Engineer
Phone: 248-909-2769
Email: moore.tyler@goflyball.com
On Tue, Jan 10, 2017 at 9:17 AM, Otto Fowler <ot...@gmail.com>
wrote:
> I have seen where memory pressure can cause some of the tests to fail on my
> laptop ( for example I have had to shut down safari when I have had a
> massive number of tabs open ).
>
> Is there any chance that this is resource related?
>
>
> On January 9, 2017 at 22:58:08, Michael Miklavcic (
> michael.miklavcic@gmail.com) wrote:
>
> Hi Tyler, You shouldn't have to skip the unit tests as we ensure every
> commit runs with 100% of tests passing. There are rare instances that this
> is not true, but in general all tests should pass. I just pulled down
> latest master and re-ran the following successfully.
>
> mvn clean install -PHDP-2.5.0.0
>
> Which branch or commit are you working from, and what command did you run
> for the build? Be aware that Maven will ignore misspelled profile names and
> instead use the default profile without warning.
>
> On Mon, Jan 9, 2017 at 10:32 AM, Tyler Moore <tm...@goflyball.com> wrote:
>
> > Seems like it's just the unit tests that are failing for pcap backend.
> Was
> > able to build successfully using new came with -DskipTests option along
> > with the profile argument.
> >
> > Regards,
> >
> > Tyler Moore
> > IT Specialist
> > tyler.mathieu@yahoo.com
> > 248-909-2769
> >
> > > On Jan 9, 2017, at 02:29, Michael Miklavcic <
> michael.miklavcic@gmail.com>
>
> > wrote:
> > >
> > > Hi Tyler, I don't recall seeing any failures the last time I ran this.
> I
> > > will take a look.
> > >
> > > On Jan 8, 2017 9:56 PM, "Tyler Moore" <tm...@goflyball.com> wrote:
> > >
> > > Michael,
> > >
> > > I am receiving error when trying to build with HPD profile, have you
> had
> > > any problems with pcap-backend tests failing when building with hdp
> > profile?
> > > Errors and log files provided below:
> > >
> > > Results :
> > >
> > > Failed tests:
> > > PcapTopologyIntegrationTest.testTimestampInKey:152->testTopology:388->
> > assertInOrder:542
> > > null
> > > PcapTopologyIntegrationTest.testTimestampInPacket:135->
> > testTopology:388->assertInOrder:542
> > > null
> > >
> > >
> > >
> > > Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
> > >
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [INFO] Reactor Summary:
> > > [INFO]
> > > [INFO] Metron ............................................. SUCCESS [
> > > 0.639 s]
> > > [INFO] metron-analytics ................................... SUCCESS [
> > > 0.029 s]
> > > [INFO] metron-maas-common ................................. SUCCESS [
> > > 8.771 s]
> > > [INFO] metron-platform .................................... SUCCESS [
> > > 0.057 s]
> > > [INFO] metron-test-utilities .............................. SUCCESS [
> > > 0.976 s]
> > > [INFO] metron-integration-test ............................ SUCCESS [
> > > 3.888 s]
> > > [INFO] metron-maas-service ................................ SUCCESS
> > [01:19
> > > min]
> > > [INFO] metron-common ...................................... SUCCESS [
> > > 41.718 s]
> > > [INFO] metron-statistics .................................. SUCCESS [
> > > 30.713 s]
> > > [INFO] metron-hbase ....................................... SUCCESS [
> > > 37.286 s]
> > > [INFO] metron-profiler-common ............................. SUCCESS [
> > > 7.967 s]
> > > [INFO] metron-profiler-client ............................. SUCCESS [
> > > 54.088 s]
> > > [INFO] metron-profiler .................................... SUCCESS
> > [02:28
> > > min]
> > > [INFO] metron-writer ...................................... SUCCESS [
> > > 7.151 s]
> > > [INFO] metron-enrichment .................................. SUCCESS [
> > > 57.076 s]
> > > [INFO] metron-indexing .................................... SUCCESS [
> > > 7.028 s]
> > > [INFO] metron-solr ........................................ SUCCESS [
> > > 36.695 s]
> > > [INFO] metron-pcap ........................................ SUCCESS [
> > > 1.123 s]
> > > [INFO] metron-parsers ..................................... SUCCESS
> > [01:58
> > > min]
> > > [INFO] metron-pcap-backend ................................ FAILURE [
> > > 49.928 s]
> > > [INFO] metron-data-management ............................. SKIPPED
> > > [INFO] metron-api ......................................... SKIPPED
> > > [INFO] metron-management .................................. SKIPPED
> > > [INFO] elasticsearch-shaded ............................... SKIPPED
> > > [INFO] metron-elasticsearch ............................... SKIPPED
> > > [INFO] metron-deployment .................................. SKIPPED
> > > [INFO] Metron Ambari Management Pack ...................... SKIPPED
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [INFO] BUILD FAILURE
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [INFO] Total time: 11:31 min
> > > [INFO] Finished at: 2017-01-08T22:23:59-05:00
> > > [INFO] Final Memory: 287M/6202M
> > > [INFO] ------------------------------------------------------------
> > > ------------
> > > [ERROR] Failed to execute goal org.apache.maven.plugins:
> > > maven-surefire-plugin:2.18:test (integration-tests) on project
> > > metron-pcap-backend: There are test failures.
> > >
> > >
> > > Regards,
> > >
> > > Tyler Moore
> > > Software Engineer
> > > Phone: 248-909-2769 <(248)%20909-2769>
> > > Email: moore.tyler@goflyball.com
> > >
> > >
> > > On Thu, Jan 5, 2017 at 5:21 PM, Michael Miklavcic <
> > > michael.miklavcic@gmail.com> wrote:
> > >
> > >> Hey Tyler,
> > >>
> > >> Build Metron with the HDP profile to get the proper deps for this
> > >> -PHDP-2.5.0.0
> > >>
> > >> Hopefully that works for you.
> > >>
> > >> Best,
> > >> Mike
> > >>
> > >>
> > >>> On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com>
> > wrote:
> > >>>
> > >>> Hey all,
> > >>>
> > >>> Wondering if there is a solution to the "Offset lags for kafka not
> > >>> supported for older versions. Please update kafka spout to latest
> > >> version."
> > >>> error seen when upgrading to metron 0.3.0?
> > >>>
> > >>> I know it is due to kafka-storm dependency that needs updated, but
> what
> > >> is
> > >>> the best way to go about that? Is it as simple at changing the
> version
> > in
> > >>> the metron pom.xml and provisioning?
> > >>>
> > >>> Regards,
> > >>>
> > >>> Tyler Moore
> > >>> Software Engineer
> > >>> Phone: 248-909-2769
> > >>> Email: moore.tyler@goflyball.com
> > >>>
> > >>
> >
>
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Otto Fowler <ot...@gmail.com>.
I have seen where memory pressure can cause some of the tests to fail on my
laptop ( for example I have had to shut down safari when I have had a
massive number of tabs open ).
Is there any chance that this is resource related?
On January 9, 2017 at 22:58:08, Michael Miklavcic (
michael.miklavcic@gmail.com) wrote:
Hi Tyler, You shouldn't have to skip the unit tests as we ensure every
commit runs with 100% of tests passing. There are rare instances that this
is not true, but in general all tests should pass. I just pulled down
latest master and re-ran the following successfully.
mvn clean install -PHDP-2.5.0.0
Which branch or commit are you working from, and what command did you run
for the build? Be aware that Maven will ignore misspelled profile names and
instead use the default profile without warning.
On Mon, Jan 9, 2017 at 10:32 AM, Tyler Moore <tm...@goflyball.com> wrote:
> Seems like it's just the unit tests that are failing for pcap backend.
Was
> able to build successfully using new came with -DskipTests option along
> with the profile argument.
>
> Regards,
>
> Tyler Moore
> IT Specialist
> tyler.mathieu@yahoo.com
> 248-909-2769
>
> > On Jan 9, 2017, at 02:29, Michael Miklavcic <mi...@gmail.com>
> wrote:
> >
> > Hi Tyler, I don't recall seeing any failures the last time I ran this.
I
> > will take a look.
> >
> > On Jan 8, 2017 9:56 PM, "Tyler Moore" <tm...@goflyball.com> wrote:
> >
> > Michael,
> >
> > I am receiving error when trying to build with HPD profile, have you
had
> > any problems with pcap-backend tests failing when building with hdp
> profile?
> > Errors and log files provided below:
> >
> > Results :
> >
> > Failed tests:
> > PcapTopologyIntegrationTest.testTimestampInKey:152->testTopology:388->
> assertInOrder:542
> > null
> > PcapTopologyIntegrationTest.testTimestampInPacket:135->
> testTopology:388->assertInOrder:542
> > null
> >
> >
> >
> > Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
> >
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] Reactor Summary:
> > [INFO]
> > [INFO] Metron ............................................. SUCCESS [
> > 0.639 s]
> > [INFO] metron-analytics ................................... SUCCESS [
> > 0.029 s]
> > [INFO] metron-maas-common ................................. SUCCESS [
> > 8.771 s]
> > [INFO] metron-platform .................................... SUCCESS [
> > 0.057 s]
> > [INFO] metron-test-utilities .............................. SUCCESS [
> > 0.976 s]
> > [INFO] metron-integration-test ............................ SUCCESS [
> > 3.888 s]
> > [INFO] metron-maas-service ................................ SUCCESS
> [01:19
> > min]
> > [INFO] metron-common ...................................... SUCCESS [
> > 41.718 s]
> > [INFO] metron-statistics .................................. SUCCESS [
> > 30.713 s]
> > [INFO] metron-hbase ....................................... SUCCESS [
> > 37.286 s]
> > [INFO] metron-profiler-common ............................. SUCCESS [
> > 7.967 s]
> > [INFO] metron-profiler-client ............................. SUCCESS [
> > 54.088 s]
> > [INFO] metron-profiler .................................... SUCCESS
> [02:28
> > min]
> > [INFO] metron-writer ...................................... SUCCESS [
> > 7.151 s]
> > [INFO] metron-enrichment .................................. SUCCESS [
> > 57.076 s]
> > [INFO] metron-indexing .................................... SUCCESS [
> > 7.028 s]
> > [INFO] metron-solr ........................................ SUCCESS [
> > 36.695 s]
> > [INFO] metron-pcap ........................................ SUCCESS [
> > 1.123 s]
> > [INFO] metron-parsers ..................................... SUCCESS
> [01:58
> > min]
> > [INFO] metron-pcap-backend ................................ FAILURE [
> > 49.928 s]
> > [INFO] metron-data-management ............................. SKIPPED
> > [INFO] metron-api ......................................... SKIPPED
> > [INFO] metron-management .................................. SKIPPED
> > [INFO] elasticsearch-shaded ............................... SKIPPED
> > [INFO] metron-elasticsearch ............................... SKIPPED
> > [INFO] metron-deployment .................................. SKIPPED
> > [INFO] Metron Ambari Management Pack ...................... SKIPPED
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] BUILD FAILURE
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] Total time: 11:31 min
> > [INFO] Finished at: 2017-01-08T22:23:59-05:00
> > [INFO] Final Memory: 287M/6202M
> > [INFO] ------------------------------------------------------------
> > ------------
> > [ERROR] Failed to execute goal org.apache.maven.plugins:
> > maven-surefire-plugin:2.18:test (integration-tests) on project
> > metron-pcap-backend: There are test failures.
> >
> >
> > Regards,
> >
> > Tyler Moore
> > Software Engineer
> > Phone: 248-909-2769 <(248)%20909-2769>
> > Email: moore.tyler@goflyball.com
> >
> >
> > On Thu, Jan 5, 2017 at 5:21 PM, Michael Miklavcic <
> > michael.miklavcic@gmail.com> wrote:
> >
> >> Hey Tyler,
> >>
> >> Build Metron with the HDP profile to get the proper deps for this
> >> -PHDP-2.5.0.0
> >>
> >> Hopefully that works for you.
> >>
> >> Best,
> >> Mike
> >>
> >>
> >>> On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com>
> wrote:
> >>>
> >>> Hey all,
> >>>
> >>> Wondering if there is a solution to the "Offset lags for kafka not
> >>> supported for older versions. Please update kafka spout to latest
> >> version."
> >>> error seen when upgrading to metron 0.3.0?
> >>>
> >>> I know it is due to kafka-storm dependency that needs updated, but
what
> >> is
> >>> the best way to go about that? Is it as simple at changing the
version
> in
> >>> the metron pom.xml and provisioning?
> >>>
> >>> Regards,
> >>>
> >>> Tyler Moore
> >>> Software Engineer
> >>> Phone: 248-909-2769
> >>> Email: moore.tyler@goflyball.com
> >>>
> >>
>
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Michael Miklavcic <mi...@gmail.com>.
Hi Tyler, You shouldn't have to skip the unit tests as we ensure every
commit runs with 100% of tests passing. There are rare instances that this
is not true, but in general all tests should pass. I just pulled down
latest master and re-ran the following successfully.
mvn clean install -PHDP-2.5.0.0
Which branch or commit are you working from, and what command did you run
for the build? Be aware that Maven will ignore misspelled profile names and
instead use the default profile without warning.
On Mon, Jan 9, 2017 at 10:32 AM, Tyler Moore <tm...@goflyball.com> wrote:
> Seems like it's just the unit tests that are failing for pcap backend. Was
> able to build successfully using new came with -DskipTests option along
> with the profile argument.
>
> Regards,
>
> Tyler Moore
> IT Specialist
> tyler.mathieu@yahoo.com
> 248-909-2769
>
> > On Jan 9, 2017, at 02:29, Michael Miklavcic <mi...@gmail.com>
> wrote:
> >
> > Hi Tyler, I don't recall seeing any failures the last time I ran this. I
> > will take a look.
> >
> > On Jan 8, 2017 9:56 PM, "Tyler Moore" <tm...@goflyball.com> wrote:
> >
> > Michael,
> >
> > I am receiving error when trying to build with HPD profile, have you had
> > any problems with pcap-backend tests failing when building with hdp
> profile?
> > Errors and log files provided below:
> >
> > Results :
> >
> > Failed tests:
> > PcapTopologyIntegrationTest.testTimestampInKey:152->testTopology:388->
> assertInOrder:542
> > null
> > PcapTopologyIntegrationTest.testTimestampInPacket:135->
> testTopology:388->assertInOrder:542
> > null
> >
> >
> >
> > Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
> >
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] Reactor Summary:
> > [INFO]
> > [INFO] Metron ............................................. SUCCESS [
> > 0.639 s]
> > [INFO] metron-analytics ................................... SUCCESS [
> > 0.029 s]
> > [INFO] metron-maas-common ................................. SUCCESS [
> > 8.771 s]
> > [INFO] metron-platform .................................... SUCCESS [
> > 0.057 s]
> > [INFO] metron-test-utilities .............................. SUCCESS [
> > 0.976 s]
> > [INFO] metron-integration-test ............................ SUCCESS [
> > 3.888 s]
> > [INFO] metron-maas-service ................................ SUCCESS
> [01:19
> > min]
> > [INFO] metron-common ...................................... SUCCESS [
> > 41.718 s]
> > [INFO] metron-statistics .................................. SUCCESS [
> > 30.713 s]
> > [INFO] metron-hbase ....................................... SUCCESS [
> > 37.286 s]
> > [INFO] metron-profiler-common ............................. SUCCESS [
> > 7.967 s]
> > [INFO] metron-profiler-client ............................. SUCCESS [
> > 54.088 s]
> > [INFO] metron-profiler .................................... SUCCESS
> [02:28
> > min]
> > [INFO] metron-writer ...................................... SUCCESS [
> > 7.151 s]
> > [INFO] metron-enrichment .................................. SUCCESS [
> > 57.076 s]
> > [INFO] metron-indexing .................................... SUCCESS [
> > 7.028 s]
> > [INFO] metron-solr ........................................ SUCCESS [
> > 36.695 s]
> > [INFO] metron-pcap ........................................ SUCCESS [
> > 1.123 s]
> > [INFO] metron-parsers ..................................... SUCCESS
> [01:58
> > min]
> > [INFO] metron-pcap-backend ................................ FAILURE [
> > 49.928 s]
> > [INFO] metron-data-management ............................. SKIPPED
> > [INFO] metron-api ......................................... SKIPPED
> > [INFO] metron-management .................................. SKIPPED
> > [INFO] elasticsearch-shaded ............................... SKIPPED
> > [INFO] metron-elasticsearch ............................... SKIPPED
> > [INFO] metron-deployment .................................. SKIPPED
> > [INFO] Metron Ambari Management Pack ...................... SKIPPED
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] BUILD FAILURE
> > [INFO] ------------------------------------------------------------
> > ------------
> > [INFO] Total time: 11:31 min
> > [INFO] Finished at: 2017-01-08T22:23:59-05:00
> > [INFO] Final Memory: 287M/6202M
> > [INFO] ------------------------------------------------------------
> > ------------
> > [ERROR] Failed to execute goal org.apache.maven.plugins:
> > maven-surefire-plugin:2.18:test (integration-tests) on project
> > metron-pcap-backend: There are test failures.
> >
> >
> > Regards,
> >
> > Tyler Moore
> > Software Engineer
> > Phone: 248-909-2769 <(248)%20909-2769>
> > Email: moore.tyler@goflyball.com
> >
> >
> > On Thu, Jan 5, 2017 at 5:21 PM, Michael Miklavcic <
> > michael.miklavcic@gmail.com> wrote:
> >
> >> Hey Tyler,
> >>
> >> Build Metron with the HDP profile to get the proper deps for this
> >> -PHDP-2.5.0.0
> >>
> >> Hopefully that works for you.
> >>
> >> Best,
> >> Mike
> >>
> >>
> >>> On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com>
> wrote:
> >>>
> >>> Hey all,
> >>>
> >>> Wondering if there is a solution to the "Offset lags for kafka not
> >>> supported for older versions. Please update kafka spout to latest
> >> version."
> >>> error seen when upgrading to metron 0.3.0?
> >>>
> >>> I know it is due to kafka-storm dependency that needs updated, but what
> >> is
> >>> the best way to go about that? Is it as simple at changing the version
> in
> >>> the metron pom.xml and provisioning?
> >>>
> >>> Regards,
> >>>
> >>> Tyler Moore
> >>> Software Engineer
> >>> Phone: 248-909-2769
> >>> Email: moore.tyler@goflyball.com
> >>>
> >>
>
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Tyler Moore <tm...@goflyball.com>.
Seems like it's just the unit tests that are failing for pcap backend. Was able to build successfully using new came with -DskipTests option along with the profile argument.
Regards,
Tyler Moore
IT Specialist
tyler.mathieu@yahoo.com
248-909-2769
> On Jan 9, 2017, at 02:29, Michael Miklavcic <mi...@gmail.com> wrote:
>
> Hi Tyler, I don't recall seeing any failures the last time I ran this. I
> will take a look.
>
> On Jan 8, 2017 9:56 PM, "Tyler Moore" <tm...@goflyball.com> wrote:
>
> Michael,
>
> I am receiving error when trying to build with HPD profile, have you had
> any problems with pcap-backend tests failing when building with hdp profile?
> Errors and log files provided below:
>
> Results :
>
> Failed tests:
> PcapTopologyIntegrationTest.testTimestampInKey:152->testTopology:388->assertInOrder:542
> null
> PcapTopologyIntegrationTest.testTimestampInPacket:135->testTopology:388->assertInOrder:542
> null
>
>
>
> Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
>
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Metron ............................................. SUCCESS [
> 0.639 s]
> [INFO] metron-analytics ................................... SUCCESS [
> 0.029 s]
> [INFO] metron-maas-common ................................. SUCCESS [
> 8.771 s]
> [INFO] metron-platform .................................... SUCCESS [
> 0.057 s]
> [INFO] metron-test-utilities .............................. SUCCESS [
> 0.976 s]
> [INFO] metron-integration-test ............................ SUCCESS [
> 3.888 s]
> [INFO] metron-maas-service ................................ SUCCESS [01:19
> min]
> [INFO] metron-common ...................................... SUCCESS [
> 41.718 s]
> [INFO] metron-statistics .................................. SUCCESS [
> 30.713 s]
> [INFO] metron-hbase ....................................... SUCCESS [
> 37.286 s]
> [INFO] metron-profiler-common ............................. SUCCESS [
> 7.967 s]
> [INFO] metron-profiler-client ............................. SUCCESS [
> 54.088 s]
> [INFO] metron-profiler .................................... SUCCESS [02:28
> min]
> [INFO] metron-writer ...................................... SUCCESS [
> 7.151 s]
> [INFO] metron-enrichment .................................. SUCCESS [
> 57.076 s]
> [INFO] metron-indexing .................................... SUCCESS [
> 7.028 s]
> [INFO] metron-solr ........................................ SUCCESS [
> 36.695 s]
> [INFO] metron-pcap ........................................ SUCCESS [
> 1.123 s]
> [INFO] metron-parsers ..................................... SUCCESS [01:58
> min]
> [INFO] metron-pcap-backend ................................ FAILURE [
> 49.928 s]
> [INFO] metron-data-management ............................. SKIPPED
> [INFO] metron-api ......................................... SKIPPED
> [INFO] metron-management .................................. SKIPPED
> [INFO] elasticsearch-shaded ............................... SKIPPED
> [INFO] metron-elasticsearch ............................... SKIPPED
> [INFO] metron-deployment .................................. SKIPPED
> [INFO] Metron Ambari Management Pack ...................... SKIPPED
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Total time: 11:31 min
> [INFO] Finished at: 2017-01-08T22:23:59-05:00
> [INFO] Final Memory: 287M/6202M
> [INFO] ------------------------------------------------------------
> ------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:
> maven-surefire-plugin:2.18:test (integration-tests) on project
> metron-pcap-backend: There are test failures.
>
>
> Regards,
>
> Tyler Moore
> Software Engineer
> Phone: 248-909-2769 <(248)%20909-2769>
> Email: moore.tyler@goflyball.com
>
>
> On Thu, Jan 5, 2017 at 5:21 PM, Michael Miklavcic <
> michael.miklavcic@gmail.com> wrote:
>
>> Hey Tyler,
>>
>> Build Metron with the HDP profile to get the proper deps for this
>> -PHDP-2.5.0.0
>>
>> Hopefully that works for you.
>>
>> Best,
>> Mike
>>
>>
>>> On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com> wrote:
>>>
>>> Hey all,
>>>
>>> Wondering if there is a solution to the "Offset lags for kafka not
>>> supported for older versions. Please update kafka spout to latest
>> version."
>>> error seen when upgrading to metron 0.3.0?
>>>
>>> I know it is due to kafka-storm dependency that needs updated, but what
>> is
>>> the best way to go about that? Is it as simple at changing the version in
>>> the metron pom.xml and provisioning?
>>>
>>> Regards,
>>>
>>> Tyler Moore
>>> Software Engineer
>>> Phone: 248-909-2769
>>> Email: moore.tyler@goflyball.com
>>>
>>
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Michael Miklavcic <mi...@gmail.com>.
Hi Tyler, I don't recall seeing any failures the last time I ran this. I
will take a look.
On Jan 8, 2017 9:56 PM, "Tyler Moore" <tm...@goflyball.com> wrote:
Michael,
I am receiving error when trying to build with HPD profile, have you had
any problems with pcap-backend tests failing when building with hdp profile?
Errors and log files provided below:
Results :
Failed tests:
PcapTopologyIntegrationTest.testTimestampInKey:152->testTopology:388->assertInOrder:542
null
PcapTopologyIntegrationTest.testTimestampInPacket:135->testTopology:388->assertInOrder:542
null
Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
[INFO] ------------------------------------------------------------
------------
[INFO] Reactor Summary:
[INFO]
[INFO] Metron ............................................. SUCCESS [
0.639 s]
[INFO] metron-analytics ................................... SUCCESS [
0.029 s]
[INFO] metron-maas-common ................................. SUCCESS [
8.771 s]
[INFO] metron-platform .................................... SUCCESS [
0.057 s]
[INFO] metron-test-utilities .............................. SUCCESS [
0.976 s]
[INFO] metron-integration-test ............................ SUCCESS [
3.888 s]
[INFO] metron-maas-service ................................ SUCCESS [01:19
min]
[INFO] metron-common ...................................... SUCCESS [
41.718 s]
[INFO] metron-statistics .................................. SUCCESS [
30.713 s]
[INFO] metron-hbase ....................................... SUCCESS [
37.286 s]
[INFO] metron-profiler-common ............................. SUCCESS [
7.967 s]
[INFO] metron-profiler-client ............................. SUCCESS [
54.088 s]
[INFO] metron-profiler .................................... SUCCESS [02:28
min]
[INFO] metron-writer ...................................... SUCCESS [
7.151 s]
[INFO] metron-enrichment .................................. SUCCESS [
57.076 s]
[INFO] metron-indexing .................................... SUCCESS [
7.028 s]
[INFO] metron-solr ........................................ SUCCESS [
36.695 s]
[INFO] metron-pcap ........................................ SUCCESS [
1.123 s]
[INFO] metron-parsers ..................................... SUCCESS [01:58
min]
[INFO] metron-pcap-backend ................................ FAILURE [
49.928 s]
[INFO] metron-data-management ............................. SKIPPED
[INFO] metron-api ......................................... SKIPPED
[INFO] metron-management .................................. SKIPPED
[INFO] elasticsearch-shaded ............................... SKIPPED
[INFO] metron-elasticsearch ............................... SKIPPED
[INFO] metron-deployment .................................. SKIPPED
[INFO] Metron Ambari Management Pack ...................... SKIPPED
[INFO] ------------------------------------------------------------
------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------
------------
[INFO] Total time: 11:31 min
[INFO] Finished at: 2017-01-08T22:23:59-05:00
[INFO] Final Memory: 287M/6202M
[INFO] ------------------------------------------------------------
------------
[ERROR] Failed to execute goal org.apache.maven.plugins:
maven-surefire-plugin:2.18:test (integration-tests) on project
metron-pcap-backend: There are test failures.
Regards,
Tyler Moore
Software Engineer
Phone: 248-909-2769 <(248)%20909-2769>
Email: moore.tyler@goflyball.com
On Thu, Jan 5, 2017 at 5:21 PM, Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Hey Tyler,
>
> Build Metron with the HDP profile to get the proper deps for this
> -PHDP-2.5.0.0
>
> Hopefully that works for you.
>
> Best,
> Mike
>
>
> On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com> wrote:
>
> > Hey all,
> >
> > Wondering if there is a solution to the "Offset lags for kafka not
> > supported for older versions. Please update kafka spout to latest
> version."
> > error seen when upgrading to metron 0.3.0?
> >
> > I know it is due to kafka-storm dependency that needs updated, but what
> is
> > the best way to go about that? Is it as simple at changing the version in
> > the metron pom.xml and provisioning?
> >
> > Regards,
> >
> > Tyler Moore
> > Software Engineer
> > Phone: 248-909-2769
> > Email: moore.tyler@goflyball.com
> >
>
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Tyler Moore <tm...@goflyball.com>.
Michael,
I am receiving error when trying to build with HPD profile, have you had
any problems with pcap-backend tests failing when building with hdp profile?
Errors and log files provided below:
Results :
Failed tests:
PcapTopologyIntegrationTest.testTimestampInKey:152->testTopology:388->assertInOrder:542
null
PcapTopologyIntegrationTest.testTimestampInPacket:135->testTopology:388->assertInOrder:542
null
Tests run: 2, Failures: 2, Errors: 0, Skipped: 0
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Metron ............................................. SUCCESS [
0.639 s]
[INFO] metron-analytics ................................... SUCCESS [
0.029 s]
[INFO] metron-maas-common ................................. SUCCESS [
8.771 s]
[INFO] metron-platform .................................... SUCCESS [
0.057 s]
[INFO] metron-test-utilities .............................. SUCCESS [
0.976 s]
[INFO] metron-integration-test ............................ SUCCESS [
3.888 s]
[INFO] metron-maas-service ................................ SUCCESS [01:19
min]
[INFO] metron-common ...................................... SUCCESS [
41.718 s]
[INFO] metron-statistics .................................. SUCCESS [
30.713 s]
[INFO] metron-hbase ....................................... SUCCESS [
37.286 s]
[INFO] metron-profiler-common ............................. SUCCESS [
7.967 s]
[INFO] metron-profiler-client ............................. SUCCESS [
54.088 s]
[INFO] metron-profiler .................................... SUCCESS [02:28
min]
[INFO] metron-writer ...................................... SUCCESS [
7.151 s]
[INFO] metron-enrichment .................................. SUCCESS [
57.076 s]
[INFO] metron-indexing .................................... SUCCESS [
7.028 s]
[INFO] metron-solr ........................................ SUCCESS [
36.695 s]
[INFO] metron-pcap ........................................ SUCCESS [
1.123 s]
[INFO] metron-parsers ..................................... SUCCESS [01:58
min]
[INFO] metron-pcap-backend ................................ FAILURE [
49.928 s]
[INFO] metron-data-management ............................. SKIPPED
[INFO] metron-api ......................................... SKIPPED
[INFO] metron-management .................................. SKIPPED
[INFO] elasticsearch-shaded ............................... SKIPPED
[INFO] metron-elasticsearch ............................... SKIPPED
[INFO] metron-deployment .................................. SKIPPED
[INFO] Metron Ambari Management Pack ...................... SKIPPED
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 11:31 min
[INFO] Finished at: 2017-01-08T22:23:59-05:00
[INFO] Final Memory: 287M/6202M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.18:test
(integration-tests) on project metron-pcap-backend: There are test failures.
Regards,
Tyler Moore
Software Engineer
Phone: 248-909-2769
Email: moore.tyler@goflyball.com
On Thu, Jan 5, 2017 at 5:21 PM, Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Hey Tyler,
>
> Build Metron with the HDP profile to get the proper deps for this
> -PHDP-2.5.0.0
>
> Hopefully that works for you.
>
> Best,
> Mike
>
>
> On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com> wrote:
>
> > Hey all,
> >
> > Wondering if there is a solution to the "Offset lags for kafka not
> > supported for older versions. Please update kafka spout to latest
> version."
> > error seen when upgrading to metron 0.3.0?
> >
> > I know it is due to kafka-storm dependency that needs updated, but what
> is
> > the best way to go about that? Is it as simple at changing the version in
> > the metron pom.xml and provisioning?
> >
> > Regards,
> >
> > Tyler Moore
> > Software Engineer
> > Phone: 248-909-2769
> > Email: moore.tyler@goflyball.com
> >
>
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Tyler Moore <tm...@goflyball.com>.
I'll give it a shot thx!
Regards,
Tyler Moore
Software Engineer
Phone: 248-909-2769
Email: moore.tyler@goflyball.com
On Thu, Jan 5, 2017 at 5:21 PM, Michael Miklavcic <
michael.miklavcic@gmail.com> wrote:
> Hey Tyler,
>
> Build Metron with the HDP profile to get the proper deps for this
> -PHDP-2.5.0.0
>
> Hopefully that works for you.
>
> Best,
> Mike
>
>
> On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com> wrote:
>
> > Hey all,
> >
> > Wondering if there is a solution to the "Offset lags for kafka not
> > supported for older versions. Please update kafka spout to latest
> version."
> > error seen when upgrading to metron 0.3.0?
> >
> > I know it is due to kafka-storm dependency that needs updated, but what
> is
> > the best way to go about that? Is it as simple at changing the version in
> > the metron pom.xml and provisioning?
> >
> > Regards,
> >
> > Tyler Moore
> > Software Engineer
> > Phone: 248-909-2769
> > Email: moore.tyler@goflyball.com
> >
>
Re: Kafka error when upgrading to Metron 0.3.0
Posted by Michael Miklavcic <mi...@gmail.com>.
Hey Tyler,
Build Metron with the HDP profile to get the proper deps for this
-PHDP-2.5.0.0
Hopefully that works for you.
Best,
Mike
On Thu, Jan 5, 2017 at 3:17 PM, Tyler Moore <tm...@goflyball.com> wrote:
> Hey all,
>
> Wondering if there is a solution to the "Offset lags for kafka not
> supported for older versions. Please update kafka spout to latest version."
> error seen when upgrading to metron 0.3.0?
>
> I know it is due to kafka-storm dependency that needs updated, but what is
> the best way to go about that? Is it as simple at changing the version in
> the metron pom.xml and provisioning?
>
> Regards,
>
> Tyler Moore
> Software Engineer
> Phone: 248-909-2769
> Email: moore.tyler@goflyball.com
>