You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Lewis John McGibbney (JIRA)" <ji...@apache.org> on 2017/03/08 06:55:38 UTC
[jira] [Commented] (NUTCH-2292) Mavenize the build for nutch-core
and nutch-plugins
[ https://issues.apache.org/jira/browse/NUTCH-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15900814#comment-15900814 ]
Lewis John McGibbney commented on NUTCH-2292:
---------------------------------------------
Hi [~markus17] and [~thammegowda] I've just pushed code to https://github.com/apache/nutch/tree/NUTCH-2292 which gets us a step closer here. Right now our tests fail as follows
{code}
Running org.apache.nutch.util.TestURLUtil
2017-03-07 22:46:58,119 WARN domain.DomainSuffixes (DomainSuffixes.java:<init>(50)) - java.net.MalformedURLException
at java.net.URL.<init>(URL.java:627)
at java.net.URL.<init>(URL.java:490)
at java.net.URL.<init>(URL.java:439)
at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(Unknown Source)
at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at org.apache.nutch.util.domain.DomainSuffixesReader.read(DomainSuffixesReader.java:56)
at org.apache.nutch.util.domain.DomainSuffixes.<init>(DomainSuffixes.java:48)
at org.apache.nutch.util.domain.DomainSuffixes.getInstance(DomainSuffixes.java:61)
at org.apache.nutch.util.URLUtil.getDomainSuffix(URLUtil.java:202)
at org.apache.nutch.util.TestURLUtil.testGetDomainSuffix(TestURLUtil.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.junit.runners.Suite.runChild(Suite.java:127)
at org.junit.runners.Suite.runChild(Suite.java:26)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:161)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
Caused by: java.lang.NullPointerException
at java.net.URL.<init>(URL.java:532)
... 48 more
Tests run: 8, Failures: 2, Errors: 1, Skipped: 0, Time elapsed: 0 sec <<< FAILURE! - in org.apache.nutch.util.TestURLUtil
testGetDomainSuffix(org.apache.nutch.util.TestURLUtil) Time elapsed: 0 sec <<< ERROR!
java.lang.NullPointerException
at org.apache.nutch.util.TestURLUtil.testGetDomainSuffix(TestURLUtil.java:82)
testGetDomainName(org.apache.nutch.util.TestURLUtil) Time elapsed: 0 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<[apache.]org> but was:<[]org>
at org.apache.nutch.util.TestURLUtil.testGetDomainName(TestURLUtil.java:34)
testChooseRepr(org.apache.nutch.util.TestURLUtil) Time elapsed: 0 sec <<< FAILURE!
org.junit.ComparisonFailure: expected:<http://www.[b].com> but was:<http://www.[a].com>
at org.apache.nutch.util.TestURLUtil.testChooseRepr(TestURLUtil.java:173)
Results :
Failed tests:
TestSegmentMerger.testLargeMerge:106 expected:<1> but was:<2>
TestSegmentMergerCrawlDatums.testEndsWithRedirect:216->merge:345 expected:<1> but was:<2>
TestSegmentMergerCrawlDatums.testFixedSequence:173->merge:345 expected:<1> but was:<2>
TestSegmentMergerCrawlDatums.testMostlyRedirects:110->merge:345 expected:<1> but was:<2>
TestSegmentMergerCrawlDatums.testRandomTestSequenceWithRedirects:151->executeSequence:276->merge:345 expected:<1> but was:<2>
TestSegmentMergerCrawlDatums.testRandomizedSequences:133->executeSequence:276->merge:345 expected:<1> but was:<2>
TestSegmentMergerCrawlDatums.testRedirFetchInOneSegment:194->merge:345 expected:<1> but was:<2>
TestSegmentMergerCrawlDatums.testSingleRandomSequence:78->executeSequence:276->merge:345 expected:<1> but was:<2>
TestURLUtil.testChooseRepr:173 expected:<http://www.[b].com> but was:<http://www.[a].com>
TestURLUtil.testGetDomainName:34 expected:<[apache.]org> but was:<[]org>
Tests in error:
TestURLUtil.testGetDomainSuffix:82 NullPointer
Tests run: 69, Failures: 10, Errors: 1, Skipped: 0
{code}
> Mavenize the build for nutch-core and nutch-plugins
> ---------------------------------------------------
>
> Key: NUTCH-2292
> URL: https://issues.apache.org/jira/browse/NUTCH-2292
> Project: Nutch
> Issue Type: Improvement
> Components: build
> Reporter: Thamme Gowda
>
> Convert the build system of nutch-core as well as plugins to Apache Maven.
> *Plan :*
> Create multi-module maven project with the following structure
> {code}
> nutch-parent
> |-- pom.xml (POM)
> |-- nutch-core
> | |-- pom.xml (JAR)
> | |--src : sources
> |-- nutch-plugins
> |-- pom.xml (POM)
> |-- plugin1
> | |-- pom.xml (JAR)
> | .....
> |-- pluginN
> |-- pom.xml (JAR)
> {code}
> NOTE: watch out for cyclic dependencies bwteen nutch-core and plugins, introduce another POM to break the cycle if required.
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)