You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Lewis John McGibbney (Created) (JIRA)" <ji...@apache.org> on 2011/11/19 13:06:51 UTC
[jira] [Created] (NUTCH-1205) Upgrade gora modules to 0.2-SNAPSHOT
Upgrade gora modules to 0.2-SNAPSHOT
------------------------------------
Key: NUTCH-1205
URL: https://issues.apache.org/jira/browse/NUTCH-1205
Project: Nutch
Issue Type: Improvement
Components: storage
Affects Versions: nutchgora
Reporter: Lewis John McGibbney
Assignee: Lewis John McGibbney
Priority: Minor
Fix For: nutchgora
Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT
Posted by "Julien Nioche (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156645#comment-13156645 ]
Julien Nioche commented on NUTCH-1205:
--------------------------------------
Why upgrading the sub-dependencies such as commons-lang explicitely? Aren't they pulled transitively along with GORA?
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263596#comment-13263596 ]
Ferdy Galema commented on NUTCH-1205:
-------------------------------------
(Also I reformatted the ivy.xml to only include spaces as indentation. This is the policy right? If so, could anyone editing xml files double check their editor settings.)
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205-v7.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema resolved NUTCH-1205.
---------------------------------
Resolution: Fixed
Attached new patch v11. Committed.
-Fixed the jdom issue. (Added test dep again).
-Added a single global exclusion for hsqldb. (The deps can have the exclusion removed).
-Tests succeed.
-Build a sqlstore runtime and played around doing some local crawls succesfully.
I did not test a deployment for the other stores. (When there is something wrong with one of them dependency-wise, we can always create new issues).
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v11.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to
0.2-incubating in ivy/ivy.xml
Posted by "Ferdy Galema (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: (was: NUTCH-1205-v3.patch)
> Upgrade gora modules to 0.2-incubating in ivy/ivy.xml
> -----------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v10.patch
The tests now work and TestGoraStorage uses a proper standalone database. (Integrating issue NUTCH-902). I think it's good to do a final check of what's to be included as dependencies in the ivy.xml.
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v8.patch
Oops there still was a failure in a test later on. (TestProtocolHttpClient). This was because of multiple ant jars. I noticed that this was caused by removing the global exclude but adding excludes to hadoop deps. (This was not sufficient obviously).
New version of patch succeeds all tests.
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205-v7.patch, NUTCH-1205-v8.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2-SNAPSHOT
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Attachment: NUTCH-1205-v2.patch
new patch which acknowledges Juliens comments regarding other dependencies. This patch ONLY upgrades the gora-module dependencies to 0.2-incubating
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on NUTCH-1205 started by Lewis John McGibbney.
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to
0.2-incubating in ivy/ivy.xml
Posted by "Ferdy Galema (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v3.patch
Reattaching because of "grant license to ASF"..
> Upgrade gora modules to 0.2-incubating in ivy/ivy.xml
> -----------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT in ivy/ivy.xml
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212031#comment-13212031 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
One last thing, I've unchecked all of the gora dependencies, not as this will be default configuration, but because we need to be 100% certain that we can gather them all, that everything compiles, tests and functions properly. Currently with this patch in place this doesn't seem to be the case.
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (NUTCH-1205) Upgrade gora modules to
0.2 in ivy/ivy.xml
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263196#comment-13263196 ]
Lewis John McGibbney edited comment on NUTCH-1205 at 4/26/12 10:15 PM:
-----------------------------------------------------------------------
Hi Ferdy. If you would be so good to look at the patch. If you also observe the way in which we do the following in AbstractNutchTest
{code}
52 DataStoreFactory.properties.setProperty("gora.sqlstore.jdbc.password","");
53 webPageStore = StorageUtils.createWebStore(conf, String.class,
54 WebPage.class);
{code}
you will see that we are not passing the correct parameters to StorageUtils.createWebStore. This makes all tests that rely on AbstractNutchTest fail misserably complaining about connection refused errors. (I'm hoping that this is correct, because if not then I am pretty much stumped.)
was (Author: lewismc):
Hi Ferdy. If you would be so good to look at the patch. If you also observe the way in which we do the following
{code} 52 DataStoreFactory.properties.setProperty("gora.sqlstore.jdbc.password","");
53 webPageStore = StorageUtils.createWebStore(conf, String.class,
54 WebPage.class);
{code}
you will see that were not passing the correct parameters. This makes all tests that rely on AbstractNutchTest fail misserably complaining about connection refused errors. (I'm hoping that this is correct, because if not then I am pretty much stumped.)
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156723#comment-13156723 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
Yeah they probably are Julien. How would I find this out for sure?
Also did you have a look at the errors? I've not the time to do this today but I'll try to get on to it at a later stage in the week. Thanks
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2-SNAPSHOT
in ivy/ivy.xml
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Attachment: NUTCH-1205-v5.patch
NUTCH-1205-v5.patch
This is getting laughable now. I've overcome god knows how many problems, but now there is a problem with the actual gora-core-0.2-SNAPSHOT jar which we pull from the nexus snapshot repository. For some reason we are pulling the test jar and not the functional one!!!
When applying the patch to Nutchgora and running
{code}
$ant compile > compile.txt
{code}
I get
{code}
compile-core:
[javac] /home/lewis/ASF/nutchgora/build.xml:97: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 170 source files to /home/lewis/ASF/nutchgora/build/classes
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/apache-cassandra-clientutil-1.0.2.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/apache-cassandra-thrift-1.0.2.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/bcel.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/dom4j-full.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/findbugs.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/plastic.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/jaxb-api.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/activation.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/jsr173_1.0_api.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/jaxb1-impl.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/xercesImpl.jar": no such file or directory
[javac] warning: [path] bad path element "/home/lewis/ASF/nutchgora/build/lib/xml-apis.jar": no such file or directory
{code}
When you look in your /build directory and open the gora-core-0.2-SNAPSHOT.jar you'll see that it's the test jar right enough.
For tonight I'm giving it a by, but will try and pick this up tomorrow at some stage.
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2-SNAPSHOT
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Attachment: NUTCH-1205.patch
This patch is breaking by build but it is a work in progress. In short the patch,
1) Upgrades all gora dependencies to 0.2-incubating
2) Upgrades all other dependencies which were also upgraded in current gora trunk.
The build fails as follows
{code}
compile-core:
[javac] /home/lewis/ASF/nutchgora/build.xml:97: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac] Compiling 171 source files to /home/lewis/ASF/nutchgora/build/classes
[javac] /home/lewis/ASF/nutchgora/src/java/org/apache/nutch/storage/StorageUtils.java:43: cannot find symbol
[javac] symbol : method createDataStore(java.lang.Class<capture#763 of ? extends org.apache.gora.store.DataStore<K,V>>,java.lang.Class<K>,java.lang.Class<V>)
[javac] location: class org.apache.gora.store.DataStoreFactory
[javac] return DataStoreFactory.createDataStore(dataStoreClass,
[javac] ^
[javac] /home/lewis/ASF/nutchgora/src/java/org/apache/nutch/storage/StorageUtils.java:59: cannot find symbol
[javac] symbol : method createDataStore(java.lang.Class<capture#496 of ? extends org.apache.gora.store.DataStore<K,V>>,java.lang.Class<K>,java.lang.Class<V>,java.lang.String)
[javac] location: class org.apache.gora.store.DataStoreFactory
[javac] return DataStoreFactory.createDataStore(dataStoreClass,
[javac] ^
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 2 errors
BUILD FAILED
{code}
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267581#comment-13267581 ]
Ferdy Galema commented on NUTCH-1205:
-------------------------------------
I committed a minor addition, that fixes the maven-plugins error when uncommenting another store.
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v11.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT in ivy/ivy.xml
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206933#comment-13206933 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
OK when I apply the patch, I'm seeing
{code}
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve] [FAILED ] maven-plugins#maven-cobertura-plugin;1.3!maven-cobertura-plugin.plugin: (0ms)
[ivy:resolve] ==== local: tried
[ivy:resolve] /home/lewis/.ivy2/local/maven-plugins/maven-cobertura-plugin/1.3/plugins/maven-cobertura-plugin.plugin
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/maven-plugins/maven-cobertura-plugin/1.3/maven-cobertura-plugin-1.3.plugin
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] http://repository.apache.org/content/groups/snapshots-group/maven-plugins/maven-cobertura-plugin/1.3/maven-cobertura-plugin-1.3.plugin
[ivy:resolve] [FAILED ] maven-plugins#maven-findbugs-plugin;1.3.1!maven-findbugs-plugin.plugin: (0ms)
[ivy:resolve] ==== local: tried
[ivy:resolve] /home/lewis/.ivy2/local/maven-plugins/maven-findbugs-plugin/1.3.1/plugins/maven-findbugs-plugin.plugin
[ivy:resolve] ==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/maven-plugins/maven-findbugs-plugin/1.3.1/maven-findbugs-plugin-1.3.1.plugin
[ivy:resolve] ==== apache-snapshot: tried
[ivy:resolve] http://repository.apache.org/content/groups/snapshots-group/maven-plugins/maven-findbugs-plugin/1.3.1/maven-findbugs-plugin-1.3.1.plugin
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: FAILED DOWNLOADS ::
[ivy:resolve] :: ^ see resolution messages for details ^ ::
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve] :: maven-plugins#maven-cobertura-plugin;1.3!maven-cobertura-plugin.plugin
[ivy:resolve] :: maven-plugins#maven-findbugs-plugin;1.3.1!maven-findbugs-plugin.plugin
[ivy:resolve] ::::::::::::::::::::::::::::::::::::::::::::::
{code}
There is a really weird extension for the plugins e.g.
{code}
maven-cobertura-plugin.plugin
{code}
I've tried excluding these as both individual exclusions for the Gora atrifacts and as global exclusions for maven-plugins but noneof this works. Been doing some reading on ivysettings on the ant/ivy website but there is a bit fo documentation so it's taking a while.
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2-SNAPSHOT
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Patch Info: Patch Available
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v9.patch
NUTCH-1205-v9.patch
This is getting hilarious. The tests are working but TestGoraStorage multiproces actually just connects to local mem sql stores.
So, to recap the current state of affairs:
The tests extending AbstractNutchText work validly. (Because they simply need a mem sql store). TestGoraStorage works but actually does not test multi-process. Somehow setting the sql url properties does not seem to reflect on the sql store.
Any suggestions?
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205-v7.patch, NUTCH-1205-v8.patch, NUTCH-1205-v9.patch, NUTCH-1205-v9.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155898#comment-13155898 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
This can't be progressed with until we get the Gora 0.2-SNAPSHOT's loaded to
http://repo1.maven.org/maven2/org/apache/gora/
I'll work on this
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: (was: NUTCH-1205-v7.patch)
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Attachment: NUTCH-1205-v6.patch
Hi Ferdy. If you would be so good to look at the patch. If you also observe the way in which we do the following
{code} 52 DataStoreFactory.properties.setProperty("gora.sqlstore.jdbc.password","");
53 webPageStore = StorageUtils.createWebStore(conf, String.class,
54 WebPage.class);
{code}
you will see that were not passing the correct parameters. This makes all tests that rely on AbstractNutchTest fail misserably complaining about connection refused errors. (I'm hoping that this is correct, because if not then I am pretty much stumped.)
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v11-addition.patch
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v11-addition.patch, NUTCH-1205-v11.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2-SNAPSHOT
in ivy/ivy.xml
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Priority: Blocker (was: Minor)
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: (was: NUTCH-1205-v9.patch)
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: (was: NUTCH-1205-v8.patch)
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT in ivy/ivy.xml
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206961#comment-13206961 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
Yeah it's another kettle of fish altogether. I'll get on it and hopefully get it sorted out. I'll ensure that the final patch includes the hsqldb upgrage as well Ferdy. Thanks for now.
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13268112#comment-13268112 ]
Hudson commented on NUTCH-1205:
-------------------------------
Integrated in Nutch-nutchgora #244 (See [https://builds.apache.org/job/Nutch-nutchgora/244/])
NUTCH-1205 Upgrade gora modules to 0.2 in ivy/ivy.xml (addition) (Revision 1333551)
NUTCH-1205 Upgrade gora modules to 0.2 in ivy/ivy.xml (addition) (Revision 1333547)
NUTCH-1205 Upgrade gora modules to 0.2 in ivy/ivy.xml (Revision 1333435)
Result = SUCCESS
ferdy :
Files :
* /nutch/branches/nutchgora/ivy/ivy.xml
ferdy :
Files :
* /nutch/branches/nutchgora/ivy/ivy.xml
ferdy :
Files :
* /nutch/branches/nutchgora/CHANGES.txt
* /nutch/branches/nutchgora/build.xml
* /nutch/branches/nutchgora/conf/gora.properties
* /nutch/branches/nutchgora/ivy/ivy.xml
* /nutch/branches/nutchgora/src/java/org/apache/nutch/storage/StorageUtils.java
* /nutch/branches/nutchgora/src/test/gora.properties
* /nutch/branches/nutchgora/src/test/org/apache/nutch/storage/TestGoraStorage.java
* /nutch/branches/nutchgora/src/test/org/apache/nutch/util/AbstractNutchTest.java
* /nutch/branches/nutchgora/src/testprocess
* /nutch/branches/nutchgora/src/testprocess/gora.properties
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v11-addition.patch, NUTCH-1205-v11.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Summary: Upgrade gora modules to 0.2 in ivy/ivy.xml (was: Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml)
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT in ivy/ivy.xml
Posted by "Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206929#comment-13206929 ]
Ferdy Galema commented on NUTCH-1205:
-------------------------------------
Could you please elaborate on what exactly the problems are and how to reproduce them?
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v11.patch
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v11.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT in ivy/ivy.xml
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206939#comment-13206939 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
To add to this, I can confirm that we are pulling the most up to date maven artifacts from the apache snapshots repository, so at least we are using bleeding edge here.
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-incubating in ivy/ivy.xml
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13176224#comment-13176224 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
for reference to fix the above problems
http://stackoverflow.com/questions/197986/what-causes-javac-to-issue-the-uses-unchecked-or-unsafe-operations-warning
> Upgrade gora modules to 0.2-incubating in ivy/ivy.xml
> -----------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266492#comment-13266492 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
Hi Ferdy, I am very happy with this. It's been a bit of a sticky one as I'm sure you'll agree. Anyway, one comment I have is that, I removed the jdom deps because I thought somewhere down the line a transitive dependency was mucking things up. As we now know this is not the case, I notice that you added one of the jdom dependencies back in. If you look at the current ivy.xml file in our svn area, you'll see the presence of two jdom dependencies, one appears to be for tests and the other I assume for runtime configuration.
I've tested this patch against our test suite e.g. ant test, and confirm that all tests pass successfully. I've also run the benchmark but will comment about this elsewhere where it is relevant.
If we can clear up the jdom discrepancy I note above then I am +1 on this one. Great work Ferdy.
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2-SNAPSHOT
in ivy/ivy.xml
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Summary: Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml (was: Upgrade gora modules to 0.2-incubating in ivy/ivy.xml)
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to
0.2-incubating in ivy/ivy.xml
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Summary: Upgrade gora modules to 0.2-incubating in ivy/ivy.xml (was: Upgrade gora modules to 0.2-SNAPSHOT)
> Upgrade gora modules to 0.2-incubating in ivy/ivy.xml
> -----------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT
Posted by "Chris A. Mattmann (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13155926#comment-13155926 ]
Chris A. Mattmann commented on NUTCH-1205:
------------------------------------------
See my comment on the gora-dev list. I don't think this is explicitly needed or desired. SNAPSHOTS should go to http://repository.apache.org and then referenced according to the "How do I do this" section of:
http://maven.apache.org/guides/development/guide-testing-development-plugins.html
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Ferdy Galema (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v7.patch
OK I got the tests working now. The problem is the fact that properties object are not correctly handled throughout the tests. This is a problem currently in Gora. In short, it means that Properties are not properly loaded in GoraMapper from dynamic properties but ALWAYS from static gora.properties. (Will shortly open issue for that). A consequence to make tests work now with Gora 0.2 is that the default gora.properties now uses a hsqldb memstore instead of a standalone hsqldb server.
Lewis, I noticed that you excluded jdom in the ivy.xml? Why is that? I included it again because the SqlStore needs it to reads it mapping.
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205-v7.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (NUTCH-1205) Upgrade gora modules to 0.2 in
ivy/ivy.xml
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney closed NUTCH-1205.
---------------------------------------
Excellent Ferdy. I'll buy you some nice whisky if we get a chance to catch up at some stage in the future :0)
Closing this one off.
Good work.
> Upgrade gora modules to 0.2 in ivy/ivy.xml
> ------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Blocker
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v10.patch, NUTCH-1205-v11-addition.patch, NUTCH-1205-v11.patch, NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205-v5.patch, NUTCH-1205-v5.patch, NUTCH-1205-v6.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT in ivy/ivy.xml
Posted by "Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206949#comment-13206949 ]
Ferdy Galema commented on NUTCH-1205:
-------------------------------------
Haven't got the slightest clue about why this error appears, sorry.. I simply do not understand how this ivy/maven repos stuff works.
> Upgrade gora modules to 0.2-SNAPSHOT in ivy/ivy.xml
> ---------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to
0.2-incubating in ivy/ivy.xml
Posted by "Ferdy Galema (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1205:
--------------------------------
Attachment: NUTCH-1205-v3.patch
There, this patch fixes the two "createDataStore" compile errors. (The existing changes of v2 are untouched.)
> Upgrade gora modules to 0.2-incubating in ivy/ivy.xml
> -----------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1205) Upgrade gora modules to
0.2-incubating in ivy/ivy.xml
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated NUTCH-1205:
----------------------------------------
Attachment: NUTCH-1205-v4.patch
This patch updates the deprecated ivy resolver enabling us to utilise the bleeding edge Gora stuff. There is a problem with two maven-plugin dependencies, I don't know where they are coming from so I thought I would put this patch up to see if anyone can resolve it themself! Thanks
> Upgrade gora modules to 0.2-incubating in ivy/ivy.xml
> -----------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205-v4.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-SNAPSHOT
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13157946#comment-13157946 ]
Lewis John McGibbney commented on NUTCH-1205:
---------------------------------------------
Patch aside... even if I _only_ change ivy/ivy.xml to
{code}
<dependency org="org.apache.gora" name="gora-core" rev="latest.integration" conf="*->compile"/>
<dependency org="org.apache.gora" name="gora-sql" rev="latest.integration" conf="*->compile"/>
{code}
the above errors still occur. Is anyone familiar with o.a.n.s.StorageUtils and I can attempt to walk my way through this one with them? My main target here is to get Nutchgora working with Gora 0.2-incubating.
>From reading the code, would it not make more sense to implement the getDataStore method rather than the createDataDtore method, this is the point of abstracting this stuff away from Nutch? If this is the case then there is some editing of methods in StorageUtils class. Something like this
{code}
@SuppressWarnings("unchecked")
public static <K, V extends Persistent> DataStore<K, V> createDataStore(Configuration conf,
Class<K> keyClass, Class<V> persistentClass) throws ClassNotFoundException, GoraException {
Class<? extends DataStore<K, V>> dataStoreClass =
(Class<? extends DataStore<K, V>>) getDataStoreClass(conf);
return DataStoreFactory.*getDataStore*(dataStoreClass, <<<<<
keyClass, persistentClass, *remaining parameters* );
}
{code}
> Upgrade gora modules to 0.2-SNAPSHOT
> ------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1205) Upgrade gora modules to
0.2-incubating in ivy/ivy.xml
Posted by "Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13189857#comment-13189857 ]
Ferdy Galema commented on NUTCH-1205:
-------------------------------------
Just to clarify, after applying v3 patch (and making sure that the right 0.2-SNAPSHOT or 0.2-incubating is installed in your local ivy2 repo) the TestGoraStorage multithreaded test fails. This is because of SqlStore not being thread-safe, see details in GORA-84.
> Upgrade gora modules to 0.2-incubating in ivy/ivy.xml
> -----------------------------------------------------
>
> Key: NUTCH-1205
> URL: https://issues.apache.org/jira/browse/NUTCH-1205
> Project: Nutch
> Issue Type: Improvement
> Components: storage
> Affects Versions: nutchgora
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Minor
> Fix For: nutchgora
>
> Attachments: NUTCH-1205-v2.patch, NUTCH-1205-v3.patch, NUTCH-1205.patch
>
>
> Although gora trunk is unstable, work is ongoing to get this fixed. For the time being, I think Nutchgora should use gora trunk as this will identify more vulnerabilities. I'll get the trivial patch submitted shortly.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira