You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by "Lewis John McGibbney (Created) (JIRA)" <ji...@apache.org> on 2012/01/12 17:03:39 UTC
[jira] [Created] (GORA-76) Upgrade to Hadoop 1.0
Upgrade to Hadoop 1.0
---------------------
Key: GORA-76
URL: https://issues.apache.org/jira/browse/GORA-76
Project: Gora
Issue Type: Improvement
Components: build process, storage
Affects Versions: 0.2-incubating
Reporter: Lewis John McGibbney
Fix For: 0.2-incubating
This was driven entirely by Juliens recent issue over in Nutchland.
Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (GORA-76) Upgrade to Hadoop 1.0.0
Posted by "Ferdy Galema (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218101#comment-13218101 ]
Ferdy Galema edited comment on GORA-76 at 2/28/12 12:45 PM:
------------------------------------------------------------
Why is this a blocked by GORA-94 exactly? Mind you that Nutchgora is able to run with Hadoop 1.0 and Avro 1.3.3 succesfully. It seems that Avro is not used directly in hadoop-core 1.0.0.
When simply changing the Hadoop version to 1.0.0 I do get some errors though when running tests, namely caused by:
Caused by: java.lang.NoSuchMethodError: org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
But have we not solved this one yet in Nutchgora? I vaguely recall that a similar error occured there when upgrading Hadoop.
was (Author: ferdy.g):
Why is this a blocker for GORA-76 exactly? Mind you that Nutchgora is able to run with Hadoop 1.0 and Avro 1.3.3 succesfully. It seems that Avro is not used directly in hadoop-core 1.0.0.
When simply changing the Hadoop version to 1.0.0 I do get some errors though when running tests, namely caused by:
Caused by: java.lang.NoSuchMethodError: org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
But have we not solved this one yet in Nutchgora? I vaguely recall that a similar error occured there when upgrading Hadoop.
> Upgrade to Hadoop 1.0.0
> -----------------------
>
> Key: GORA-76
> URL: https://issues.apache.org/jira/browse/GORA-76
> Project: Apache Gora
> Issue Type: Improvement
> Components: build process, storage
> Affects Versions: 0.2
> Reporter: Lewis John McGibbney
> Fix For: 0.2
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-76) Upgrade to Hadoop 1.0.0
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated GORA-76:
-------------------------------------
Summary: Upgrade to Hadoop 1.0.0 (was: Upgrade to Hadoop 1.0)
> Upgrade to Hadoop 1.0.0
> -----------------------
>
> Key: GORA-76
> URL: https://issues.apache.org/jira/browse/GORA-76
> Project: Apache Gora
> Issue Type: Improvement
> Components: build process, storage
> Affects Versions: 0.2
> Reporter: Lewis John McGibbney
> Fix For: 0.2
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0
Posted by "Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218139#comment-13218139 ]
Ferdy Galema commented on GORA-76:
----------------------------------
Ok I will try to use the same packages for Gora. Will get back at this.
> Upgrade to Hadoop 1.0.0
> -----------------------
>
> Key: GORA-76
> URL: https://issues.apache.org/jira/browse/GORA-76
> Project: Apache Gora
> Issue Type: Improvement
> Components: build process, storage
> Affects Versions: 0.2
> Reporter: Lewis John McGibbney
> Fix For: 0.2
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13209645#comment-13209645 ]
Lewis John McGibbney commented on GORA-76:
------------------------------------------
This is not a simple upgrade. We currently ship with Avro 1.3.3 which was released June 7 2010. Since then Avro has spun out from Hadoop, this obviously brings changes to the codebase...
I see upgrading Avro as a separate issue altogether so I'll open an issue accordingly.
> Upgrade to Hadoop 1.0
> ---------------------
>
> Key: GORA-76
> URL: https://issues.apache.org/jira/browse/GORA-76
> Project: Apache Gora
> Issue Type: Improvement
> Components: build process, storage
> Affects Versions: 0.2
> Reporter: Lewis John McGibbney
> Fix For: 0.2
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0
Posted by "Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218101#comment-13218101 ]
Ferdy Galema commented on GORA-76:
----------------------------------
Why is this a blocker for GORA-76 exactly? Mind you that Nutchgora is able to run with Hadoop 1.0 and Avro 1.3.3 succesfully. It seems that Avro is not used directly in hadoop-core 1.0.0.
When simply changing the Hadoop version to 1.0.0 I do get some errors though when running tests, namely caused by:
Caused by: java.lang.NoSuchMethodError: org.codehaus.jackson.JsonFactory.enable(Lorg/codehaus/jackson/JsonParser$Feature;)Lorg/codehaus/jackson/JsonFactory;
But have we not solved this one yet in Nutchgora? I vaguely recall that a similar error occured there when upgrading Hadoop.
> Upgrade to Hadoop 1.0.0
> -----------------------
>
> Key: GORA-76
> URL: https://issues.apache.org/jira/browse/GORA-76
> Project: Apache Gora
> Issue Type: Improvement
> Components: build process, storage
> Affects Versions: 0.2
> Reporter: Lewis John McGibbney
> Fix For: 0.2
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13218124#comment-13218124 ]
Lewis John McGibbney commented on GORA-76:
------------------------------------------
Hi Ferdy. Maybe blocker was the wrong tag to be using... I don't know.
What I do know is that Avro 1.3.3 was included in org.apache.hadoop (before it became TLP), whereas now Avro 1.6.2 is org.apache.avro, I'm therefore drawing the conclusion that my thinking revolved around, upgrade one upgrade the other...
If you are able to upgrade Hadoop libs to 1.0.0, and you wish to do so then that would be great. I was experiencing some problems, which in hindsight I should have documented. This lead me to opening GORA-94.
Regarding your final comment, I'm really not sure about Nutchgora. I remember asking Markus during a conversation about what is required to upgrade the Hadoop libs and he said that it was relatively straightforward. From your description above it seems that it might also be the case with Gora.
So are you able to compile Gora by updating Hadoop libs? Did you also upgrade the test dependency? When upgrading Ivy deps in Nutchgora I noticed that trunk uses hadoop-core 1.0.0 and 0.20.205.0 tests! Don't really know why but for consistency I shadowed this in Nutchgora upgrade.
> Upgrade to Hadoop 1.0.0
> -----------------------
>
> Key: GORA-76
> URL: https://issues.apache.org/jira/browse/GORA-76
> Project: Apache Gora
> Issue Type: Improvement
> Components: build process, storage
> Affects Versions: 0.2
> Reporter: Lewis John McGibbney
> Fix For: 0.2
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira