You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jena.apache.org by ajs6f <gi...@git.apache.org> on 2015/07/21 16:01:17 UTC
[GitHub] jena pull request: Bump versions of Hadoop, Guava for Elephas
GitHub user ajs6f opened a pull request:
https://github.com/apache/jena/pull/87
Bump versions of Hadoop, Guava for Elephas
https://issues.apache.org/jira/browse/JENA-994
Possibly off-target or too late for this ticket, but bumping Hadoop to latest release enables bringing Guava to latest release as well, a nice bonus.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ajs6f/jena ElephasBump
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/jena/pull/87.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #87
----
commit dbb11f4c4127749bd297af11b19ba6ff3c2297e0
Author: ajs6f <aj...@virginia.edu>
Date: 2015-07-21T13:59:55Z
Bump versions of Hadoop, Guava for Elephas
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of Hadoop, Guava for Elephas
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123373621
I brought this back to just a bump for Guava (to an old, but less old, version) and JUnit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123455323
I know nothing of the story behind the chosen JUnit version, but I can tell you that the current version of Guava used in the main codebase (18.0) will not work with the version of Hadoop in use in Elephas now. (That's actually how I ended up trying to bump Hadoop.) This (16.0.1) is the latest version I could find that would. Guava 18.0 _will_ work with a later version of Hadoop, but I don't know how much later-- I only tried with 2.7.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by rvesse <gi...@git.apache.org>.
Github user rvesse commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123743814
It may be possible to remove the explicit dependency on guava as a stop-gap
If memory serves the reason behind this was that dependency management in the Jena parents was causing a version conflict. Since guava is no longer a dependency of other Jena modules (because it is shaded) it may be possible to simply remove the explicit dependency on Guava
Also completely agree with Andy, even if the tests pass the real test is that the demo application runs correctly on an actual Hadoop cluster where various dependencies are provided by the cluster
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/jena/pull/87
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of Hadoop, Guava for Elephas
Posted by rvesse <gi...@git.apache.org>.
Github user rvesse commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123360536
The existing profile is what I would define as a reliable default
We could perhaps move to 2.6.x as that is starting to make it into the latest builds of CDH, Hortonworks etc but as Andy notes people tend to upgrade infrequently in this space
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of Hadoop, Guava for Elephas
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123349424
Hadoop 2.7.0 was first released in April of this year and marked as not a production release. 2.7.1 was this month (July). AFAICT neither CDH nor HDP are using 2.7.x at all.
Hadoop installations don't get upgraded very frequently; it's "non-trivial". IMO it is better to wait for users to ask for an update to get the pacing right.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123468369
On 21/07/15 16:00, Rob Vesse wrote:
> The existing profile is what I would define as a reliable default
>
> We could perhaps move to 2.6.x as that is starting to make it into the
> latest builds of CDH, Hortonworks etc but as Andy notes people tend to
> upgrade infrequently in this space
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/jena/pull/87#issuecomment-123360536>.
>
Rob - the current dependency is already Hadoop 2.6.0
Andy
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123745671
Is that something that can be automated as a system test (test-driving the demo app in a simple cluster)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of Hadoop, Guava for Elephas
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123350955
Have users asked for upgrades in the past?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123468596
org.apache.hadoop:hadoop-common:jar:2.6.0 depends on com.google.guava:guava:jar:11.0.2 (provided).
Just because Jena tests work isn't evidence Guava 16.0.1 works unfortunately. Jena is a library and may be co-resident with other code. See HADOOP-10101 including the initial attempt at migration to 15.0 which did not go in. At this very late stage, it needs a strong advantage for users for change.
(and this is why we shade guava for Jena!)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123767963
Makes sense - then the choice of guava direct is whatever Hadoop dictates.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of Hadoop, Guava for Elephas
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123352349
The version of Hadoop in use is controlled by a Maven profile. Perhaps we could offer a "reliable" default profile and a "latest release" profile for those looking to explore?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of Hadoop, Guava for Elephas
Posted by ajs6f <gi...@git.apache.org>.
Github user ajs6f commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123365451
Okay, I'll back off on Hadoop and just offer a simple bump for Guava up to the last version that is compatible with the current Hadoop.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by rvesse <gi...@git.apache.org>.
Github user rvesse commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123754862
Agreed, that reminds me that I am intending to strip Guava out of Airline anyway
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123454124
Why does Elephas have it's own version management of JUnit and of jena-arq (aligned to the main build)? Could this be left from original development?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123747484
```
[INFO] +- com.github.rvesse:airline:jar:0.9.2:compile
[INFO] | +- javax.inject:javax.inject:jar:1:compile
[INFO] | +- com.google.guava:guava:jar:11.0.2:compile
[INFO] | \- org.apache.commons:commons-lang3:jar:3.3.2:compile
```
Presumably the explicit `dependencyManagement` is causing that version to be choosen over com.github.rvesse:airline.
I've fixed the JUnit and jena-arq dependency management.
Jena itself does need to depend on a runtime Guava because of shading (looking like a better decision all the time!).
I don't see this as a change needed for 3.0.0. It can be sorted out at leisure.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] jena pull request: Bump versions of JUnit, Guava for Elephas
Posted by afs <gi...@git.apache.org>.
Github user afs commented on the pull request:
https://github.com/apache/jena/pull/87#issuecomment-123476169
Junit version is now the same as the rest of Jena. Version of jena-arq managed in dependencyManagement with an explicit version, not a property, because that keeps the release plugin happy (well, happier).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---