You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Bill Graham (JIRA)" <ji...@apache.org> on 2011/01/11 08:23:45 UTC
[jira] Created: (HBASE-3436) HBase start scripts should not include
maven repos jars if they exist in lib
HBase start scripts should not include maven repos jars if they exist in lib
----------------------------------------------------------------------------
Key: HBASE-3436
URL: https://issues.apache.org/jira/browse/HBASE-3436
Project: HBase
Issue Type: Bug
Reporter: Bill Graham
When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HBASE-3436) HBase start scripts should not include
maven repos jars if they exist in lib
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-3436:
-------------------------
Priority: Critical (was: Major)
Fix Version/s: 0.90.1
I think this issue will manufacture support requests up on list so I made it critical fix for 0.90.1.
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986800#action_12986800 ]
Bill Graham commented on HBASE-3436:
------------------------------------
Looking more closely at bin/hbase it seems the maven deps are added if the pom.xml file is found under $HBASE_HOME. That seems to imply to the script that this is a developer setup.
The issue with this approach is that the pom.xml file is included in the distro though, so by default this will be the case if someone untars and starts. In a production RPM we'd probably rebuild and not include the pom.xml, but this might not be the case for others.
Instead I think injecting jars into what would be the "obvious" classpath should only be done when a developer takes an action, like setting a config or creating a target/ dir.
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham resolved HBASE-3436.
--------------------------------
Resolution: Duplicate
Sorry for the unnecessary churn on this guys, but this was fixed in HBASE-3383, which was after 0.90.0-rc1, which I'm still on (yes, I will upgrade :)).
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980273#action_12980273 ]
Bill Graham commented on HBASE-3436:
------------------------------------
I'm don't recall running mvn install, but I do see this jar in my maven repos:
~/.m2/repository/org/apache/hadoop/hadoop-core/0.20-append-r1042285/hadoop-core-0.20-append-r1042285.jar
Which is the same one that is packaged the HBase lib/ dir. What I recall is doing the following:
1. Downloading the HBase tar and extracting it
2. Modifying some configs and running it from by home dir
3. Copying the distro with custom configs to /var/opt/habase, where we now run it.
I'm not sure how the append jar got to my maven repo, but maybe I ran some target within HBase that put it there? I suspect my target/cached_classpath.txt file came along for the ride in step 3 above.
Either way, I've deleted the cached_classpath file and I can no longer reproduce, meaning I can't manage to get the maven jars into my classpath when starting/stoping the master. This is a good thing.
Feel free to mark this issue as invalid if it seems like something I might have done. We should try to make this maven/classpath functionality more obvious though, since I was hung up on this for a long time. Maybe better log messaging that these classes are being inserted into the cp would help.
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980258#action_12980258 ]
stack commented on HBASE-3436:
------------------------------
Yeah, I suppose. That check is there:
{code}
# Detect if we are in hbase sources dir
in_dev_env=false
if [ -d "${HBASE_HOME}/target" ]; then
in_dev_env=true
fi
{code}
Bill, anything else particular about your environment. e.g. did you download a built package and then inside in it do a new build?
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "Ted Dunning (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986786#action_12986786 ]
Ted Dunning commented on HBASE-3436:
------------------------------------
An approach that I am experimenting with is to mark hadoop and similar dependencies as being in the "provided" scope. My experimental platform right now is ycsb which is one step further down the food chain than hbase, but it has the same problem (but substitute hbase for hadoop in the discussion).
So far, this approach seems to work well. It creates a jar-with-dependencies that lacks the provided dependencies and their dependencies. At runtime, I simply inject hbase/lib/*.jar into the class path and all is good. Executing with a class path provided by maven should work just as well but I haven't tried it.
Based on my still formative success, it seems that this might be a reasonable path forward for hbase as well.
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "ryan rawson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12986908#action_12986908 ]
ryan rawson commented on HBASE-3436:
------------------------------------
we've enjoyed shipping 'build in place' tar.gz in the past, and i'd
like to keep that going if at all possible, so going with the [ -x
'target' ] kind of logic seems good to me.
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980230#action_12980230 ]
Todd Lipcon commented on HBASE-3436:
------------------------------------
I thought we only injected the maven jars if we saw a "target" directory -- ie that it's a build/dev environment instead of an installed one?
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HBASE-3436) HBase start scripts should not
include maven repos jars if they exist in lib
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-3436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980259#action_12980259 ]
stack commented on HBASE-3436:
------------------------------
Bill, that is, did you change your hbase download from a clean binary install into a 'dev environment' by doing 'mvn install' or something. Thanks.
> HBase start scripts should not include maven repos jars if they exist in lib
> ----------------------------------------------------------------------------
>
> Key: HBASE-3436
> URL: https://issues.apache.org/jira/browse/HBASE-3436
> Project: HBase
> Issue Type: Bug
> Reporter: Bill Graham
> Priority: Critical
> Fix For: 0.90.1
>
>
> When starting the master, the jars of the users maven repos get injected into the classpath as a convenience to developers. This can cause quite a debugging headache when hadoop jars don't match what's on the cluster.
> We should change the start scripts to not do this if the jars exist in the lib dir. Or better yet, we would only include maven repos jars if an optional param exists in hbase-env.sh.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.