You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@metron.apache.org by nickwallen <gi...@git.apache.org> on 2018/10/08 20:47:13 UTC
[GitHub] metron pull request #1232: METRON-1813 Stellar REPL Not Initialized with Cli...
GitHub user nickwallen opened a pull request:
https://github.com/apache/metron/pull/1232
METRON-1813 Stellar REPL Not Initialized with Client JAAS
Running a function like `KAFKA_GET` in the Stellar REPL does not work in a kerberized environment.
* When the Stellar REPL is launched in a Kerberized environment, it needs to have the Client JAAS passed to it so that Stellar functions can access resources like Kafka.
* The JVM running the REPL never gets passed the `-Djava.security.auth.login.config=$METRON_HOME/client_jaas.conf` JVM arg.
## Testing
1. Launch the development environment.
1. Kerberize the development environment.
1. Launch the REPL.
```
source /etc/default/metron
$METRON_HOME/bin/stellar -z $ZOOKEEPER
```
1. Retrieve messages using Kafka.
```
conf := { "group.id":"bro_parser","security.protocol":"SASL_PLAINTEXT" }
KAFKA_GET("bro", 10, conf)
```
1. If you can retrieve messages, the fix worked.
## Pull Request Checklist
- [ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at [Metron Jira](https://issues.apache.org/jira/browse/METRON/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel).
- [ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
- [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
- [ ] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
- [ ] Have you included steps or a guide to how the change may be verified and tested manually?
- [ ] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
- [ ] Have you written or updated unit tests and or integration tests to verify your changes?
- [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nickwallen/metron METRON-1813
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/metron/pull/1232.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1232
----
commit d74dac2ff80159022a55e327a6e7ae83af535259
Author: Nick Allen <ni...@...>
Date: 2018-10-08T20:40:20Z
METRON-1813 Stellar REPL Not Initialized with Client JAAS
----
---
[GitHub] metron issue #1232: METRON-1813 Stellar REPL Not Initialized with Client JAA...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on the issue:
https://github.com/apache/metron/pull/1232
> @mmiklavc: just curious about the proposed test which says to source the metron env script.
You don't have to source /etc/default/metron before running the stellar script. I only added that so that I could reference $METRON_HOME and such in these test instructions. It is not required to do so.
---
[GitHub] metron issue #1232: METRON-1813 Stellar REPL Not Initialized with Client JAA...
Posted by mmiklavc <gi...@git.apache.org>.
Github user mmiklavc commented on the issue:
https://github.com/apache/metron/pull/1232
@nickwallen, just curious about the proposed test which says to source the metron env script. In the stellar shell script you introduced a change that does exactly that - `source $METRON_SYSCONFIG`. Am I reading that right? The proposed changes lgtm. +1 by inspection.
---
[GitHub] metron pull request #1232: METRON-1813 Stellar REPL Not Initialized with Cli...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/1232#discussion_r223506784
--- Diff: metron-platform/metron-common/src/main/scripts/stellar ---
@@ -28,10 +28,15 @@ elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then
. /usr/lib/bigtop-utils/bigtop-detect-javahome
fi
+export METRON_SYSCONFIG="/etc/default/metron"
+if [ -f "$METRON_SYSCONFIG" ]; then
+ source $METRON_SYSCONFIG
--- End diff --
In `/etc/default/metron`, `METRON_JVMFLAGS` will be set to either...
1. Empty if the cluster is not kerberized
1. Set to `-Djava.security.auth.login.config=$METRON_HOME/client_jaas.conf` if kerberos has been enabled.
---
[GitHub] metron pull request #1232: METRON-1813 Stellar REPL Not Initialized with Cli...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/1232#discussion_r223506424
--- Diff: metron-platform/metron-common/src/main/scripts/stellar ---
@@ -28,10 +28,15 @@ elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then
. /usr/lib/bigtop-utils/bigtop-detect-javahome
fi
+export METRON_SYSCONFIG="/etc/default/metron"
+if [ -f "$METRON_SYSCONFIG" ]; then
+ source $METRON_SYSCONFIG
+fi
+
+export METRON_VERSION="${METRON_VERSION:-${project.version}}"
+export METRON_HOME="${METRON_HOME:-/usr/metron/$METRON_VERSION}"
export HBASE_CONFIGS=$(hbase classpath)
-export METRON_VERSION=${project.version}
-export METRON_HOME=/usr/metron/$METRON_VERSION
export STELLAR_LIB=$(find $METRON_HOME/lib/ -name metron-parsers*.jar)
export MANAGEMENT_LIB=$(find $METRON_HOME/lib/ -name metron-management*.jar)
export PROFILER_LIB=$(find $METRON_HOME/lib/ -name metron-profiler-repl*.jar)
-java $JVMFLAGS -cp "${CONTRIB:-$METRON_HOME/contrib/*}:$STELLAR_LIB:$MANAGEMENT_LIB:$PROFILER_LIB:$HBASE_CONFIGS" org.apache.metron.stellar.common.shell.cli.StellarShell "$@"
+java $METRON_JVMFLAGS -cp "${CONTRIB:-$METRON_HOME/contrib/*}:$STELLAR_LIB:$MANAGEMENT_LIB:$PROFILER_LIB:$HBASE_CONFIGS" org.apache.metron.stellar.common.shell.cli.StellarShell "$@"
--- End diff --
The script referenced `JVMFLAGS`, but I don't know where that is being set. I assume `METRON_JVMFLAGS` is what was originally intended.
---
[GitHub] metron pull request #1232: METRON-1813 Stellar REPL Not Initialized with Cli...
Posted by nickwallen <gi...@git.apache.org>.
Github user nickwallen commented on a diff in the pull request:
https://github.com/apache/metron/pull/1232#discussion_r223507156
--- Diff: metron-platform/metron-common/src/main/scripts/stellar ---
@@ -28,10 +28,15 @@ elif [ -e /usr/lib/bigtop-utils/bigtop-detect-javahome ]; then
. /usr/lib/bigtop-utils/bigtop-detect-javahome
fi
+export METRON_SYSCONFIG="/etc/default/metron"
+if [ -f "$METRON_SYSCONFIG" ]; then
+ source $METRON_SYSCONFIG
+fi
+
+export METRON_VERSION="${METRON_VERSION:-${project.version}}"
+export METRON_HOME="${METRON_HOME:-/usr/metron/$METRON_VERSION}"
--- End diff --
Even though we source `/etc/default/metron` which does define `METRON_HOME` and `METRON_VERSION`, I chose to continue to set a value for these should `/etc/default/metron` not exist.
---
[GitHub] metron pull request #1232: METRON-1813 Stellar REPL Not Initialized with Cli...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/metron/pull/1232
---