You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flume.apache.org by pk...@apache.org on 2023/12/21 09:24:00 UTC
(flume) branch trunk updated (2e6141172 -> 8f12c906a)
This is an automated email from the ASF dual-hosted git repository.
pkarwasz pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/flume.git
from 2e6141172 Merge pull request #414 from apache/dependabot/maven/flume-parent/com.google.guava-guava-29.0-jre
new e8893444f Normalize line endings
new 8f12c906a Update Maven Wrapper script
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../conf/stopwords.txt => .gitattributes | 11 +-
.mvn/wrapper/MavenWrapperDownloader.java | 117 ---
.mvn/wrapper/maven-wrapper.properties | 8 +-
bin/flume-ng.ps1 | 814 ++++++++++-----------
.../test-documents/sample-statuses-20120906-141433 | 6 +-
.../test/resources/test-documents/testBMPfp.txt | 6 +-
.../src/test/resources/test-documents/testRFC822 | 82 +--
mvnw | 218 +++---
mvnw.cmd | 31 +-
9 files changed, 597 insertions(+), 696 deletions(-)
copy flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/solr/collection1/conf/stopwords.txt => .gitattributes (71%)
delete mode 100644 .mvn/wrapper/MavenWrapperDownloader.java
(flume) 02/02: Update Maven Wrapper script
Posted by pk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pkarwasz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/flume.git
commit 8f12c906a2914d9146c6e2fda234390ae8d1a93c
Author: Piotr P. Karwasz <pi...@karwasz.org>
AuthorDate: Thu Dec 21 09:38:45 2023 +0100
Update Maven Wrapper script
---
.mvn/wrapper/MavenWrapperDownloader.java | 117 -----------------
.mvn/wrapper/maven-wrapper.properties | 8 +-
mvnw | 218 +++++++++++++++----------------
mvnw.cmd | 31 ++++-
4 files changed, 133 insertions(+), 241 deletions(-)
diff --git a/.mvn/wrapper/MavenWrapperDownloader.java b/.mvn/wrapper/MavenWrapperDownloader.java
deleted file mode 100644
index b901097f2..000000000
--- a/.mvn/wrapper/MavenWrapperDownloader.java
+++ /dev/null
@@ -1,117 +0,0 @@
-/*
- * Copyright 2007-present the original author or authors.
- *
- * Licensed under the Apache License, Version 2.0 (the "License");
- * you may not use this file except in compliance with the License.
- * You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-import java.net.*;
-import java.io.*;
-import java.nio.channels.*;
-import java.util.Properties;
-
-public class MavenWrapperDownloader {
-
- private static final String WRAPPER_VERSION = "0.5.6";
- /**
- * Default URL to download the maven-wrapper.jar from, if no 'downloadUrl' is provided.
- */
- private static final String DEFAULT_DOWNLOAD_URL = "https://repo.maven.apache.org/maven2/io/takari/maven-wrapper/"
- + WRAPPER_VERSION + "/maven-wrapper-" + WRAPPER_VERSION + ".jar";
-
- /**
- * Path to the maven-wrapper.properties file, which might contain a downloadUrl property to
- * use instead of the default one.
- */
- private static final String MAVEN_WRAPPER_PROPERTIES_PATH =
- ".mvn/wrapper/maven-wrapper.properties";
-
- /**
- * Path where the maven-wrapper.jar will be saved to.
- */
- private static final String MAVEN_WRAPPER_JAR_PATH =
- ".mvn/wrapper/maven-wrapper.jar";
-
- /**
- * Name of the property which should be used to override the default download url for the wrapper.
- */
- private static final String PROPERTY_NAME_WRAPPER_URL = "wrapperUrl";
-
- public static void main(String args[]) {
- System.out.println("- Downloader started");
- File baseDirectory = new File(args[0]);
- System.out.println("- Using base directory: " + baseDirectory.getAbsolutePath());
-
- // If the maven-wrapper.properties exists, read it and check if it contains a custom
- // wrapperUrl parameter.
- File mavenWrapperPropertyFile = new File(baseDirectory, MAVEN_WRAPPER_PROPERTIES_PATH);
- String url = DEFAULT_DOWNLOAD_URL;
- if(mavenWrapperPropertyFile.exists()) {
- FileInputStream mavenWrapperPropertyFileInputStream = null;
- try {
- mavenWrapperPropertyFileInputStream = new FileInputStream(mavenWrapperPropertyFile);
- Properties mavenWrapperProperties = new Properties();
- mavenWrapperProperties.load(mavenWrapperPropertyFileInputStream);
- url = mavenWrapperProperties.getProperty(PROPERTY_NAME_WRAPPER_URL, url);
- } catch (IOException e) {
- System.out.println("- ERROR loading '" + MAVEN_WRAPPER_PROPERTIES_PATH + "'");
- } finally {
- try {
- if(mavenWrapperPropertyFileInputStream != null) {
- mavenWrapperPropertyFileInputStream.close();
- }
- } catch (IOException e) {
- // Ignore ...
- }
- }
- }
- System.out.println("- Downloading from: " + url);
-
- File outputFile = new File(baseDirectory.getAbsolutePath(), MAVEN_WRAPPER_JAR_PATH);
- if(!outputFile.getParentFile().exists()) {
- if(!outputFile.getParentFile().mkdirs()) {
- System.out.println(
- "- ERROR creating output directory '" + outputFile.getParentFile().getAbsolutePath() + "'");
- }
- }
- System.out.println("- Downloading to: " + outputFile.getAbsolutePath());
- try {
- downloadFileFromURL(url, outputFile);
- System.out.println("Done");
- System.exit(0);
- } catch (Throwable e) {
- System.out.println("- Error downloading");
- e.printStackTrace();
- System.exit(1);
- }
- }
-
- private static void downloadFileFromURL(String urlString, File destination) throws Exception {
- if (System.getenv("MVNW_USERNAME") != null && System.getenv("MVNW_PASSWORD") != null) {
- String username = System.getenv("MVNW_USERNAME");
- char[] password = System.getenv("MVNW_PASSWORD").toCharArray();
- Authenticator.setDefault(new Authenticator() {
- @Override
- protected PasswordAuthentication getPasswordAuthentication() {
- return new PasswordAuthentication(username, password);
- }
- });
- }
- URL website = new URL(urlString);
- ReadableByteChannel rbc;
- rbc = Channels.newChannel(website.openStream());
- FileOutputStream fos = new FileOutputStream(destination);
- fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
- fos.close();
- rbc.close();
- }
-
-}
diff --git a/.mvn/wrapper/maven-wrapper.properties b/.mvn/wrapper/maven-wrapper.properties
index 8c79a83ae..346d645fd 100644
--- a/.mvn/wrapper/maven-wrapper.properties
+++ b/.mvn/wrapper/maven-wrapper.properties
@@ -5,14 +5,14 @@
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
-#
+#
# http://www.apache.org/licenses/LICENSE-2.0
-#
+#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
-distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.8.4/apache-maven-3.8.4-bin.zip
-wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar
+distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.6/apache-maven-3.9.6-bin.zip
+wrapperUrl=https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar
diff --git a/mvnw b/mvnw
index 5643201c7..8d937f4c1 100755
--- a/mvnw
+++ b/mvnw
@@ -19,7 +19,7 @@
# ----------------------------------------------------------------------------
# ----------------------------------------------------------------------------
-# Maven Start Up Batch script
+# Apache Maven Wrapper startup batch script, version 3.2.0
#
# Required ENV vars:
# ------------------
@@ -27,7 +27,6 @@
#
# Optional ENV vars
# -----------------
-# M2_HOME - location of maven2's installed home dir
# MAVEN_OPTS - parameters passed to the Java VM when running Maven
# e.g. to debug Maven itself, use
# set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=8000
@@ -54,7 +53,7 @@ fi
cygwin=false;
darwin=false;
mingw=false
-case "`uname`" in
+case "$(uname)" in
CYGWIN*) cygwin=true ;;
MINGW*) mingw=true;;
Darwin*) darwin=true
@@ -62,9 +61,9 @@ case "`uname`" in
# See https://developer.apple.com/library/mac/qa/qa1170/_index.html
if [ -z "$JAVA_HOME" ]; then
if [ -x "/usr/libexec/java_home" ]; then
- export JAVA_HOME="`/usr/libexec/java_home`"
+ JAVA_HOME="$(/usr/libexec/java_home)"; export JAVA_HOME
else
- export JAVA_HOME="/Library/Java/Home"
+ JAVA_HOME="/Library/Java/Home"; export JAVA_HOME
fi
fi
;;
@@ -72,68 +71,38 @@ esac
if [ -z "$JAVA_HOME" ] ; then
if [ -r /etc/gentoo-release ] ; then
- JAVA_HOME=`java-config --jre-home`
+ JAVA_HOME=$(java-config --jre-home)
fi
fi
-if [ -z "$M2_HOME" ] ; then
- ## resolve links - $0 may be a link to maven's home
- PRG="$0"
-
- # need this for relative symlinks
- while [ -h "$PRG" ] ; do
- ls=`ls -ld "$PRG"`
- link=`expr "$ls" : '.*-> \(.*\)$'`
- if expr "$link" : '/.*' > /dev/null; then
- PRG="$link"
- else
- PRG="`dirname "$PRG"`/$link"
- fi
- done
-
- saveddir=`pwd`
-
- M2_HOME=`dirname "$PRG"`/..
-
- # make it fully qualified
- M2_HOME=`cd "$M2_HOME" && pwd`
-
- cd "$saveddir"
- # echo Using m2 at $M2_HOME
-fi
-
# For Cygwin, ensure paths are in UNIX format before anything is touched
if $cygwin ; then
- [ -n "$M2_HOME" ] &&
- M2_HOME=`cygpath --unix "$M2_HOME"`
[ -n "$JAVA_HOME" ] &&
- JAVA_HOME=`cygpath --unix "$JAVA_HOME"`
+ JAVA_HOME=$(cygpath --unix "$JAVA_HOME")
[ -n "$CLASSPATH" ] &&
- CLASSPATH=`cygpath --path --unix "$CLASSPATH"`
+ CLASSPATH=$(cygpath --path --unix "$CLASSPATH")
fi
# For Mingw, ensure paths are in UNIX format before anything is touched
if $mingw ; then
- [ -n "$M2_HOME" ] &&
- M2_HOME="`(cd "$M2_HOME"; pwd)`"
- [ -n "$JAVA_HOME" ] &&
- JAVA_HOME="`(cd "$JAVA_HOME"; pwd)`"
+ [ -n "$JAVA_HOME" ] && [ -d "$JAVA_HOME" ] &&
+ JAVA_HOME="$(cd "$JAVA_HOME" || (echo "cannot cd into $JAVA_HOME."; exit 1); pwd)"
fi
if [ -z "$JAVA_HOME" ]; then
- javaExecutable="`which javac`"
- if [ -n "$javaExecutable" ] && ! [ "`expr \"$javaExecutable\" : '\([^ ]*\)'`" = "no" ]; then
+ javaExecutable="$(which javac)"
+ if [ -n "$javaExecutable" ] && ! [ "$(expr "\"$javaExecutable\"" : '\([^ ]*\)')" = "no" ]; then
# readlink(1) is not available as standard on Solaris 10.
- readLink=`which readlink`
- if [ ! `expr "$readLink" : '\([^ ]*\)'` = "no" ]; then
+ readLink=$(which readlink)
+ if [ ! "$(expr "$readLink" : '\([^ ]*\)')" = "no" ]; then
if $darwin ; then
- javaHome="`dirname \"$javaExecutable\"`"
- javaExecutable="`cd \"$javaHome\" && pwd -P`/javac"
+ javaHome="$(dirname "\"$javaExecutable\"")"
+ javaExecutable="$(cd "\"$javaHome\"" && pwd -P)/javac"
else
- javaExecutable="`readlink -f \"$javaExecutable\"`"
+ javaExecutable="$(readlink -f "\"$javaExecutable\"")"
fi
- javaHome="`dirname \"$javaExecutable\"`"
- javaHome=`expr "$javaHome" : '\(.*\)/bin'`
+ javaHome="$(dirname "\"$javaExecutable\"")"
+ javaHome=$(expr "$javaHome" : '\(.*\)/bin')
JAVA_HOME="$javaHome"
export JAVA_HOME
fi
@@ -149,7 +118,7 @@ if [ -z "$JAVACMD" ] ; then
JAVACMD="$JAVA_HOME/bin/java"
fi
else
- JAVACMD="`\\unset -f command; \\command -v java`"
+ JAVACMD="$(\unset -f command 2>/dev/null; \command -v java)"
fi
fi
@@ -163,12 +132,9 @@ if [ -z "$JAVA_HOME" ] ; then
echo "Warning: JAVA_HOME environment variable is not set."
fi
-CLASSWORLDS_LAUNCHER=org.codehaus.plexus.classworlds.launcher.Launcher
-
# traverses directory structure from process work directory to filesystem root
# first directory with .mvn subdirectory is considered project base directory
find_maven_basedir() {
-
if [ -z "$1" ]
then
echo "Path not specified to find_maven_basedir"
@@ -184,96 +150,99 @@ find_maven_basedir() {
fi
# workaround for JBEAP-8937 (on Solaris 10/Sparc)
if [ -d "${wdir}" ]; then
- wdir=`cd "$wdir/.."; pwd`
+ wdir=$(cd "$wdir/.." || exit 1; pwd)
fi
# end of workaround
done
- echo "${basedir}"
+ printf '%s' "$(cd "$basedir" || exit 1; pwd)"
}
# concatenates all lines of a file
concat_lines() {
if [ -f "$1" ]; then
- echo "$(tr -s '\n' ' ' < "$1")"
+ # Remove \r in case we run on Windows within Git Bash
+ # and check out the repository with auto CRLF management
+ # enabled. Otherwise, we may read lines that are delimited with
+ # \r\n and produce $'-Xarg\r' rather than -Xarg due to word
+ # splitting rules.
+ tr -s '\r\n' ' ' < "$1"
+ fi
+}
+
+log() {
+ if [ "$MVNW_VERBOSE" = true ]; then
+ printf '%s\n' "$1"
fi
}
-BASE_DIR=`find_maven_basedir "$(pwd)"`
+BASE_DIR=$(find_maven_basedir "$(dirname "$0")")
if [ -z "$BASE_DIR" ]; then
exit 1;
fi
+MAVEN_PROJECTBASEDIR=${MAVEN_BASEDIR:-"$BASE_DIR"}; export MAVEN_PROJECTBASEDIR
+log "$MAVEN_PROJECTBASEDIR"
+
##########################################################################################
# Extension to allow automatically downloading the maven-wrapper.jar from Maven-central
# This allows using the maven wrapper in projects that prohibit checking in binary data.
##########################################################################################
-if [ -r "$BASE_DIR/.mvn/wrapper/maven-wrapper.jar" ]; then
- if [ "$MVNW_VERBOSE" = true ]; then
- echo "Found .mvn/wrapper/maven-wrapper.jar"
- fi
+wrapperJarPath="$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.jar"
+if [ -r "$wrapperJarPath" ]; then
+ log "Found $wrapperJarPath"
else
- if [ "$MVNW_VERBOSE" = true ]; then
- echo "Couldn't find .mvn/wrapper/maven-wrapper.jar, downloading it ..."
- fi
+ log "Couldn't find $wrapperJarPath, downloading it ..."
+
if [ -n "$MVNW_REPOURL" ]; then
- jarUrl="$MVNW_REPOURL/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
+ wrapperUrl="$MVNW_REPOURL/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
else
- jarUrl="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
+ wrapperUrl="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
fi
- while IFS="=" read key value; do
- case "$key" in (wrapperUrl) jarUrl="$value"; break ;;
+ while IFS="=" read -r key value; do
+ # Remove '\r' from value to allow usage on windows as IFS does not consider '\r' as a separator ( considers space, tab, new line ('\n'), and custom '=' )
+ safeValue=$(echo "$value" | tr -d '\r')
+ case "$key" in (wrapperUrl) wrapperUrl="$safeValue"; break ;;
esac
- done < "$BASE_DIR/.mvn/wrapper/maven-wrapper.properties"
- if [ "$MVNW_VERBOSE" = true ]; then
- echo "Downloading from: $jarUrl"
- fi
- wrapperJarPath="$BASE_DIR/.mvn/wrapper/maven-wrapper.jar"
+ done < "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.properties"
+ log "Downloading from: $wrapperUrl"
+
if $cygwin; then
- wrapperJarPath=`cygpath --path --windows "$wrapperJarPath"`
+ wrapperJarPath=$(cygpath --path --windows "$wrapperJarPath")
fi
if command -v wget > /dev/null; then
- if [ "$MVNW_VERBOSE" = true ]; then
- echo "Found wget ... using wget"
- fi
+ log "Found wget ... using wget"
+ [ "$MVNW_VERBOSE" = true ] && QUIET="" || QUIET="--quiet"
if [ -z "$MVNW_USERNAME" ] || [ -z "$MVNW_PASSWORD" ]; then
- wget "$jarUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
+ wget $QUIET "$wrapperUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
else
- wget --http-user=$MVNW_USERNAME --http-password=$MVNW_PASSWORD "$jarUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
+ wget $QUIET --http-user="$MVNW_USERNAME" --http-password="$MVNW_PASSWORD" "$wrapperUrl" -O "$wrapperJarPath" || rm -f "$wrapperJarPath"
fi
elif command -v curl > /dev/null; then
- if [ "$MVNW_VERBOSE" = true ]; then
- echo "Found curl ... using curl"
- fi
+ log "Found curl ... using curl"
+ [ "$MVNW_VERBOSE" = true ] && QUIET="" || QUIET="--silent"
if [ -z "$MVNW_USERNAME" ] || [ -z "$MVNW_PASSWORD" ]; then
- curl -o "$wrapperJarPath" "$jarUrl" -f
+ curl $QUIET -o "$wrapperJarPath" "$wrapperUrl" -f -L || rm -f "$wrapperJarPath"
else
- curl --user $MVNW_USERNAME:$MVNW_PASSWORD -o "$wrapperJarPath" "$jarUrl" -f
+ curl $QUIET --user "$MVNW_USERNAME:$MVNW_PASSWORD" -o "$wrapperJarPath" "$wrapperUrl" -f -L || rm -f "$wrapperJarPath"
fi
-
else
- if [ "$MVNW_VERBOSE" = true ]; then
- echo "Falling back to using Java to download"
- fi
- javaClass="$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.java"
+ log "Falling back to using Java to download"
+ javaSource="$MAVEN_PROJECTBASEDIR/.mvn/wrapper/MavenWrapperDownloader.java"
+ javaClass="$MAVEN_PROJECTBASEDIR/.mvn/wrapper/MavenWrapperDownloader.class"
# For Cygwin, switch paths to Windows format before running javac
if $cygwin; then
- javaClass=`cygpath --path --windows "$javaClass"`
+ javaSource=$(cygpath --path --windows "$javaSource")
+ javaClass=$(cygpath --path --windows "$javaClass")
fi
- if [ -e "$javaClass" ]; then
- if [ ! -e "$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.class" ]; then
- if [ "$MVNW_VERBOSE" = true ]; then
- echo " - Compiling MavenWrapperDownloader.java ..."
- fi
- # Compiling the Java class
- ("$JAVA_HOME/bin/javac" "$javaClass")
+ if [ -e "$javaSource" ]; then
+ if [ ! -e "$javaClass" ]; then
+ log " - Compiling MavenWrapperDownloader.java ..."
+ ("$JAVA_HOME/bin/javac" "$javaSource")
fi
- if [ -e "$BASE_DIR/.mvn/wrapper/MavenWrapperDownloader.class" ]; then
- # Running the downloader
- if [ "$MVNW_VERBOSE" = true ]; then
- echo " - Running MavenWrapperDownloader.java ..."
- fi
- ("$JAVA_HOME/bin/java" -cp .mvn/wrapper MavenWrapperDownloader "$MAVEN_PROJECTBASEDIR")
+ if [ -e "$javaClass" ]; then
+ log " - Running MavenWrapperDownloader.java ..."
+ ("$JAVA_HOME/bin/java" -cp .mvn/wrapper MavenWrapperDownloader "$wrapperUrl" "$wrapperJarPath") || rm -f "$wrapperJarPath"
fi
fi
fi
@@ -282,35 +251,58 @@ fi
# End of extension
##########################################################################################
-export MAVEN_PROJECTBASEDIR=${MAVEN_BASEDIR:-"$BASE_DIR"}
-if [ "$MVNW_VERBOSE" = true ]; then
- echo $MAVEN_PROJECTBASEDIR
+# If specified, validate the SHA-256 sum of the Maven wrapper jar file
+wrapperSha256Sum=""
+while IFS="=" read -r key value; do
+ case "$key" in (wrapperSha256Sum) wrapperSha256Sum=$value; break ;;
+ esac
+done < "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.properties"
+if [ -n "$wrapperSha256Sum" ]; then
+ wrapperSha256Result=false
+ if command -v sha256sum > /dev/null; then
+ if echo "$wrapperSha256Sum $wrapperJarPath" | sha256sum -c > /dev/null 2>&1; then
+ wrapperSha256Result=true
+ fi
+ elif command -v shasum > /dev/null; then
+ if echo "$wrapperSha256Sum $wrapperJarPath" | shasum -a 256 -c > /dev/null 2>&1; then
+ wrapperSha256Result=true
+ fi
+ else
+ echo "Checksum validation was requested but neither 'sha256sum' or 'shasum' are available."
+ echo "Please install either command, or disable validation by removing 'wrapperSha256Sum' from your maven-wrapper.properties."
+ exit 1
+ fi
+ if [ $wrapperSha256Result = false ]; then
+ echo "Error: Failed to validate Maven wrapper SHA-256, your Maven wrapper might be compromised." >&2
+ echo "Investigate or delete $wrapperJarPath to attempt a clean download." >&2
+ echo "If you updated your Maven version, you need to update the specified wrapperSha256Sum property." >&2
+ exit 1
+ fi
fi
+
MAVEN_OPTS="$(concat_lines "$MAVEN_PROJECTBASEDIR/.mvn/jvm.config") $MAVEN_OPTS"
# For Cygwin, switch paths to Windows format before running java
if $cygwin; then
- [ -n "$M2_HOME" ] &&
- M2_HOME=`cygpath --path --windows "$M2_HOME"`
[ -n "$JAVA_HOME" ] &&
- JAVA_HOME=`cygpath --path --windows "$JAVA_HOME"`
+ JAVA_HOME=$(cygpath --path --windows "$JAVA_HOME")
[ -n "$CLASSPATH" ] &&
- CLASSPATH=`cygpath --path --windows "$CLASSPATH"`
+ CLASSPATH=$(cygpath --path --windows "$CLASSPATH")
[ -n "$MAVEN_PROJECTBASEDIR" ] &&
- MAVEN_PROJECTBASEDIR=`cygpath --path --windows "$MAVEN_PROJECTBASEDIR"`
+ MAVEN_PROJECTBASEDIR=$(cygpath --path --windows "$MAVEN_PROJECTBASEDIR")
fi
# Provide a "standardized" way to retrieve the CLI args that will
# work with both Windows and non-Windows executions.
-MAVEN_CMD_LINE_ARGS="$MAVEN_CONFIG $@"
+MAVEN_CMD_LINE_ARGS="$MAVEN_CONFIG $*"
export MAVEN_CMD_LINE_ARGS
WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain
+# shellcheck disable=SC2086 # safe args
exec "$JAVACMD" \
$MAVEN_OPTS \
$MAVEN_DEBUG_OPTS \
-classpath "$MAVEN_PROJECTBASEDIR/.mvn/wrapper/maven-wrapper.jar" \
- "-Dmaven.home=${M2_HOME}" \
"-Dmaven.multiModuleProjectDirectory=${MAVEN_PROJECTBASEDIR}" \
${WRAPPER_LAUNCHER} $MAVEN_CONFIG "$@"
diff --git a/mvnw.cmd b/mvnw.cmd
index 23b7079a3..f80fbad3e 100644
--- a/mvnw.cmd
+++ b/mvnw.cmd
@@ -18,13 +18,12 @@
@REM ----------------------------------------------------------------------------
@REM ----------------------------------------------------------------------------
-@REM Maven Start Up Batch script
+@REM Apache Maven Wrapper startup batch script, version 3.2.0
@REM
@REM Required ENV vars:
@REM JAVA_HOME - location of a JDK home dir
@REM
@REM Optional ENV vars
-@REM M2_HOME - location of maven2's installed home dir
@REM MAVEN_BATCH_ECHO - set to 'on' to enable the echoing of the batch commands
@REM MAVEN_BATCH_PAUSE - set to 'on' to wait for a keystroke before ending
@REM MAVEN_OPTS - parameters passed to the Java VM when running Maven
@@ -120,10 +119,10 @@ SET MAVEN_JAVA_EXE="%JAVA_HOME%\bin\java.exe"
set WRAPPER_JAR="%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.jar"
set WRAPPER_LAUNCHER=org.apache.maven.wrapper.MavenWrapperMain
-set DOWNLOAD_URL="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
+set WRAPPER_URL="https://repo.maven.apache.org/maven2/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
FOR /F "usebackq tokens=1,2 delims==" %%A IN ("%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.properties") DO (
- IF "%%A"=="wrapperUrl" SET DOWNLOAD_URL=%%B
+ IF "%%A"=="wrapperUrl" SET WRAPPER_URL=%%B
)
@REM Extension to allow automatically downloading the maven-wrapper.jar from Maven-central
@@ -134,11 +133,11 @@ if exist %WRAPPER_JAR% (
)
) else (
if not "%MVNW_REPOURL%" == "" (
- SET DOWNLOAD_URL="%MVNW_REPOURL%/org/apache/maven/wrapper/maven-wrapper/3.1.0/maven-wrapper-3.1.0.jar"
+ SET WRAPPER_URL="%MVNW_REPOURL%/org/apache/maven/wrapper/maven-wrapper/3.2.0/maven-wrapper-3.2.0.jar"
)
if "%MVNW_VERBOSE%" == "true" (
echo Couldn't find %WRAPPER_JAR%, downloading it ...
- echo Downloading from: %DOWNLOAD_URL%
+ echo Downloading from: %WRAPPER_URL%
)
powershell -Command "&{"^
@@ -146,7 +145,7 @@ if exist %WRAPPER_JAR% (
"if (-not ([string]::IsNullOrEmpty('%MVNW_USERNAME%') -and [string]::IsNullOrEmpty('%MVNW_PASSWORD%'))) {"^
"$webclient.Credentials = new-object System.Net.NetworkCredential('%MVNW_USERNAME%', '%MVNW_PASSWORD%');"^
"}"^
- "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; $webclient.DownloadFile('%DOWNLOAD_URL%', '%WRAPPER_JAR%')"^
+ "[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12; $webclient.DownloadFile('%WRAPPER_URL%', '%WRAPPER_JAR%')"^
"}"
if "%MVNW_VERBOSE%" == "true" (
echo Finished downloading %WRAPPER_JAR%
@@ -154,6 +153,24 @@ if exist %WRAPPER_JAR% (
)
@REM End of extension
+@REM If specified, validate the SHA-256 sum of the Maven wrapper jar file
+SET WRAPPER_SHA_256_SUM=""
+FOR /F "usebackq tokens=1,2 delims==" %%A IN ("%MAVEN_PROJECTBASEDIR%\.mvn\wrapper\maven-wrapper.properties") DO (
+ IF "%%A"=="wrapperSha256Sum" SET WRAPPER_SHA_256_SUM=%%B
+)
+IF NOT %WRAPPER_SHA_256_SUM%=="" (
+ powershell -Command "&{"^
+ "$hash = (Get-FileHash \"%WRAPPER_JAR%\" -Algorithm SHA256).Hash.ToLower();"^
+ "If('%WRAPPER_SHA_256_SUM%' -ne $hash){"^
+ " Write-Output 'Error: Failed to validate Maven wrapper SHA-256, your Maven wrapper might be compromised.';"^
+ " Write-Output 'Investigate or delete %WRAPPER_JAR% to attempt a clean download.';"^
+ " Write-Output 'If you updated your Maven version, you need to update the specified wrapperSha256Sum property.';"^
+ " exit 1;"^
+ "}"^
+ "}"
+ if ERRORLEVEL 1 goto error
+)
+
@REM Provide a "standardized" way to retrieve the CLI args that will
@REM work with both Windows and non-Windows executions.
set MAVEN_CMD_LINE_ARGS=%*
(flume) 01/02: Normalize line endings
Posted by pk...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
pkarwasz pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/flume.git
commit e8893444f17c60add3edbf62390ced7dd92fc0fd
Author: Piotr P. Karwasz <pi...@karwasz.org>
AuthorDate: Thu Dec 21 09:32:35 2023 +0100
Normalize line endings
---
.gitattributes | 23 +
bin/flume-ng.ps1 | 814 ++++++++++-----------
.../test-documents/sample-statuses-20120906-141433 | 6 +-
.../test/resources/test-documents/testBMPfp.txt | 6 +-
.../src/test/resources/test-documents/testRFC822 | 82 +--
5 files changed, 477 insertions(+), 454 deletions(-)
diff --git a/.gitattributes b/.gitattributes
new file mode 100644
index 000000000..eaa5806c4
--- /dev/null
+++ b/.gitattributes
@@ -0,0 +1,23 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# All text files with LF line endings
+* text=auto eol=lf
+# PS1 files are Windows specific
+*.ps1 eol=crlf
+# Maven Wrapper cmd script
+/mvnw.cmd eol=crlf
+# Maven Wrapper need LF line endings
+/.mvn/wrapper/maven-wrapper.properties eol=lf
diff --git a/bin/flume-ng.ps1 b/bin/flume-ng.ps1
index 4778717b7..a826737c3 100755
--- a/bin/flume-ng.ps1
+++ b/bin/flume-ng.ps1
@@ -1,407 +1,407 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-param(
- [Parameter(Position=0, Mandatory=$true)]
- [ValidateSet('help','agent','avro-client','version')] $command,
- [Parameter()] [string] $conf,
- [Parameter()] [Alias('C')] [string] $classPath,
- [Parameter()] [Alias('property')] [string] $javaProperties,
- [Parameter()] [Alias('d')] [switch] $dryrun,
- [Parameter()] [Alias('plugins-path')] [string] $pluginsPath,
-
- [Parameter()] [Alias('f',"conf-file")] [string] $confFile,
- [Parameter()] [Alias('n')] [string] $name,
-
- [Parameter()] [string] $rpcProps,
- [Parameter()] [Alias('H',"host")] [string] $avroHost,
- [Parameter()] [Alias('p',"port")] [string] $avroPort,
- [Parameter()] [Alias("dirname")] [string] $avroDirName,
- [Parameter()] [Alias("filename")] [string] $avroFileName,
- [Parameter()] [Alias('R',"headerFile")] [string] $avroHeaderFile
-)
-
-# when invoked from cmd shell, array arguments are treated as array with one string value
-# so we accept classpath and javaProperty as string argument and split explicitly into array
-
-if ($classPath -ne "") {
- $classPathArr = $classPath.Split(";")
-} else {
- $classPathArr = @()
-}
-
-if($javaProperties -ne "") {
- $javaPropertiesArr = $javaProperties.Split(";") # properties may have embedded comma, so using ; as delim
-} else {
- $javaPropertiesArr = @()
-}
-
-# FUNCTIONS
-Function PrintUsage {
- Write-Host "
-Usage: flume-ng <command> [options]...
-
-commands:
- help display this help text
- agent run a Flume agent
- avro-client run an avro Flume client
- version show Flume version info
-
-global options:
- -conf <conf> use configs in <conf> directory
- -classpath,-C ""value1;value2; .."" append to the classpath
- -property ""name1=val;name2=val; .."" sets a JDK system property value
- -dryrun,-d do not actually start Flume (test)
- -plugins-path ""dir1;dir2"" semi-colon-separated list of plugins.d directories. See the
- plugins.d section in the user guide for more details.
- Default: \$FLUME_HOME/plugins.d
-
-agent options:
- -conf-file,-f <file> specify a config file (required)
- -name,-n <name> the name of this agent (required)
-
-avro-client options:
- -rpcProps <file> RPC client properties file with server connection params
- -host,-H <host> hostname to which events will be sent (required)
- -port,-p <port> port of the avro source (required)
- -dirname <dir> directory to stream to avro source
- -filename <file> text file to stream to avro source [default: std input]
- -headerFile,-R <file> headerFile containing headers as key/value pairs on each new line
-
- Either --rpcProps or both --host and --port must be specified.
-
-Note that if <conf> directory is specified, then it is always included first
-in the classpath."
-}
-
-Function EnumerateJars($path) {
- return Get-ChildItem $path -Filter *.jar | ? { "$_" -notmatch "slf4j-(api|log4j12).*jar" } | % { $_.FullName}
-}
-
-Function GetHadoopHome {
- if($env:HADOOP_PREFIX) {
- $hadoopBin = "$env:HADOOP_PREFIX\bin;"
- } elseif ($env:HADOOP_HOME) {
- $hadoopBin = "$env:HADOOP_HOME\bin;"
- }
-
- #Searches for hadoop.cmd in the HADOOP_HOME, current directory and path
- [String[]] $hadoopPaths = ("$hadoopBin;.;$env:PATH").Split(";") |
- ? { "$_" -ne "" -and (Test-Path $_ )} |
- ? { Test-Path ( Join-Path $_ "hadoop.cmd" ) }
- if($hadoopPaths -ne $Null ) {
- $binDir = $hadoopPaths[0] + "\.."
- return Resolve-Path $binDir
- }
- Write-Host "WARN: HADOOP_PREFIX or HADOOP_HOME not found"
- return $Null
-}
-
-Function GetHbaseHome() {
- if($env:HBASE_HOME) {
- return $env:HBASE_HOME
- }
-
- #Searches for hbase.cmd in the HBASE_HOME, current directory and path
- [String[]] $hbasePaths = (".;$env:PATH").Split(";") |
- ? { "$_" -ne "" -and (Test-Path $_) } |
- ? { Test-Path (Join-Path $_ "hbase.cmd") }
-
- if($hbasePaths -ne $Null) {
- return $hbasePaths[0]
- }
- Write-Host "WARN: HBASE_HOME not found"
- return $Null
-}
-
-Function GetHiveHome() {
- if($env:HIVE_HOME) {
- return $env:HIVE_HOME
- }
-
- #Searches for hive.cmd in the HIVE_HOME, current directory and path
- [String[]] $hivePaths = (".;$env:PATH").Split(";") |
- ? { "$_" -ne "" -and (Test-Path $_) } |
- ? { Test-Path (Join-Path $_ "hive.cmd") }
-
- if($hivePaths -ne $Null) {
- return $hivePaths[0]
- }
- Write-Host "WARN: HIVE_HOME not found"
- return $Null
-}
-
-Function GetJavaLibraryPath ($cmd, $flumeLibDir) {
- $flumeCoreJar = EnumerateJars( $flumeLibDir ) | ? { $_ -match "flume-ng-core.*jar" }
- $output = & "$cmd" -classpath $flumeCoreJar org.apache.flume.tools.GetJavaProperty java.library.path
-
- # look for the line that has the desired property value
- if( $output -match "(?m)^java\.library\.path=(.*)$" ) {
- return $Matches[1].split(";") | % { $_ -replace "\\$" , ""} # trim trailing \ char
- }
- return @();
-}
-
-
-Function GetClassPath ($cmd) {
- $output = & "$cmd" "classpath"
- return $output;
-}
-
-Function GetJavaPath {
- if ($env:JAVA_HOME) {
- return "$env:JAVA_HOME\bin\java.exe" }
- Write-Host "WARN: JAVA_HOME not set"
- return '"' + (Resolve-Path "java.exe").Path + '"'
-}
-
-
-function runFlume($javaClassPath, $javaLibraryPath, $javaOptions, $class, $javaProcessArgumentList) {
- [string]$javaPath = GetJavaPath
- [string]$fullJavaCommand = "-classpath $javaClassPath -Djava.library.path=$javaLibraryPath $javaOptions $class $javaProcessArgumentList"
- if ($dryrun) {
- Write-Host 'Dry run mode enabled (will not actually initiate startup)'
- Write-Host "$javaPath $fullJavaCommand"
- } else {
- Write-Host "
- Running FLUME $command :
- class: $class
- arguments: $javaProcessArgumentList
- "
-
- $ErrorActionPreference = "Continue"
- $x = Start-Process $javaPath -ArgumentList "$fullJavaCommand" -Wait -NoNewWindow
- }
-}
-
-
-# The script will terminate if any steps fail
-$ErrorActionPreference = "Stop"
-
-#SWITCH FOR THE DIFFERENT COMMANDS
-[string] $javaProcessArgumentList
-switch ($command)
-{
- 'help' {
- PrintUsage
- return }
-
- 'agent' {
- if (!$Name) {
- PrintUsage
- Write-Host "ERROR: Name parameter missing"
- return
- }
- if (!$ConfFile) {
- PrintUsage
- Write-Host "ERROR: ConfFile parameter missing"
- return
- }
- $class='org.apache.flume.node.Application'
- $confFile = '"' + (Resolve-Path $confFile).Path + '"'
- $javaProcessArgumentList = "-n $name -f $confFile" }
-
- 'avro-client' {
- $class='org.apache.flume.client.avro.AvroCLIClient'
- if("$rpcProps" -eq "") {
- if (!$AvroHost) {
- PrintUsage
- Write-Host "ERROR: Avro Host parameter missing"
- return
- }
- if (!$AvroPort) {
- PrintUsage
- Write-Host "ERROR: Avro Port parameter missing"
- return
- }
- $javaProcessArgumentList = " -H $AvroHost -p $AvroPort"
- } else {
- $javaProcessArgumentList += " --rpcProps ""$rpcProps"""
- }
-
- if ($avroHeaderFile) {
- $avroHeaderFile = (Resolve-Path $avroHeaderFile).Path
- $javaProcessArgumentList += " -R $avroHeaderFile" }
- if ($avroFileName) {
- $avroFileName = (Resolve-Path $avroFileName).Path
- $javaProcessArgumentList += " -F $avroFileName" }
- if($avroDirName) {
- $avroDirName = (Resolve-Path $avroDirName).Path
- $javaProcessArgumentList += " --dirname ""$avroDirName""" } }
-
- 'version' {
- $class='org.apache.flume.tools.VersionInfo'
- $javaProcessArgumentList = "" }
-
- default {
- PrintUsage
- Write-Host "ERROR: Invalid command '$command'"
- return }
-}
-
-$FlumeHome = $null
-if($env:FLUME_HOME) {
- $FlumeHome = $env:FLUME_HOME
-} else {
- $ScriptPath = Split-Path -Parent $MyInvocation.MyCommand.Path
- $FlumeHome = Split-Path -Parent $ScriptPath
-}
-
-########### Source flume-env.ps1 ##############
-
-# allow users to override the default env vars via conf\flume-env.ps1
-if( "$conf" -eq "" ) {
- if( Test-path ("$FlumeHome\conf") ) {
- $conf = "$FlumeHome\conf"
- Write-Host "WARN: Config directory not set. Defaulting to $conf"
- }
-}
-if ( "$conf" -ne "" ) {
- Write-Host "Sourcing environment configuration script $conf\flume-env.ps1"
- if ( Test-path "$conf\flume-env.ps1" ) {
- . "$conf\flume-env.ps1"
- } else {
- Write-Host "WARN: Did not find $conf\flume-env.ps1"
- }
-} else {
- Write-Host "WARN: No configuration directory found! Use --conf <dir> to set."
-}
-
-########### Setup JAVA_OPTS ##############
-
-[string]$javaOptions="$JAVA_OPTS"
-foreach ($opt in $javaPropertiesArr) {
- $javaOptions = "$javaOptions -D$opt"
-}
-
-########### Setup Classpath ###############
-# flume\conf ; flume_home\lib\* ; cmdline ; env.ps1 ; plugins.d ; hadoop.cpath ; hbase.cpath ; hive.cpath
-
-[string]$javaClassPath='"' + $conf + '"'
-[string]$flumeLibJars=""
-[string]$flumeLibDir = Resolve-Path "$FlumeHome\lib"
-
-# Add FlumeHome\lib\* to class path
-$javaClassPath = "$javaClassPath;""$flumeLibDir\*"""
-$flumeLibJars = "$flumeLibDir\*"""
-
-# Add classpath from cmd line & FLUME_CLASSPATH in flume-env.ps1
-if ( $FLUME_CLASSPATH ) {
- $classPathArr = $FLUME_CLASSPATH.Split(";")
-}
-foreach ($path in $classPathArr) {
- $fullPath = (Resolve-Path $path).Path
- $javaClassPath = "$javaClassPath;""$fullPath"""
-}
-
-$javaLibraryPath = ""
-
-# Add plugins.d into classpath and libpath
-if ("$pluginsPath" -eq "") {
- $pluginsPath = "$FlumeHome\plugins.d"
-}
-
-foreach($plugin in $pluginsPath.Split(";") ) {
- if ( Test-path "$plugin" ) {
- $pluginTmp1 = (@(Get-ChildItem "$plugin\*\lib") -join "\*"";""")
- if( "$pluginTmp1" -ne "" ) {
- $javaClassPath="$javaClassPath;""" + $pluginTmp1 + "\*"";"
- }
-
- $pluginTmp2 = (@(Get-ChildItem "$plugin\*\libext") -join "\*"";""")
- if( "$pluginTmp2" -ne "" ) {
- $javaClassPath="$javaClassPath;""" + $pluginTmp2 + "\*"";"
- }
-
- $javaLibraryPathTmp = (@(Get-ChildItem "$plugin\*\native") -join "\*"";""")
- if( "$javaLibraryPathTmp" -ne "" ) {
- $javaLibraryPath= "$javaLibraryPath""" + "$javaLibraryPathTmp" + "\*"";"
- }
- }
-}
-
-# Add Hadoop classpath & java.library.path
-$hadoopHome = GetHadoopHome
-if("$hadoopHome" -ne "") {
- $hadoopCmd = "$hadoopHome\bin\hadoop.cmd"
- if( Test-Path $hadoopCmd ) {
- Write-Host "Including Hadoop libraries found in ($hadoopHome) for DFS access"
- $javaClassPath = "$javaClassPath;""$hadoopHome\conf""";
- foreach ($path in GetClassPath $hadoopCmd) {
- $javaClassPath = "$javaClassPath;""$path"""
- }
-
- foreach ( $path in GetJavaLibraryPath $hadoopCmd $flumeLibDir ) {
- $javaLibraryPath = "$javaLibraryPath""$path"";"
- }
- } else {
- Write-Host "WARN: $hadoopCmd not be found. Unable to include Hadoop's classpath & java.library.path"
- }
-} else {
- Write-Host "WARN: HADOOP_PREFIX not set. Unable to include Hadoop's classpath & java.library.path"
-}
-
-# Add HBase classpath & java.library.path
-$hbaseHome = GetHbaseHome
-if( "$hbaseHome" -ne "" ) {
- $hbaseCmd = "$hbaseHome\bin\hbase.cmd"
- if( Test-Path $hbaseCmd ) {
- Write-Host "Including HBase libraries found via ($hbaseHome) for HBase access"
- foreach ( $path in GetClassPath $hbaseCmd ) {
- $javaClassPath = "$javaClassPath;""$path"""
- }
- $javaClassPath = "$javaClassPath;""$hbaseHome\conf"""
-
- foreach ( $path in GetJavaLibraryPath $hbaseCmd $flumeLibDir ) {
- $javaLibraryPath = "$javaLibraryPath""$path"";"
- }
- } else {
- Write-Host "WARN: $hbaseCmd not be found. Unable to include HBase classpath and java.library.path"
- }
-}
-
-# Add Hive classpath
-$hiveHome = GetHiveHome
-if( "$hiveHome" -ne "" ) {
- $hiveLib = "$hiveHome\lib"
- if( Test-Path $hiveLib ) {
- Write-Host "Including Hive libraries found via ($hiveHome) for Hive access"
- $javaClassPath = "$javaClassPath;""$hiveLib\*"""
- } else {
- Write-Host "WARN: $hiveLib not found. Unable to include Hive into classpath"
- }
-
- $hiveConf = "$hiveHome\conf"
- if( Test-Path $hiveConf ) {
- Write-Host "Including Hive conf dir ($hiveConf) in classpath for Hive access"
- $javaClassPath = "$javaClassPath;""$hiveConf"""
- } else {
- Write-Host "WARN: $hiveConf not found. Unable to include it into classpath"
- }
-
- $hcatLib = "$hiveHome\hcatalog\share\hcatalog"
- if( Test-Path $hcatLib ) {
- Write-Host "Including HCatalog libraries ($hcatLib) for Hive access"
- $javaClassPath = "$javaClassPath;""$hcatLib\*"""
- } else {
- Write-Host "WARN: $hcatLib not found. Unable to include HCatalog into classpath"
- }
-}
-
-runFlume $javaClassPath $javaLibraryPath $javaOptions $class $javaProcessArgumentList
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+param(
+ [Parameter(Position=0, Mandatory=$true)]
+ [ValidateSet('help','agent','avro-client','version')] $command,
+ [Parameter()] [string] $conf,
+ [Parameter()] [Alias('C')] [string] $classPath,
+ [Parameter()] [Alias('property')] [string] $javaProperties,
+ [Parameter()] [Alias('d')] [switch] $dryrun,
+ [Parameter()] [Alias('plugins-path')] [string] $pluginsPath,
+
+ [Parameter()] [Alias('f',"conf-file")] [string] $confFile,
+ [Parameter()] [Alias('n')] [string] $name,
+
+ [Parameter()] [string] $rpcProps,
+ [Parameter()] [Alias('H',"host")] [string] $avroHost,
+ [Parameter()] [Alias('p',"port")] [string] $avroPort,
+ [Parameter()] [Alias("dirname")] [string] $avroDirName,
+ [Parameter()] [Alias("filename")] [string] $avroFileName,
+ [Parameter()] [Alias('R',"headerFile")] [string] $avroHeaderFile
+)
+
+# when invoked from cmd shell, array arguments are treated as array with one string value
+# so we accept classpath and javaProperty as string argument and split explicitly into array
+
+if ($classPath -ne "") {
+ $classPathArr = $classPath.Split(";")
+} else {
+ $classPathArr = @()
+}
+
+if($javaProperties -ne "") {
+ $javaPropertiesArr = $javaProperties.Split(";") # properties may have embedded comma, so using ; as delim
+} else {
+ $javaPropertiesArr = @()
+}
+
+# FUNCTIONS
+Function PrintUsage {
+ Write-Host "
+Usage: flume-ng <command> [options]...
+
+commands:
+ help display this help text
+ agent run a Flume agent
+ avro-client run an avro Flume client
+ version show Flume version info
+
+global options:
+ -conf <conf> use configs in <conf> directory
+ -classpath,-C ""value1;value2; .."" append to the classpath
+ -property ""name1=val;name2=val; .."" sets a JDK system property value
+ -dryrun,-d do not actually start Flume (test)
+ -plugins-path ""dir1;dir2"" semi-colon-separated list of plugins.d directories. See the
+ plugins.d section in the user guide for more details.
+ Default: \$FLUME_HOME/plugins.d
+
+agent options:
+ -conf-file,-f <file> specify a config file (required)
+ -name,-n <name> the name of this agent (required)
+
+avro-client options:
+ -rpcProps <file> RPC client properties file with server connection params
+ -host,-H <host> hostname to which events will be sent (required)
+ -port,-p <port> port of the avro source (required)
+ -dirname <dir> directory to stream to avro source
+ -filename <file> text file to stream to avro source [default: std input]
+ -headerFile,-R <file> headerFile containing headers as key/value pairs on each new line
+
+ Either --rpcProps or both --host and --port must be specified.
+
+Note that if <conf> directory is specified, then it is always included first
+in the classpath."
+}
+
+Function EnumerateJars($path) {
+ return Get-ChildItem $path -Filter *.jar | ? { "$_" -notmatch "slf4j-(api|log4j12).*jar" } | % { $_.FullName}
+}
+
+Function GetHadoopHome {
+ if($env:HADOOP_PREFIX) {
+ $hadoopBin = "$env:HADOOP_PREFIX\bin;"
+ } elseif ($env:HADOOP_HOME) {
+ $hadoopBin = "$env:HADOOP_HOME\bin;"
+ }
+
+ #Searches for hadoop.cmd in the HADOOP_HOME, current directory and path
+ [String[]] $hadoopPaths = ("$hadoopBin;.;$env:PATH").Split(";") |
+ ? { "$_" -ne "" -and (Test-Path $_ )} |
+ ? { Test-Path ( Join-Path $_ "hadoop.cmd" ) }
+ if($hadoopPaths -ne $Null ) {
+ $binDir = $hadoopPaths[0] + "\.."
+ return Resolve-Path $binDir
+ }
+ Write-Host "WARN: HADOOP_PREFIX or HADOOP_HOME not found"
+ return $Null
+}
+
+Function GetHbaseHome() {
+ if($env:HBASE_HOME) {
+ return $env:HBASE_HOME
+ }
+
+ #Searches for hbase.cmd in the HBASE_HOME, current directory and path
+ [String[]] $hbasePaths = (".;$env:PATH").Split(";") |
+ ? { "$_" -ne "" -and (Test-Path $_) } |
+ ? { Test-Path (Join-Path $_ "hbase.cmd") }
+
+ if($hbasePaths -ne $Null) {
+ return $hbasePaths[0]
+ }
+ Write-Host "WARN: HBASE_HOME not found"
+ return $Null
+}
+
+Function GetHiveHome() {
+ if($env:HIVE_HOME) {
+ return $env:HIVE_HOME
+ }
+
+ #Searches for hive.cmd in the HIVE_HOME, current directory and path
+ [String[]] $hivePaths = (".;$env:PATH").Split(";") |
+ ? { "$_" -ne "" -and (Test-Path $_) } |
+ ? { Test-Path (Join-Path $_ "hive.cmd") }
+
+ if($hivePaths -ne $Null) {
+ return $hivePaths[0]
+ }
+ Write-Host "WARN: HIVE_HOME not found"
+ return $Null
+}
+
+Function GetJavaLibraryPath ($cmd, $flumeLibDir) {
+ $flumeCoreJar = EnumerateJars( $flumeLibDir ) | ? { $_ -match "flume-ng-core.*jar" }
+ $output = & "$cmd" -classpath $flumeCoreJar org.apache.flume.tools.GetJavaProperty java.library.path
+
+ # look for the line that has the desired property value
+ if( $output -match "(?m)^java\.library\.path=(.*)$" ) {
+ return $Matches[1].split(";") | % { $_ -replace "\\$" , ""} # trim trailing \ char
+ }
+ return @();
+}
+
+
+Function GetClassPath ($cmd) {
+ $output = & "$cmd" "classpath"
+ return $output;
+}
+
+Function GetJavaPath {
+ if ($env:JAVA_HOME) {
+ return "$env:JAVA_HOME\bin\java.exe" }
+ Write-Host "WARN: JAVA_HOME not set"
+ return '"' + (Resolve-Path "java.exe").Path + '"'
+}
+
+
+function runFlume($javaClassPath, $javaLibraryPath, $javaOptions, $class, $javaProcessArgumentList) {
+ [string]$javaPath = GetJavaPath
+ [string]$fullJavaCommand = "-classpath $javaClassPath -Djava.library.path=$javaLibraryPath $javaOptions $class $javaProcessArgumentList"
+ if ($dryrun) {
+ Write-Host 'Dry run mode enabled (will not actually initiate startup)'
+ Write-Host "$javaPath $fullJavaCommand"
+ } else {
+ Write-Host "
+ Running FLUME $command :
+ class: $class
+ arguments: $javaProcessArgumentList
+ "
+
+ $ErrorActionPreference = "Continue"
+ $x = Start-Process $javaPath -ArgumentList "$fullJavaCommand" -Wait -NoNewWindow
+ }
+}
+
+
+# The script will terminate if any steps fail
+$ErrorActionPreference = "Stop"
+
+#SWITCH FOR THE DIFFERENT COMMANDS
+[string] $javaProcessArgumentList
+switch ($command)
+{
+ 'help' {
+ PrintUsage
+ return }
+
+ 'agent' {
+ if (!$Name) {
+ PrintUsage
+ Write-Host "ERROR: Name parameter missing"
+ return
+ }
+ if (!$ConfFile) {
+ PrintUsage
+ Write-Host "ERROR: ConfFile parameter missing"
+ return
+ }
+ $class='org.apache.flume.node.Application'
+ $confFile = '"' + (Resolve-Path $confFile).Path + '"'
+ $javaProcessArgumentList = "-n $name -f $confFile" }
+
+ 'avro-client' {
+ $class='org.apache.flume.client.avro.AvroCLIClient'
+ if("$rpcProps" -eq "") {
+ if (!$AvroHost) {
+ PrintUsage
+ Write-Host "ERROR: Avro Host parameter missing"
+ return
+ }
+ if (!$AvroPort) {
+ PrintUsage
+ Write-Host "ERROR: Avro Port parameter missing"
+ return
+ }
+ $javaProcessArgumentList = " -H $AvroHost -p $AvroPort"
+ } else {
+ $javaProcessArgumentList += " --rpcProps ""$rpcProps"""
+ }
+
+ if ($avroHeaderFile) {
+ $avroHeaderFile = (Resolve-Path $avroHeaderFile).Path
+ $javaProcessArgumentList += " -R $avroHeaderFile" }
+ if ($avroFileName) {
+ $avroFileName = (Resolve-Path $avroFileName).Path
+ $javaProcessArgumentList += " -F $avroFileName" }
+ if($avroDirName) {
+ $avroDirName = (Resolve-Path $avroDirName).Path
+ $javaProcessArgumentList += " --dirname ""$avroDirName""" } }
+
+ 'version' {
+ $class='org.apache.flume.tools.VersionInfo'
+ $javaProcessArgumentList = "" }
+
+ default {
+ PrintUsage
+ Write-Host "ERROR: Invalid command '$command'"
+ return }
+}
+
+$FlumeHome = $null
+if($env:FLUME_HOME) {
+ $FlumeHome = $env:FLUME_HOME
+} else {
+ $ScriptPath = Split-Path -Parent $MyInvocation.MyCommand.Path
+ $FlumeHome = Split-Path -Parent $ScriptPath
+}
+
+########### Source flume-env.ps1 ##############
+
+# allow users to override the default env vars via conf\flume-env.ps1
+if( "$conf" -eq "" ) {
+ if( Test-path ("$FlumeHome\conf") ) {
+ $conf = "$FlumeHome\conf"
+ Write-Host "WARN: Config directory not set. Defaulting to $conf"
+ }
+}
+if ( "$conf" -ne "" ) {
+ Write-Host "Sourcing environment configuration script $conf\flume-env.ps1"
+ if ( Test-path "$conf\flume-env.ps1" ) {
+ . "$conf\flume-env.ps1"
+ } else {
+ Write-Host "WARN: Did not find $conf\flume-env.ps1"
+ }
+} else {
+ Write-Host "WARN: No configuration directory found! Use --conf <dir> to set."
+}
+
+########### Setup JAVA_OPTS ##############
+
+[string]$javaOptions="$JAVA_OPTS"
+foreach ($opt in $javaPropertiesArr) {
+ $javaOptions = "$javaOptions -D$opt"
+}
+
+########### Setup Classpath ###############
+# flume\conf ; flume_home\lib\* ; cmdline ; env.ps1 ; plugins.d ; hadoop.cpath ; hbase.cpath ; hive.cpath
+
+[string]$javaClassPath='"' + $conf + '"'
+[string]$flumeLibJars=""
+[string]$flumeLibDir = Resolve-Path "$FlumeHome\lib"
+
+# Add FlumeHome\lib\* to class path
+$javaClassPath = "$javaClassPath;""$flumeLibDir\*"""
+$flumeLibJars = "$flumeLibDir\*"""
+
+# Add classpath from cmd line & FLUME_CLASSPATH in flume-env.ps1
+if ( $FLUME_CLASSPATH ) {
+ $classPathArr = $FLUME_CLASSPATH.Split(";")
+}
+foreach ($path in $classPathArr) {
+ $fullPath = (Resolve-Path $path).Path
+ $javaClassPath = "$javaClassPath;""$fullPath"""
+}
+
+$javaLibraryPath = ""
+
+# Add plugins.d into classpath and libpath
+if ("$pluginsPath" -eq "") {
+ $pluginsPath = "$FlumeHome\plugins.d"
+}
+
+foreach($plugin in $pluginsPath.Split(";") ) {
+ if ( Test-path "$plugin" ) {
+ $pluginTmp1 = (@(Get-ChildItem "$plugin\*\lib") -join "\*"";""")
+ if( "$pluginTmp1" -ne "" ) {
+ $javaClassPath="$javaClassPath;""" + $pluginTmp1 + "\*"";"
+ }
+
+ $pluginTmp2 = (@(Get-ChildItem "$plugin\*\libext") -join "\*"";""")
+ if( "$pluginTmp2" -ne "" ) {
+ $javaClassPath="$javaClassPath;""" + $pluginTmp2 + "\*"";"
+ }
+
+ $javaLibraryPathTmp = (@(Get-ChildItem "$plugin\*\native") -join "\*"";""")
+ if( "$javaLibraryPathTmp" -ne "" ) {
+ $javaLibraryPath= "$javaLibraryPath""" + "$javaLibraryPathTmp" + "\*"";"
+ }
+ }
+}
+
+# Add Hadoop classpath & java.library.path
+$hadoopHome = GetHadoopHome
+if("$hadoopHome" -ne "") {
+ $hadoopCmd = "$hadoopHome\bin\hadoop.cmd"
+ if( Test-Path $hadoopCmd ) {
+ Write-Host "Including Hadoop libraries found in ($hadoopHome) for DFS access"
+ $javaClassPath = "$javaClassPath;""$hadoopHome\conf""";
+ foreach ($path in GetClassPath $hadoopCmd) {
+ $javaClassPath = "$javaClassPath;""$path"""
+ }
+
+ foreach ( $path in GetJavaLibraryPath $hadoopCmd $flumeLibDir ) {
+ $javaLibraryPath = "$javaLibraryPath""$path"";"
+ }
+ } else {
+ Write-Host "WARN: $hadoopCmd not be found. Unable to include Hadoop's classpath & java.library.path"
+ }
+} else {
+ Write-Host "WARN: HADOOP_PREFIX not set. Unable to include Hadoop's classpath & java.library.path"
+}
+
+# Add HBase classpath & java.library.path
+$hbaseHome = GetHbaseHome
+if( "$hbaseHome" -ne "" ) {
+ $hbaseCmd = "$hbaseHome\bin\hbase.cmd"
+ if( Test-Path $hbaseCmd ) {
+ Write-Host "Including HBase libraries found via ($hbaseHome) for HBase access"
+ foreach ( $path in GetClassPath $hbaseCmd ) {
+ $javaClassPath = "$javaClassPath;""$path"""
+ }
+ $javaClassPath = "$javaClassPath;""$hbaseHome\conf"""
+
+ foreach ( $path in GetJavaLibraryPath $hbaseCmd $flumeLibDir ) {
+ $javaLibraryPath = "$javaLibraryPath""$path"";"
+ }
+ } else {
+ Write-Host "WARN: $hbaseCmd not be found. Unable to include HBase classpath and java.library.path"
+ }
+}
+
+# Add Hive classpath
+$hiveHome = GetHiveHome
+if( "$hiveHome" -ne "" ) {
+ $hiveLib = "$hiveHome\lib"
+ if( Test-Path $hiveLib ) {
+ Write-Host "Including Hive libraries found via ($hiveHome) for Hive access"
+ $javaClassPath = "$javaClassPath;""$hiveLib\*"""
+ } else {
+ Write-Host "WARN: $hiveLib not found. Unable to include Hive into classpath"
+ }
+
+ $hiveConf = "$hiveHome\conf"
+ if( Test-Path $hiveConf ) {
+ Write-Host "Including Hive conf dir ($hiveConf) in classpath for Hive access"
+ $javaClassPath = "$javaClassPath;""$hiveConf"""
+ } else {
+ Write-Host "WARN: $hiveConf not found. Unable to include it into classpath"
+ }
+
+ $hcatLib = "$hiveHome\hcatalog\share\hcatalog"
+ if( Test-Path $hcatLib ) {
+ Write-Host "Including HCatalog libraries ($hcatLib) for Hive access"
+ $javaClassPath = "$javaClassPath;""$hcatLib\*"""
+ } else {
+ Write-Host "WARN: $hcatLib not found. Unable to include HCatalog into classpath"
+ }
+}
+
+runFlume $javaClassPath $javaLibraryPath $javaOptions $class $javaProcessArgumentList
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433
index 380ab6ebe..e633a1f71 100644
--- a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433
@@ -1,4 +1,4 @@
-1000
-{"text":"sample tweet one","retweet_count":0,"in_reply_to_user_id":null,"retweeted":false,"truncated":false,"source":"href=\"http:\/\/sample.com\"","id_str":"1234567891","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_status_id":null,"place":null,"in_reply_to_status_id_str":null,"coordinates":null,"created_at":"Wed Sep 05 01:01:01 +0000 1985","in_reply_to_screen_name":null,"favorited":false,"in_reply_to_user_id_str":null,"user":{"default_profile_image":false,"friends [...]
-2000
+1000
+{"text":"sample tweet one","retweet_count":0,"in_reply_to_user_id":null,"retweeted":false,"truncated":false,"source":"href=\"http:\/\/sample.com\"","id_str":"1234567891","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_status_id":null,"place":null,"in_reply_to_status_id_str":null,"coordinates":null,"created_at":"Wed Sep 05 01:01:01 +0000 1985","in_reply_to_screen_name":null,"favorited":false,"in_reply_to_user_id_str":null,"user":{"default_profile_image":false,"friends [...]
+2000
{"text":"sample tweet two","retweet_count":0,"in_reply_to_user_id":null,"retweeted":false,"truncated":false,"source":"href=\"http:\/\/sample.com\"","id_str":"2345678902","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_status_id":null,"place":null,"in_reply_to_status_id_str":null,"coordinates":null,"created_at":"Wed Sep 05 02:14:34 +0000 1985","in_reply_to_screen_name":null,"favorited":false,"in_reply_to_user_id_str":null,"user":{"default_profile_image":false,"friends [...]
\ No newline at end of file
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt
index b0f1a68ff..1da2966d4 100644
--- a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt
@@ -1,3 +1,3 @@
-BMW to Make Hybrid Sports Car
- By CHRISTOPH RAUWALD .
-LEIPZIG, Germany—German car maker BMW AG said Friday it will start series production of a new plug-in hybrid sports car in 2013, to be based on the Vision EfficientDynamics Concept car shown at the Frankfurt auto show in September last year. Chief Executive Norbert Reithofer said the car will be produced in Germany but didn't provide details on the price. The BMW Vision EfficientDynamics Concept car is a sporty plug-in, full hybrid with a turbo-diesel engine, four seats and upward-pivoti [...]
+BMW to Make Hybrid Sports Car
+ By CHRISTOPH RAUWALD .
+LEIPZIG, Germany—German car maker BMW AG said Friday it will start series production of a new plug-in hybrid sports car in 2013, to be based on the Vision EfficientDynamics Concept car shown at the Frankfurt auto show in September last year. Chief Executive Norbert Reithofer said the car will be produced in Germany but didn't provide details on the price. The BMW Vision EfficientDynamics Concept car is a sporty plug-in, full hybrid with a turbo-diesel engine, four seats and upward-pivoti [...]
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testRFC822 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testRFC822
index 22fbf15c4..9ce423a18 100644
--- a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testRFC822
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testRFC822
@@ -1,41 +1,41 @@
-From: "Julien Nioche (JIRA)" <ji...@apache.org>
-To: dev@tika.apache.org
-Subject: [jira] Commented: (TIKA-461) RFC822 messages not parsed
-Reply-To: dev@tika.apache.org
-Delivered-To: mailing list dev@tika.apache.org
-Date: Mon, 6 Sep 2010 05:25:34 -0400 (EDT)
-In-Reply-To: <60...@thor>
-MIME-Version: 1.0
-Content-Type: text/plain; charset=utf-8
-Content-Transfer-Encoding: 7bit
-X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
-X-Virus-Checked: Checked by ClamAV on apache.org
-
-
- [ https://issues.apache.org/jira/browse/TIKA-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906468#action_12906468 ]
-
-Julien Nioche commented on TIKA-461:
-------------------------------------
-
-I'll have a look at mime4j and try to use it in Tika
-
-> RFC822 messages not parsed
-> --------------------------
->
-> Key: TIKA-461
-> URL: https://issues.apache.org/jira/browse/TIKA-461
-> Project: Tika
-> Issue Type: Bug
-> Components: parser
-> Affects Versions: 0.7
-> Reporter: Joshua Turner
-> Assignee: Julien Nioche
->
-> Presented with an RFC822 message exported from Thunderbird, AutodetectParser produces an empty body, and a Metadata containing only one key-value pair: "Content-Type=message/rfc822". Directly calling MboxParser likewise gives an empty body, but with two metadata pairs: "Content-Encoding=us-ascii Content-Type=application/mbox".
-> A quick peek at the source of MboxParser shows that the implementation is pretty naive. If the wiring can be sorted out, something like Apache James' mime4j might be a better bet.
-
---
-This message is automatically generated by JIRA.
--
-You can reply to this email to add a comment to the issue online.
-
+From: "Julien Nioche (JIRA)" <ji...@apache.org>
+To: dev@tika.apache.org
+Subject: [jira] Commented: (TIKA-461) RFC822 messages not parsed
+Reply-To: dev@tika.apache.org
+Delivered-To: mailing list dev@tika.apache.org
+Date: Mon, 6 Sep 2010 05:25:34 -0400 (EDT)
+In-Reply-To: <60...@thor>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=utf-8
+Content-Transfer-Encoding: 7bit
+X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
+X-Virus-Checked: Checked by ClamAV on apache.org
+
+
+ [ https://issues.apache.org/jira/browse/TIKA-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906468#action_12906468 ]
+
+Julien Nioche commented on TIKA-461:
+------------------------------------
+
+I'll have a look at mime4j and try to use it in Tika
+
+> RFC822 messages not parsed
+> --------------------------
+>
+> Key: TIKA-461
+> URL: https://issues.apache.org/jira/browse/TIKA-461
+> Project: Tika
+> Issue Type: Bug
+> Components: parser
+> Affects Versions: 0.7
+> Reporter: Joshua Turner
+> Assignee: Julien Nioche
+>
+> Presented with an RFC822 message exported from Thunderbird, AutodetectParser produces an empty body, and a Metadata containing only one key-value pair: "Content-Type=message/rfc822". Directly calling MboxParser likewise gives an empty body, but with two metadata pairs: "Content-Encoding=us-ascii Content-Type=application/mbox".
+> A quick peek at the source of MboxParser shows that the implementation is pretty naive. If the wiring can be sorted out, something like Apache James' mime4j might be a better bet.
+
+--
+This message is automatically generated by JIRA.
+-
+You can reply to this email to add a comment to the issue online.
+