You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flume.apache.org by hs...@apache.org on 2013/06/18 09:35:20 UTC
[4/9] FLUME-2070. Add a Flume Morphline Solr Sink.
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox
new file mode 100644
index 0000000..2aa4828
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/complex.mbox
@@ -0,0 +1,291 @@
+From core-user-return-14700-apmail-hadoop-core-user-archive=hadoop.apache.org@hadoop.apache.org Mon Jun 01 04:28:28 2009
+Return-Path: <co...@hadoop.apache.org>
+Delivered-To: apmail-hadoop-core-user-archive@www.apache.org
+Received: (qmail 19921 invoked from network); 1 Jun 2009 04:28:28 -0000
+Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
+ by minotaur.apache.org with SMTP; 1 Jun 2009 04:28:28 -0000
+Received: (qmail 84995 invoked by uid 500); 1 Jun 2009 04:28:38 -0000
+Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org
+Received: (qmail 84895 invoked by uid 500); 1 Jun 2009 04:28:38 -0000
+Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
+Precedence: bulk
+List-Help: <ma...@hadoop.apache.org>
+List-Unsubscribe: <ma...@hadoop.apache.org>
+List-Post: <ma...@hadoop.apache.org>
+List-Id: <core-user.hadoop.apache.org>
+Reply-To: core-user@hadoop.apache.org
+Delivered-To: mailing list core-user@hadoop.apache.org
+Received: (qmail 84885 invoked by uid 99); 1 Jun 2009 04:28:38 -0000
+Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
+ by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 04:28:38 +0000
+X-ASF-Spam-Status: No, hits=1.2 required=10.0
+ tests=SPF_NEUTRAL
+X-Spam-Check-By: apache.org
+Received-SPF: neutral (athena.apache.org: local policy)
+Received: from [69.147.107.21] (HELO mrout2-b.corp.re1.yahoo.com) (69.147.107.21)
+ by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 04:28:26 +0000
+Received: from SNV-EXPF01.ds.corp.yahoo.com (snv-expf01.ds.corp.yahoo.com [207.126.227.250])
+ by mrout2-b.corp.re1.yahoo.com (8.13.8/8.13.8/y.out) with ESMTP id n514QYA6099963
+ for <co...@hadoop.apache.org>; Sun, 31 May 2009 21:26:35 -0700 (PDT)
+DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
+ h=received:user-agent:date:subject:from:to:message-id:
+ thread-topic:thread-index:in-reply-to:mime-version:content-type:
+ content-transfer-encoding:x-originalarrivaltime;
+ b=YVtSNdgjeeSBS1yY3XDolul49i+HrgNG7QszMo9LzGnrwejjgsl5+iUM6EiQgEpV
+Received: from SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.9]) by SNV-EXPF01.ds.corp.yahoo.com with Microsoft SMTPSVC(6.0.3790.3959);
+ Sun, 31 May 2009 21:26:34 -0700
+Received: from 10.66.92.213 ([10.66.92.213]) by SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.58]) with Microsoft Exchange Server HTTP-DAV ;
+ Mon, 1 Jun 2009 04:26:33 +0000
+User-Agent: Microsoft-Entourage/12.17.0.090302
+Date: Mon, 01 Jun 2009 09:56:31 +0530
+Subject: Re: question about when shuffle/sort start working
+From: Jothi Padmanabhan <jo...@yahoo-inc.com>
+To: <co...@hadoop.apache.org>
+Message-ID: <C6...@yahoo-inc.com>
+Thread-Topic: question about when shuffle/sort start working
+Thread-Index: AcnicSNoBw19cMU8UEaXwAdZ1YYhuw==
+In-Reply-To: <44...@web111005.mail.gq1.yahoo.com>
+Mime-version: 1.0
+Content-type: text/plain;
+ charset="US-ASCII"
+Content-transfer-encoding: 7bit
+X-OriginalArrivalTime: 01 Jun 2009 04:26:34.0501 (UTC) FILETIME=[257EAB50:01C9E271]
+X-Virus-Checked: Checked by ClamAV on apache.org
+
+When a Mapper completes, MapCompletionEvents are generated. Reducers try to
+fetch map outputs for a given map only on the receipt of such events.
+
+Jothi
+
+
+On 5/30/09 10:00 AM, "Jianmin Woo" <ji...@yahoo.com> wrote:
+
+> Hi,
+> I am being confused by the protocol between mapper and reducer. When mapper
+> emitting the (key,value) pair done, is there any signal the mapper send out to
+> hadoop framework in protocol to indicate that map is done and the shuffle/sort
+> can begin for reducer? If there is no this signal in protocol, when the
+> framework begin the shuffle/sort?
+>
+> Thanks,
+> Jianmin
+>
+>
+>
+>
+
+
+From core-user-return-14701-apmail-hadoop-core-user-archive=hadoop.apache.org@hadoop.apache.org Mon Jun 01 05:31:14 2009
+Return-Path: <co...@hadoop.apache.org>
+Delivered-To: apmail-hadoop-core-user-archive@www.apache.org
+Received: (qmail 38243 invoked from network); 1 Jun 2009 05:31:14 -0000
+Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
+ by minotaur.apache.org with SMTP; 1 Jun 2009 05:31:14 -0000
+Received: (qmail 15621 invoked by uid 500); 1 Jun 2009 05:31:24 -0000
+Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org
+Received: (qmail 15557 invoked by uid 500); 1 Jun 2009 05:31:24 -0000
+Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
+Precedence: bulk
+List-Help: <ma...@hadoop.apache.org>
+List-Unsubscribe: <ma...@hadoop.apache.org>
+List-Post: <ma...@hadoop.apache.org>
+List-Id: <core-user.hadoop.apache.org>
+Reply-To: core-user@hadoop.apache.org
+Delivered-To: mailing list core-user@hadoop.apache.org
+Received: (qmail 15547 invoked by uid 99); 1 Jun 2009 05:31:24 -0000
+Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
+ by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 05:31:24 +0000
+X-ASF-Spam-Status: No, hits=2.2 required=10.0
+ tests=HTML_MESSAGE,SPF_PASS
+X-Spam-Check-By: apache.org
+Received-SPF: pass (nike.apache.org: local policy)
+Received: from [68.142.237.94] (HELO n9.bullet.re3.yahoo.com) (68.142.237.94)
+ by apache.org (qpsmtpd/0.29) with SMTP; Mon, 01 Jun 2009 05:31:11 +0000
+Received: from [68.142.237.88] by n9.bullet.re3.yahoo.com with NNFMP; 01 Jun 2009 05:30:50 -0000
+Received: from [67.195.9.82] by t4.bullet.re3.yahoo.com with NNFMP; 01 Jun 2009 05:30:49 -0000
+Received: from [67.195.9.99] by t2.bullet.mail.gq1.yahoo.com with NNFMP; 01 Jun 2009 05:30:49 -0000
+Received: from [127.0.0.1] by omp103.mail.gq1.yahoo.com with NNFMP; 01 Jun 2009 05:28:01 -0000
+X-Yahoo-Newman-Property: ymail-3
+X-Yahoo-Newman-Id: 796121.97519.bm@omp103.mail.gq1.yahoo.com
+Received: (qmail 35264 invoked by uid 60001); 1 Jun 2009 05:30:49 -0000
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1243834249; bh=R8qzdi/IbLyO8UwpnaujDpT9E+6bJ7nkmZN2803EmRk=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=vq4c6RIDbkuLPYd8mirusIXf6DqTb/IeT55In7W00Y5Sxx1ZiXBb78yE9+TDfXJ0elsEZvqv4ocyvolGE0eGtyYeJA0mZikpRNu6pidxPNpCplOcLHBRz7YQ7iERwv3TagRlWy2Xd3oD9ZeV0A05P7WUOiNNX1PUUJD1IVdrEZo=
+DomainKey-Signature:a=rsa-sha1; q=dns; c=nofws;
+ s=s1024; d=yahoo.com;
+ h=Message-ID:X-YMail-OSG:Received:X-Mailer:References:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type;
+ b=6HXZV98ON5vBwmE/xS8stVD0D2F4dkMY7a0suX5KVTb736JdR8G59mqBq/dWcpbFTLiCLtxi18LMb/dU1RKRGOEdn3l3j/jKXhBrhIgfg3qtNskPedXDKBvn7JGXiSkqpA/tUtPjvc0Uuk8/LaA01SQTz40Engg7nD8/EJdIAhA=;
+Message-ID: <59...@web111010.mail.gq1.yahoo.com>
+X-YMail-OSG: KzhhrJYVM1m.MCS6vRpRP2ZZO2PrfnbngosELDCIa91ZqvhJph4RdmzfUW0jw9W04RCSch1K730bPohwNpNBIk2QR_zt4_mfbhfq7YEPkSoz9LSXG90P9vIo5Fc8qyZN0U6vA9gtdyGQTpN5ahvillUH9nAF0TMWv2SvZJLjPlQ0Z0p8oK8ltBwGTgLrM8Jtdn9D29yoRyi3_EpVOfdD9OP.EK50Vr1XwSUYMbnpZ0WGHMwd.Yig7A6Elwadm3YVbfOdx2mfrG.jQsUAxQjRBNvbrOM57.FaE11kHTe9aoBWSeihNg--
+Received: from [216.145.54.7] by web111010.mail.gq1.yahoo.com via HTTP; Sun, 31 May 2009 22:30:49 PDT
+X-Mailer: YahooMailRC/1277.43 YahooMailWebService/0.7.289.10
+References: <C6...@yahoo-inc.com>
+Date: Sun, 31 May 2009 22:30:49 -0700 (PDT)
+From: Jianmin Woo <ji...@yahoo.com>
+Subject: Re: question about when shuffle/sort start working
+To: core-user@hadoop.apache.org
+In-Reply-To: <C6...@yahoo-inc.com>
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="0-1193839393-1243834249=:35091"
+X-Virus-Checked: Checked by ClamAV on apache.org
+
+--0-1193839393-1243834249=:35091
+Content-Type: text/plain; charset=us-ascii
+
+Thanks a lot for your explanation, Jothi.
+
+So is this event generated by hadoop framework? Is there any API in mapper to fire this event? Actually, I am thinking to implement a mapper that will emit some <key, value> pairs, then fire this event to let the reducer works, the same mapper task then emit some other <key, value> pairs and repeat. Do you think is this logic feasible by current API?
+
+Thanks,
+Jianmin
+
+
+
+
+
+________________________________
+From: Jothi Padmanabhan <jo...@yahoo-inc.com>
+To: core-user@hadoop.apache.org
+Sent: Monday, June 1, 2009 12:26:31 PM
+Subject: Re: question about when shuffle/sort start working
+
+When a Mapper completes, MapCompletionEvents are generated. Reducers try to
+fetch map outputs for a given map only on the receipt of such events.
+
+Jothi
+
+
+On 5/30/09 10:00 AM, "Jianmin Woo" <ji...@yahoo.com> wrote:
+
+> Hi,
+> I am being confused by the protocol between mapper and reducer. When mapper
+> emitting the (key,value) pair done, is there any signal the mapper send out to
+> hadoop framework in protocol to indicate that map is done and the shuffle/sort
+> can begin for reducer? If there is no this signal in protocol, when the
+> framework begin the shuffle/sort?
+>
+> Thanks,
+> Jianmin
+>
+>
+>
+>
+
+
+
+--0-1193839393-1243834249=:35091--
+
+
+From core-user-return-14702-apmail-hadoop-core-user-archive=hadoop.apache.org@hadoop.apache.org Mon Jun 01 06:04:30 2009
+Return-Path: <co...@hadoop.apache.org>
+Delivered-To: apmail-hadoop-core-user-archive@www.apache.org
+Received: (qmail 53387 invoked from network); 1 Jun 2009 06:04:29 -0000
+Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3)
+ by minotaur.apache.org with SMTP; 1 Jun 2009 06:04:29 -0000
+Received: (qmail 39066 invoked by uid 500); 1 Jun 2009 06:04:39 -0000
+Delivered-To: apmail-hadoop-core-user-archive@hadoop.apache.org
+Received: (qmail 38970 invoked by uid 500); 1 Jun 2009 06:04:39 -0000
+Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
+Precedence: bulk
+List-Help: <ma...@hadoop.apache.org>
+List-Unsubscribe: <ma...@hadoop.apache.org>
+List-Post: <ma...@hadoop.apache.org>
+List-Id: <core-user.hadoop.apache.org>
+Reply-To: core-user@hadoop.apache.org
+Delivered-To: mailing list core-user@hadoop.apache.org
+Received: (qmail 38955 invoked by uid 99); 1 Jun 2009 06:04:39 -0000
+Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
+ by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 06:04:39 +0000
+X-ASF-Spam-Status: No, hits=1.2 required=10.0
+ tests=SPF_NEUTRAL
+X-Spam-Check-By: apache.org
+Received-SPF: neutral (athena.apache.org: local policy)
+Received: from [216.145.54.172] (HELO mrout2.yahoo.com) (216.145.54.172)
+ by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 01 Jun 2009 06:04:28 +0000
+Received: from SNV-EXBH01.ds.corp.yahoo.com (snv-exbh01.ds.corp.yahoo.com [207.126.227.249])
+ by mrout2.yahoo.com (8.13.6/8.13.6/y.out) with ESMTP id n5163FGq038852
+ for <co...@hadoop.apache.org>; Sun, 31 May 2009 23:03:15 -0700 (PDT)
+DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
+ h=received:user-agent:date:subject:from:to:message-id:
+ thread-topic:thread-index:in-reply-to:mime-version:content-type:
+ content-transfer-encoding:x-originalarrivaltime;
+ b=rChE4SCnwtWaZpjhovkiXDKfDiVNdRRvsadSGG9S9bgvOexn/9/5JjEQx1pOR7Nb
+Received: from SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.9]) by SNV-EXBH01.ds.corp.yahoo.com with Microsoft SMTPSVC(6.0.3790.3959);
+ Sun, 31 May 2009 23:03:15 -0700
+Received: from 10.66.92.213 ([10.66.92.213]) by SNV-EXVS08.ds.corp.yahoo.com ([207.126.227.58]) with Microsoft Exchange Server HTTP-DAV ;
+ Mon, 1 Jun 2009 06:03:15 +0000
+User-Agent: Microsoft-Entourage/12.17.0.090302
+Date: Mon, 01 Jun 2009 11:33:13 +0530
+Subject: Re: question about when shuffle/sort start working
+From: Jothi Padmanabhan <jo...@yahoo-inc.com>
+To: <co...@hadoop.apache.org>
+Message-ID: <C6...@yahoo-inc.com>
+Thread-Topic: question about when shuffle/sort start working
+Thread-Index: AcnifqWrLG6N7GAk7kqy9QalVWfegQ==
+In-Reply-To: <59...@web111010.mail.gq1.yahoo.com>
+Mime-version: 1.0
+Content-type: text/plain;
+ charset="US-ASCII"
+Content-transfer-encoding: 7bit
+X-OriginalArrivalTime: 01 Jun 2009 06:03:15.0462 (UTC) FILETIME=[A7231260:01C9E27E]
+X-Virus-Checked: Checked by ClamAV on apache.org
+
+
+No you cannot raise this event yourself, this event is generated internally
+by the framework.
+
+I am guessing that what you probably want is to have a chain of MapReduce
+Jobs where the output of one is automatically fed as input to another. You
+can look at these classes: JobControl and ChainMapper/ChainReducer.
+
+Jothi
+
+On 6/1/09 11:00 AM, "Jianmin Woo" <ji...@yahoo.com> wrote:
+
+> Thanks a lot for your explanation, Jothi.
+>
+> So is this event generated by hadoop framework? Is there any API in mapper to
+> fire this event? Actually, I am thinking to implement a mapper that will emit
+> some <key, value> pairs, then fire this event to let the reducer works, the
+> same mapper task then emit some other <key, value> pairs and repeat. Do you
+> think is this logic feasible by current API?
+>
+> Thanks,
+> Jianmin
+>
+>
+>
+>
+>
+> ________________________________
+> From: Jothi Padmanabhan <jo...@yahoo-inc.com>
+> To: core-user@hadoop.apache.org
+> Sent: Monday, June 1, 2009 12:26:31 PM
+> Subject: Re: question about when shuffle/sort start working
+>
+> When a Mapper completes, MapCompletionEvents are generated. Reducers try to
+> fetch map outputs for a given map only on the receipt of such events.
+>
+> Jothi
+>
+>
+> On 5/30/09 10:00 AM, "Jianmin Woo" <ji...@yahoo.com> wrote:
+>
+>> Hi,
+>> I am being confused by the protocol between mapper and reducer. When mapper
+>> emitting the (key,value) pair done, is there any signal the mapper send out
+>> to
+>> hadoop framework in protocol to indicate that map is done and the
+>> shuffle/sort
+>> can begin for reducer? If there is no this signal in protocol, when the
+>> framework begin the shuffle/sort?
+>>
+>> Thanks,
+>> Jianmin
+>>
+>>
+>>
+>>
+>
+>
+>
+
+
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log
new file mode 100644
index 0000000..633a15c
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-sessions.log
@@ -0,0 +1,9 @@
+Started GET /foo
+ Foo Started GET as HTML
+Completed 401 Unauthorized in 0ms
+
+
+Started GET /bar
+ Bar as HTML
+Completed 200 OK in 339ms
+Started GET /baz
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log
new file mode 100644
index 0000000..419f799
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace-expected-long-event.log
@@ -0,0 +1,25 @@
+Grave: Timer task com.base2services.jenkins.SqsQueueHandler@32eea79d failed
+com.amazonaws.AmazonClientException: Unable to calculate a request signature: Unable to calculate a request signature: Empty key
+ at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:71)
+ at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:55)
+ at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:83)
+ at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:46)
+ at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:238)
+ at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170)
+ at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:776)
+ at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:564)
+ at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:732)
+ at com.base2services.jenkins.SqsProfile.createQueue(SqsProfile.java:72)
+ at com.base2services.jenkins.SqsProfile.getQueueUrl(SqsProfile.java:62)
+ at com.base2services.jenkins.SqsQueueHandler.doRun(SqsQueueHandler.java:37)
+ at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
+ at java.util.TimerThread.mainLoop(Timer.java:555)
+ at java.util.TimerThread.run(Timer.java:505)
+Caused by: com.amazonaws.AmazonClientException: Unable to calculate a request signature: Empty key
+ at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:90)
+ at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:68)
+ ... 14 more
+Caused by: java.lang.IllegalArgumentException: Empty key
+ at javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:96)
+ at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:87)
+ ... 15 more
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log
new file mode 100644
index 0000000..b860012
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/multiline-stacktrace.log
@@ -0,0 +1,30 @@
+juil. 25, 2012 10:49:46 AM hudson.triggers.SafeTimerTask run
+Grave: Timer task com.base2services.jenkins.SqsQueueHandler@32eea79d failed
+com.amazonaws.AmazonClientException: Unable to calculate a request signature: Unable to calculate a request signature: Empty key
+ at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:71)
+ at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:55)
+ at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:83)
+ at com.amazonaws.auth.QueryStringSigner.sign(QueryStringSigner.java:46)
+ at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:238)
+ at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170)
+ at com.amazonaws.services.sqs.AmazonSQSClient.invoke(AmazonSQSClient.java:776)
+ at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:564)
+ at com.amazonaws.services.sqs.AmazonSQSClient.listQueues(AmazonSQSClient.java:732)
+ at com.base2services.jenkins.SqsProfile.createQueue(SqsProfile.java:72)
+ at com.base2services.jenkins.SqsProfile.getQueueUrl(SqsProfile.java:62)
+ at com.base2services.jenkins.SqsQueueHandler.doRun(SqsQueueHandler.java:37)
+ at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:54)
+ at java.util.TimerThread.mainLoop(Timer.java:555)
+ at java.util.TimerThread.run(Timer.java:505)
+Caused by: com.amazonaws.AmazonClientException: Unable to calculate a request signature: Empty key
+ at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:90)
+ at com.amazonaws.auth.AbstractAWSSigner.signAndBase64Encode(AbstractAWSSigner.java:68)
+ ... 14 more
+Caused by: java.lang.IllegalArgumentException: Empty key
+ at javax.crypto.spec.SecretKeySpec.<init>(SecretKeySpec.java:96)
+ at com.amazonaws.auth.AbstractAWSSigner.sign(AbstractAWSSigner.java:87)
+ ... 15 more
+
+
+juil. 25, 2012 10:49:54 AM hudson.slaves.SlaveComputer tryReconnect
+Infos: Attempting to reconnect CentosVagrant
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz
new file mode 100644
index 0000000..e5c0f7a
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/non-length-delimited-20130430-234145-tweets.json.gz differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss
new file mode 100644
index 0000000..758f6a1
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/rsstest.rss
@@ -0,0 +1,36 @@
+<?xml version="1.0" encoding="ISO-8859-1" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<rss version="0.91">
+ <channel>
+ <title>TestChannel</title>
+ <link>http://test.channel.com/</link>
+ <description>Sample RSS File for Junit test</description>
+ <language>en-us</language>
+
+ <item>
+ <title>Home Page of Chris Mattmann</title>
+ <link>http://www-scf.usc.edu/~mattmann/</link>
+ <description>Chris Mattmann's home page</description>
+ </item>
+ <item>
+ <title>Awesome Open Source Search Engine</title>
+ <link>http://www.nutch.org/</link>
+ <description>Yup, that's what it is</description>
+ </item>
+ </channel>
+</rss>
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433
new file mode 100644
index 0000000..380ab6e
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433
@@ -0,0 +1,4 @@
+1000
+{"text":"sample tweet one","retweet_count":0,"in_reply_to_user_id":null,"retweeted":false,"truncated":false,"source":"href=\"http:\/\/sample.com\"","id_str":"1234567891","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_status_id":null,"place":null,"in_reply_to_status_id_str":null,"coordinates":null,"created_at":"Wed Sep 05 01:01:01 +0000 1985","in_reply_to_screen_name":null,"favorited":false,"in_reply_to_user_id_str":null,"user":{"default_profile_image":false,"friends_count":111,"profile_background_color":"3C0C29","location":"Palo Alto","is_translator":false,"profile_background_tile":true,"favourites_count":11,"verified":false,"profile_sidebar_fill_color":"efefef","follow_request_sent":null,"contributors_enabled":false,"description":"desc1","profile_sidebar_border_color":"eeeeee","profile_image_url_https":"https:\/\/si0.twimg.com\/profile_images\/1\/normal.jpg","id_str":"1111111","listed_count":1,"lang":"en","screen_name":"fake_user1","show_all_inline_media":fals
e,"profile_use_background_image":true,"profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/1111111\/normal.jpg","default_profile":false,"statuses_count":11111,"created_at":"Thu Apr 07 11:04:54 +0000 1985","profile_text_color":"333333","followers_count":111,"protected":false,"following":null,"notifications":null,"profile_background_image_url":"http:\/\/a0.twimg.com\/images\/themes\/theme1\/bg.gif","time_zone":null,"url":null,"name":"name1","geo_enabled":false,"profile_link_color":"009999","id":1111112,"profile_background_image_url_https":"https:\/\/si0.twimg.com\/images\/themes\/theme1\/bg.gif","utc_offset":null},"id":11111112,"contributors":null,"geo":null}
+2000
+{"text":"sample tweet two","retweet_count":0,"in_reply_to_user_id":null,"retweeted":false,"truncated":false,"source":"href=\"http:\/\/sample.com\"","id_str":"2345678902","entities":{"user_mentions":[],"hashtags":[],"urls":[]},"in_reply_to_status_id":null,"place":null,"in_reply_to_status_id_str":null,"coordinates":null,"created_at":"Wed Sep 05 02:14:34 +0000 1985","in_reply_to_screen_name":null,"favorited":false,"in_reply_to_user_id_str":null,"user":{"default_profile_image":false,"friends_count":222,"profile_background_color":"3C0C29","location":"San Francisco","is_translator":false,"profile_background_tile":false,"favourites_count":22,"verified":false,"profile_sidebar_fill_color":"B2D948","follow_request_sent":null,"contributors_enabled":false,"description":"desc2","profile_sidebar_border_color":"8EC63D","profile_image_url_https":"https:\/\/si0.twimg.com\/profile_images\/22222222\/image_normal.jpg","id_str":"2222222","listed_count":0,"lang":"en","screen_name":"fake_user2","show_all_
inline_media":false,"profile_use_background_image":true,"profile_image_url":"http:\/\/a0.twimg.com\/profile_images\/2222222\/image_normal.jpg","default_profile":false,"statuses_count":222222,"created_at":"Thu Aug 04 11:33:28 +0000 1985","profile_text_color":"444444","followers_count":222,"protected":false,"following":null,"notifications":null,"profile_background_image_url":"http:\/\/a0.twimg.com\/profile_background_images\/222222\/222222.jpg","time_zone":"Central Time (US & Canada)","url":null,"name":"name2","geo_enabled":false,"profile_link_color":"9A0057","id":2222223,"profile_background_image_url_https":"https:\/\/si0.twimg.com\/profile_background_images\/2222222\/22222.jpg","utc_offset":-21600},"id":222223,"contributors":null,"geo":null}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro
new file mode 100644
index 0000000..900507c
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-medium.avro differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc
new file mode 100644
index 0000000..cd64717
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433-subschema.avsc
@@ -0,0 +1,12 @@
+{
+ "type" : "record",
+ "name" : "Doc",
+ "doc" : "adoc",
+ "fields" : [ {
+ "name" : "id",
+ "type" : "string"
+ }, {
+ "name" : "text",
+ "type" : [ "string", "null" ]
+ } ]
+}
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro
new file mode 100644
index 0000000..4dbf180
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avro differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc
new file mode 100644
index 0000000..9e4529f
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.avsc
@@ -0,0 +1,57 @@
+{
+ "type" : "record",
+ "name" : "Doc",
+ "doc" : "adoc",
+ "fields" : [ {
+ "name" : "id",
+ "type" : "string"
+ }, {
+ "name" : "user_friends_count",
+ "type" : [ "int", "null" ]
+ }, {
+ "name" : "user_location",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "user_description",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "user_statuses_count",
+ "type" : [ "int", "null" ]
+ }, {
+ "name" : "user_followers_count",
+ "type" : [ "int", "null" ]
+ }, {
+ "name" : "user_name",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "user_screen_name",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "created_at",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "text",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "retweet_count",
+ "type" : [ "int", "null" ]
+ }, {
+ "name" : "retweeted",
+ "type" : [ "boolean", "null" ]
+ }, {
+ "name" : "in_reply_to_user_id",
+ "type" : [ "long", "null" ]
+ }, {
+ "name" : "source",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "in_reply_to_status_id",
+ "type" : [ "long", "null" ]
+ }, {
+ "name" : "media_url_https",
+ "type" : [ "string", "null" ]
+ }, {
+ "name" : "expanded_url",
+ "type" : [ "string", "null" ]
+ } ]
+}
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2
new file mode 100644
index 0000000..a4a9159
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.bz2 differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz
new file mode 100644
index 0000000..3e7a44c
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/sample-statuses-20120906-141433.gz differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z
new file mode 100644
index 0000000..94d62d3
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.7z differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio
new file mode 100644
index 0000000..c13a1cb
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.cpio differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar
new file mode 100644
index 0000000..3076a58
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tar differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2 b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2
new file mode 100644
index 0000000..21488d3
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tbz2 differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz
new file mode 100644
index 0000000..baca6bb
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.tgz differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip
new file mode 100644
index 0000000..27d600d
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-documents.zip differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg
new file mode 100644
index 0000000..c975c0c
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-outlook.msg differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip
new file mode 100644
index 0000000..f6b3edc
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/test-zip-of-zip.zip differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif
new file mode 100644
index 0000000..97eac1d
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testAIFF.aif differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp
new file mode 100644
index 0000000..c017615
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMP.bmp differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt
new file mode 100644
index 0000000..b0f1a68
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testBMPfp.txt
@@ -0,0 +1,3 @@
+BMW to Make Hybrid Sports Car
+ By CHRISTOPH RAUWALD .
+LEIPZIG, Germany—German car maker BMW AG said Friday it will start series production of a new plug-in hybrid sports car in 2013, to be based on the Vision EfficientDynamics Concept car shown at the Frankfurt auto show in September last year. Chief Executive Norbert Reithofer said the car will be produced in Germany but didn't provide details on the price. The BMW Vision EfficientDynamics Concept car is a sporty plug-in, full hybrid with a turbo-diesel engine, four seats and upward-pivoting doors. BMW executive board member Klaus Draeger told reporters he expects to achieve "a significant sales volume" with the new high-performance sports car. Asked whether annual sales could exceed 1,000 vehicles, Mr. Draeger said, "You said this and I'm not saying this is wrong." In March, Mr. Reithofer indicated that the concept car was set to make it into series production. "I like the car. And you know what it means when I say I like the car—it means I will drive it. It's not just a concept
car," he told analysts during a presentation in Munich. The car will be designed for sale in all major global markets, which according to Mr. Draeger might require offering a gasoline engine instead of the prototype's three-cylinder diesel engine. Diesel cars account for roughly half of the European market, but are significantly less popular in the U.S. and hardly present at all in China. Mr. Draeger declined to comment on the vehicle's price tag, but noted that in order to achieve substantial sales volumes the price mustn't be too high. He said the same goes for BMW's planned Megacity Vehicle. A price tag of €60,000 ($85,242) or more would certainly limit potential sales volumes, he said.
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita
new file mode 100644
index 0000000..b68da7b
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testDITA.dita
@@ -0,0 +1,34 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE task PUBLIC "-//OASIS//DTD DITA Task//EN" "task.dtd">
+<task id="apache-tika">
+ <title>Apache Tika</title>
+ <shortdesc>Apache Tika - a content analysis toolkit.</shortdesc>
+ <prolog>
+ <author>Apache Software Foundation</author>
+ <copyright>
+ <copyryear year="2011"/>
+ <copyrholder>Apache Software Foundation</copyrholder>
+ </copyright>
+ <metadata>
+ <audience experiencelevel="expert" job="Customizing" type="Coder"/>
+ <category>Metadata</category>
+ <keywords>
+ <keyword>Tika</keyword>
+ <keyword>Content</keyword>
+ </keywords>
+ <prodinfo>
+ <prodname>Apache Tika</prodname>
+ <vrmlist>
+ <vrm version="1.x" release="Final" modification="2011/11/11"/>
+ </vrmlist>
+ </prodinfo>
+ </metadata>
+ </prolog>
+ <taskbody>
+ <context>
+ <p>The Apache Tika toolkit detects and extracts metadata and structured text content from various documents using existing parser libraries. You can find the latest release on the download page. See the Getting Started guide for instructions on how to start using Tika.</p>
+
+ <p>Tika is a project of the Apache Software Foundation, and was formerly a subproject of Apache Lucene.</p>
+ </context>
+ </taskbody>
+</task>
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx
new file mode 100644
index 0000000..d9a7126
--- /dev/null
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEMLX.emlx
@@ -0,0 +1,55 @@
+1795
+From: "Julien Nioche (JIRA)" <ji...@apache.org>
+To: dev@tika.apache.org
+Subject: [jira] Commented: (TIKA-461) RFC822 messages not parsed
+Reply-To: dev@tika.apache.org
+Delivered-To: mailing list dev@tika.apache.org
+Date: Mon, 6 Sep 2010 05:25:34 -0400 (EDT)
+In-Reply-To: <60...@thor>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=utf-8
+Content-Transfer-Encoding: 7bit
+X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
+X-Virus-Checked: Checked by ClamAV on apache.org
+
+
+ [ https://issues.apache.org/jira/browse/TIKA-461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12906468#action_12906468 ]
+
+Julien Nioche commented on TIKA-461:
+------------------------------------
+
+I'll have a look at mime4j and try to use it in Tika
+
+> RFC822 messages not parsed
+> --------------------------
+>
+> Key: TIKA-461
+> URL: https://issues.apache.org/jira/browse/TIKA-461
+> Project: Tika
+> Issue Type: Bug
+> Components: parser
+> Affects Versions: 0.7
+> Reporter: Joshua Turner
+> Assignee: Julien Nioche
+>
+> Presented with an RFC822 message exported from Thunderbird, AutodetectParser produces an empty body, and a Metadata containing only one key-value pair: "Content-Type=message/rfc822". Directly calling MboxParser likewise gives an empty body, but with two metadata pairs: "Content-Encoding=us-ascii Content-Type=application/mbox".
+> A quick peek at the source of MboxParser shows that the implementation is pretty naive. If the wiring can be sorted out, something like Apache James' mime4j might be a better bet.
+
+--
+This message is automatically generated by JIRA.
+-
+You can reply to this email to add a comment to the issue online.
+
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+ <key>flags</key>
+ <integer>0</integer>
+ <key>sender</key>
+ <string>"Julien Nioche (JIRA)" <jira@apache.org></string>
+ <key>subject</key>
+ <string>[jira] Commented: (TIKA-461) RFC822 messages not parsed</string>
+ <key>to</key>
+ <string>dev@tika.apache.org</string></dict>
+</plist>
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls
new file mode 100644
index 0000000..86b2916
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xls differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx
new file mode 100644
index 0000000..8d5169f
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testEXCEL.xlsx differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac
new file mode 100644
index 0000000..ccec947
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.flac differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga
new file mode 100644
index 0000000..37a1247
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLAC.oga differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv
new file mode 100644
index 0000000..d35e9bb
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testFLV.flv differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif
new file mode 100644
index 0000000..e09e641
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testGIF.gif differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar
new file mode 100644
index 0000000..4677a62
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJAR.jar differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg
new file mode 100644
index 0000000..1b93e77
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz
new file mode 100644
index 0000000..2ee8e9c
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.gz differ
http://git-wip-us.apache.org/repos/asf/flume/blob/cf629841/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz
----------------------------------------------------------------------
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz
new file mode 100644
index 0000000..3f35102
Binary files /dev/null and b/flume-ng-sinks/flume-ng-morphline-solr-sink/src/test/resources/test-documents/testJPEG_EXIF.jpg.tar.gz differ