You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flume.apache.org by Eran Kutner <er...@gigya.com> on 2011/08/08 16:41:52 UTC
Collector going crazy...
Hi,
I have 3 collectors running, all with this configuration:
collectorSource(54001) | collector(600000) {
escapedFormatDfs("hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec")) }
Right now tall three are going crazy, opening multiple HDFS files per
second. All failing with this error: "OutputFormat instance can only write
to the same OutputStream".
This configuration used to work fine, for a few days, but I've seen this
wild behavior happen a few times before. Usually restarting the service was
enough to stop it. Now that doesn't seem to help.
The logs look like this:
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
'WEB514 [INFO Sun Aug 07 09:51:43 EDT 2011] { City : } { CookieID : } {
Country : US } { Date : (long)-721216654471344504 (string) '??????F #??'
(double)-2.2850700827524182E260 } { ErrCode : 0 '' } { Event : 637534208 '&'
} { GigyaUID : } { IP : 75.107.140.48 } { IPNum :
(long)5434591513138954240 (string) 'Kk???0' (double)2.110829276777375E55 }
{ Info : } { PartnerData : } { PartnerData_hash : (long)0 (string) ''
(double)0.0 } { PartnerID : (long)8768254487304339456 (string) 'y??? '
(double)1.3781410651737394E278 } { ProviderUID : } { Referrer :
http://www.heavy.com/action/girls/2011/08/the-20-hottest-photos-of-jessica-jane-clement/}
{ ReqID : d3b9c9f27a1848919003e25268b90c34 } { RetryCount : 0 '' } {
RowType : GSEvent } { Server : WEB514 } { SiteAccountID : } {
SiteAccountSNID : 0 '' } { State : } { Status : 50331648 ' ' } { TotalTime
: 0 '' } { Type : js } { UCID : tSUUKaA13PGc62n9R9xn4PbqBwHxfeeuV15smIXotZY=
} { UUID : } { UserAge : 48 } { UserGender : 0 '' } { Val : 0 '' } { dbTime
: 0 '' } { log : } { memcacheTime : 0 '' } { provider : 0 '' } { request :
} { siteID : (long)8919959604146733056 (string) '{??? '
(double)1.9842753626727537E288 } { source : 1315925605 'None' } ' with
error: OutputFormat instance can only write to the same OutputStream
11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink 'escapedFormatDfs(
"hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
11/08/08 10:33:12 INFO rolling.RollSink: opening RollSink
'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
11/08/08 10:33:12 INFO debug.InsistentOpenDecorator: Opened MaskDecorator on
try 0
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
'WEB204 [INFO Mon Aug 08 10:13:54 EDT 2011] { City : } { CookieID : } {
Country : DE } { Date : (long)6591305623858826888 (string) '[y??????#??'
(double)4.440113505901932E132 } { IP : 93.231.131.246 } { IPNum :
(long)6766522057732653056 (string) ']??????' (double)2.2940434972655672E144
} { Referrer : } { RowType : GSLoad } { Server : WEB204 } { State : } {
UCID : Jyg2gzYokIn3id2wDfv71Q== } { isHTTPS : 48 } { partnerID :
(long)-7147478690450898944 (string) '??????' (double)-6.42864763300399E-170
} { siteID : (long)5554354511524397056 (string) 'M'
(double)2.162938607572601E63 } ' with error: OutputFormat instance can only
write to the same OutputStream
11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink 'escapedFormatDfs(
"hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
"events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
-eran
Re: Collector going crazy...
Posted by Eran Kutner <er...@gigya.com>.
Great!
By the way, I've opened FLUME-734 about this issue, it has some more log
data there.
Thanks.
-eran
On Wed, Aug 10, 2011 at 16:55, Jonathan Hsieh <jo...@cloudera.com> wrote:
> Eran,
>
> I've dug into this and I think I've found the root cause of this bug. I'll
> try to patch this today.
>
> Jon.
>
>
> On Mon, Aug 8, 2011 at 7:41 AM, Eran Kutner <er...@gigya.com> wrote:
>
>> Hi,
>> I have 3 collectors running, all with this configuration:
>> collectorSource(54001) | collector(600000) {
>> escapedFormatDfs("hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
>> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec")) }
>>
>> Right now tall three are going crazy, opening multiple HDFS files per
>> second. All failing with this error: "OutputFormat instance can only write
>> to the same OutputStream".
>> This configuration used to work fine, for a few days, but I've seen this
>> wild behavior happen a few times before. Usually restarting the service was
>> enough to stop it. Now that doesn't seem to help.
>>
>> The logs look like this:
>>
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
>> 11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
>> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
>> 11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
>> 'WEB514 [INFO Sun Aug 07 09:51:43 EDT 2011] { City : } { CookieID : } {
>> Country : US } { Date : (long)-721216654471344504 (string) '??????F #??'
>> (double)-2.2850700827524182E260 } { ErrCode : 0 '' } { Event : 637534208 '&'
>> } { GigyaUID : } { IP : 75.107.140.48 } { IPNum :
>> (long)5434591513138954240 (string) 'Kk???0' (double)2.110829276777375E55 }
>> { Info : } { PartnerData : } { PartnerData_hash : (long)0 (string) ''
>> (double)0.0 } { PartnerID : (long)8768254487304339456 (string) 'y??? '
>> (double)1.3781410651737394E278 } { ProviderUID : } { Referrer :
>> http://www.heavy.com/action/girls/2011/08/the-20-hottest-photos-of-jessica-jane-clement/} { ReqID : d3b9c9f27a1848919003e25268b90c34 } { RetryCount : 0 '' } {
>> RowType : GSEvent } { Server : WEB514 } { SiteAccountID : } {
>> SiteAccountSNID : 0 '' } { State : } { Status : 50331648 ' ' } { TotalTime
>> : 0 '' } { Type : js } { UCID : tSUUKaA13PGc62n9R9xn4PbqBwHxfeeuV15smIXotZY=
>> } { UUID : } { UserAge : 48 } { UserGender : 0 '' } { Val : 0 '' } { dbTime
>> : 0 '' } { log : } { memcacheTime : 0 '' } { provider : 0 '' } { request :
>> } { siteID : (long)8919959604146733056 (string) '{??? '
>> (double)1.9842753626727537E288 } { source : 1315925605 'None' } ' with
>> error: OutputFormat instance can only write to the same OutputStream
>> 11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink
>> 'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
>> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
>> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
>> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
>> 11/08/08 10:33:12 INFO rolling.RollSink: opening RollSink
>> 'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
>> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
>> 11/08/08 10:33:12 INFO debug.InsistentOpenDecorator: Opened MaskDecorator
>> on try 0
>> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
>> 11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
>> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
>> 11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
>> 'WEB204 [INFO Mon Aug 08 10:13:54 EDT 2011] { City : } { CookieID : } {
>> Country : DE } { Date : (long)6591305623858826888 (string) '[y??????#??'
>> (double)4.440113505901932E132 } { IP : 93.231.131.246 } { IPNum :
>> (long)6766522057732653056 (string) ']??????' (double)2.2940434972655672E144
>> } { Referrer : } { RowType : GSLoad } { Server : WEB204 } { State : } {
>> UCID : Jyg2gzYokIn3id2wDfv71Q== } { isHTTPS : 48 } { partnerID :
>> (long)-7147478690450898944 (string) '??????' (double)-6.42864763300399E-170
>> } { siteID : (long)5554354511524397056 (string) 'M'
>> (double)2.162938607572601E63 } ' with error: OutputFormat instance can only
>> write to the same OutputStream
>> 11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink
>> 'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
>> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
>> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
>> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
>> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
>>
>>
>>
>>
>>
>> -eran
>>
>>
>
>
> --
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>
>
>
Re: Collector going crazy...
Posted by Jonathan Hsieh <jo...@cloudera.com>.
Eran,
I've dug into this and I think I've found the root cause of this bug. I'll
try to patch this today.
Jon.
On Mon, Aug 8, 2011 at 7:41 AM, Eran Kutner <er...@gigya.com> wrote:
> Hi,
> I have 3 collectors running, all with this configuration:
> collectorSource(54001) | collector(600000) {
> escapedFormatDfs("hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec")) }
>
> Right now tall three are going crazy, opening multiple HDFS files per
> second. All failing with this error: "OutputFormat instance can only write
> to the same OutputStream".
> This configuration used to work fine, for a few days, but I've seen this
> wild behavior happen a few times before. Usually restarting the service was
> enough to stop it. Now that doesn't seem to help.
>
> The logs look like this:
>
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
> 11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
> 11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
> 'WEB514 [INFO Sun Aug 07 09:51:43 EDT 2011] { City : } { CookieID : } {
> Country : US } { Date : (long)-721216654471344504 (string) '??????F #??'
> (double)-2.2850700827524182E260 } { ErrCode : 0 '' } { Event : 637534208 '&'
> } { GigyaUID : } { IP : 75.107.140.48 } { IPNum :
> (long)5434591513138954240 (string) 'Kk???0' (double)2.110829276777375E55 }
> { Info : } { PartnerData : } { PartnerData_hash : (long)0 (string) ''
> (double)0.0 } { PartnerID : (long)8768254487304339456 (string) 'y??? '
> (double)1.3781410651737394E278 } { ProviderUID : } { Referrer :
> http://www.heavy.com/action/girls/2011/08/the-20-hottest-photos-of-jessica-jane-clement/} { ReqID : d3b9c9f27a1848919003e25268b90c34 } { RetryCount : 0 '' } {
> RowType : GSEvent } { Server : WEB514 } { SiteAccountID : } {
> SiteAccountSNID : 0 '' } { State : } { Status : 50331648 ' ' } { TotalTime
> : 0 '' } { Type : js } { UCID : tSUUKaA13PGc62n9R9xn4PbqBwHxfeeuV15smIXotZY=
> } { UUID : } { UserAge : 48 } { UserGender : 0 '' } { Val : 0 '' } { dbTime
> : 0 '' } { log : } { memcacheTime : 0 '' } { provider : 0 '' } { request :
> } { siteID : (long)8919959604146733056 (string) '{??? '
> (double)1.9842753626727537E288 } { source : 1315925605 'None' } ' with
> error: OutputFormat instance can only write to the same OutputStream
> 11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink
> 'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312089-0400.5436285984294774.00000040-col1.snappy.tmp
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
> 11/08/08 10:33:12 INFO rolling.RollSink: opening RollSink
> 'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
> 11/08/08 10:33:12 INFO debug.InsistentOpenDecorator: Opened MaskDecorator
> on try 0
> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
> hdfs://hadoop1-m1:8020/raw-events/2011-08-07/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
> 11/08/08 10:33:12 INFO compress.CodecPool: Got brand-new compressor
> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Opening
> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Creating HDFS file:
> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
> 11/08/08 10:33:12 INFO debug.StubbornAppendSink: append failed on event
> 'WEB204 [INFO Mon Aug 08 10:13:54 EDT 2011] { City : } { CookieID : } {
> Country : DE } { Date : (long)6591305623858826888 (string) '[y??????#??'
> (double)4.440113505901932E132 } { IP : 93.231.131.246 } { IPNum :
> (long)6766522057732653056 (string) ']??????' (double)2.2940434972655672E144
> } { Referrer : } { RowType : GSLoad } { Server : WEB204 } { State : } {
> UCID : Jyg2gzYokIn3id2wDfv71Q== } { isHTTPS : 48 } { partnerID :
> (long)-7147478690450898944 (string) '??????' (double)-6.42864763300399E-170
> } { siteID : (long)5554354511524397056 (string) 'M'
> (double)2.162938607572601E63 } ' with error: OutputFormat instance can only
> write to the same OutputStream
> 11/08/08 10:33:12 INFO rolling.RollSink: closing RollSink
> 'escapedFormatDfs( "hdfs://hadoop1-m1:8020/raw-events/%Y-%m-%d/",
> "events-%{rolltag}-col1.snappy", seqfile("SnappyCodec") )'
> 11/08/08 10:33:12 INFO hdfs.EscapedCustomDfsSink: Closing
> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: Closing HDFS file:
> hdfs://hadoop1-m1:8020/raw-events/2011-08-08/events-20110808-103312478-0400.5436286373283774.00000040-col1.snappy.tmp
> 11/08/08 10:33:12 INFO hdfs.CustomDfsSink: done writing raw file to hdfs
>
>
>
>
>
> -eran
>
>
--
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com