You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2013/12/04 04:56:58 UTC
issue about read file from HDFS
hi,mailist:
when HDFS 's file is in appending ,no other reader can get
data from this file,so when i do some statistics action every five minuters
use hive read hdfs file ,i can not read data ,any good way can offer? thanks
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
Ok, so you go for yourself.
On Wed, Dec 4, 2013 at 12:15 PM, ch huang <ju...@gmail.com> wrote:
> it seems not a good suggestion,get lot of partition dir and data file will
> be a big compact to NN
>
>
> On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> One suggestion is change your hive partition, add a hive partition every
>> five minutes, and your HDFS file also roller every five minutes.
>>
>>
>> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,mailist:
>>> when HDFS 's file is in appending ,no other reader can
>>> get data from this file,so when i do some statistics action every five
>>> minuters use hive read hdfs file ,i can not read data ,any good way can
>>> offer? thanks
>>>
>>
>>
>
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
Ok, so you go for yourself.
On Wed, Dec 4, 2013 at 12:15 PM, ch huang <ju...@gmail.com> wrote:
> it seems not a good suggestion,get lot of partition dir and data file will
> be a big compact to NN
>
>
> On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> One suggestion is change your hive partition, add a hive partition every
>> five minutes, and your HDFS file also roller every five minutes.
>>
>>
>> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,mailist:
>>> when HDFS 's file is in appending ,no other reader can
>>> get data from this file,so when i do some statistics action every five
>>> minuters use hive read hdfs file ,i can not read data ,any good way can
>>> offer? thanks
>>>
>>
>>
>
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
Ok, so you go for yourself.
On Wed, Dec 4, 2013 at 12:15 PM, ch huang <ju...@gmail.com> wrote:
> it seems not a good suggestion,get lot of partition dir and data file will
> be a big compact to NN
>
>
> On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> One suggestion is change your hive partition, add a hive partition every
>> five minutes, and your HDFS file also roller every five minutes.
>>
>>
>> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,mailist:
>>> when HDFS 's file is in appending ,no other reader can
>>> get data from this file,so when i do some statistics action every five
>>> minuters use hive read hdfs file ,i can not read data ,any good way can
>>> offer? thanks
>>>
>>
>>
>
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
Ok, so you go for yourself.
On Wed, Dec 4, 2013 at 12:15 PM, ch huang <ju...@gmail.com> wrote:
> it seems not a good suggestion,get lot of partition dir and data file will
> be a big compact to NN
>
>
> On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> One suggestion is change your hive partition, add a hive partition every
>> five minutes, and your HDFS file also roller every five minutes.
>>
>>
>> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,mailist:
>>> when HDFS 's file is in appending ,no other reader can
>>> get data from this file,so when i do some statistics action every five
>>> minuters use hive read hdfs file ,i can not read data ,any good way can
>>> offer? thanks
>>>
>>
>>
>
Re: issue about read file from HDFS
Posted by ch huang <ju...@gmail.com>.
it seems not a good suggestion,get lot of partition dir and data file will
be a big compact to NN
On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
> One suggestion is change your hive partition, add a hive partition every
> five minutes, and your HDFS file also roller every five minutes.
>
>
> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>
>> hi,mailist:
>> when HDFS 's file is in appending ,no other reader can get
>> data from this file,so when i do some statistics action every five minuters
>> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>>
>
>
Re: issue about read file from HDFS
Posted by ch huang <ju...@gmail.com>.
it seems not a good suggestion,get lot of partition dir and data file will
be a big compact to NN
On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
> One suggestion is change your hive partition, add a hive partition every
> five minutes, and your HDFS file also roller every five minutes.
>
>
> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>
>> hi,mailist:
>> when HDFS 's file is in appending ,no other reader can get
>> data from this file,so when i do some statistics action every five minuters
>> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>>
>
>
Re: issue about read file from HDFS
Posted by ch huang <ju...@gmail.com>.
it seems not a good suggestion,get lot of partition dir and data file will
be a big compact to NN
On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
> One suggestion is change your hive partition, add a hive partition every
> five minutes, and your HDFS file also roller every five minutes.
>
>
> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>
>> hi,mailist:
>> when HDFS 's file is in appending ,no other reader can get
>> data from this file,so when i do some statistics action every five minuters
>> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>>
>
>
Re: issue about read file from HDFS
Posted by ch huang <ju...@gmail.com>.
it seems not a good suggestion,get lot of partition dir and data file will
be a big compact to NN
On Wed, Dec 4, 2013 at 12:08 PM, Azuryy Yu <az...@gmail.com> wrote:
> One suggestion is change your hive partition, add a hive partition every
> five minutes, and your HDFS file also roller every five minutes.
>
>
> On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
>
>> hi,mailist:
>> when HDFS 's file is in appending ,no other reader can get
>> data from this file,so when i do some statistics action every five minuters
>> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>>
>
>
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
One suggestion is change your hive partition, add a hive partition every
five minutes, and your HDFS file also roller every five minutes.
On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
> when HDFS 's file is in appending ,no other reader can get
> data from this file,so when i do some statistics action every five minuters
> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
One suggestion is change your hive partition, add a hive partition every
five minutes, and your HDFS file also roller every five minutes.
On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
> when HDFS 's file is in appending ,no other reader can get
> data from this file,so when i do some statistics action every five minuters
> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
One suggestion is change your hive partition, add a hive partition every
five minutes, and your HDFS file also roller every five minutes.
On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
> when HDFS 's file is in appending ,no other reader can get
> data from this file,so when i do some statistics action every five minuters
> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>
Re: issue about read file from HDFS
Posted by Azuryy Yu <az...@gmail.com>.
One suggestion is change your hive partition, add a hive partition every
five minutes, and your HDFS file also roller every five minutes.
On Wed, Dec 4, 2013 at 11:56 AM, ch huang <ju...@gmail.com> wrote:
> hi,mailist:
> when HDFS 's file is in appending ,no other reader can get
> data from this file,so when i do some statistics action every five minuters
> use hive read hdfs file ,i can not read data ,any good way can offer? thanks
>