You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Rex X <dn...@gmail.com> on 2016/01/22 06:30:18 UTC
What is the best way to locate the offset and length of all fields in
a Hadoop sequential text file?
The given sequential files correspond to an external Hive table.
They are stored in
/tableName/part-00000
/tableName/part-00001
...
There are about 2000 attributes in the table. Now I want to process the
data using Hadoop streaming and mapReduce. The first step is to find the
offset and length for each attribute.
What is the best way to get this information?
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Rex X <dn...@gmail.com>.
Hi LLoyd,
The metadata of this table are STRINGs, BIGINTs, and TINYINTs, 2000
attributes in total.
I need to transform the data that cannot be done with built-in functions of
Hive.
Thank you.
Best,
Rex
On Fri, Jan 22, 2016 at 1:58 AM, Namikaze Minato <ll...@gmail.com>
wrote:
> Hello. We don't have any information about your data.
>
> I don't think we can help you with this. Also, I cannot understand what
> you are trying to achieve. Please also tell us why you are using hadoop
> streaming instead of hive to do your operations.
>
> Regards,
> LLoyd
>
> On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
>
>> The given sequential files correspond to an external Hive table.
>>
>> They are stored in
>> /tableName/part-00000
>> /tableName/part-00001
>> ...
>>
>> There are about 2000 attributes in the table. Now I want to process the
>> data using Hadoop streaming and mapReduce. The first step is to find the
>> offset and length for each attribute.
>>
>> What is the best way to get this information?
>>
>>
>>
>
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Rex X <dn...@gmail.com>.
Hi LLoyd,
The metadata of this table are STRINGs, BIGINTs, and TINYINTs, 2000
attributes in total.
I need to transform the data that cannot be done with built-in functions of
Hive.
Thank you.
Best,
Rex
On Fri, Jan 22, 2016 at 1:58 AM, Namikaze Minato <ll...@gmail.com>
wrote:
> Hello. We don't have any information about your data.
>
> I don't think we can help you with this. Also, I cannot understand what
> you are trying to achieve. Please also tell us why you are using hadoop
> streaming instead of hive to do your operations.
>
> Regards,
> LLoyd
>
> On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
>
>> The given sequential files correspond to an external Hive table.
>>
>> They are stored in
>> /tableName/part-00000
>> /tableName/part-00001
>> ...
>>
>> There are about 2000 attributes in the table. Now I want to process the
>> data using Hadoop streaming and mapReduce. The first step is to find the
>> offset and length for each attribute.
>>
>> What is the best way to get this information?
>>
>>
>>
>
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Rex X <dn...@gmail.com>.
Hi LLoyd,
The metadata of this table are STRINGs, BIGINTs, and TINYINTs, 2000
attributes in total.
I need to transform the data that cannot be done with built-in functions of
Hive.
Thank you.
Best,
Rex
On Fri, Jan 22, 2016 at 1:58 AM, Namikaze Minato <ll...@gmail.com>
wrote:
> Hello. We don't have any information about your data.
>
> I don't think we can help you with this. Also, I cannot understand what
> you are trying to achieve. Please also tell us why you are using hadoop
> streaming instead of hive to do your operations.
>
> Regards,
> LLoyd
>
> On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
>
>> The given sequential files correspond to an external Hive table.
>>
>> They are stored in
>> /tableName/part-00000
>> /tableName/part-00001
>> ...
>>
>> There are about 2000 attributes in the table. Now I want to process the
>> data using Hadoop streaming and mapReduce. The first step is to find the
>> offset and length for each attribute.
>>
>> What is the best way to get this information?
>>
>>
>>
>
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Rex X <dn...@gmail.com>.
Hi LLoyd,
The metadata of this table are STRINGs, BIGINTs, and TINYINTs, 2000
attributes in total.
I need to transform the data that cannot be done with built-in functions of
Hive.
Thank you.
Best,
Rex
On Fri, Jan 22, 2016 at 1:58 AM, Namikaze Minato <ll...@gmail.com>
wrote:
> Hello. We don't have any information about your data.
>
> I don't think we can help you with this. Also, I cannot understand what
> you are trying to achieve. Please also tell us why you are using hadoop
> streaming instead of hive to do your operations.
>
> Regards,
> LLoyd
>
> On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
>
>> The given sequential files correspond to an external Hive table.
>>
>> They are stored in
>> /tableName/part-00000
>> /tableName/part-00001
>> ...
>>
>> There are about 2000 attributes in the table. Now I want to process the
>> data using Hadoop streaming and mapReduce. The first step is to find the
>> offset and length for each attribute.
>>
>> What is the best way to get this information?
>>
>>
>>
>
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Namikaze Minato <ll...@gmail.com>.
Hello. We don't have any information about your data.
I don't think we can help you with this. Also, I cannot understand what you
are trying to achieve. Please also tell us why you are using hadoop
streaming instead of hive to do your operations.
Regards,
LLoyd
On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
> The given sequential files correspond to an external Hive table.
>
> They are stored in
> /tableName/part-00000
> /tableName/part-00001
> ...
>
> There are about 2000 attributes in the table. Now I want to process the
> data using Hadoop streaming and mapReduce. The first step is to find the
> offset and length for each attribute.
>
> What is the best way to get this information?
>
>
>
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Namikaze Minato <ll...@gmail.com>.
Hello. We don't have any information about your data.
I don't think we can help you with this. Also, I cannot understand what you
are trying to achieve. Please also tell us why you are using hadoop
streaming instead of hive to do your operations.
Regards,
LLoyd
On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
> The given sequential files correspond to an external Hive table.
>
> They are stored in
> /tableName/part-00000
> /tableName/part-00001
> ...
>
> There are about 2000 attributes in the table. Now I want to process the
> data using Hadoop streaming and mapReduce. The first step is to find the
> offset and length for each attribute.
>
> What is the best way to get this information?
>
>
>
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Namikaze Minato <ll...@gmail.com>.
Hello. We don't have any information about your data.
I don't think we can help you with this. Also, I cannot understand what you
are trying to achieve. Please also tell us why you are using hadoop
streaming instead of hive to do your operations.
Regards,
LLoyd
On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
> The given sequential files correspond to an external Hive table.
>
> They are stored in
> /tableName/part-00000
> /tableName/part-00001
> ...
>
> There are about 2000 attributes in the table. Now I want to process the
> data using Hadoop streaming and mapReduce. The first step is to find the
> offset and length for each attribute.
>
> What is the best way to get this information?
>
>
>
Re: What is the best way to locate the offset and length of all
fields in a Hadoop sequential text file?
Posted by Namikaze Minato <ll...@gmail.com>.
Hello. We don't have any information about your data.
I don't think we can help you with this. Also, I cannot understand what you
are trying to achieve. Please also tell us why you are using hadoop
streaming instead of hive to do your operations.
Regards,
LLoyd
On 22 January 2016 at 06:30, Rex X <dn...@gmail.com> wrote:
> The given sequential files correspond to an external Hive table.
>
> They are stored in
> /tableName/part-00000
> /tableName/part-00001
> ...
>
> There are about 2000 attributes in the table. Now I want to process the
> data using Hadoop streaming and mapReduce. The first step is to find the
> offset and length for each attribute.
>
> What is the best way to get this information?
>
>
>