You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Manish Verma <ma...@gmail.com> on 2014/03/26 02:15:10 UTC
Using MS DOS File in Hive
I am trying to build a hive table on a MS DOS File (record ends with CRLF
character). Could some know how to do this?
Thanks
Manish
Re: Using MS DOS File in Hive
Posted by Manish Verma <ma...@gmail.com>.
Hi Andrew,
Some of the field values in this file have LF in them. I was trying to find
a way which does not require processing the file to make it conform to Unix
style file. I believe that by writing your own File Format/Splitter classes
you could use any delimiter in map reduce input file. I did not find how to
specify CRLF as record delimiter in Hive.
Thanks
Manish
On Tue, Mar 25, 2014 at 7:36 PM, Andrew Ehrlich <an...@aehrlich.com> wrote:
> I think you can only use "\n" to denote new lines in Hive. What if you
> replaced the CRLF character with a "\n" in the data pipeline into HDFS or
> with a mapreduce job after the files are in HDFS?
>
>
> On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
>
>> I am trying to build a hive table on a MS DOS File (record ends with CRLF
>> character). Could some know how to do this?
>>
>> Thanks
>> Manish
>>
>
>
Re: Using MS DOS File in Hive
Posted by Manish Verma <ma...@gmail.com>.
Hi Andrew,
Some of the field values in this file have LF in them. I was trying to find
a way which does not require processing the file to make it conform to Unix
style file. I believe that by writing your own File Format/Splitter classes
you could use any delimiter in map reduce input file. I did not find how to
specify CRLF as record delimiter in Hive.
Thanks
Manish
On Tue, Mar 25, 2014 at 7:36 PM, Andrew Ehrlich <an...@aehrlich.com> wrote:
> I think you can only use "\n" to denote new lines in Hive. What if you
> replaced the CRLF character with a "\n" in the data pipeline into HDFS or
> with a mapreduce job after the files are in HDFS?
>
>
> On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
>
>> I am trying to build a hive table on a MS DOS File (record ends with CRLF
>> character). Could some know how to do this?
>>
>> Thanks
>> Manish
>>
>
>
Re: Using MS DOS File in Hive
Posted by Manish Verma <ma...@gmail.com>.
Hi Andrew,
Some of the field values in this file have LF in them. I was trying to find
a way which does not require processing the file to make it conform to Unix
style file. I believe that by writing your own File Format/Splitter classes
you could use any delimiter in map reduce input file. I did not find how to
specify CRLF as record delimiter in Hive.
Thanks
Manish
On Tue, Mar 25, 2014 at 7:36 PM, Andrew Ehrlich <an...@aehrlich.com> wrote:
> I think you can only use "\n" to denote new lines in Hive. What if you
> replaced the CRLF character with a "\n" in the data pipeline into HDFS or
> with a mapreduce job after the files are in HDFS?
>
>
> On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
>
>> I am trying to build a hive table on a MS DOS File (record ends with CRLF
>> character). Could some know how to do this?
>>
>> Thanks
>> Manish
>>
>
>
Re: Using MS DOS File in Hive
Posted by Manish Verma <ma...@gmail.com>.
Hi Andrew,
Some of the field values in this file have LF in them. I was trying to find
a way which does not require processing the file to make it conform to Unix
style file. I believe that by writing your own File Format/Splitter classes
you could use any delimiter in map reduce input file. I did not find how to
specify CRLF as record delimiter in Hive.
Thanks
Manish
On Tue, Mar 25, 2014 at 7:36 PM, Andrew Ehrlich <an...@aehrlich.com> wrote:
> I think you can only use "\n" to denote new lines in Hive. What if you
> replaced the CRLF character with a "\n" in the data pipeline into HDFS or
> with a mapreduce job after the files are in HDFS?
>
>
> On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
>
>> I am trying to build a hive table on a MS DOS File (record ends with CRLF
>> character). Could some know how to do this?
>>
>> Thanks
>> Manish
>>
>
>
Re: Using MS DOS File in Hive
Posted by Andrew Ehrlich <an...@aehrlich.com>.
I think you can only use "\n" to denote new lines in Hive. What if you
replaced the CRLF character with a "\n" in the data pipeline into HDFS or
with a mapreduce job after the files are in HDFS?
On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
> I am trying to build a hive table on a MS DOS File (record ends with CRLF
> character). Could some know how to do this?
>
> Thanks
> Manish
>
Re: Using MS DOS File in Hive
Posted by Andrew Ehrlich <an...@aehrlich.com>.
I think you can only use "\n" to denote new lines in Hive. What if you
replaced the CRLF character with a "\n" in the data pipeline into HDFS or
with a mapreduce job after the files are in HDFS?
On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
> I am trying to build a hive table on a MS DOS File (record ends with CRLF
> character). Could some know how to do this?
>
> Thanks
> Manish
>
Re: Using MS DOS File in Hive
Posted by Andrew Ehrlich <an...@aehrlich.com>.
I think you can only use "\n" to denote new lines in Hive. What if you
replaced the CRLF character with a "\n" in the data pipeline into HDFS or
with a mapreduce job after the files are in HDFS?
On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
> I am trying to build a hive table on a MS DOS File (record ends with CRLF
> character). Could some know how to do this?
>
> Thanks
> Manish
>
Re: Using MS DOS File in Hive
Posted by Andrew Ehrlich <an...@aehrlich.com>.
I think you can only use "\n" to denote new lines in Hive. What if you
replaced the CRLF character with a "\n" in the data pipeline into HDFS or
with a mapreduce job after the files are in HDFS?
On Tue, Mar 25, 2014 at 6:15 PM, Manish Verma <ma...@gmail.com>wrote:
> I am trying to build a hive table on a MS DOS File (record ends with CRLF
> character). Could some know how to do this?
>
> Thanks
> Manish
>