You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Kumar Jayapal <kj...@gmail.com> on 2015/05/01 05:47:28 UTC
parque table
Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR
smallint, FISCAL_PERIOD smallint)
STORED AS TEXTFILE;
and loaded it with data.
LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE raw;
I have to load it to parque table
when I say select * from raw it shows all null values.
NULLNULLNULLNULLNULLNULLNULLNULL
NULLNULLNULLNULLNULLNULLNULLNULL
NULLNULLNULLNULLNULLNULLNULLNULL
NULLNULLNULLNULLNULLNULLNULLNULL
Why is not show showing the actual data in file. will it show once I load
it to parque table?
Please let me know if I am doing anything wrong.
Thanks
jay
Re: parque table
Posted by "Bhagwan S. Soni" <bh...@gmail.com>.
Please mention partition as well, while loading data into a partitioned
table.
On Fri, May 1, 2015 at 8:22 PM, Sean Busbey <bu...@cloudera.com> wrote:
> -user@hadoop to bcc
>
> Kumar,
>
> I'm copying your question over to the Apache Hive user list (
> user@hive.apache.org). Please keep your questions about using Hive there.
> The Hadoop user list (user@hadoop.apache.org) is just for that project.
>
> On Fri, May 1, 2015 at 9:32 AM, Asit Parija <as...@sigmoidanalytics.com>
> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by
>> default it should be able to read the .gz files ( if they are comma
>> delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
>> wrote:
>>
>>> Hello Nitin,
>>>
>>> Dint understand what you mean. Are you telling me to set
>>> COMPRESSION_CODEC=gzip ?
>>>
>>> thanks
>>> Jay
>>>
>>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>>> wrote:
>>>
>>>> You loaded a gz file in a table stored as text file
>>>> either define compression format or uncompress the file and load it
>>>>
>>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>>> wrote:
>>>>
>>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>>> STORED AS TEXTFILE;
>>>>>
>>>>> and loaded it with data.
>>>>>
>>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>>> TABLE raw;
>>>>>
>>>>> I have to load it to parque table
>>>>>
>>>>> when I say select * from raw it shows all null values.
>>>>>
>>>>>
>>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>>> Why is not show showing the actual data in file. will it show once I
>>>>> load it to parque table?
>>>>>
>>>>> Please let me know if I am doing anything wrong.
>>>>>
>>>>>
>>>>> Thanks
>>>>> jay
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Nitin Pawar
>>>>
>>>
>>>
>>
>
>
> --
> Sean
>
Re: parque table
Posted by Sean Busbey <bu...@cloudera.com>.
-user@hadoop to bcc
Kumar,
I'm copying your question over to the Apache Hive user list (
user@hive.apache.org). Please keep your questions about using Hive there.
The Hadoop user list (user@hadoop.apache.org) is just for that project.
On Fri, May 1, 2015 at 9:32 AM, Asit Parija <as...@sigmoidanalytics.com>
wrote:
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
--
Sean
Re: parque table
Posted by Sean Busbey <bu...@cloudera.com>.
-user@hadoop to bcc
Kumar,
I'm copying your question over to the Apache Hive user list (
user@hive.apache.org). Please keep your questions about using Hive there.
The Hadoop user list (user@hadoop.apache.org) is just for that project.
On Fri, May 1, 2015 at 9:32 AM, Asit Parija <as...@sigmoidanalytics.com>
wrote:
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
--
Sean
Re: parque table
Posted by Sean Busbey <bu...@cloudera.com>.
-user@hadoop to bcc
Kumar,
I'm copying your question over to the Apache Hive user list (
user@hive.apache.org). Please keep your questions about using Hive there.
The Hadoop user list (user@hadoop.apache.org) is just for that project.
On Fri, May 1, 2015 at 9:32 AM, Asit Parija <as...@sigmoidanalytics.com>
wrote:
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
--
Sean
Re: parque table
Posted by Sean Busbey <bu...@cloudera.com>.
-user@hadoop to bcc
Kumar,
I'm copying your question over to the Apache Hive user list (
user@hive.apache.org). Please keep your questions about using Hive there.
The Hadoop user list (user@hadoop.apache.org) is just for that project.
On Fri, May 1, 2015 at 9:32 AM, Asit Parija <as...@sigmoidanalytics.com>
wrote:
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
--
Sean
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
I can't reproduce your problem. Here's what happens on hive-1.1.0-cdh5.4.0:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table rawpartition (fiscal_year=2015, fiscal_period=1);
Loading data to table db_apw.raw partition (fiscal_year=2015, fiscal_period=1)
Partition db_apw.raw{fiscal_year=2015, fiscal_period=1} stats: [numFiles=1, numRows=0, totalSize=74, rawDataSize=0]
OK
hive> select * from raw;
OK
106,"2003-02-03",20,2,"A","2","2","037" 2015 1
106,"2003-02-03",20,3,"A","2","2","037" 2015 1
106,"2003-02-03",8,2,"A","2","2","037" 2015 1
As already mentioned in this thread, you need to specify the partition clause (in red above), or you get an error:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table raw;
FAILED: SemanticException [Error 10062]: Need to specify partition columns because the destination table is partitioned
hth
Gabriel Balan
On 5/1/2015 11:21 AM, Hadoop User wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
If you're quoted fields may contain commas, you must use RegexSerDe to parse each line into fields.
create table foo(c0 string, c1 string, c2 string, c3 string, c4 string, c5 string, c6 string, c7 string)
row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
with serdeproperties
("input.regex" = "^([^,]*),\"([^\"]*)\",([^,]*),([^,]*),\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\"$");
--here I assumed some fields are always quoted, and some fields are always unquoted. You may need something fancier for the general case.
load DATA local inpath 'log.txt.gz' into table foo;
select * from foo;
OK
106 2003-02-03 20 2 A 2 2 037
106 2003-02-03 20 3 A 2 2 037
106 2003-02-03 8 2 A 2 2 037
If you're sure there are no commas in your quoted fields, then you could try putting a view on top of the table, and have the view use UDFs to strip the quotes.
hth
Gabriel Balan
On 5/2/2015 1:04 AM, Kumar Jayapal wrote:6
> Hi,
>
> When I am loading this data I am getting " " inserted into the table how to load with out " "it.
>
>
> Inline image 1
>
>
>
> thanks
> jay
>
>
>
>
>
>
>
>
>
>
> Thanks
> Jay
>
> On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
If you're quoted fields may contain commas, you must use RegexSerDe to parse each line into fields.
create table foo(c0 string, c1 string, c2 string, c3 string, c4 string, c5 string, c6 string, c7 string)
row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
with serdeproperties
("input.regex" = "^([^,]*),\"([^\"]*)\",([^,]*),([^,]*),\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\"$");
--here I assumed some fields are always quoted, and some fields are always unquoted. You may need something fancier for the general case.
load DATA local inpath 'log.txt.gz' into table foo;
select * from foo;
OK
106 2003-02-03 20 2 A 2 2 037
106 2003-02-03 20 3 A 2 2 037
106 2003-02-03 8 2 A 2 2 037
If you're sure there are no commas in your quoted fields, then you could try putting a view on top of the table, and have the view use UDFs to strip the quotes.
hth
Gabriel Balan
On 5/2/2015 1:04 AM, Kumar Jayapal wrote:6
> Hi,
>
> When I am loading this data I am getting " " inserted into the table how to load with out " "it.
>
>
> Inline image 1
>
>
>
> thanks
> jay
>
>
>
>
>
>
>
>
>
>
> Thanks
> Jay
>
> On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
If you're quoted fields may contain commas, you must use RegexSerDe to parse each line into fields.
create table foo(c0 string, c1 string, c2 string, c3 string, c4 string, c5 string, c6 string, c7 string)
row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
with serdeproperties
("input.regex" = "^([^,]*),\"([^\"]*)\",([^,]*),([^,]*),\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\"$");
--here I assumed some fields are always quoted, and some fields are always unquoted. You may need something fancier for the general case.
load DATA local inpath 'log.txt.gz' into table foo;
select * from foo;
OK
106 2003-02-03 20 2 A 2 2 037
106 2003-02-03 20 3 A 2 2 037
106 2003-02-03 8 2 A 2 2 037
If you're sure there are no commas in your quoted fields, then you could try putting a view on top of the table, and have the view use UDFs to strip the quotes.
hth
Gabriel Balan
On 5/2/2015 1:04 AM, Kumar Jayapal wrote:6
> Hi,
>
> When I am loading this data I am getting " " inserted into the table how to load with out " "it.
>
>
> Inline image 1
>
>
>
> thanks
> jay
>
>
>
>
>
>
>
>
>
>
> Thanks
> Jay
>
> On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
If you're quoted fields may contain commas, you must use RegexSerDe to parse each line into fields.
create table foo(c0 string, c1 string, c2 string, c3 string, c4 string, c5 string, c6 string, c7 string)
row format serde 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
with serdeproperties
("input.regex" = "^([^,]*),\"([^\"]*)\",([^,]*),([^,]*),\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\",\"([^\"]*)\"$");
--here I assumed some fields are always quoted, and some fields are always unquoted. You may need something fancier for the general case.
load DATA local inpath 'log.txt.gz' into table foo;
select * from foo;
OK
106 2003-02-03 20 2 A 2 2 037
106 2003-02-03 20 3 A 2 2 037
106 2003-02-03 8 2 A 2 2 037
If you're sure there are no commas in your quoted fields, then you could try putting a view on top of the table, and have the view use UDFs to strip the quotes.
hth
Gabriel Balan
On 5/2/2015 1:04 AM, Kumar Jayapal wrote:6
> Hi,
>
> When I am loading this data I am getting " " inserted into the table how to load with out " "it.
>
>
> Inline image 1
>
>
>
> thanks
> jay
>
>
>
>
>
>
>
>
>
>
> Thanks
> Jay
>
> On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hi,
When I am loading this data I am getting " " inserted into the table how
to load with out " "it.
[image: Inline image 1]
thanks
jay
Thanks
Jay
On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kj...@gmail.com> wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
I can't reproduce your problem. Here's what happens on hive-1.1.0-cdh5.4.0:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table rawpartition (fiscal_year=2015, fiscal_period=1);
Loading data to table db_apw.raw partition (fiscal_year=2015, fiscal_period=1)
Partition db_apw.raw{fiscal_year=2015, fiscal_period=1} stats: [numFiles=1, numRows=0, totalSize=74, rawDataSize=0]
OK
hive> select * from raw;
OK
106,"2003-02-03",20,2,"A","2","2","037" 2015 1
106,"2003-02-03",20,3,"A","2","2","037" 2015 1
106,"2003-02-03",8,2,"A","2","2","037" 2015 1
As already mentioned in this thread, you need to specify the partition clause (in red above), or you get an error:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table raw;
FAILED: SemanticException [Error 10062]: Need to specify partition columns because the destination table is partitioned
hth
Gabriel Balan
On 5/1/2015 11:21 AM, Hadoop User wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
I can't reproduce your problem. Here's what happens on hive-1.1.0-cdh5.4.0:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table rawpartition (fiscal_year=2015, fiscal_period=1);
Loading data to table db_apw.raw partition (fiscal_year=2015, fiscal_period=1)
Partition db_apw.raw{fiscal_year=2015, fiscal_period=1} stats: [numFiles=1, numRows=0, totalSize=74, rawDataSize=0]
OK
hive> select * from raw;
OK
106,"2003-02-03",20,2,"A","2","2","037" 2015 1
106,"2003-02-03",20,3,"A","2","2","037" 2015 1
106,"2003-02-03",8,2,"A","2","2","037" 2015 1
As already mentioned in this thread, you need to specify the partition clause (in red above), or you get an error:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table raw;
FAILED: SemanticException [Error 10062]: Need to specify partition columns because the destination table is partitioned
hth
Gabriel Balan
On 5/1/2015 11:21 AM, Hadoop User wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by gabriel balan <ga...@oracle.com>.
Hi
I can't reproduce your problem. Here's what happens on hive-1.1.0-cdh5.4.0:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table rawpartition (fiscal_year=2015, fiscal_period=1);
Loading data to table db_apw.raw partition (fiscal_year=2015, fiscal_period=1)
Partition db_apw.raw{fiscal_year=2015, fiscal_period=1} stats: [numFiles=1, numRows=0, totalSize=74, rawDataSize=0]
OK
hive> select * from raw;
OK
106,"2003-02-03",20,2,"A","2","2","037" 2015 1
106,"2003-02-03",20,3,"A","2","2","037" 2015 1
106,"2003-02-03",8,2,"A","2","2","037" 2015 1
As already mentioned in this thread, you need to specify the partition clause (in red above), or you get an error:
hive> LOAD DATA LOCAL INPATH 'access.log.gz' into table raw;
FAILED: SemanticException [Error 10062]: Need to specify partition columns because the destination table is partitioned
hth
Gabriel Balan
On 5/1/2015 11:21 AM, Hadoop User wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <asit@sigmoidanalytics.com <ma...@sigmoidanalytics.com>> wrote:
>
>> Hi Kumar ,
>> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>>
>>
>> Thanks
>> Asit
>>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <nitinpawar432@gmail.com <ma...@gmail.com>> wrote:
>>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kjayapal17@gmail.com <ma...@gmail.com>> wrote:
>>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> |LOAD DATA LOCAL INPATH ||'/tmp/weblogs/20090603-access.log.gz'||INTO TABLE raw;|
>> |
>> |
>> |I have to load it to parque table|
>> |
>> |
>> |when I say select * from raw it shows all null values.|
>> |
>> |
>> |
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> NULL NULL NULL NULL NULL NULL NULL NULL
>>
>> |
>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>> |
>> |
>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>>
>>
--
The statements and opinions expressed here are my own and do not necessarily represent those of Oracle Corporation.
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hi,
When I am loading this data I am getting " " inserted into the table how
to load with out " "it.
[image: Inline image 1]
thanks
jay
Thanks
Jay
On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kj...@gmail.com> wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hi,
When I am loading this data I am getting " " inserted into the table how
to load with out " "it.
[image: Inline image 1]
thanks
jay
Thanks
Jay
On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kj...@gmail.com> wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hi,
When I am loading this data I am getting " " inserted into the table how
to load with out " "it.
[image: Inline image 1]
thanks
jay
Thanks
Jay
On Fri, May 1, 2015 at 8:21 AM, Hadoop User <kj...@gmail.com> wrote:
> Here is the content of the file once it's unzip
>
> 106,"2003-02-03",20,2,"A","2","2","037"
> 106,"2003-02-03",20,3,"A","2","2","037"
> 106,"2003-02-03",8,2,"A","2","2","037"
>
>
>
>
>
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
Re: parque table
Posted by Hadoop User <kj...@gmail.com>.
Here is the content of the file once it's unzip
106,"2003-02-03",20,2,"A","2","2","037"
106,"2003-02-03",20,3,"A","2","2","037"
106,"2003-02-03",8,2,"A","2","2","037"
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>
>
> Thanks
> Asit
>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com> wrote:
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>
Re: parque table
Posted by Sean Busbey <bu...@cloudera.com>.
-user@hadoop to bcc
Kumar,
I'm copying your question over to the Apache Hive user list (
user@hive.apache.org). Please keep your questions about using Hive there.
The Hadoop user list (user@hadoop.apache.org) is just for that project.
On Fri, May 1, 2015 at 9:32 AM, Asit Parija <as...@sigmoidanalytics.com>
wrote:
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by
> default it should be able to read the .gz files ( if they are comma
> delimited csv files ) .
>
>
> Thanks
> Asit
>
> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set
>> COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
>> wrote:
>>
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>>> wrote:
>>>
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO
>>>> TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>>> Why is not show showing the actual data in file. will it show once I
>>>> load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
--
Sean
Re: parque table
Posted by Hadoop User <kj...@gmail.com>.
Here is the content of the file once it's unzip
106,"2003-02-03",20,2,"A","2","2","037"
106,"2003-02-03",20,3,"A","2","2","037"
106,"2003-02-03",8,2,"A","2","2","037"
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>
>
> Thanks
> Asit
>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com> wrote:
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>
Re: parque table
Posted by Hadoop User <kj...@gmail.com>.
Here is the content of the file once it's unzip
106,"2003-02-03",20,2,"A","2","2","037"
106,"2003-02-03",20,3,"A","2","2","037"
106,"2003-02-03",8,2,"A","2","2","037"
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>
>
> Thanks
> Asit
>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com> wrote:
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>
Re: parque table
Posted by Hadoop User <kj...@gmail.com>.
Here is the content of the file once it's unzip
106,"2003-02-03",20,2,"A","2","2","037"
106,"2003-02-03",20,3,"A","2","2","037"
106,"2003-02-03",8,2,"A","2","2","037"
> On May 1, 2015, at 7:32 AM, Asit Parija <as...@sigmoidanalytics.com> wrote:
>
> Hi Kumar ,
> You can remove the stored as text file part and then try that out by default it should be able to read the .gz files ( if they are comma delimited csv files ) .
>
>
> Thanks
> Asit
>
>> On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>> Hello Nitin,
>>
>> Dint understand what you mean. Are you telling me to set COMPRESSION_CODEC=gzip ?
>>
>> thanks
>> Jay
>>
>>> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com> wrote:
>>> You loaded a gz file in a table stored as text file
>>> either define compression format or uncompress the file and load it
>>>
>>>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
>>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>>> STORED AS TEXTFILE;
>>>>
>>>> and loaded it with data.
>>>>
>>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE raw;
>>>>
>>>> I have to load it to parque table
>>>>
>>>> when I say select * from raw it shows all null values.
>>>>
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>>
>>>> NULL NULL NULL NULL NULL NULL NULL NULL
>>>> Why is not show showing the actual data in file. will it show once I load it to parque table?
>>>>
>>>> Please let me know if I am doing anything wrong.
>>>>
>>>>
>>>> Thanks
>>>> jay
>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>
Re: parque table
Posted by Asit Parija <as...@sigmoidanalytics.com>.
Hi Kumar ,
You can remove the stored as text file part and then try that out by
default it should be able to read the .gz files ( if they are comma
delimited csv files ) .
Thanks
Asit
On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Hello Nitin,
>
> Dint understand what you mean. Are you telling me to set
> COMPRESSION_CODEC=gzip ?
>
> thanks
> Jay
>
> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
> wrote:
>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>> wrote:
>>
>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>> STORED AS TEXTFILE;
>>>
>>> and loaded it with data.
>>>
>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>>> raw;
>>>
>>> I have to load it to parque table
>>>
>>> when I say select * from raw it shows all null values.
>>>
>>>
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> Why is not show showing the actual data in file. will it show once I
>>> load it to parque table?
>>>
>>> Please let me know if I am doing anything wrong.
>>>
>>>
>>> Thanks
>>> jay
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
Re: parque table
Posted by Asit Parija <as...@sigmoidanalytics.com>.
Hi Kumar ,
You can remove the stored as text file part and then try that out by
default it should be able to read the .gz files ( if they are comma
delimited csv files ) .
Thanks
Asit
On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Hello Nitin,
>
> Dint understand what you mean. Are you telling me to set
> COMPRESSION_CODEC=gzip ?
>
> thanks
> Jay
>
> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
> wrote:
>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>> wrote:
>>
>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>> STORED AS TEXTFILE;
>>>
>>> and loaded it with data.
>>>
>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>>> raw;
>>>
>>> I have to load it to parque table
>>>
>>> when I say select * from raw it shows all null values.
>>>
>>>
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> Why is not show showing the actual data in file. will it show once I
>>> load it to parque table?
>>>
>>> Please let me know if I am doing anything wrong.
>>>
>>>
>>> Thanks
>>> jay
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
Re: parque table
Posted by Asit Parija <as...@sigmoidanalytics.com>.
Hi Kumar ,
You can remove the stored as text file part and then try that out by
default it should be able to read the .gz files ( if they are comma
delimited csv files ) .
Thanks
Asit
On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Hello Nitin,
>
> Dint understand what you mean. Are you telling me to set
> COMPRESSION_CODEC=gzip ?
>
> thanks
> Jay
>
> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
> wrote:
>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>> wrote:
>>
>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>> STORED AS TEXTFILE;
>>>
>>> and loaded it with data.
>>>
>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>>> raw;
>>>
>>> I have to load it to parque table
>>>
>>> when I say select * from raw it shows all null values.
>>>
>>>
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> Why is not show showing the actual data in file. will it show once I
>>> load it to parque table?
>>>
>>> Please let me know if I am doing anything wrong.
>>>
>>>
>>> Thanks
>>> jay
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
Re: parque table
Posted by Asit Parija <as...@sigmoidanalytics.com>.
Hi Kumar ,
You can remove the stored as text file part and then try that out by
default it should be able to read the .gz files ( if they are comma
delimited csv files ) .
Thanks
Asit
On Fri, May 1, 2015 at 10:55 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Hello Nitin,
>
> Dint understand what you mean. Are you telling me to set
> COMPRESSION_CODEC=gzip ?
>
> thanks
> Jay
>
> On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
> wrote:
>
>> You loaded a gz file in a table stored as text file
>> either define compression format or uncompress the file and load it
>>
>> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
>> wrote:
>>
>>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>>> STORED AS TEXTFILE;
>>>
>>> and loaded it with data.
>>>
>>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>>> raw;
>>>
>>> I have to load it to parque table
>>>
>>> when I say select * from raw it shows all null values.
>>>
>>>
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> NULLNULLNULLNULLNULLNULLNULLNULL
>>> Why is not show showing the actual data in file. will it show once I
>>> load it to parque table?
>>>
>>> Please let me know if I am doing anything wrong.
>>>
>>>
>>> Thanks
>>> jay
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hello Nitin,
Dint understand what you mean. Are you telling me to set
COMPRESSION_CODEC=gzip ?
thanks
Jay
On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
wrote:
> You loaded a gz file in a table stored as text file
> either define compression format or uncompress the file and load it
>
> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>> raw;
>>
>> I have to load it to parque table
>>
>> when I say select * from raw it shows all null values.
>>
>>
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> Why is not show showing the actual data in file. will it show once I load
>> it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>>
>>
>
>
> --
> Nitin Pawar
>
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hello Nitin,
Dint understand what you mean. Are you telling me to set
COMPRESSION_CODEC=gzip ?
thanks
Jay
On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
wrote:
> You loaded a gz file in a table stored as text file
> either define compression format or uncompress the file and load it
>
> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>> raw;
>>
>> I have to load it to parque table
>>
>> when I say select * from raw it shows all null values.
>>
>>
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> Why is not show showing the actual data in file. will it show once I load
>> it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>>
>>
>
>
> --
> Nitin Pawar
>
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hello Nitin,
Dint understand what you mean. Are you telling me to set
COMPRESSION_CODEC=gzip ?
thanks
Jay
On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
wrote:
> You loaded a gz file in a table stored as text file
> either define compression format or uncompress the file and load it
>
> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>> raw;
>>
>> I have to load it to parque table
>>
>> when I say select * from raw it shows all null values.
>>
>>
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> Why is not show showing the actual data in file. will it show once I load
>> it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>>
>>
>
>
> --
> Nitin Pawar
>
Re: parque table
Posted by Kumar Jayapal <kj...@gmail.com>.
Hello Nitin,
Dint understand what you mean. Are you telling me to set
COMPRESSION_CODEC=gzip ?
thanks
Jay
On Thu, Apr 30, 2015 at 10:02 PM, Nitin Pawar <ni...@gmail.com>
wrote:
> You loaded a gz file in a table stored as text file
> either define compression format or uncompress the file and load it
>
> On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com>
> wrote:
>
>> Created table CREATE TABLE raw (line STRING) PARTITIONED BY
>> (FISCAL_YEAR smallint, FISCAL_PERIOD smallint)
>> STORED AS TEXTFILE;
>>
>> and loaded it with data.
>>
>> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
>> raw;
>>
>> I have to load it to parque table
>>
>> when I say select * from raw it shows all null values.
>>
>>
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> NULLNULLNULLNULLNULLNULLNULLNULL
>> Why is not show showing the actual data in file. will it show once I load
>> it to parque table?
>>
>> Please let me know if I am doing anything wrong.
>>
>>
>> Thanks
>> jay
>>
>>
>
>
> --
> Nitin Pawar
>
Re: parque table
Posted by Nitin Pawar <ni...@gmail.com>.
You loaded a gz file in a table stored as text file
either define compression format or uncompress the file and load it
On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR
> smallint, FISCAL_PERIOD smallint)
> STORED AS TEXTFILE;
>
> and loaded it with data.
>
> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
> raw;
>
> I have to load it to parque table
>
> when I say select * from raw it shows all null values.
>
>
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> Why is not show showing the actual data in file. will it show once I load
> it to parque table?
>
> Please let me know if I am doing anything wrong.
>
>
> Thanks
> jay
>
>
--
Nitin Pawar
Re: parque table
Posted by Nitin Pawar <ni...@gmail.com>.
You loaded a gz file in a table stored as text file
either define compression format or uncompress the file and load it
On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR
> smallint, FISCAL_PERIOD smallint)
> STORED AS TEXTFILE;
>
> and loaded it with data.
>
> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
> raw;
>
> I have to load it to parque table
>
> when I say select * from raw it shows all null values.
>
>
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> Why is not show showing the actual data in file. will it show once I load
> it to parque table?
>
> Please let me know if I am doing anything wrong.
>
>
> Thanks
> jay
>
>
--
Nitin Pawar
Re: parque table
Posted by Nitin Pawar <ni...@gmail.com>.
You loaded a gz file in a table stored as text file
either define compression format or uncompress the file and load it
On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR
> smallint, FISCAL_PERIOD smallint)
> STORED AS TEXTFILE;
>
> and loaded it with data.
>
> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
> raw;
>
> I have to load it to parque table
>
> when I say select * from raw it shows all null values.
>
>
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> Why is not show showing the actual data in file. will it show once I load
> it to parque table?
>
> Please let me know if I am doing anything wrong.
>
>
> Thanks
> jay
>
>
--
Nitin Pawar
Re: parque table
Posted by Nitin Pawar <ni...@gmail.com>.
You loaded a gz file in a table stored as text file
either define compression format or uncompress the file and load it
On Fri, May 1, 2015 at 9:17 AM, Kumar Jayapal <kj...@gmail.com> wrote:
> Created table CREATE TABLE raw (line STRING) PARTITIONED BY (FISCAL_YEAR
> smallint, FISCAL_PERIOD smallint)
> STORED AS TEXTFILE;
>
> and loaded it with data.
>
> LOAD DATA LOCAL INPATH '/tmp/weblogs/20090603-access.log.gz' INTO TABLE
> raw;
>
> I have to load it to parque table
>
> when I say select * from raw it shows all null values.
>
>
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> NULLNULLNULLNULLNULLNULLNULLNULL
> Why is not show showing the actual data in file. will it show once I load
> it to parque table?
>
> Please let me know if I am doing anything wrong.
>
>
> Thanks
> jay
>
>
--
Nitin Pawar