Re: Problems of loading data from hdfs

[Zheng Shao <zs...@gmail.com>]

There is no way to go around that right now.
Note that the partition column is actually not in the data - it's
appended to each row at runtime.

Hive uses the directory structure to figure out the partition values.




On Mon, Apr 5, 2010 at 9:24 PM, Fan Yang <ya...@ebupt.com> wrote:
> I’ve found out the root cause is not the data and table format. It’s because
> of the name of file on hdfs. I used “.data” as the file name and load data
> command ignore the file when its name started with ‘.’
>
>
>
> Anyway, thanks very much.
>
>
>
> BTW, when create a table with partitions. It seems the partition column must
> be the last column of the table. How can I use some other column which is
> not the last one to mark partition?
>
>
>
> Fan
>
>
>
> From: zhu weimin [mailto:xim-shu@tsm.kddilabs.jp]
> Sent: Tuesday, April 06, 2010 10:53 AM
> To: hive-user@hadoop.apache.org
> Subject: RE: Problems of loading data from hdfs
>
>
>
> I think your data probably terminated by comma character.
>
> If is that, recreate table like the following
>
> CREATE TABLE table_name{
>
> …
>
> }
>
>   ROW FORMAT DELIMITED
>
>   FIELDS TERMINATED BY ',';
>
>
>
> If is not, review your table describe and data samples
>
>
>
> Zhu weimin
>
>
>
> From: Fan Yang [mailto:yangfan@ebupt.com]
> Sent: Tuesday, April 06, 2010 11:19 AM
> To: hive-user@hadoop.apache.org
> Subject: Problems of loading data from hdfs
>
>
>
> Hi
>
>
>
> When I try to load data into table from hdfs, it seems successful and gives
> no warnings. But using select clause can’t get any result from the table.
>
> However, after I load data from local file which is download from hdfs by
> using ‘hadoop dfs get’ command, I can select out records.
>
> Note, these two files are exactly the same and no errors were given during
> the procedure.
>
>
>
> Thanks in advance.
>
>
>
> Fan



-- 
Yours,
Zheng
http://www.linkedin.com/in/zshao