You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "kaducangica ." <ka...@gmail.com> on 2017/12/06 14:43:04 UTC
Fwd: Hive - Json Serde - ORC
Hi all,
i have a very complex json that i need to insert in a hive table. A json
example follws attached.
First of all i read a json file with Spark to make some data processing and
then i write to a stage table with no Serde and with no any kind of
compression and format.
Then i do an INSERT/SELECT into the "jsonTable" (create table attached)
with no problems. This table use a json Serde
(org.openx.data.jsonserde.JsonSerDe)
and a ORC format and is also particioned by date and timezone.
The problem is that after all this process every time a try to make a
simple "select * from jsonTable" query i got this error message:
"Failed with exception java.io.IOException:java.io.IOException: Error
reading file: hdfs://ip-xxx-xxx-xxx-xxx.sa-east-1.compute.int
ernal:8020/user/hive/warehouse/jsonTable/data_posicao_short=2017-12-02/
veitimezone=America-Sao_Paulo/000000_0"
Actually i do not know if it is possible to use Serde, ORC and partition in
the same table.
Someone could help me?
Thanks in advance.
Best regards
Carlos.
Re: Hive - Json Serde - ORC
Posted by Wei-Chiu Chuang <we...@cloudera.com>.
Hi I think you are better off asking this question at the hive mailing list.
Best
On Wed, Dec 6, 2017 at 6:43 AM, kaducangica . <ka...@gmail.com> wrote:
> Hi all,
>
> i have a very complex json that i need to insert in a hive table. A json
> example follws attached.
>
> First of all i read a json file with Spark to make some data processing
> and then i write to a stage table with no Serde and with no any kind of
> compression and format.
>
> Then i do an INSERT/SELECT into the "jsonTable" (create table attached)
> with no problems. This table use a json Serde (org.openx.data.jsonserde.JsonSerDe)
> and a ORC format and is also particioned by date and timezone.
>
> The problem is that after all this process every time a try to make a
> simple "select * from jsonTable" query i got this error message:
>
> "Failed with exception java.io.IOException:java.io.IOException: Error
> reading file: hdfs://ip-xxx-xxx-xxx-xxx.sa-east-1.compute.int
> ernal:8020/user/hive/warehouse/jsonTable/data_posicao_short=
> 2017-12-02/veitimezone=America-Sao_Paulo/000000_0"
>
> Actually i do not know if it is possible to use Serde, ORC and partition
> in the same table.
>
> Someone could help me?
>
> Thanks in advance.
> Best regards
>
> Carlos.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: user-help@hadoop.apache.org
>
--
A very happy Clouderan