You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Marcelo Valle <ma...@ktech.com> on 2019/05/24 11:57:49 UTC
Re: Exporting parquet, issues with schema
Hi,
I know this thread is a little old, I found the original discussion here:
https://grokbase.com/t/sqoop/user/15c94s4vjx/exporting-parquet-issues-with-schema
But I am having the same problem now, more than 3 years later, and scoop 2
seems to have been deprecated.
Are there any plans of not needing this metadata folder anymore?
Thanks,
Marcelo.
This email is confidential [and may be protected by legal privilege]. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt.
KTech Services Ltd is registered in England as company number 10704940.
Registered Office: The River Building, 1 Cousin Lane, London EC4R 3TE, United Kingdom
Re: Exporting parquet, issues with schema
Posted by Marcelo Valle <ma...@ktech.com>.
This assumes I have a hive table. Then I have to generate one for my
parquet.
Parquet files have schemas inside themselves, why should I need a metadata
folder or hive metadata? Couldn't sqoop read the parquet metadata directly
from the file?
On Fri, 24 May 2019 at 13:40, Markus Kemper <ma...@cloudera.com> wrote:
>
> Hello Marcelo,
>
> Have you tried using hcatalog to export Parquet?
>
> $ sqoop export --connect $MYSQL_CONN --username $MYSQL_USER --password
> $MYSQL_PWD --table $MYSQL_TABLE --num-mappers 1 --hcatalog-database
> $HIVE_DATABASE --hcatalog-table $HIVE_PARQUET_TABLE
>
> Markus Kemper
> Cloudera Support
>
>
>
>
> On Fri, May 24, 2019 at 7:58 AM Marcelo Valle <ma...@ktech.com>
> wrote:
>
>> Hi,
>>
>> I know this thread is a little old, I found the original discussion here:
>> https://grokbase.com/t/sqoop/user/15c94s4vjx/exporting-parquet-issues-with-schema
>>
>> But I am having the same problem now, more than 3 years later, and scoop
>> 2 seems to have been deprecated.
>>
>> Are there any plans of not needing this metadata folder anymore?
>>
>> Thanks,
>> Marcelo.
>>
>>
>>
>> This email is confidential [and may be protected by legal privilege]. If
>> you are not the intended recipient, please do not copy or disclose its
>> content but contact the sender immediately upon receipt.
>>
>> KTech Services Ltd is registered in England as company number 10704940.
>>
>> Registered Office: The River Building, 1 Cousin Lane, London EC4R 3TE,
>> United Kingdom
>>
>
This email is confidential [and may be protected by legal privilege]. If you are not the intended recipient, please do not copy or disclose its content but contact the sender immediately upon receipt.
KTech Services Ltd is registered in England as company number 10704940.
Registered Office: The River Building, 1 Cousin Lane, London EC4R 3TE, United Kingdom
Re: Exporting parquet, issues with schema
Posted by Markus Kemper <ma...@cloudera.com>.
Hello Marcelo,
Have you tried using hcatalog to export Parquet?
$ sqoop export --connect $MYSQL_CONN --username $MYSQL_USER --password
$MYSQL_PWD --table $MYSQL_TABLE --num-mappers 1 --hcatalog-database
$HIVE_DATABASE --hcatalog-table $HIVE_PARQUET_TABLE
Markus Kemper
Cloudera Support
On Fri, May 24, 2019 at 7:58 AM Marcelo Valle <ma...@ktech.com>
wrote:
> Hi,
>
> I know this thread is a little old, I found the original discussion here:
> https://grokbase.com/t/sqoop/user/15c94s4vjx/exporting-parquet-issues-with-schema
>
> But I am having the same problem now, more than 3 years later, and scoop 2
> seems to have been deprecated.
>
> Are there any plans of not needing this metadata folder anymore?
>
> Thanks,
> Marcelo.
>
>
>
> This email is confidential [and may be protected by legal privilege]. If
> you are not the intended recipient, please do not copy or disclose its
> content but contact the sender immediately upon receipt.
>
> KTech Services Ltd is registered in England as company number 10704940.
>
> Registered Office: The River Building, 1 Cousin Lane, London EC4R 3TE,
> United Kingdom
>