[jira] [Commented] (SQOOP-2907) Export parquet files to RDBMS: don't require .metadata for parquet files

["Ram (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17186551#comment-17186551 ] 

Ram commented on SQOOP-2907:
----------------------------

[~yuan_zac] [~sanysandish@gmail.com] [~vasas]

We are using *sqoop 1.4.7* to upload parquet data that is stored in HDFS - *Plain parquet files and NOT a Hive table*

**We're still facing the same issue - 

 
{code:java}
20/08/28 13:37:02 ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetIOException: Cannot access descriptor location: hdfs:///<location>/part-00000-f9f92493-36a1-4714-bcc6-291c118cf599-c000/snappy/parquet/.metadata
org.kitesdk.data.DatasetIOException: Cannot access descriptor location:  hdfs:///<location>/part-00000-f9f92493-36a1-4714-bcc6-291c118cf599-c000/snappy/parquet/.metadata{code}
The command we're running - 

 
{code:java}
/sqoop-1.4.7.bin__hadoop-2.6.0/bin/sqoop export --connect jdbc:postgresql://<postgres_db_details> --username <username> --password <password> --table <table_name> --export-dir hdfs:///<location>/part-00000-f9f92493-36a1-4714-bcc6-291c118cf599-c000.parquet
{code}
Postgres JAR - postgresql-42.2.11.jar

Please do suggest a solution ASAP.

 

 

> Export parquet files to RDBMS: don't require .metadata for parquet files
> ------------------------------------------------------------------------
>
>                 Key: SQOOP-2907
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2907
>             Project: Sqoop
>          Issue Type: Improvement
>          Components: metastore
>    Affects Versions: 1.4.6
>         Environment: sqoop 1.4.6
> export parquet files to Oracle
>            Reporter: Ruslan Dautkhanov
>            Assignee: Sandish Kumar HN
>            Priority: Major
>              Labels: sqoop
>         Attachments: SQOOP-2907-3.patch, SQOOP-2907.patch, SQOOP-2907.patch1, SQOOP-2907.patch2
>
>
> Kite currently requires .metadata.
> Parquet files have their own metadata stored along data files.
> It would be great for Export operation on parquet files to RDBMS not to require .metadata.
> We have most of the files created by Spark and Hive, and they don't create .metadata, it only Kite that does.
> It makes sqoop export of parquet files usability very limited.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)