[jira] [Commented] (SQOOP-2453) Running sqoop with --as-parquet option always fails the container with a memory usage error

["Ron Gonzalez (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SQOOP-2453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14648035#comment-14648035 ] 

Ron Gonzalez commented on SQOOP-2453:
-------------------------------------

The entire data set with compression (I think default is snappy) was around 1 GB. By running it as a LocalJobRunner, I was able to get it to complete.

> Running sqoop with --as-parquet option always fails the container with a memory usage error
> -------------------------------------------------------------------------------------------
>
>                 Key: SQOOP-2453
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2453
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.6
>         Environment: Ubuntu Linux 64 bit
>            Reporter: Ron Gonzalez
>
> I tried invoking sqoop with the --as-parquet option. The default container size with the --as-avro option could handle any sized data load, but with the parquet option, it always fails with Yarn killing the mapper containers. I've increased the container size to 2 GB to no avail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)