You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@sqoop.apache.org by Eric <en...@gmail.com> on 2015/07/27 11:59:03 UTC

Export error on Amazon EMR : Tried to end inactive copy

Hi!

I tried to run a sqoop export command on Amazon EMR. I succeed running it
on a small dataset, but with a bigger one, I get this error on the cluster:

Error: java.io.IOException: org.postgresql.util.PSQLException: Tried to end
inactive copy
    at
org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:145)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:160)
    at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170)
Caused by: org.postgresql.util.PSQLException: Tried to end inactive copy
    at
org.postgresql.core.v3.QueryExecutorImpl.endCopy(QueryExecutorImpl.java:814)
    at org.postgresql.core.v3.CopyInImpl.endCopy(CopyInImpl.java:59)
    at
org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:142)
    ... 9 more

And my command is : ./sqoop export --direct --connect
jdbc:postgresql://xxxx --username xxxx --table updates_test --export-dir
s3://xxx  -P --fields-terminated-by ',' --enclosed-by '\"' --null-string ''
--null-non-string ''

I was wondering if I missed anything ?

Thank you for your help.

Re: Export error on Amazon EMR : Tried to end inactive copy

Posted by Abraham Elmahrek <ab...@cloudera.com>.
Hey man,

I haven't tried this on EMR, but I feel like the copy has been closed
before. Maybe cleanup is called multiple times. Could you file a Jira and
don't use --direct mode for now?

-Abe

On Mon, Jul 27, 2015 at 2:59 AM, Eric <en...@gmail.com> wrote:

> Hi!
>
> I tried to run a sqoop export command on Amazon EMR. I succeed running it
> on a small dataset, but with a bigger one, I get this error on the cluster:
>
> Error: java.io.IOException: org.postgresql.util.PSQLException: Tried to
> end inactive copy
>     at
> org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:145)
>     at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:160)
>     at
> org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
>     at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773)
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>     at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175)
>     at java.security.AccessController.doPrivileged(Native Method)
>     at javax.security.auth.Subject.doAs(Subject.java:415)
>     at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
>     at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170)
> Caused by: org.postgresql.util.PSQLException: Tried to end inactive copy
>     at
> org.postgresql.core.v3.QueryExecutorImpl.endCopy(QueryExecutorImpl.java:814)
>     at org.postgresql.core.v3.CopyInImpl.endCopy(CopyInImpl.java:59)
>     at
> org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:142)
>     ... 9 more
>
> And my command is : ./sqoop export --direct --connect
> jdbc:postgresql://xxxx --username xxxx --table updates_test --export-dir
> s3://xxx  -P --fields-terminated-by ',' --enclosed-by '\"' --null-string ''
> --null-non-string ''
>
> I was wondering if I missed anything ?
>
> Thank you for your help.
>