You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Benjamin BONNET <be...@m4x.org> on 2016/10/06 09:26:37 UTC
Re: Review Request 50655: SQOOP-2991 : endless failed Netezza import
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50655/
-----------------------------------------------------------
(Updated oct. 6, 2016, 9:26 matin)
Review request for Sqoop, David Robson, Jarek Cecho, and Kathleen Ting.
Changes
-------
fix compilation issue, thanks Liz
Repository: sqoop-trunk
Description
-------
See on JIRA https://issues.apache.org/jira/browse/SQOOP-2991
Diffs (updated)
-----
src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java aa058d1
src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java 2efea53
src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd23
Diff: https://reviews.apache.org/r/50655/diff/
Testing
-------
Tested with requests that throw errors on Netezza side : importing with a user that has not enough rights to create an external table. Without patch, import fails but map reduce job never ends, with patch, import fails, map reduce job ends with an IOException.
Thanks,
Benjamin BONNET
Re: Review Request 50655: SQOOP-2991 : endless failed Netezza import
Posted by Erzsebet Szilagyi <er...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50655/#review155806
-----------------------------------------------------------
Ship it!
Thanks for fixing it!
- Erzsebet Szilagyi
On Oct. 6, 2016, 11:26 a.m., Benjamin BONNET wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50655/
> -----------------------------------------------------------
>
> (Updated Oct. 6, 2016, 11:26 a.m.)
>
>
> Review request for Sqoop, David Robson, Jarek Cecho, and Kathleen Ting.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> See on JIRA https://issues.apache.org/jira/browse/SQOOP-2991
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java aa058d1
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java 2efea53
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd23
>
> Diff: https://reviews.apache.org/r/50655/diff/
>
>
> Testing
> -------
>
> Tested with requests that throw errors on Netezza side : importing with a user that has not enough rights to create an external table. Without patch, import fails but map reduce job never ends, with patch, import fails, map reduce job ends with an IOException.
>
>
> Thanks,
>
> Benjamin BONNET
>
>
Re: Review Request 50655: SQOOP-2991 : endless failed Netezza import
Posted by Benjamin BONNET <be...@m4x.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50655/
-----------------------------------------------------------
(Updated juin 27, 2017, 8:27 après-midi)
Review request for Sqoop, David Robson, Jarek Cecho, and Kathleen Ting.
Repository: sqoop-trunk
Description
-------
See on JIRA https://issues.apache.org/jira/browse/SQOOP-2991
Diffs (updated)
-----
src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java aa058d14e
src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java 2efea53af
src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd235c
Diff: https://reviews.apache.org/r/50655/diff/4/
Changes: https://reviews.apache.org/r/50655/diff/3-4/
Testing
-------
Tested with requests that throw errors on Netezza side : importing with a user that has not enough rights to create an external table. Without patch, import fails but map reduce job never ends, with patch, import fails, map reduce job ends with an IOException.
Thanks,
Benjamin BONNET
Re: Review Request 50655: SQOOP-2991 : endless failed Netezza import
Posted by Benjamin BONNET <be...@m4x.org>.
> On d�c. 20, 2016, 11 matin, Attila Szabo wrote:
> > src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java, line 102
> > <https://reviews.apache.org/r/50655/diff/3/?file=1525232#file1525232line102>
> >
> > Hey Benjamin,
> >
> > Could you please describe why are we using LF,CR characters here?
> >
> > I'm a bit concerned, because AFAIK this line ending form is only supported by Microsoft Windows systems. On the other hand (if I'm not mistaken) there it's in the opposite direction (so CR,LF). Are we sure that this will also work on Unix systems? (Like Linux, BSD, OSX, etc.) Or is Netezza using this quite platform dependent way to end lines?
> >
> > Thanks for the clarification!
> >
> > Attila
Hi Attila,
Thanks for reviewing.
Actually, we just need to have something written into the fifo to "unlock" the reader and let the exception pop. This is not related to Netezza, it is just about named pipes.
You are right for the order (\r\n rather than \n\r) and a plain \n should do the trick for Unix. Actually, the code was tested on Linux only (with \n\r).
By the way I wonder if Sqoop is supposed to work on Windows when using Netezza direct mode : direct mode explicitly uses Unix native functions (mkfifo/mknod).
Regards
- Benjamin
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50655/#review159696
-----------------------------------------------------------
On oct. 6, 2016, 9:26 matin, Benjamin BONNET wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50655/
> -----------------------------------------------------------
>
> (Updated oct. 6, 2016, 9:26 matin)
>
>
> Review request for Sqoop, David Robson, Jarek Cecho, and Kathleen Ting.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> See on JIRA https://issues.apache.org/jira/browse/SQOOP-2991
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java aa058d1
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java 2efea53
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd23
>
> Diff: https://reviews.apache.org/r/50655/diff/
>
>
> Testing
> -------
>
> Tested with requests that throw errors on Netezza side : importing with a user that has not enough rights to create an external table. Without patch, import fails but map reduce job never ends, with patch, import fails, map reduce job ends with an IOException.
>
>
> Thanks,
>
> Benjamin BONNET
>
>
Re: Review Request 50655: SQOOP-2991 : endless failed Netezza import
Posted by Attila Szabo <ma...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50655/#review159696
-----------------------------------------------------------
src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java (line 101)
<https://reviews.apache.org/r/50655/#comment230718>
Hey Benjamin,
Could you please describe why are we using LF,CR characters here?
I'm a bit concerned, because AFAIK this line ending form is only supported by Microsoft Windows systems. On the other hand (if I'm not mistaken) there it's in the opposite direction (so CR,LF). Are we sure that this will also work on Unix systems? (Like Linux, BSD, OSX, etc.) Or is Netezza using this quite platform dependent way to end lines?
Thanks for the clarification!
Attila
- Attila Szabo
On Oct. 6, 2016, 9:26 a.m., Benjamin BONNET wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50655/
> -----------------------------------------------------------
>
> (Updated Oct. 6, 2016, 9:26 a.m.)
>
>
> Review request for Sqoop, David Robson, Jarek Cecho, and Kathleen Ting.
>
>
> Repository: sqoop-trunk
>
>
> Description
> -------
>
> See on JIRA https://issues.apache.org/jira/browse/SQOOP-2991
>
>
> Diffs
> -----
>
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java aa058d1
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java 2efea53
> src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd23
>
> Diff: https://reviews.apache.org/r/50655/diff/
>
>
> Testing
> -------
>
> Tested with requests that throw errors on Netezza side : importing with a user that has not enough rights to create an external table. Without patch, import fails but map reduce job never ends, with patch, import fails, map reduce job ends with an IOException.
>
>
> Thanks,
>
> Benjamin BONNET
>
>