You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Boris Partensky (Created) (JIRA)" <ji...@apache.org> on 2012/02/29 13:43:57 UTC
[jira] [Created] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
boolean fields get nullified during postgres direct import into hive.
---------------------------------------------------------------------
Key: SQOOP-448
URL: https://issues.apache.org/jira/browse/SQOOP-448
Project: Sqoop
Issue Type: Bug
Components: connectors/postgresql
Affects Versions: 1.3.0
Reporter: Boris Partensky
Fix For: 1.3.0
These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13219720#comment-13219720 ]
Bilung Lee commented on SQOOP-448:
----------------------------------
Thanks for your patch, Boris! Could you please upload your patch to the review board (http://reviews.apache.org)? This would facilitate the review process. Thanks!
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bilung Lee resolved SQOOP-448.
------------------------------
Resolution: Fixed
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.4.2-incubating
>
> Attachments: sqoop.patch, sqoop_diffs1.txt, sqoop_diffs2.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Partensky updated SQOOP-448:
----------------------------------
Status: Patch Available (was: Open)
fix
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225662#comment-13225662 ]
Boris Partensky commented on SQOOP-448:
---------------------------------------
Thanks.
https://reviews.apache.org/r/4263
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225501#comment-13225501 ]
Bilung Lee commented on SQOOP-448:
----------------------------------
This might be related to how you generate your patch.
Below are the steps you may follow:
1. Check out the code from the trunk as shown in the link below:
http://cwiki.apache.org/confluence/display/SQOOP/Setting+up+Development+Environment#SettingupDevelopmentEnvironment-BuildingtheSources
2. After making the changes, generate the patch as shown in the link below:
http://cwiki.apache.org/confluence/display/SQOOP/How+to+Contribute#HowtoContribute-ProvidingPatches
3. Upload the patch to the review board:
- If you are using SVN to generate patch, choose "Sqoop" as repository and type in "." as base directory.
- If you are using GIT to generate patch, choose "sqoop-git" as repository.
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222612#comment-13222612 ]
Boris Partensky commented on SQOOP-448:
---------------------------------------
Thanks a log Bilung for this pointer. I changed my code to use type names instead and currently testing it here internally. Will upload the diffs as soon as they are verified here. I tried merging with SQOOP-352 diffs but turns out that those were created against totally different packages - I have
com.cloudera.sqoop and those are made in org.apache. And the code is slightly different.
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230387#comment-13230387 ]
Hudson commented on SQOOP-448:
------------------------------
Integrated in Sqoop-ant-jdk-1.6 #98 (See [https://builds.apache.org/job/Sqoop-ant-jdk-1.6/98/])
SQOOP-448 boolean fields get nullified during postgres direct import into hive.
(fix a problem in previous patch) (Revision 1301118)
Result = SUCCESS
blee : http://svn.apache.org/viewvc/?view=rev&rev=1301118
Files :
* /incubator/sqoop/trunk/src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.4.2-incubating
>
> Attachments: sqoop.patch, sqoop_diffs1.txt, sqoop_diffs2.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Partensky updated SQOOP-448:
----------------------------------
Attachment: sqoop_diffs2.txt
address a string length logic bug
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.4.2-incubating
>
> Attachments: sqoop.patch, sqoop_diffs1.txt, sqoop_diffs2.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bilung Lee updated SQOOP-448:
-----------------------------
Fix Version/s: (was: 1.3.0)
1.4.2-incubating
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.4.2-incubating
>
> Attachments: sqoop.patch, sqoop_diffs1.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bilung Lee reassigned SQOOP-448:
--------------------------------
Assignee: Boris Partensky
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Reopened) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Partensky reopened SQOOP-448:
-----------------------------------
a bug has been discovered in OR condition in getCopyCommand method.
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.4.2-incubating
>
> Attachments: sqoop.patch, sqoop_diffs1.txt, sqoop_diffs2.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Partensky updated SQOOP-448:
----------------------------------
Attachment: sqoop_diffs1.txt
final patch, reviewed on rb
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch, sqoop_diffs1.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bilung Lee updated SQOOP-448:
-----------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed. Thanks, Boris!
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch, sqoop_diffs1.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13228717#comment-13228717 ]
Hudson commented on SQOOP-448:
------------------------------
Integrated in Sqoop-ant-jdk-1.6 #97 (See [https://builds.apache.org/job/Sqoop-ant-jdk-1.6/97/])
SQOOP-448 boolean fields get nullified during postgres direct import into hive. (Revision 1300345)
Result = SUCCESS
blee : http://svn.apache.org/viewvc/?view=rev&rev=1300345
Files :
* /incubator/sqoop/trunk/src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch, sqoop_diffs1.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230359#comment-13230359 ]
Bilung Lee commented on SQOOP-448:
----------------------------------
Thanks for the quick response and update. The problem is corrected now. Thanks!
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.4.2-incubating
>
> Attachments: sqoop.patch, sqoop_diffs1.txt, sqoop_diffs2.txt
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Partensky updated SQOOP-448:
----------------------------------
Attachment: sqoop.patch
diffs for the fix
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Bilung Lee (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13222699#comment-13222699 ]
Bilung Lee commented on SQOOP-448:
----------------------------------
Thanks for your contribution! The patch will eventually be committed into the trunk, and thus the convention is to base the patch on the version of code in the trunk.
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Partensky updated SQOOP-448:
----------------------------------
Status: Patch Available (was: Open)
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Partensky updated SQOOP-448:
----------------------------------
Status: Open (was: Patch Available)
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (SQOOP-448) boolean fields get nullified during
postgres direct import into hive.
Posted by "Boris Partensky (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SQOOP-448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225435#comment-13225435 ]
Boris Partensky commented on SQOOP-448:
---------------------------------------
Hi, I am having problems submitting my diffs to reviews.apache.org.
I specified Sqoop as repository, "https://svn.apache.org/repos/asf/incubator/sqoop/trunk/src/java" as base, that's where in my working copy I generated the diffs. When I try to submit the diffs, it tells me -
The file 'https://svn.apache.org/repos/asf/incubator/sqoop/trunk/src/java/org/apache/sqoop/manager/DirectPostgresqlManager.java' (r1298521) could not be found in the repository
> boolean fields get nullified during postgres direct import into hive.
> ---------------------------------------------------------------------
>
> Key: SQOOP-448
> URL: https://issues.apache.org/jira/browse/SQOOP-448
> Project: Sqoop
> Issue Type: Bug
> Components: connectors/postgresql
> Affects Versions: 1.3.0
> Reporter: Boris Partensky
> Assignee: Boris Partensky
> Labels: patch
> Fix For: 1.3.0
>
> Attachments: sqoop.patch
>
>
> These diffs address this issue - https://issues.cloudera.org/browse/SQOOP-43.
> I changed DirectPostgresManager.java to special-case rewrite COPY.... command for boolean fields so that hive sees "true"/"false". The only shortcoming with my patch is that because PG jdbc driver can not tell bit data types from booleans, direct import will break with SQL cast errors if someone has real bit fields. Not sure how often people use those, but it will break direct import for that one person who does. If you have an idea on how I can make this better, let me know. Diffs are attached
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira