You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Adline Dsilva <ad...@mimos.my> on 2015/09/04 05:20:43 UTC
SQOOP2 : delimter option for JDBC connector
Hi,
Is there any way to remove/replace delimiters in sqoop2 with Generic JDBC connector.
Currently Sqoop1 has option
--input-enclosed-by <char> Sets a required field encloser
--input-escaped-by <char> Sets the input escape character
--input-fields-terminated-by <char> Sets the input field separator
--input-lines-terminated-by <char> Sets the input end-of-line character
--input-optionally-enclosed-by <char> Sets a field enclosing character
--hive-drop-import-delims Drops \n, \r, and \01 from string fields when importing to Hive.
--hive-delims-replacement Replace \n, \r, and \01 from string fields with user defined string when importing to Hive.
Is there any similar option in sqoop2. Or anyway to implement this feature
Thanks & Regards,
Adline
________________________________
DISCLAIMER:
This e-mail (including any attachments) is for the addressee(s) only and may be confidential, especially as regards personal data. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message (including any attachments).
MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e-mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law.
RE: SQOOP2 : delimter option for JDBC connector
Posted by Adline Dsilva <ad...@mimos.my>.
Hi Abe,
I would like to take up this task. Also It would be great if you guys can give some pointers on where to start.
Regards,
Adline
________________________________________
From: Abraham Elmahrek [abe@cloudera.com]
Sent: 04 September 2015 17:11
To: dev@sqoop.apache.org; user@sqoop.apache.org
Subject: Re: SQOOP2 : delimter option for JDBC connector
Hey there,
These features hasn't been implemented yet, but it definitely can be
implemented (https://issues.apache.org/jira/browse/SQOOP-1522 to start). If
you're interested, I'd ask Richard if you could pick up this Jira.
-Abe
On Thu, Sep 3, 2015 at 8:20 PM Adline Dsilva <ad...@mimos.my> wrote:
> Hi,
>
> Is there any way to remove/replace delimiters in sqoop2 with Generic
> JDBC connector.
>
> Currently Sqoop1 has option
>
> --input-enclosed-by <char> Sets a required field encloser --input-escaped-by
> <char> Sets the input escape character --input-fields-terminated-by <char> Sets
> the input field separator --input-lines-terminated-by <char> Sets the
> input end-of-line character --input-optionally-enclosed-by <char> Sets a
> field enclosing character
>
>
> --hive-drop-import-delims Drops *\n*, *\r*, and *\01* from string fields
> when importing to Hive. --hive-delims-replacement Replace *\n*, *\r*, and
> *\01* from string fields with user defined string when importing to Hive.
>
>
>
>
> Is there any similar option in sqoop2. Or anyway to implement this
> feature
>
>
>
> Thanks & Regards,
>
> Adline
>
> ------------------------------
> *DISCLAIMER:*
>
> This e-mail (including any attachments) is for the addressee(s) only and
> may be confidential, especially as regards personal data. If you are not
> the intended recipient, please note that any dealing, review, distribution,
> printing, copying or use of this e-mail is strictly prohibited. If you have
> received this email in error, please notify the sender immediately and
> delete the original message (including any attachments).
>
> MIMOS Berhad is a research and development institution under the purview
> of the Malaysian Ministry of Science, Technology and Innovation. Opinions,
> conclusions and other information in this e-mail that do not relate to the
> official business of MIMOS Berhad and/or its subsidiaries shall be
> understood as neither given nor endorsed by MIMOS Berhad and/or its
> subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts
> responsibility for the same. All liability arising from or in connection
> with computer viruses and/or corrupted e-mails is excluded to the fullest
> extent permitted by law.
>
________________________________
DISCLAIMER:
This e-mail (including any attachments) is for the addressee(s) only and may be confidential, especially as regards personal data. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message (including any attachments).
MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e-mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law.
RE: SQOOP2 : delimter option for JDBC connector
Posted by Adline Dsilva <ad...@mimos.my>.
Hi Abe,
I would like to take up this task. Also It would be great if you guys can give some pointers on where to start.
Regards,
Adline
________________________________________
From: Abraham Elmahrek [abe@cloudera.com]
Sent: 04 September 2015 17:11
To: dev@sqoop.apache.org; user@sqoop.apache.org
Subject: Re: SQOOP2 : delimter option for JDBC connector
Hey there,
These features hasn't been implemented yet, but it definitely can be
implemented (https://issues.apache.org/jira/browse/SQOOP-1522 to start). If
you're interested, I'd ask Richard if you could pick up this Jira.
-Abe
On Thu, Sep 3, 2015 at 8:20 PM Adline Dsilva <ad...@mimos.my> wrote:
> Hi,
>
> Is there any way to remove/replace delimiters in sqoop2 with Generic
> JDBC connector.
>
> Currently Sqoop1 has option
>
> --input-enclosed-by <char> Sets a required field encloser --input-escaped-by
> <char> Sets the input escape character --input-fields-terminated-by <char> Sets
> the input field separator --input-lines-terminated-by <char> Sets the
> input end-of-line character --input-optionally-enclosed-by <char> Sets a
> field enclosing character
>
>
> --hive-drop-import-delims Drops *\n*, *\r*, and *\01* from string fields
> when importing to Hive. --hive-delims-replacement Replace *\n*, *\r*, and
> *\01* from string fields with user defined string when importing to Hive.
>
>
>
>
> Is there any similar option in sqoop2. Or anyway to implement this
> feature
>
>
>
> Thanks & Regards,
>
> Adline
>
> ------------------------------
> *DISCLAIMER:*
>
> This e-mail (including any attachments) is for the addressee(s) only and
> may be confidential, especially as regards personal data. If you are not
> the intended recipient, please note that any dealing, review, distribution,
> printing, copying or use of this e-mail is strictly prohibited. If you have
> received this email in error, please notify the sender immediately and
> delete the original message (including any attachments).
>
> MIMOS Berhad is a research and development institution under the purview
> of the Malaysian Ministry of Science, Technology and Innovation. Opinions,
> conclusions and other information in this e-mail that do not relate to the
> official business of MIMOS Berhad and/or its subsidiaries shall be
> understood as neither given nor endorsed by MIMOS Berhad and/or its
> subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts
> responsibility for the same. All liability arising from or in connection
> with computer viruses and/or corrupted e-mails is excluded to the fullest
> extent permitted by law.
>
________________________________
DISCLAIMER:
This e-mail (including any attachments) is for the addressee(s) only and may be confidential, especially as regards personal data. If you are not the intended recipient, please note that any dealing, review, distribution, printing, copying or use of this e-mail is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original message (including any attachments).
MIMOS Berhad is a research and development institution under the purview of the Malaysian Ministry of Science, Technology and Innovation. Opinions, conclusions and other information in this e-mail that do not relate to the official business of MIMOS Berhad and/or its subsidiaries shall be understood as neither given nor endorsed by MIMOS Berhad and/or its subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts responsibility for the same. All liability arising from or in connection with computer viruses and/or corrupted e-mails is excluded to the fullest extent permitted by law.
Re: SQOOP2 : delimter option for JDBC connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
Hey there,
These features hasn't been implemented yet, but it definitely can be
implemented (https://issues.apache.org/jira/browse/SQOOP-1522 to start). If
you're interested, I'd ask Richard if you could pick up this Jira.
-Abe
On Thu, Sep 3, 2015 at 8:20 PM Adline Dsilva <ad...@mimos.my> wrote:
> Hi,
>
> Is there any way to remove/replace delimiters in sqoop2 with Generic
> JDBC connector.
>
> Currently Sqoop1 has option
>
> --input-enclosed-by <char> Sets a required field encloser --input-escaped-by
> <char> Sets the input escape character --input-fields-terminated-by <char> Sets
> the input field separator --input-lines-terminated-by <char> Sets the
> input end-of-line character --input-optionally-enclosed-by <char> Sets a
> field enclosing character
>
>
> --hive-drop-import-delims Drops *\n*, *\r*, and *\01* from string fields
> when importing to Hive. --hive-delims-replacement Replace *\n*, *\r*, and
> *\01* from string fields with user defined string when importing to Hive.
>
>
>
>
> Is there any similar option in sqoop2. Or anyway to implement this
> feature
>
>
>
> Thanks & Regards,
>
> Adline
>
> ------------------------------
> *DISCLAIMER:*
>
> This e-mail (including any attachments) is for the addressee(s) only and
> may be confidential, especially as regards personal data. If you are not
> the intended recipient, please note that any dealing, review, distribution,
> printing, copying or use of this e-mail is strictly prohibited. If you have
> received this email in error, please notify the sender immediately and
> delete the original message (including any attachments).
>
> MIMOS Berhad is a research and development institution under the purview
> of the Malaysian Ministry of Science, Technology and Innovation. Opinions,
> conclusions and other information in this e-mail that do not relate to the
> official business of MIMOS Berhad and/or its subsidiaries shall be
> understood as neither given nor endorsed by MIMOS Berhad and/or its
> subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts
> responsibility for the same. All liability arising from or in connection
> with computer viruses and/or corrupted e-mails is excluded to the fullest
> extent permitted by law.
>
Re: SQOOP2 : delimter option for JDBC connector
Posted by Abraham Elmahrek <ab...@cloudera.com>.
Hey there,
These features hasn't been implemented yet, but it definitely can be
implemented (https://issues.apache.org/jira/browse/SQOOP-1522 to start). If
you're interested, I'd ask Richard if you could pick up this Jira.
-Abe
On Thu, Sep 3, 2015 at 8:20 PM Adline Dsilva <ad...@mimos.my> wrote:
> Hi,
>
> Is there any way to remove/replace delimiters in sqoop2 with Generic
> JDBC connector.
>
> Currently Sqoop1 has option
>
> --input-enclosed-by <char> Sets a required field encloser --input-escaped-by
> <char> Sets the input escape character --input-fields-terminated-by <char> Sets
> the input field separator --input-lines-terminated-by <char> Sets the
> input end-of-line character --input-optionally-enclosed-by <char> Sets a
> field enclosing character
>
>
> --hive-drop-import-delims Drops *\n*, *\r*, and *\01* from string fields
> when importing to Hive. --hive-delims-replacement Replace *\n*, *\r*, and
> *\01* from string fields with user defined string when importing to Hive.
>
>
>
>
> Is there any similar option in sqoop2. Or anyway to implement this
> feature
>
>
>
> Thanks & Regards,
>
> Adline
>
> ------------------------------
> *DISCLAIMER:*
>
> This e-mail (including any attachments) is for the addressee(s) only and
> may be confidential, especially as regards personal data. If you are not
> the intended recipient, please note that any dealing, review, distribution,
> printing, copying or use of this e-mail is strictly prohibited. If you have
> received this email in error, please notify the sender immediately and
> delete the original message (including any attachments).
>
> MIMOS Berhad is a research and development institution under the purview
> of the Malaysian Ministry of Science, Technology and Innovation. Opinions,
> conclusions and other information in this e-mail that do not relate to the
> official business of MIMOS Berhad and/or its subsidiaries shall be
> understood as neither given nor endorsed by MIMOS Berhad and/or its
> subsidiaries and neither MIMOS Berhad nor its subsidiaries accepts
> responsibility for the same. All liability arising from or in connection
> with computer viruses and/or corrupted e-mails is excluded to the fullest
> extent permitted by law.
>