You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Ted Yu <yu...@gmail.com> on 2016/01/26 16:49:59 UTC

Re: org.netezza.error.NzSQLException: ERROR: Invalid datatype - TEXT

Please take a look at getJDBCType() in:
sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala

You can register dialect for Netezza as shown
in sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala

Cheers

On Tue, Jan 26, 2016 at 7:26 AM, Kali.tummala@gmail.com <
Kali.tummala@gmail.com> wrote:

> Hi All,
>
> I am using Spark jdbc df to store data into Netezza , I think spark is
> trying to create table using data type TEXT for string column , netezza
> doesn't support data type text.
>
> how to overwrite spark method to use VARCHAR instead of data type text ?
>
> val
>
> sourcedfmode=sourcedf.persist(StorageLevel.MEMORY_AND_DISK).write.mode("overwrite")
> sourcedfmode.jdbc(TargetDBinfo.url,TargetDBinfo.table,targetprops)
>
> Thanks
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/org-netezza-error-NzSQLException-ERROR-Invalid-datatype-TEXT-tp26072.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Re: org.netezza.error.NzSQLException: ERROR: Invalid datatype - TEXT

Posted by Sri <ka...@gmail.com>.

Thanks Ted trick worked, need to commit this feature in next spark release.

Thanks
Sri

Sent from my iPhone

> On 26 Jan 2016, at 15:49, Ted Yu <yu...@gmail.com> wrote:
> 
> Please take a look at getJDBCType() in:
> sql/core/src/main/scala/org/apache/spark/sql/jdbc/PostgresDialect.scala
> 
> You can register dialect for Netezza as shown in sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala
> 
> Cheers
> 
>> On Tue, Jan 26, 2016 at 7:26 AM, Kali.tummala@gmail.com <Ka...@gmail.com> wrote:
>> Hi All,
>> 
>> I am using Spark jdbc df to store data into Netezza , I think spark is
>> trying to create table using data type TEXT for string column , netezza
>> doesn't support data type text.
>> 
>> how to overwrite spark method to use VARCHAR instead of data type text ?
>> 
>> val
>> sourcedfmode=sourcedf.persist(StorageLevel.MEMORY_AND_DISK).write.mode("overwrite")
>> sourcedfmode.jdbc(TargetDBinfo.url,TargetDBinfo.table,targetprops)
>> 
>> Thanks
>> 
>> 
>> 
>> --
>> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/org-netezza-error-NzSQLException-ERROR-Invalid-datatype-TEXT-tp26072.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>