You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by MooseSpark <Pa...@gmail.com> on 2015/08/18 09:40:54 UTC

Changed Column order in DataFrame.Columns call and insertIntoJDBC

I have a RDD which I am using to create the data frame based on one POJO, but
when Dataframe is created, the sequence of column order get changed.

DataFrame df=sqlCtx.createDataFrame(rdd, Pojo.class);

String[] columns=df.columns();
//columns here are of different order what has been defined in pojo
//in pojo properties are p1,p2,p3
//but in columns it is p3 p1 p2 and same is being saved into jdbc 
 
df.insertIntoJDBC("jdbc:sqlserver://xx.yyy.00.11:PORT;databaseName=spark_gpeh;user=saw;password=password@123;",
"Test", false);

any idea ?








--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Changed-Column-order-in-DataFrame-Columns-call-and-insertIntoJDBC-tp24309.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re:Changed Column order in DataFrame.Columns call and insertIntoJDBC

Posted by Todd <bi...@163.com>.
Take a look at the doc for the method:

 
      /**
   * Applies a schema to an RDD of Java Beans.
   *
   * WARNING: Since there is no guaranteed ordering for fields in a Java Bean,
   *          SELECT * queries will return the columns in an undefined order.
   * @group dataframes
   * @since 1.3.0
   */




At 2015-08-18 15:40:54, "MooseSpark" <Pa...@gmail.com> wrote:
>I have a RDD which I am using to create the data frame based on one POJO, but
>when Dataframe is created, the sequence of column order get changed.
>
>DataFrame df=sqlCtx.createDataFrame(rdd, Pojo.class);
>
>String[] columns=df.columns();
>//columns here are of different order what has been defined in pojo
>//in pojo properties are p1,p2,p3
>//but in columns it is p3 p1 p2 and same is being saved into jdbc 
> 
>df.insertIntoJDBC("jdbc:sqlserver://xx.yyy.00.11:PORT;databaseName=spark_gpeh;user=saw;password=password@123;",
>"Test", false);
>
>any idea ?
>
>
>
>
>
>
>
>
>--
>View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Changed-Column-order-in-DataFrame-Columns-call-and-insertIntoJDBC-tp24309.html
>Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>For additional commands, e-mail: user-help@spark.apache.org
>