[jira] [Resolved] (SPARK-21989) createDataset and the schema of encoder class

["Hyukjin Kwon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-21989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Hyukjin Kwon resolved SPARK-21989.
----------------------------------
    Resolution: Invalid

That's due to missing setter and getter for Java Bean. It works if both are set:

{code}
public class SampleData implements Serializable {
  private String str;

  public void setStr(String str) {
    this.str = str;
  }
  public String getStr() {
    return this.str;
  }
}
{code}

> createDataset and the schema of encoder class
> ---------------------------------------------
>
>                 Key: SPARK-21989
>                 URL: https://issues.apache.org/jira/browse/SPARK-21989
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.2.0
>            Reporter: taiho choi
>
> Hello.
> public class SampleData implements Serializable {
>     public String str;
> }
>         ArrayList<String> arr= new ArrayList();
>         arr.add("{\"str\": \"everyone\"}");
>         arr.add("{\"str\": \"Hello\"}");
>         JavaRDD<SampleData> data2 = sc.parallelize(arr).map(v -> {return new Gson().fromJson(v, SampleData.class);});
>         Dataset<SampleData> df = sqc.createDataset(data2.rdd(), Encoders.bean(SampleData.class));
>         df.printSchema();
> expected result of printSchema is str field of sampleData class.
> actual result is following.
> root
> and if i call df.show() it displays like following.
> ++
> ||
> ++
> ||
> ||
> ++
> what i expected is , "hello", "everyone" will be displayed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org