You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2017/09/16 11:42:01 UTC
[jira] [Resolved] (SPARK-22019) JavaBean int type property
[ https://issues.apache.org/jira/browse/SPARK-22019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-22019.
----------------------------------
Resolution: Invalid
> JavaBean int type property
> ---------------------------
>
> Key: SPARK-22019
> URL: https://issues.apache.org/jira/browse/SPARK-22019
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.0
> Reporter: taiho choi
>
> when the type of SampleData's id is int, following code generates errors.
> when long, it's ok.
>
> {code:java}
> @Test
> public void testDataSet2() {
> ArrayList<String> arr= new ArrayList();
> arr.add("{\"str\": \"everyone\", \"id\": 1}");
> arr.add("{\"str\": \"Hello\", \"id\": 1}");
> //1.read array and change to string dataset.
> JavaRDD<String> data = sc.parallelize(arr);
> Dataset<String> stringdataset = sqc.createDataset(data.rdd(), Encoders.STRING());
> stringdataset.show(); //PASS
> //2. convert string dataset to sampledata dataset
> Dataset<SampleData> df = sqc.read().json(stringdataset).as(Encoders.bean(SampleData.class));
> df.show();//PASS
> df.printSchema();//PASS
> Dataset<SampleDataFlat> fad = df.flatMap(SampleDataFlat::flatMap, Encoders.bean(SampleDataFlat.class));
> fad.show(); //ERROR
> fad.printSchema();
> }
> public static class SampleData implements Serializable {
> public String getStr() {
> return str;
> }
> public void setStr(String str) {
> this.str = str;
> }
> public int getId() {
> return id;
> }
> public void setId(int id) {
> this.id = id;
> }
> String str;
> int id;
> }
> public static class SampleDataFlat {
> String str;
> public String getStr() {
> return str;
> }
> public void setStr(String str) {
> this.str = str;
> }
> public SampleDataFlat(String str, long id) {
> this.str = str;
> }
> public static Iterator<SampleDataFlat> flatMap(SampleData data) {
> ArrayList<SampleDataFlat> arr = new ArrayList<>();
> arr.add(new SampleDataFlat(data.getStr(), data.getId()));
> arr.add(new SampleDataFlat(data.getStr(), data.getId()+1));
> arr.add(new SampleDataFlat(data.getStr(), data.getId()+2));
> return arr.iterator();
> }
> }
> {code}
> ==Error message==
> Caused by: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 38, Column 16: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 38, Column 16: No applicable constructor/method found for actual parameters "long"; candidates are: "public void SparkUnitTest$SampleData.setId(int)"
> /* 024 */ public java.lang.Object apply(java.lang.Object _i) {
> /* 025 */ InternalRow i = (InternalRow) _i;
> /* 026 */
> /* 027 */ final SparkUnitTest$SampleData value1 = false ? null : new SparkUnitTest$SampleData();
> /* 028 */ this.javaBean = value1;
> /* 029 */ if (!false) {
> /* 030 */
> /* 031 */
> /* 032 */ boolean isNull3 = i.isNullAt(0);
> /* 033 */ long value3 = isNull3 ? -1L : (i.getLong(0));
> /* 034 */
> /* 035 */ if (isNull3) {
> /* 036 */ throw new NullPointerException(((java.lang.String) references[0]));
> /* 037 */ }
> /* 038 */ javaBean.setId(value3);
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org