You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Guiju Zhang (JIRA)" <ji...@apache.org> on 2019/03/20 07:44:00 UTC
[jira] [Created] (SPARK-27211) cast error when select column from
Row
Guiju Zhang created SPARK-27211:
-----------------------------------
Summary: cast error when select column from Row
Key: SPARK-27211
URL: https://issues.apache.org/jira/browse/SPARK-27211
Project: Spark
Issue Type: Question
Components: Java API
Affects Versions: 2.3.1, 2.3.0
Reporter: Guiju Zhang
(1) RawLogPlayload has an field: long timestamp
(2)
extractedRawTc.printSchema(); // output1
Dataset<RawLogPayload> extractedRawW3cFilled = extractedRawW3c.alias("extractedRawW3c")
.join(extractedRawTc.alias("extractedRawTc"), functions.col("extractedRawW3c.rawsessionid").equalTo(functions.col("extractedRawTc.rawsessionid")), "inner")
.select(functions.col("extractedRawW3c.df_logdatetime"), functions.col("extractedRawW3c.rawsessionid"), functions.col("extractedRawTc.uid"),
functions.col("extractedRawW3c.time"),functions.col("extractedRawW3c.T"),functions.col("extractedRawW3c.url"),functions.col("extractedRawW3c.wid"),
functions.col("extractedRawW3c.tid"), functions.col("extractedRawW3c.fid"),functions.col("extractedRawW3c.string1"),
functions.col("extractedRawW3c.curWindow"), *functions.col("extractedRawW3c.timestamp")*)
.as(Encoders.bean(RawLogPayload.class));
extractedRawW3cFilled.printSchema(); // output2
(4) cast exception
2019-03-20 15:28:31 ERROR CodeGenerator:91 ## failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 103, Column 32: No applicable constructor/method found for actual parameters "org.apache.spark.unsafe.types.UTF8String"; candidates are: "public void com.microsoft.datamining.spartan.api.core.RawLogPayload.setTimestamp(long)"
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 103, Column 32: *No applicable constructor/method found for actual parameters "org.apache.spark.unsafe.types.UTF8String"; candidates are: "public void com.**xxxx**.**xxxx**.spartan.api.core.RawLogPayload.setTimestamp(long)"*
at org.codehaus.janino.UnitCompiler.compileError(UnitCompiler.java:11821)
at org.codehaus.janino.UnitCompiler.findMostSpecificIInvocable(UnitCompiler.java:8910)
Output1 extractedRawTc schema
root
|-- curWindow: string (nullable = true)
|-- df_logdatetime: string (nullable = true)
|-- fid: string (nullable = true)
|-- rawsessionid: string (nullable = true)
|-- string1: string (nullable = true)
|-- t: string (nullable = true)
|-- tid: string (nullable = true)
|-- time: string (nullable = true)
|-- *timestamp: long (nullable = true)*
|-- uid: string (nullable = true)
|-- url: string (nullable = true)
|-- wid: string (nullable = true)
Output2 extractedRawW3cFilled schema
root
|-- df_logdatetime: string (nullable = true)
|-- rawsessionid: string (nullable = true)
|-- uid: string (nullable = true)
|-- time: string (nullable = true)
|-- T: string (nullable = true)
|-- url: string (nullable = true)
|-- wid: string (nullable = true)
|-- tid: string (nullable = true)
|-- fid: string (nullable = true)
|-- string1: string (nullable = true)
|-- curWindow: string (nullable = true)
|-- *timestamp: long (nullable = true)*
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org