Error when trying to filter

[Marco Mans <ma...@telemans.de>]

Hi!

I'm trying to execute this code:

StructField datetime = new StructField("DateTime", DataTypes.DateType,
true, Metadata.empty());
StructField tagname = new StructField("Tagname", DataTypes.StringType,
true, Metadata.empty());
StructField value = new StructField("Value", DataTypes.DoubleType,
false, Metadata.empty());
StructField quality = new StructField("Quality",
DataTypes.IntegerType, true, Metadata.empty());

StructType schema = new StructType(new StructField[]{datetime,
tagname, value, quality});

Dataset<Row> allData = spark.read().option("header",
"true").option("dateFormat", "yyyy-MM-dd
HH:mm:ss.SSS").option("comment",
"-").schema(schema).csv("/ingest/davis/landing");

allData = allData.filter((Row value1) -> {
    // SOME COOL FILTER-CODE.....
    return true;
});

allData.show();

I get this error on the executors:

java.lang.NoSuchMethodError:
org.apache.commons.lang3.time.FastDateFormat.parse(Ljava/lang/String;)Ljava/util/Date;


I'm running spark 2.0.0.cloudera1


Does anyone know why this error occurs?


Regards,

Marco