Create dataset from data frame with missing columns

[to...@gmail.com]

Is it possible to concisely create a dataset from a dataframe with missing columns? Specifically, suppose I create a dataframe with:
val df: DataFrame  = Seq(("v1"),("v2")).toDF("f1")
 
Then, I have a case class for a dataset defined as:
case class CC(f1: String, f2: Option[String] = None)
 
I’d like to use df.as[CC] to get an instance of the case class, but this gives me the following error:
org.apache.spark.sql.AnalysisException: cannot resolve '`f2`' given input columns: [f1];
 
Is there a concise way to use the default values as defined by the case class?

Warmest Regards,
Jason Tokayer, PhD
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org