[GitHub] [spark] mazeboard opened a new pull request #24367: [SPARK-27457][SQL] modify bean encoder to support avro objects

[GitBox <gi...@apache.org>]

mazeboard opened a new pull request #24367: [SPARK-27457][SQL] modify bean encoder to support avro objects
URL: https://github.com/apache/spark/pull/24367
 
 
   ## What changes were proposed in this pull request?
   Currently we modified JavaTypeInference to be able to create encoders for Avro objects; we have now two solutions, the one in the PR https://github.com/apache/spark/pull/24299 and this PR (fewer code changes); which one is better? 
   
   ## How was this patch tested?
   
   We added one test in ExpressionencoderSuite and used the following program to test it locally:
   
         implicit val avroExampleEncoder = Encoders.bean[AvroExample1](classOf[AvroExample1]).asInstanceOf[ExpressionEncoder[AvroExample1]]
         val input: AvroExample1 = AvroExample1.newBuilder()
           .setMyarray(List("Foo", "Bar").asJava)
           .setMyboolean(true)
           .setMybytes(java.nio.ByteBuffer.wrap("MyBytes".getBytes()))
           .setMydouble(2.5)
           .setMyfixed(new Magic("magic".getBytes))
           .setMyfloat(25.0F)
           .setMyint(100)
           .setMylong(10L)
           .setMystring("hello")
           .setMymap(Map(
             "foo" -> new java.lang.Integer(1),
             "bar" -> new java.lang.Integer(2)).asJava)
           .setMymoney(Money.newBuilder().setAmount(100.0F).setCurrency(Currency.EUR).build())
           .build()
     
         val row: InternalRow = avroExampleEncoder.toRow(input)
     
         val output: AvroExample1 = avroExampleEncoder.resolveAndBind().fromRow(row)
     
         val ds: Dataset[AvroExample1] = List(input).toDS()
     
         println(ds.schema)
         println(ds.collect().toList)
     
         ds.write.format("avro").save("example1")
     
         val fooDF = spark.read.format("avro").load("example1")
     
         val fooDS = fooDF.as[AvroExample1]
     
         println(fooDS.collect().toList)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org