[GitHub] [spark] mazeboard edited a comment on issue #24367: [SPARK-27457][SQL] modify bean encoder to support avro objects

[GitBox <gi...@apache.org>]

mazeboard edited a comment on issue #24367: [SPARK-27457][SQL] modify bean encoder to support avro objects
URL: https://github.com/apache/spark/pull/24367#issuecomment-493660597
 
 
   The following example shows why I prefer the solution that modifies ScalaReflection (https://github.com/apache/spark/pull/24299)
   
   In the example below, If we use bean encoder we must declare a tuple encoder as shown:
   ```
       implicit val barcodeEncoder = Encoders.bean[Barcode](classOf[Barcode]).asInstanceOf[ExpressionEncoder[Barcode]]
       implicit val tuplEncoder = Encoders.tuple[String, Money, CrpAddDesc](
         Encoders.STRING,
         Encoders.bean[Money](classOf[Money]).asInstanceOf[ExpressionEncoder[Money]],
         Encoders.bean[CrpAddDesc](classOf[CrpAddDesc]).asInstanceOf[ExpressionEncoder[CrpAddDesc]])
   ...
   ...
       val ds: Dataset[Barcode] = List(barcode).toDS()
       val x: Dataset[(String, Money, CrpAddDesc)] = ds.map(a => {
         (
           a.getBarcode,
           a.getPrdTaxVal,
           a.getCrpAttributes.get(0).getCrpAddDesc.get(0))
       })
   ```
   
   if we do not declare the tuple encoder we get an error `No Encoder found for common.lib.v1.Money`
   
   But with the ScalaReflection (https://github.com/apache/spark/pull/24299) modification we just need to declare an encoder for Barcode as shown below:
   
   ```
       implicit val barcodeEncoder = ExpressionEncoder[Barcode]()
         ...
         ...
       val ds: Dataset[Barcode] = List(barcode).toDS()
       val x: Dataset[(String, Money, CrpAddDesc)] = ds.map(a => {
         (
           a.getBarcode,
           a.getPrdTaxVal,
           a.getCrpAttributes.get(0).getCrpAddDesc.get(0))
       })
   ```
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org