[jira] [Commented] (CRUNCH-293) Injection of reader into AvroRecordReader

["Josh Wills (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CRUNCH-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822808#comment-13822808 ] 

Josh Wills commented on CRUNCH-293:
-----------------------------------

I'd prefer to merge them into one, but yeah, there will be some Scrunch changes required to support that. Scrunch has its own impl of ReflectData to support Scala collection types (e.g., Buffer). The key bit of code that is causing some trouble is in PTypeFamily.scala:

object Avros extends PTypeFamily {
  override def ptf = AvroTypeFamily.getInstance()

  CAvros.REFLECT_DATA_FACTORY = new ScalaReflectDataFactory()

  def reflects[T: ClassManifest]() = CAvros.reflects(classManifest[T].erasure)
}

That CAvros.REFLECT_DATA_FACTORY line would need to be updated w/a ScalaReflectDataFactory that supported the new interface.

> Injection of reader into AvroRecordReader
> -----------------------------------------
>
>                 Key: CRUNCH-293
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-293
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.7.0, 0.8.0
>            Reporter: Micah Whitacre
>            Assignee: Micah Whitacre
>         Attachments: CRUNCH-293.patch
>
>
> With CRUNCH-243, I wanted to support injecting custom readers to handle the cases like passivity between Avro Schema.  The changes made however were not complete as we also need to be able to inject a reader into the AvroRecordReader which constructs its own SpecificDatumReader.
> We could create a SpecificDataFactory which emulates the ReflectDataFactory.  Or simplify to a single DataFactory which will create either Reflect/Specific/Generic.  Thoughts?



--
This message was sent by Atlassian JIRA
(v6.1#6144)