You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2021/05/08 08:45:15 UTC

[GitHub] [hudi] sathyaprakashg commented on pull request #2012: [HUDI-1129] Deltastreamer Add support for schema evolution

sathyaprakashg commented on pull request #2012:
URL: https://github.com/apache/hudi/pull/2012#issuecomment-835216648


   > @sathyaprakashg and others: trying to understand the use-case here. I understand its related to deltastreamer receiving events in old schema after Hudi's dataset schema got evolved. what's the schema from schema provider when source is producing events in old schema (after schema got evolved w/ hudi dataset)? if the schema provider's schema is updated, I guess there is no need to store the writer schema w/ payload.
   > AvroConversionUtils.createDataFrame() will ensure to convert the JavaRDD w/ old schema to Dataset w/ new schema if schemaProvider.SourceSchema() has the evolved schema.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org