You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@devlake.apache.org by GitBox <gi...@apache.org> on 2022/10/25 03:26:00 UTC

[GitHub] [incubator-devlake] hezyin commented on issue #3123: [Feature][Framework][POC] Streamed data collection using Singer Spec

hezyin commented on issue #3123:
URL: https://github.com/apache/incubator-devlake/issues/3123#issuecomment-1289926488

   @keon94 Thanks for the summary, Keon.
   
   Regarding the current singer-spec framework implementation: it's worth considering bringing back the raw data layer for two reasons: easier e2e testing and improved dev efficiency when modifying extraction/conversion logic. It's also not too much work as singer tap naturally spits out JSON messages and we just need to dump those JSON into our raw layer. Airbyte's taking a similar approach I believe.
   
   
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@devlake.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org