You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hop.apache.org by "CarlosJuncher03 (via GitHub)" <gi...@apache.org> on 2026/02/10 12:57:34 UTC

[I] [Feature Request]: Dev Cache Mode / Step Checkpoints (development only) (hop)

CarlosJuncher03 opened a new issue, #6537:
URL: https://github.com/apache/hop/issues/6537

   ### What would you like to happen?
   
   Goal
   Speed up development and debugging cycles in Apache Hop pipelines by reusing a step output as a “checkpoint”, avoiding reprocessing the entire upstream on every run.
   
   Concept (proposal)
   
   The user marks a step as a checkpoint.
   
   Hop stores the step output (rows/dataset) in a local cache.
   
   On later runs, Hop can skip upstream steps and start from the checkpoint, loading data from cache.
   
   The user explicitly chooses:
   
   Run from the beginning
   
   Run from the last checkpoint
   
   Note
   Unlike a “test pipeline”, the idea is not to create dataset metadata. This is a development-only cache feature to speed up iteration, especially when upstream extraction (e.g., from a database) is slow. The user can define where to store the cache and how many rows to save. At runtime, they can run from the beginning or resume from the last checkpoint, without creating extra metadata or separate test pipelines.
   
   ### Issue Priority
   
   Priority: 3
   
   ### Issue Component
   
   Component: Hop Gui


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature Request]: Dev Cache Mode / Step Checkpoints (development only) (hop)

Posted by "CarlosJuncher03 (via GitHub)" <gi...@apache.org>.
CarlosJuncher03 commented on issue #6537:
URL: https://github.com/apache/hop/issues/6537#issuecomment-3877447908

   The idea behind this activity is to validate whether this is possible. It would be like an advanced development mode, similar to what some tools allow, which is to run from a specific saved point.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Feature Request]: Dev Cache Mode / Step Checkpoints (development only) (hop)

Posted by "el-wiss (via GitHub)" <gi...@apache.org>.
el-wiss commented on issue #6537:
URL: https://github.com/apache/hop/issues/6537#issuecomment-4250252799

   As a long-term user of a proprietary ETL product (FME), I'd consider this a HUGE usability boost for developing ETL pipelines in Hop! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org