You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hop.apache.org by "dmainou (via GitHub)" <gi...@apache.org> on 2025/09/02 00:54:35 UTC
[I] [Feature Request]: Merge diff "changed" fields enhancement (hop)
dmainou opened a new issue, #5660:
URL: https://github.com/apache/hop/issues/5660
### What would you like to happen?
# Story:
Upgrade the merge-diff step to include the ability to:
1. add a 'changes' output field that contains a json with the changes that resulted in a 'changed' flag field.
2. select whether the 'changes' field contains the reference, compare or both key value pairs.
3. select whether the 'changes' field contains the keys used to compare
# Uses
1 When validating 2 sources it would enable to identify the offending fields that led to a row flagged as changed without heavy lifting.
2. When validating the status of an API it would enable to then update the 'remote' system by putting/patching the api with the fields that differ from the internal system.
Sample output 1 "Both" : { "changes" : [ "fieldA" : { "from" : 111, "to" : 222 }, ... ] }
Sample output 2 "Reference"/"Compare" : {"changes": [fieldId": 1, "fieldName" : "updated name"......]}
Sample output 3 "Key"+"Reference" : {"changes": ["key1" : 1 , fieldId": 1, "fieldName" : "updated name"......]}
Benefit, the above will avoid having to:
- Avoid heavy use of JavaScript step
For example, to replicate sample output 3 above you would need to:
- Use a Switch step to isolate the changed rows
- Use a Merge Join step to bring the 'remote' side data back into the equation
- Use a custom JavaScript to
-- Identify the keys
-- Identify and compare the "reference" and "comparison" fields
-- Build and output a JSON string.
As always, thank you for all your amazing work!
### Issue Priority
Priority: 3
### Issue Component
Component: Hop Gui
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature Request]: Merge diff "changed" fields enhancement (hop)
Posted by "mattcasters (via GitHub)" <gi...@apache.org>.
mattcasters commented on issue #5660:
URL: https://github.com/apache/hop/issues/5660#issuecomment-3259711890
[mergediff_unittest.txt](https://github.com/user-attachments/files/22181131/mergediff_unittest.txt) Shows a possible issue after review, by @dmainou
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature Request]: Merge diff "changed" fields enhancement (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva commented on issue #5660:
URL: https://github.com/apache/hop/issues/5660#issuecomment-3244004967
Duplicate of https://github.com/apache/hop/issues/2209 but this one is written out better so closing the older ticket
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature Request]: Merge diff "changed" fields enhancement (hop)
Posted by "dmainou (via GitHub)" <gi...@apache.org>.
dmainou commented on issue #5660:
URL: https://github.com/apache/hop/issues/5660#issuecomment-3259748548
@mattcasters I'll test asap.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature Request]: Merge diff "changed" fields enhancement (hop)
Posted by "mattcasters (via GitHub)" <gi...@apache.org>.
mattcasters commented on issue #5660:
URL: https://github.com/apache/hop/issues/5660#issuecomment-3259737086
@dmainou just fill in the reference and compare transforms in the Merge Rows transform of the attached pipeline and you should be OK.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature Request]: Merge diff "changed" fields enhancement (hop)
Posted by "mattcasters (via GitHub)" <gi...@apache.org>.
mattcasters closed issue #5660: [Feature Request]: Merge diff "changed" fields enhancement
URL: https://github.com/apache/hop/issues/5660
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org