You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hop.apache.org by "arif-basri (via GitHub)" <gi...@apache.org> on 2026/04/14 03:43:13 UTC
[I] [Bug]: Duplicate column name generated in Database Lookup step causes incorrect output schema (hop)
arif-basri opened a new issue, #6995:
URL: https://github.com/apache/hop/issues/6995
### Apache Hop version?
2.17.0
### Java version?
17.0.18
### Operating system
Linux
### What happened?
<img width="1623" height="810" alt="Image" src="https://github.com/user-attachments/assets/3ec29358-f774-40d8-9391-cee74707a1d0" />
<h2>Problem Statement</h2>
<p>
Database Lookup</strong> step produces inconsistent and incorrect column naming
when compared across identical transformation logic.
</p>
<p><strong>Left pipeline flow (working correctly):</strong></p>
<ul>
<li>Table Input → Database Lookup → Database Lookup 2</li>
<li>Final output schema:</li>
</ul>
<pre>
| brand | model | brand_1 | brand_2 |
</pre>
<p><strong>Right pipeline flow (buggy behavior):</strong></p>
<ul>
<li>Table Input 2 → Database Lookup 3 → Select Values → Database Lookup 2 2</li>
<li>Final output schema:</li>
</ul>
<pre>
| brand | model | brand |
</pre>
<hr/>
<h2>Expected Behavior</h2>
<p>
The final step <strong>Database lookup 2 2</strong> should produce:
</p>
<pre>
| brand | model | brand_1 |
</pre>
<p>
Column names should be automatically disambiguated (e.g., <code>brand_1</code>) when a field with the same name already exists.
</p>
<hr/>
<h2>Actual Behavior</h2>
<ul>
<li>Duplicate column name <code>brand</code> appears in the output</li>
<li>Expected alias (<code>brand_1</code>) is not applied</li>
<li>Results in ambiguous schema and potential downstream errors</li>
</ul>
<hr/>
<h2>Impact</h2>
<ul>
<li>Breaks deterministic pipeline behavior</li>
<li>Causes schema conflicts in downstream steps</li>
<li>Makes pipelines harder to debug and maintain</li>
<li>Inconsistent behavior across equivalent transformations</li>
</ul>
<hr/>
<h2>Additional Observations</h2>
<ul>
<li>The issue occurs only in the right-side flow</li>
<li>Possible relation to the <strong>Select Values</strong> step affecting metadata propagation</li>
<li>Same lookup logic works correctly without the intermediate step</li>
</ul>
<hr/>
<h2>Suggested Investigation Areas</h2>
<ul>
<li>Field name collision handling in Database Lookup</li>
<li>Metadata propagation after Select Values</li>
<li>Alias generation logic for duplicate fields (e.g., _1, _2)</li>
<li>Differences in row metadata structure between both flows</li>
</ul>
<h2>Atached files to repro</h2>
<ul>
<li>sqlite.json - rdbms metadata</li>
<li>debug_dup_field_setup.hpl - setup test table and insert test data</li>
<li>debug_dup_field_name.hpl - contain the steps in the screenshot</li>
[repro_files.zip](https://github.com/user-attachments/files/26695840/repro_files.zip)
</ul>
### Issue Priority
Priority: 0
### Issue Component
Component: Pipelines
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Duplicate column name generated in Database Lookup step causes incorrect output schema (hop)
Posted by "leehaut (via GitHub)" <gi...@apache.org>.
leehaut commented on issue #6995:
URL: https://github.com/apache/hop/issues/6995#issuecomment-4241318051
Hi, [repro_files.zip](https://github.com/user-attachments/files/26695840/repro_files.zip) 404 not found.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Duplicate column name generated in Database Lookup step causes incorrect output schema (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva commented on issue #6995:
URL: https://github.com/apache/hop/issues/6995#issuecomment-4241637125
I'll take a look if it's still an issue .take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Duplicate column name generated in Database Lookup step causes incorrect output schema (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva closed issue #6995: [Bug]: Duplicate column name generated in Database Lookup step causes incorrect output schema
URL: https://github.com/apache/hop/issues/6995
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Duplicate column name generated in Database Lookup step causes incorrect output schema (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva commented on issue #6995:
URL: https://github.com/apache/hop/issues/6995#issuecomment-4241453928
Definitely not a P0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org