You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hop.apache.org by "mycloudy127 (via GitHub)" <gi...@apache.org> on 2026/08/27 15:30:18 UTC
[I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
mycloudy127 opened a new issue, #8130:
URL: https://github.com/apache/hop/issues/8130
### Apache Hop version?
2.19.0
### Java version?
OpenJDK Corretto 21.0.12.8.1 (Java 21.0.12 LTS, 64-bit)
### Operating system
Windows
### What happened?
## Problem
The **Dimension Lookup/Update** transform fails when it inserts a new dimension row into PostgreSQL and the technical-key column is `numeric(38,0)`.
Our existing PDI implementation and DW schema use PostgreSQL sequences for technical keys and `numeric(38,0)` for the key columns. Changing the database schema is not an option for us.
Hop detects the technical key as `BigNumber(38)`, while PostgreSQL `nextval()` returns a Java `Long`. When a new dimension row is required, the transform fails with a `ClassCastException`.
Pipelines with no source rows, or rows that do not require a new dimension record, can complete successfully. The failure occurs consistently when a new technical key must be generated.
## Expected behavior
Dimension Lookup/Update should insert the new row successfully and return the generated sequence value, converting the PostgreSQL sequence `Long` correctly for a `numeric(38,0)` technical-key column.
## Actual behavior
The transform fails with:
```text
Unexpected conversion error while converting value [variant_domains_key BigNumber(38)] to a BigNumber
Caused by: java.lang.ClassCastException
Stack trace location:
org.apache.hop.core.row.value.ValueMetaBase.getBigNumber(...)
org.apache.hop.core.row.value.ValueMetaBigNumber.getNativeDataType(...)
org.apache.hop.pipeline.transforms.dimensionlookup.DimensionLookup.lookupValues(DimensionLookup.java:966)
Minimal reproduction
1. Use a PostgreSQL connection.
2. Create a table with a numeric(38,0) technical key and a sequence:
create sequence dim_test_seq;
create table dim_test (
test_key numeric(38,0) primary key,
business_key numeric(38,0) not null
);
3. Create a pipeline with a Dimension Lookup/Update transform:
- Technical key field: test_key
- Technical key creation method: Sequence
- Sequence name: dim_test_seq
- Business/natural key: business_key
4. Send a row with a new business_key value so that the transform must insert a row.
5. Run the pipeline.
The pipeline fails while converting the generated sequence value.
Additional observation
The installed Hop 2.19 Dimension Lookup code appears to detect the BigNumber/Long case and replace the return metadata with an Integer metadata type. However, it subsequently performs conversion using the previous BigNumber metadata, causing the failure.
Please let me know if a sample Hop pipeline or a full sanitized stack trace would help.
```
### Issue Priority
Priority: 3
### Issue Component
Component: Transforms
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "Fjelley85 (via GitHub)" <gi...@apache.org>.
Fjelley85 commented on issue #8130:
URL: https://github.com/apache/hop/issues/8130#issuecomment-5477827271
I see that it is closed, but we have similar issue regarding the dimension lookup/update step, but it doesn't involve technical key.
Unexpected conversion error while converting value [projects Integer(9)] to an Integer
2026/08/31 13:25:31 [Error] dim_employee.0 - class java.lang.String cannot be cast to class java.lang.Long (java.lang.String and java.lang.Long are in module java.base of loader 'bootstrap')
pentaho 9.4, postgres 18.4.
Projects is stored as an Integer in postgres.
@hansva , Was this issue also fixed in this fix?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "mycloudy127 (via GitHub)" <gi...@apache.org>.
mycloudy127 commented on issue #8130:
URL: https://github.com/apache/hop/issues/8130#issuecomment-5444625072
> Just for context: which PDI version are you migrating from? Are you using the same database (version), jdbc driver, java version etc? [@hansva](https://github.com/hansva) did quite a bit of database hardening work in 2.20.0-SNAPSHOT, so this might already be fixed, but we'll need to check.
Hi,
Thanks for your quick response. The we are using "pdi-ce-9.3.0.0-428". And Postgres database is the same, but postgres JDBC driver is "postgresql-42.2.23.jar". Through the investigation, I downloaded the source code and made one line change and verifying it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva commented on issue #8130:
URL: https://github.com/apache/hop/issues/8130#issuecomment-5461727515
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva commented on issue #8130:
URL: https://github.com/apache/hop/issues/8130#issuecomment-5477911017
Not sure, feel free to test a build from [here](https://repository.apache.org/content/repositories/snapshots/org/apache/hop/hop-client/2.20.0-SNAPSHOT/). If it is an issue, please create a ticket and a clear reproduction path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "mycloudy127 (via GitHub)" <gi...@apache.org>.
mycloudy127 commented on issue #8130:
URL: https://github.com/apache/hop/issues/8130#issuecomment-5480145061
@hansva, thanks for your quick response and I will use the 2.20 snapshot for testing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva closed issue #8130: [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence
URL: https://github.com/apache/hop/issues/8130
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "mycloudy127 (via GitHub)" <gi...@apache.org>.
mycloudy127 commented on issue #8130:
URL: https://github.com/apache/hop/issues/8130#issuecomment-5441456334
We are looking for migrating PDI to Apache Hop. And above issue happened for dimension pipelines which were converted from PDI transformations. But with PDI, there is no issue at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Dimension Lookup/Update fails for PostgreSQL numeric(38,0) technical key generated by sequence (hop)
Posted by "bamaer (via GitHub)" <gi...@apache.org>.
bamaer commented on issue #8130:
URL: https://github.com/apache/hop/issues/8130#issuecomment-5443976770
Just for context: which PDI version are you migrating from? Are you using the same database (version), jdbc driver, java version etc? @hansva did quite a bit of database hardening work in 2.20.0-SNAPSHOT, so this might already be fixed, but we'll need to check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org