[I] Missing schema/type validation between SQL and Java operators leads to runtime ClassCastException [wayang]

["makarandhinge (via GitHub)" <gi...@apache.org>]

makarandhinge opened a new issue, #747:
URL: https://github.com/apache/wayang/issues/747

   While working on a pipeline like PostgresTableSource → Join → SqlToStream → Java map → collect, I faced a runtime ClassCastException. The issue is happening because after the SQL join, the data is actually in a tuple-like structure (like Tuple2<Record, Record>), but during the SqlToStream conversion, this structure is not preserved properly and everything is treated as a Record.
   
   Because of this, when I try to use the data in the Java map step (for example accessing fields or casting types), it assumes a wrong structure and crashes at runtime (e.g., String cannot be cast to Integer). There is no validation or check between SQL and Java stages, so the mismatch is not caught early and only fails during execution.
   
   Ideally, the correct schema/type should be preserved (or validated) when moving from SQL to Java, so that such issues don’t happen at runtime.
   
   for example - 
   
   CREATE TABLE table_a (
       id INT,
       name TEXT
   );
   
   CREATE TABLE table_b (
       id TEXT,   -- different type
       value TEXT
   );
   
   
   <img width="1806" height="718" alt="Image" src="https://github.com/user-attachments/assets/11d0e4c4-c7a2-4eb1-befb-306fc9eb1d4e" />


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Missing schema/type validation between SQL and Java operators leads to runtime ClassCastException [wayang]

["AliRana30 (via GitHub)" <gi...@apache.org>]

AliRana30 commented on issue #747:
URL: https://github.com/apache/wayang/issues/747#issuecomment-5517268568

   @novatechflow I want to work on this issue. I will fix the schema/type validation between both the operators to ensure safe migrations. Are you working on this? @makarandhinge 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [I] Missing schema/type validation between SQL and Java operators leads to runtime ClassCastException [wayang]

["AliRana30 (via GitHub)" <gi...@apache.org>]

AliRana30 commented on issue #747:
URL: https://github.com/apache/wayang/issues/747#issuecomment-5471532349

   @zkaoudi I want to work on this issue. I will fix the schema/type validation between both the operators to ensure safe migrations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@wayang.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org