You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "ryankert01 (via GitHub)" <gi...@apache.org> on 2025/12/29 02:26:30 UTC

[I] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

ryankert01 opened a new issue, #765:
URL: https://github.com/apache/mahout/issues/765

   ### **Description**
   
   There is currently an inconsistency in how our system handles `null` values depending on the execution mode. To ensure predictable behavior and data integrity, we should align these strategies.
   
   ### **Current Behavior**
   
   * **Batch Mode:** Null values are automatically coerced/set to `0`.
   * **Streaming Mode:** Null values trigger a runtime error.
   
   ### **Proposed Change**
   
   We need to decide on a unified strategy for handling `null` values across both modes. Potential directions include:
   
   1. **Strict Error Handling:** Both modes return an error to prevent silent data corruption.
   2. **Default Coercion:** Both modes default to `0` (or a type-appropriate default).
   3. **Configurable Policy:** Allow users to define the `on_null` behavior (e.g., `ignore`, `fill_zero`, or `raise_error`).
   
   ### **Context**
   
   https://github.com/apache/mahout/pull/753#discussion_r2650061909
   
   ### **To-Do / Discussion Points**
   
   * [ ] Research performance implications of adding null-checks in the streaming hot path.
   * [ ] Determine if "Default to 0" is safe for all supported data types.
   * [ ] Reach a consensus on the unified default behavior.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "manvendra5345 (via GitHub)" <gi...@apache.org>.
manvendra5345 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698631316

   Hi @ryankert01 , I'm a new open-source contributor and I'm interested in taking on this issue. I’ve reviewed the description regarding the inconsistency between Batch and Streaming modes. Could you point me toward the specific modules or files where the null-handling logic for these modes is defined? I want to make sure I'm looking at the right place before I start drafting a fix. Thanks! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "manvendra5345 (via GitHub)" <gi...@apache.org>.
manvendra5345 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3699377329

   No sir I have device which fulfil the requirements


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "manvendra5345 (via GitHub)" <gi...@apache.org>.
manvendra5345 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698541321

   Sure sir I will strat working on it right away.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "manvendra5345 (via GitHub)" <gi...@apache.org>.
manvendra5345 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3783032471

   Yes sir i have understood code base but there are some problem I am facing. So can I ask them here.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "ryankert01 (via GitHub)" <gi...@apache.org>.
ryankert01 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3781981860

   @manvendra5345 How’s it going? Any questions?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "manvendra5345 (via GitHub)" <gi...@apache.org>.
manvendra5345 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698648277

   No problem @400Ping sir as I told above my goal is to learn open contribution as I a, just strating so I have read the issue discription and undestood the changes but i am not able to locate how can I start. If you can help me.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "ryankert01 (via GitHub)" <gi...@apache.org>.
ryankert01 closed issue #765: [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes
URL: https://github.com/apache/mahout/issues/765


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "manvendra5345 (via GitHub)" <gi...@apache.org>.
manvendra5345 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698527941

   Hi @ryankert01 ,
   
   I’m interested in working on this issue as part of GSoC. I’ve gone through the issue details and related code, and I’d love to take ownership and start contributing.
   
   If it’s available, could you please assign this issue to me?
   
   Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "400Ping (via GitHub)" <gi...@apache.org>.
400Ping commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698697407

   You can start by looking into the code since it's already pretty straightforward. Also you may need a Linux system and a RTX 30s GPU or above.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "ryankert01 (via GitHub)" <gi...@apache.org>.
ryankert01 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698532994

   @manvendra5345 Sweet! Feel free! 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "400Ping (via GitHub)" <gi...@apache.org>.
400Ping commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698642179

   Btw I am not very sure if the qdp part is included in the GSoC, it is a PoC that we are working on and it requires knowledge of cuda, rust and GPUs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [QDP] Unify Null Value Handling Strategy between Batch and Streaming Modes [mahout]

Posted by "ryankert01 (via GitHub)" <gi...@apache.org>.
ryankert01 commented on issue #765:
URL: https://github.com/apache/mahout/issues/765#issuecomment-3698729681

   Good luck along with the journey! Btw, if you don't have device needed, you can use colab to run tests! We have a reference of how to install in /benchmarks/notebooks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@mahout.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org