You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by "b-enedict (via GitHub)" <gi...@apache.org> on 2026/04/23 09:07:19 UTC

[PR] [SYSTEMDS-3944] Modality Alignment, Contrastive Learning, new Data Loaders [systemds]

b-enedict opened a new pull request, #2461:
URL: https://github.com/apache/systemds/pull/2461

   # Summary
   
   This PR introduces new functionality for multimodal learning in Scuro, including a contrastive learning operator, a modality alignment operator, and additional data loaders.
   
   ## Changes
   ### Contrastive Learning Operator
   - Constructs modality pairs via a Cartesian product
   - Uses a user-defined function to label pairs as positive or negative
   - Enables dynamic generation of contrastive samples
   ### Modality Alignment Operator
   - Aligns previously unaligned modalities using feature-based similarity (e.g., ORB, perceptual hashing)
   - Outputs a matching between a primary and secondary modality
   - Matching is applied after representation learning and before fusion
   ### Data Loaders
   - PDF loader: converts document pages into NumPy arrays for OpenCV processing
   - Audio loader: converts audio to text using faster-whisper


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [SYSTEMDS-3944] Modality Alignment, Contrastive Learning, new Data Loaders [systemds]

Posted by "christinadionysio (via GitHub)" <gi...@apache.org>.
christinadionysio closed pull request #2461: [SYSTEMDS-3944] Modality Alignment, Contrastive Learning, new Data Loaders
URL: https://github.com/apache/systemds/pull/2461


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org