You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "deeeeeelan (via GitHub)" <gi...@apache.org> on 2025/05/12 02:07:21 UTC

[I] JSON Multi Value column validation [pinot]

deeeeeelan opened a new issue, #15765:
URL: https://github.com/apache/pinot/issues/15765

   I noticed on my Pinot dashboard AFTER a realtime table is created, it shows ERROR. checked on my Server component and noticed the error is as below
   
   ```
   Caused by: java.lang.IllegalStateException: There is no multi-value type for JSON
   	at org.apache.pinot.common.utils.PinotDataType.getPinotDataTypeForIngestion(PinotDataType.java:1489) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.segment.local.recordtransformer.DataTypeTransformer.<init>(DataTypeTransformer.java:56) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.getDefaultTransformers(CompositeTransformer.java:107) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.segment.local.recordtransformer.CompositeTransformer.getDefaultTransformer(CompositeTransformer.java:122) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.segment.local.segment.creator.TransformPipeline.<init>(TransformPipeline.java:63) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.segment.local.utils.IngestionUtils.getFieldsForRecordExtractor(IngestionUtils.java:335) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.core.data.manager.realtime.RealtimeSegmentDataManager.<init>(RealtimeSegmentDataManager.java:1591) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.doAddConsumingSegment(RealtimeTableDataManager.java:534) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.addConsumingSegment(RealtimeTableDataManager.java:479) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addConsumingSegment(HelixInstanceDataManager.java:266) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   	at org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeConsumingFromOffline(SegmentOnlineOfflineStateModelFactory.java:81) ~[pinot-all-1.3.0-jar-with-dependencies.jar:1.3.0-c0023da298126af6a01b802a04b66da34ba16134]
   ```
   
   
   just wonder why no validations occurs during table creation to avoid segment consumption errors rather than create first then validate later ?
   
   are there any Docs referred to this multi value column config for realtime table ?
   
   as currently im trying to work on Realtime Table Upsert table configuration on APPEND partial upsert mode. pretty frustrating when a fresh table was created then got ERROR suddenly due to bad segments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] JSON Multi Value column validation [pinot]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #15765: JSON Multi Value column validation
URL: https://github.com/apache/pinot/issues/15765


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] JSON Multi Value column validation [pinot]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15765:
URL: https://github.com/apache/pinot/issues/15765#issuecomment-4151441139

   This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] JSON Multi Value column validation [pinot]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15765:
URL: https://github.com/apache/pinot/issues/15765#issuecomment-4340080593

   This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] JSON Multi Value column validation [pinot]

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #15765:
URL: https://github.com/apache/pinot/issues/15765#issuecomment-2892383716

   Good catch! We should add this to `Schema.validate()`. Do you want to help contribute this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] JSON Multi Value column validation [pinot]

Posted by "deeeeeelan (via GitHub)" <gi...@apache.org>.
deeeeeelan commented on issue #15765:
URL: https://github.com/apache/pinot/issues/15765#issuecomment-2898036574

   > Good catch! We should add this to `Schema.validate()`. Do you want to help contribute this?
   
   to be honest since Apache Pinot was built on top of Java, it is not my strength honestly and I am just a consumer of a great OLAP engine. But if there is someone that can help me to guide for a contribution I would gladly help to contribue on this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org