You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@druid.apache.org by "anshu-makkar (via GitHub)" <gi...@apache.org> on 2025/08/12 06:43:26 UTC

[I] Missing SQL Functions for Tuple Sketches (druid)

anshu-makkar opened a new issue, #18390:
URL: https://github.com/apache/druid/issues/18390

   Our team at Samsung had added Support for SQL Based Ingestion for Tuple Sketches. 
   https://github.com/apache/druid/pull/13887 But this is not present in the main code now, as data sketches has migrated in the codebase. @abhishekagarwal87 @gianm  Could you please help us. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #18390:
URL: https://github.com/apache/druid/issues/18390#issuecomment-4597763040

   This issue has been marked as stale due to 280 days of inactivity.
   It will be closed in 4 weeks if no further activity occurs. If this issue is still
   relevant, please simply write any comment. Even if closed, you can still revive the
   issue at any time or discuss it on the dev@druid.apache.org list.
   Thank you for your contributions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #18390:
URL: https://github.com/apache/druid/issues/18390#issuecomment-4849276848

   This issue has been closed due to lack of activity. If you think that
   is incorrect, or the issue requires additional review, you can revive the issue at
   any time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "anshu-makkar (via GitHub)" <gi...@apache.org>.
anshu-makkar commented on issue #18390:
URL: https://github.com/apache/druid/issues/18390#issuecomment-3218834388

   I think I found the issue, I was using wrong signature.  Thanks for the help @abhishekagarwal87 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "abhishekagarwal87 (via GitHub)" <gi...@apache.org>.
abhishekagarwal87 commented on issue #18390:
URL: https://github.com/apache/druid/issues/18390#issuecomment-3218804390

   when you say its not present in the main code now, what do you mean? 
   https://github.com/apache/druid/blob/master/extensions-core/datasketches/src/main/java/org/apache/druid/query/aggregation/datasketches/tuple/sql/ArrayOfDoublesSketchMetricsSumEstimateOperatorConversion.java
   
   Is there some other class I am missing? 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "anshu-makkar (via GitHub)" <gi...@apache.org>.
anshu-makkar commented on issue #18390:
URL: https://github.com/apache/druid/issues/18390#issuecomment-3218746303

   Hello Team, Is there a plan to restore Tuple SKetches SQL for MSQ.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "anshu-makkar (via GitHub)" <gi...@apache.org>.
anshu-makkar commented on issue #18390:
URL: https://github.com/apache/druid/issues/18390#issuecomment-3218843357

   I see another issue in MSQ for Tuple Sketch vs Index_Parallel Job. 
   
   For the same data set, via index_parallel it takes 2 hrs to ingest the data( data set is hourly and has 24 hours of data) , with MSQ it goes on for ever like more than 5 hours, but it still keeps on going on. 
   
   On comparing the Ingestion Spec, difference is how the rollups( metrics vs aggregations) is defined 
   IN MSQ below is how the spec has metrics 
   ```
   "aggregations": [
       {
         "type": "arrayOfDoublesSketch",
         "name": "a0",
         "fieldName": "user_ifa",
         "nominalEntries": 16384,
         "metricColumns": [
           "req_count"
         ],
         "numberOfValues": 1
       },
       {
         "type": "arrayOfDoublesSketch",
         "name": "a1",
         "fieldName": "user_ifa",
         "nominalEntries": 16384,
         "metricColumns": [
           "win_rate_prob"
         ],
         "numberOfValues": 1
       },
       {
         "type": "longSum",
         "name": "a2",
         "fieldName": "req_count"
       },
       {
         "type": "doubleSum",
         "name": "a3",
         "fieldName": "win_rate_prob"
       }
     ]
   ```
   
   
   In case of `index_parallel` job
   
   ```
         "metricsSpec": [
           {
             "type": "arrayOfDoublesSketch",
             "name": "user_tuple_sketch_count",
             "fieldName": "user_ifa",
             "nominalEntries": 16384,
             "metricColumns": [
               "req_count"
             ],
             "numberOfValues": 1
           },
           {
             "type": "arrayOfDoublesSketch",
             "name": "user_tuple_sketch_probability",
             "fieldName": "user_ifa",
             "nominalEntries": 16384,
             "metricColumns": [
               "win_rate_prob"
             ],
             "numberOfValues": 1
           },
           {
             "type": "longSum",
             "name": "total_req_count",
             "fieldName": "psid_count"
           },
           {
             "type": "doubleSum",
             "name": "sum_win_probability",
             "fieldName": "win_rate_prob"
           }
         ],
   ```
   
   There is a stark difference between the time it is taken via MSQ.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "anshu-makkar (via GitHub)" <gi...@apache.org>.
anshu-makkar commented on issue #18390:
URL: https://github.com/apache/druid/issues/18390#issuecomment-3218809369

   I am unable to use `DS_TUPLE_DOUBLES` function in MSQ. It says unsupported. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org


Re: [I] Missing SQL Functions for Tuple Sketches (druid)

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #18390: Missing SQL Functions for Tuple Sketches
URL: https://github.com/apache/druid/issues/18390


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org