[GitHub] [druid] JulianJaffePinterest commented on issue #9463: Add namespaces to Druid segments within a data source

[GitBox <gi...@apache.org>]

JulianJaffePinterest commented on issue #9463: Add namespaces to Druid segments within a data source
URL: https://github.com/apache/druid/issues/9463#issuecomment-598346730
 
 
   We have both differing dimension and differing metrics between the namespaces. I investigated modifying union datasources as well, but there were a number of problems:
   
   1.  At the time (and perhaps still), union data sources were queried in serial, not parallel. This is obviously fixable.
   2. Implementing this feature with unioned data sources required implementing query logic to handle disjoin schemata, which namespacing did not.
   3. We've scaled this from a handful of namespaces per data source (where union data sources probably could work) to hundreds for a few (I'm not sure how easy managing 100+ data sources in a cluster would be, but this may not be a major concern)
   
   Since more changes were required for making this work with union datasources, we went with namespacing (and it was fairly simple to implement the changes described above). I see merit in both approaches.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org