Re: Design Review for Configurable Index Type

[Jihoon Son <ji...@apache.org>]

Hi Liran,

thanks for an interesting proposal! I believe this proposal will help to
compare different implementations.
I haven't looked through the proposal closely yet, but will try soon.

On Mon, Aug 31, 2020 at 5:22 AM Liran Funaro
<li...@verizonmedia.com.invalid> wrote:

> I will appreciate the community feedback/review on a design proposal I
> submitted: https://github.com/apache/druid/issues/10321.
>
> This design proposal introduces an API that allows users to choose which
> in-memory indexing approach to use during ingestion (e.g.,
> OnheapIncrementalIndex).
>
> As suggested by Jonathan Wei during one discussion (
> https://github.com/apache/druid/issues/5698#issuecomment-506881097), this
> API will allow extensions to offer their own in-memory index
> implementations.
>
> It is motivated by the understanding that different indexing approaches
> might have different tradeoffs. Thus, it can be beneficial to allow users
> to select the best approach for their workload. A notable example is #10001
> <https://github.com/apache/druid/pull/10001> which shows significant
> improvements in throughput and resource utilization for batch ingestion.
>
> This proposal is accompanied by a work-in-progress PR:
> https://github.com/apache/druid/pull/10335
>
> Thanks,
>
> Liran
>