Empty Data Source in Druid

[Andriod Guru <as...@gmail.com>]

Hi Team,

I am trying to automate Druid pipeline using stream sets and one of the use case demands an empty druid data source should be created for users.

Do we have opportunity to create one?

Also how can I start contributing to Druid releases. I have been using Druid since 0.14 version and have fair idea about Java.

Thanks,
Ashish

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org

Re: Empty Data Source in Druid

[Gian Merlino <gi...@apache.org>]

In Druid it's not possible to have a datasource without any segments. But
is possible, in theory, to have an empty datasource: you would need a
single segment that has no rows (the important part is that the segment
exists, not that it actually has rows in it). But there are two problems
with this:

1) Druid ingestion really tries to avoid creating segments with zero rows.
So it may not be possible without hackery.
2) Druid SQL will probably not work right on a datasource with zero rows in
it. (I don't think any of the columns, etc, will be detected properly.)

I would consider this an unsupported use case currently.


On Mon, Dec 23, 2019 at 11:05 AM David Glasser <gl...@apollographql.com>
wrote:

> Hmm, I may have answered too quickly. That will lead to no segments being
> created, so the system will not consider the data source to exist, I think.
>
> On Mon, Dec 23, 2019 at 11:04 AM David Glasser <gl...@apollographql.com>
> wrote:
>
> > You can do batch ingestion from an empty file, as documented (with
> > multiple ways of invoking it) at
> > https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html
> >
> > On Mon, Dec 23, 2019 at 6:35 AM Andriod Guru <as...@gmail.com>
> wrote:
> >
> >> Hi Team,
> >>
> >> I am trying to automate Druid pipeline using stream sets and one of the
> >> use case demands an empty druid data source should be created for users.
> >>
> >> Do we have opportunity to create one?
> >>
> >> Also how can I start contributing to Druid releases. I have been using
> >> Druid since 0.14 version and have fair idea about Java.
> >>
> >> Thanks,
> >> Ashish
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> >> For additional commands, e-mail: dev-help@druid.apache.org
> >>
> >>
>

Re: Empty Data Source in Druid

[David Glasser <gl...@apollographql.com>]

Hmm, I may have answered too quickly. That will lead to no segments being
created, so the system will not consider the data source to exist, I think.

On Mon, Dec 23, 2019 at 11:04 AM David Glasser <gl...@apollographql.com>
wrote:

> You can do batch ingestion from an empty file, as documented (with
> multiple ways of invoking it) at
> https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html
>
> On Mon, Dec 23, 2019 at 6:35 AM Andriod Guru <as...@gmail.com> wrote:
>
>> Hi Team,
>>
>> I am trying to automate Druid pipeline using stream sets and one of the
>> use case demands an empty druid data source should be created for users.
>>
>> Do we have opportunity to create one?
>>
>> Also how can I start contributing to Druid releases. I have been using
>> Druid since 0.14 version and have fair idea about Java.
>>
>> Thanks,
>> Ashish
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
>> For additional commands, e-mail: dev-help@druid.apache.org
>>
>>

Re: Empty Data Source in Druid

[David Glasser <gl...@apollographql.com>]

You can do batch ingestion from an empty file, as documented (with multiple
ways of invoking it) at
https://druid.apache.org/docs/latest/tutorials/tutorial-batch.html

On Mon, Dec 23, 2019 at 6:35 AM Andriod Guru <as...@gmail.com> wrote:

> Hi Team,
>
> I am trying to automate Druid pipeline using stream sets and one of the
> use case demands an empty druid data source should be created for users.
>
> Do we have opportunity to create one?
>
> Also how can I start contributing to Druid releases. I have been using
> Druid since 0.14 version and have fair idea about Java.
>
> Thanks,
> Ashish
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> For additional commands, e-mail: dev-help@druid.apache.org
>
>