[DISCUSS] Container-friendly test server

[Jon Malkin <jm...@apache.org>]

Hey all,

I've been playing with making a simple sketches server using embedded
Jetty. JSON interface for updates, merges, queries. It's intended to play
nicely with things like Docker, although obviously it'll run just fine on
its own.

The idea is to have a very basic system so people can play with sketches
and try things out. My current vision for it (and obviously that can change
based on community interest) is that it's more like a playground -- we're
integrated with a few real databases which will be way more performant than
anything I'm doing here. I'm not even allowing injection of new sketches
after initialization (which keeps memory bounded better). Again, if you
want a real database then use one :)  But the hope is that this provides an
easier way to test sketches without making a full commitment to adopting
them. If using Docker, it also avoids things like Java versioning issues
and lets you use whatever language you want, although there'll be a
significant performance hit by needing to do text parsing for every update.

Anyway, I was planning to ask the infra team to create an
incubator-datasketches-server repo for this. Not necessarily expecting it'd
be a release-grade repo, and it'll have dependencies on at least Jetty
which makes it less suitable for the core java repo. This seems worth
opening up to the community for discussion, though. So the questions are:

1. Does this seem like something of value? ("a few people I know say yes"
is accurate, but I don't claim to know a representative set of our
potential audience)
2. Are we ok adding a sandbox-scope repository?
   2a. If no, would your answer change if we make official releases?
3. If yes to 1 and 2, any objection to incubator-datasketches-server?

Thanks,
  jon

Re: [DISCUSS] Container-friendly test server

[leerho <le...@gmail.com>]

1. Yes
2. Yes
3. No.

I don't think we need to do official releases at this time.  It is still
new and we will be learning how people would like to use it.

If at some point in the future, if there are applications that are
long-time running and folks start depending on it, then we might need some
means to mark upgrades and enhancements to the JSON API.  But we can decide
this later.

I do think we need a way to get the word out that it exists!  We could
really use an Announcement blog on our website, or equivalent.

Thank you for doing this!

Lee.


On Tue, Oct 27, 2020 at 5:14 PM Jon Malkin <jm...@apache.org> wrote:

> Hey all,
>
> I've been playing with making a simple sketches server using embedded
> Jetty. JSON interface for updates, merges, queries. It's intended to play
> nicely with things like Docker, although obviously it'll run just fine on
> its own.
>
> The idea is to have a very basic system so people can play with sketches
> and try things out. My current vision for it (and obviously that can change
> based on community interest) is that it's more like a playground -- we're
> integrated with a few real databases which will be way more performant than
> anything I'm doing here. I'm not even allowing injection of new sketches
> after initialization (which keeps memory bounded better). Again, if you
> want a real database then use one :)  But the hope is that this provides an
> easier way to test sketches without making a full commitment to adopting
> them. If using Docker, it also avoids things like Java versioning issues
> and lets you use whatever language you want, although there'll be a
> significant performance hit by needing to do text parsing for every update.
>
> Anyway, I was planning to ask the infra team to create an
> incubator-datasketches-server repo for this. Not necessarily expecting it'd
> be a release-grade repo, and it'll have dependencies on at least Jetty
> which makes it less suitable for the core java repo. This seems worth
> opening up to the community for discussion, though. So the questions are:
>
> 1. Does this seem like something of value? ("a few people I know say yes"
> is accurate, but I don't claim to know a representative set of our
> potential audience)
> 2. Are we ok adding a sandbox-scope repository?
>    2a. If no, would your answer change if we make official releases?
> 3. If yes to 1 and 2, any objection to incubator-datasketches-server?
>
> Thanks,
>   jon
>