You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@asterixdb.apache.org by Ian Maxon <im...@apache.org> on 2025/08/21 15:06:32 UTC

[DISCUSS][APE] Containerized packaging

Hello fellow devs,

I proposed an APE to add a containerized packaging to AsterixDB. I
know we've had some one-off and demo ones (e.g. asterix-docker), but I
think it's time to have an official one. I've had this change waiting
in the wings for a while, and I figured it's time to try and push it
over the finish line.

I think this container should work for both evaluation and production
deployment. It contains a default config, but the entrypoint script
allows both a config file and any command line parameters to be given.

Here's the current patchset:
https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
and APE document:
https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+27%3A+Containerized+packaging

Looking forward to everyone's thoughts.

-Ian

Re: [DISCUSS][APE] Containerized packaging

Posted by Glenn Galvizo <gl...@couchbase.com.INVALID>.
+1 (would love to see a one-line “docker run asterixdb” command work + the ability to use compose to easily scale out as well)

> On Aug 21, 2025, at 10:35, Suryaa Charan Shivakumar <ss...@ucr.edu.invalid> wrote:
> +1 for this feature!
> 
> Thanks and Regards,
> Suryaa Charan S
> 
> On Thu, Aug 21, 2025, 10:02 Ian Maxon <im...@apache.org> wrote:
> 
>> Thanks for the +1, Mike. Yeah, I agree totally on adding better
>> Kubernetes support. I have a dependent patch on this
>> (https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18242) that adds
>> example Helm charts as a first step toward this. Using a StatefulSet
>> with either a PersistentVolume or one of the cloud/object store
>> storage back-ends achieves basic functionality for static clusters.
>> This patch comes out of some experimentation I was doing last year,
>> and I was able to do stop/start just fine if I recall correctly.
>> PersistentVolumes are really awesome, and solve a major headache that
>> both YARN and Mesos had for hosting databases or anything that needed
>> to manage data long-term, at least from my point of view.
>> 
>> I think the Right solution however is to add a Kubernetes Operator for
>> AsterixDB, and this should be the next step. This would allow resizing
>> the cluster and other things. All the support for this already exists
>> for the runtime effectively, and Kubernetes provides a lot of the
>> things in the past that had to be done on our side (e.g. NCService's
>> config distribution, or Managix's use of Zookeeper). It's roughly
>> analogous to what was necessary to put AsterixDB on YARN, but
>> Kubernetes has a lot more features.
>> 
>> On Thu, Aug 21, 2025 at 9:42 AM Mike Carey <dt...@gmail.com> wrote:
>>> +1 for this addition!  (It might be cool to also have a default
>>> Kubernetes controller that runs things in the clustered case, if there
>>> is some way to do that, so that users can have reliable clusters that
>>> they can manage and turn on and off?)
>>> On 8/21/25 8:06 AM, Ian Maxon wrote:
>>>> Hello fellow devs,
>>>> I proposed an APE to add a containerized packaging to AsterixDB. I
>>>> know we've had some one-off and demo ones (e.g. asterix-docker), but I
>>>> think it's time to have an official one. I've had this change waiting
>>>> in the wings for a while, and I figured it's time to try and push it
>>>> over the finish line.
>>>> I think this container should work for both evaluation and production
>>>> deployment. It contains a default config, but the entrypoint script
>>>> allows both a config file and any command line parameters to be given.
>>>> Here's the current patchset:
>>>> https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
>>>> and APE document:
>> https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+27%3A+Containerized+packaging
>>>> Looking forward to everyone's thoughts.
>>>> -Ian

Re: [DISCUSS][APE] Containerized packaging

Posted by Suryaa Charan Shivakumar <ss...@ucr.edu.INVALID>.
+1 for this feature!

Thanks and Regards,
Suryaa Charan S

On Thu, Aug 21, 2025, 10:02 Ian Maxon <im...@apache.org> wrote:

> Thanks for the +1, Mike. Yeah, I agree totally on adding better
> Kubernetes support. I have a dependent patch on this
> (https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18242) that adds
> example Helm charts as a first step toward this. Using a StatefulSet
> with either a PersistentVolume or one of the cloud/object store
> storage back-ends achieves basic functionality for static clusters.
> This patch comes out of some experimentation I was doing last year,
> and I was able to do stop/start just fine if I recall correctly.
> PersistentVolumes are really awesome, and solve a major headache that
> both YARN and Mesos had for hosting databases or anything that needed
> to manage data long-term, at least from my point of view.
>
> I think the Right solution however is to add a Kubernetes Operator for
> AsterixDB, and this should be the next step. This would allow resizing
> the cluster and other things. All the support for this already exists
> for the runtime effectively, and Kubernetes provides a lot of the
> things in the past that had to be done on our side (e.g. NCService's
> config distribution, or Managix's use of Zookeeper). It's roughly
> analogous to what was necessary to put AsterixDB on YARN, but
> Kubernetes has a lot more features.
>
> On Thu, Aug 21, 2025 at 9:42 AM Mike Carey <dt...@gmail.com> wrote:
> >
> > +1 for this addition!  (It might be cool to also have a default
> > Kubernetes controller that runs things in the clustered case, if there
> > is some way to do that, so that users can have reliable clusters that
> > they can manage and turn on and off?)
> >
> > On 8/21/25 8:06 AM, Ian Maxon wrote:
> > > Hello fellow devs,
> > >
> > > I proposed an APE to add a containerized packaging to AsterixDB. I
> > > know we've had some one-off and demo ones (e.g. asterix-docker), but I
> > > think it's time to have an official one. I've had this change waiting
> > > in the wings for a while, and I figured it's time to try and push it
> > > over the finish line.
> > >
> > > I think this container should work for both evaluation and production
> > > deployment. It contains a default config, but the entrypoint script
> > > allows both a config file and any command line parameters to be given.
> > >
> > > Here's the current patchset:
> > > https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
> > > and APE document:
> > >
> https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+27%3A+Containerized+packaging
> > >
> > > Looking forward to everyone's thoughts.
> > >
> > > -Ian
>

Re: [DISCUSS][APE] Containerized packaging

Posted by Ian Maxon <im...@apache.org>.
Thanks for the +1, Mike. Yeah, I agree totally on adding better
Kubernetes support. I have a dependent patch on this
(https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18242) that adds
example Helm charts as a first step toward this. Using a StatefulSet
with either a PersistentVolume or one of the cloud/object store
storage back-ends achieves basic functionality for static clusters.
This patch comes out of some experimentation I was doing last year,
and I was able to do stop/start just fine if I recall correctly.
PersistentVolumes are really awesome, and solve a major headache that
both YARN and Mesos had for hosting databases or anything that needed
to manage data long-term, at least from my point of view.

I think the Right solution however is to add a Kubernetes Operator for
AsterixDB, and this should be the next step. This would allow resizing
the cluster and other things. All the support for this already exists
for the runtime effectively, and Kubernetes provides a lot of the
things in the past that had to be done on our side (e.g. NCService's
config distribution, or Managix's use of Zookeeper). It's roughly
analogous to what was necessary to put AsterixDB on YARN, but
Kubernetes has a lot more features.

On Thu, Aug 21, 2025 at 9:42 AM Mike Carey <dt...@gmail.com> wrote:
>
> +1 for this addition!  (It might be cool to also have a default
> Kubernetes controller that runs things in the clustered case, if there
> is some way to do that, so that users can have reliable clusters that
> they can manage and turn on and off?)
>
> On 8/21/25 8:06 AM, Ian Maxon wrote:
> > Hello fellow devs,
> >
> > I proposed an APE to add a containerized packaging to AsterixDB. I
> > know we've had some one-off and demo ones (e.g. asterix-docker), but I
> > think it's time to have an official one. I've had this change waiting
> > in the wings for a while, and I figured it's time to try and push it
> > over the finish line.
> >
> > I think this container should work for both evaluation and production
> > deployment. It contains a default config, but the entrypoint script
> > allows both a config file and any command line parameters to be given.
> >
> > Here's the current patchset:
> > https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
> > and APE document:
> > https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+27%3A+Containerized+packaging
> >
> > Looking forward to everyone's thoughts.
> >
> > -Ian

Re: [DISCUSS][APE] Containerized packaging

Posted by Mike Carey <dt...@gmail.com>.
+1 for this addition!  (It might be cool to also have a default 
Kubernetes controller that runs things in the clustered case, if there 
is some way to do that, so that users can have reliable clusters that 
they can manage and turn on and off?)

On 8/21/25 8:06 AM, Ian Maxon wrote:
> Hello fellow devs,
>
> I proposed an APE to add a containerized packaging to AsterixDB. I
> know we've had some one-off and demo ones (e.g. asterix-docker), but I
> think it's time to have an official one. I've had this change waiting
> in the wings for a while, and I figured it's time to try and push it
> over the finish line.
>
> I think this container should work for both evaluation and production
> deployment. It contains a default config, but the entrypoint script
> allows both a config file and any command line parameters to be given.
>
> Here's the current patchset:
> https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
> and APE document:
> https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+27%3A+Containerized+packaging
>
> Looking forward to everyone's thoughts.
>
> -Ian

[RESULT][APE] Containerized packaging

Posted by Ian Maxon <im...@uci.edu>.
By the expedited APE acceptance process, this APE is adopted with the
following PMC members expressing their acceptance of the proposal:
+1's:
Mike Carey
Ian Maxon
Michael Blow
With 2 nonbinding +1's:
Suryaa Charan Shivakumar
Glenn Galvizo

On Fri, Aug 29, 2025 at 8:06 AM Ian Maxon <im...@uci.edu> wrote:
>
> +1
>
> On Fri, Aug 29, 2025 at 8:03 AM Michael Blow <mb...@gmail.com> wrote:
> >
> > +1, looks good to me!
> >
> > Thanks,
> >
> > -MDB
> >
> > On Thu, Aug 21, 2025 at 11:08 AM Ian Maxon <im...@apache.org> wrote:
> >
> > > Hello fellow devs,
> > >
> > > I proposed an APE to add a containerized packaging to AsterixDB. I
> > > know we've had some one-off and demo ones (e.g. asterix-docker), but I
> > > think it's time to have an official one. I've had this change waiting
> > > in the wings for a while, and I figured it's time to try and push it
> > > over the finish line.
> > >
> > > I think this container should work for both evaluation and production
> > > deployment. It contains a default config, but the entrypoint script
> > > allows both a config file and any command line parameters to be given.
> > >
> > > Here's the current patchset:
> > > https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
> > > and APE document:
> > >
> > > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/ASTERIXDB/APE*27*3A*Containerized*packaging__;KyUrKw!!CzAuKJ42GuquVTTmVmPViYEvSg!Nl6qYUt7Y4tEmsI7qklZY0uvWM_UtQ9ni5738XdyqBWiNFi8DTk_Fq3InWFL35K3MTgKbRZ2O7raJm-AKfiI$
> > >
> > > Looking forward to everyone's thoughts.
> > >
> > > -Ian
> > >

Re: [DISCUSS][APE] Containerized packaging

Posted by Ian Maxon <im...@uci.edu>.
+1

On Fri, Aug 29, 2025 at 8:03 AM Michael Blow <mb...@gmail.com> wrote:
>
> +1, looks good to me!
>
> Thanks,
>
> -MDB
>
> On Thu, Aug 21, 2025 at 11:08 AM Ian Maxon <im...@apache.org> wrote:
>
> > Hello fellow devs,
> >
> > I proposed an APE to add a containerized packaging to AsterixDB. I
> > know we've had some one-off and demo ones (e.g. asterix-docker), but I
> > think it's time to have an official one. I've had this change waiting
> > in the wings for a while, and I figured it's time to try and push it
> > over the finish line.
> >
> > I think this container should work for both evaluation and production
> > deployment. It contains a default config, but the entrypoint script
> > allows both a config file and any command line parameters to be given.
> >
> > Here's the current patchset:
> > https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
> > and APE document:
> >
> > https://urldefense.com/v3/__https://cwiki.apache.org/confluence/display/ASTERIXDB/APE*27*3A*Containerized*packaging__;KyUrKw!!CzAuKJ42GuquVTTmVmPViYEvSg!Nl6qYUt7Y4tEmsI7qklZY0uvWM_UtQ9ni5738XdyqBWiNFi8DTk_Fq3InWFL35K3MTgKbRZ2O7raJm-AKfiI$
> >
> > Looking forward to everyone's thoughts.
> >
> > -Ian
> >

Re: [DISCUSS][APE] Containerized packaging

Posted by Michael Blow <mb...@gmail.com>.
+1, looks good to me!

Thanks,

-MDB

On Thu, Aug 21, 2025 at 11:08 AM Ian Maxon <im...@apache.org> wrote:

> Hello fellow devs,
>
> I proposed an APE to add a containerized packaging to AsterixDB. I
> know we've had some one-off and demo ones (e.g. asterix-docker), but I
> think it's time to have an official one. I've had this change waiting
> in the wings for a while, and I figured it's time to try and push it
> over the finish line.
>
> I think this container should work for both evaluation and production
> deployment. It contains a default config, but the entrypoint script
> allows both a config file and any command line parameters to be given.
>
> Here's the current patchset:
> https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/18183
> and APE document:
>
> https://cwiki.apache.org/confluence/display/ASTERIXDB/APE+27%3A+Containerized+packaging
>
> Looking forward to everyone's thoughts.
>
> -Ian
>