You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@texera.apache.org by Chen Li <ch...@apache.org> on 2026/06/30 04:58:16 UTC
Running the system on a laptop with 8GB of RAM
Tanishq,
Last time you mentioned finding a way to run the system on your laptop with
8GB memory, by turning off unneeded microservices. Can you share the
information and add it to our GitHub site to benefit other developers?
Thanks,
Chen
Re: Running the system on a laptop with 8GB of RAM
Posted by Chen Li <ch...@apache.org>.
Thanks, Tanishq. Feel free to add related comments to the PR
https://github.com/apache/texera/pull/5961 .
On Sat, Jul 4, 2026 at 11:26 AM Tanishq Prashant Gandhi <ta...@uci.edu>
wrote:
> Hi Chen, Yicong, and all,
>
> Sorry for the slow reply, this email slipped past me for a week. Catching
> up now.
>
> Yicong, thanks for bin/local-dev.sh. I haven't tried it yet, but from the
> description it sounds like a lighter approach than mine.
>
> What I can share is how I got the Kubernetes setup running on my 8 GB
> laptop, in case it's useful for anyone testing the Kubernetes-specific
> behavior. Everything runs inside a mini Kubernetes cluster (minikube),
> which itself uses a good chunk of memory, so the trick was fitting the rest
> into what's left. Here's what worked:
>
> Give Docker enough memory before creating the cluster. In Docker Desktop →
> Settings → Resources, set Memory to 7 GB. Then:
> minikube start --memory=6000 --cpus=4 --driver=docker
>
> Turn off the services you don't need. I put this in a small values override
> file and passed it with -f:
> agentService:
> enabled: false # AI assistant
> litellm:
> enabled: false # AI model router
> exampleDataLoader:
> enabled: false # sample-data loader
> texeraImages:
> pullPolicy: IfNotPresent
>
> Give the install extra time:
> helm install texera k8s --namespace texera-dev --create-namespace \
> -f k8s/values-development.yaml -f k8s/values-dev-trimmed.yaml --timeout
> 20m
>
>
> With the AI services off, the whole thing fits in about 6 GB, tight, but
> stable.
>
> The reason I went the Kubernetes route is that it mirrors how Texera
> actually runs in production, so it lets me test my changes in a real
> deployment setup.
>
> Best,
> Tanishq
>
> On Fri, Jul 3, 2026 at 6:50 PM Chen Li <ch...@apache.org> wrote:
>
> > Thanks, Yicong. To finish this topic, let's use the tool in
> >
> >
> https://urldefense.com/v3/__https://github.com/apache/texera/pull/5961to__;!!CzAuKJ42GuquVTTmVmPViYEvSg!Mr3uz1cItQMYuQb489hbDyIxoTzsbU_m_HgLAr8-TtJg3XuT_oja4at4zks9Cf9c9dwEzegP3nBrg3A$
> > help developers manage the
> > micro services efficiently.
> >
> > Chen Li
> >
> > On Mon, Jun 29, 2026 at 10:46 PM Yicong Huang <yi...@cs.umass.edu>
> > wrote:
> >
> > > Hi Tanishq, Chen and all,
> > >
> > > Just wanted to add my two cents here. My local dev env has always been
> a
> > > headache. I used to use M1 macbook pro + 8GB RAM (from 2022?), and
> > > constantly got services OOM, build failure, and slowness. Because of
> > this I
> > > had to upgrade to M4 mac mini + 24GB RAM this month. The hardware
> change
> > > solved the issue to a certain extend, where many of the issues remain
> on
> > > our dev/infra tooling. For instance, there are too many services and
> I/my
> > > agent always forget to rebuild some of them after change, and it is
> > always
> > > hard to understand which version (branch/worktree/commit) of service
> was
> > > started by my agents.
> > >
> > > To solve this, I have recently introduced a new tool `bin/local-dev.sh`
> > > which can help developers to boot up a dev setup with one command. It
> can
> > > also have options to skip some services if not needed. It can also
> manage
> > > versions of services and use one command to refresh changed services.
> > Here
> > > you can read more about it
> >
> https://urldefense.com/v3/__https://github.com/apache/texera/pull/5961__;!!CzAuKJ42GuquVTTmVmPViYEvSg!Mr3uz1cItQMYuQb489hbDyIxoTzsbU_m_HgLAr8-TtJg3XuT_oja4at4zks9Cf9c9dwEzegPlYuA0A4$
> > ,
> > > and see demo videos of the usage.
> > >
> > > The main purpose of this tool was to let agents manage texera services
> > > easier during local development, by providing CLI for texera services.
> > > Before this, I would ask my agent to launch texera services locally and
> > > after 1 hour it still stucks. With this tool + a custome MCP connecting
> > to
> > > Claude Code, I can switch between branches and get services deployed
> in
> > > seconds.
> > >
> > > I believe it can also help human developers as well. For instance, I
> also
> > > provided `bin/local-dev.sh -i` which is an interactive interface where
> > you
> > > (human) can manage services, view MEM and CUP usage, and view logs.
> From
> > > what I see, local-dev mode does not need that much of memory, and we
> can
> > > further optimize the usage. I do believe running dev services on a 8GB
> > RAM
> > > machine is possible, as long as we don't do massive data processing
> > during
> > > dev.
> > >
> > > I am actively improving this tool, please feel free to try it out and
> > > create issues for bug report or feature requests. I hope this can make
> > > developers life easier.
> > >
> > >
> > > Best,
> > > Yicong Huang
> > > yiconghuang@cs.umass.edu
> > >
> > > On Jun 29, 2026 at 9:58 PM -0700, Chen Li <ch...@apache.org>, wrote:
> > > > Tanishq,
> > > >
> > > > Last time you mentioned finding a way to run the system on your
> laptop
> > > with
> > > > 8GB memory, by turning off unneeded microservices. Can you share the
> > > > information and add it to our GitHub site to benefit other
> developers?
> > > >
> > > > Thanks,
> > > >
> > > > Chen
> > >
> >
>
Re: Running the system on a laptop with 8GB of RAM
Posted by Tanishq Prashant Gandhi <ta...@uci.edu>.
Hi Chen, Yicong, and all,
Sorry for the slow reply, this email slipped past me for a week. Catching
up now.
Yicong, thanks for bin/local-dev.sh. I haven't tried it yet, but from the
description it sounds like a lighter approach than mine.
What I can share is how I got the Kubernetes setup running on my 8 GB
laptop, in case it's useful for anyone testing the Kubernetes-specific
behavior. Everything runs inside a mini Kubernetes cluster (minikube),
which itself uses a good chunk of memory, so the trick was fitting the rest
into what's left. Here's what worked:
Give Docker enough memory before creating the cluster. In Docker Desktop →
Settings → Resources, set Memory to 7 GB. Then:
minikube start --memory=6000 --cpus=4 --driver=docker
Turn off the services you don't need. I put this in a small values override
file and passed it with -f:
agentService:
enabled: false # AI assistant
litellm:
enabled: false # AI model router
exampleDataLoader:
enabled: false # sample-data loader
texeraImages:
pullPolicy: IfNotPresent
Give the install extra time:
helm install texera k8s --namespace texera-dev --create-namespace \
-f k8s/values-development.yaml -f k8s/values-dev-trimmed.yaml --timeout
20m
With the AI services off, the whole thing fits in about 6 GB, tight, but
stable.
The reason I went the Kubernetes route is that it mirrors how Texera
actually runs in production, so it lets me test my changes in a real
deployment setup.
Best,
Tanishq
On Fri, Jul 3, 2026 at 6:50 PM Chen Li <ch...@apache.org> wrote:
> Thanks, Yicong. To finish this topic, let's use the tool in
>
> https://urldefense.com/v3/__https://github.com/apache/texera/pull/5961to__;!!CzAuKJ42GuquVTTmVmPViYEvSg!Mr3uz1cItQMYuQb489hbDyIxoTzsbU_m_HgLAr8-TtJg3XuT_oja4at4zks9Cf9c9dwEzegP3nBrg3A$
> help developers manage the
> micro services efficiently.
>
> Chen Li
>
> On Mon, Jun 29, 2026 at 10:46 PM Yicong Huang <yi...@cs.umass.edu>
> wrote:
>
> > Hi Tanishq, Chen and all,
> >
> > Just wanted to add my two cents here. My local dev env has always been a
> > headache. I used to use M1 macbook pro + 8GB RAM (from 2022?), and
> > constantly got services OOM, build failure, and slowness. Because of
> this I
> > had to upgrade to M4 mac mini + 24GB RAM this month. The hardware change
> > solved the issue to a certain extend, where many of the issues remain on
> > our dev/infra tooling. For instance, there are too many services and I/my
> > agent always forget to rebuild some of them after change, and it is
> always
> > hard to understand which version (branch/worktree/commit) of service was
> > started by my agents.
> >
> > To solve this, I have recently introduced a new tool `bin/local-dev.sh`
> > which can help developers to boot up a dev setup with one command. It can
> > also have options to skip some services if not needed. It can also manage
> > versions of services and use one command to refresh changed services.
> Here
> > you can read more about it
> https://urldefense.com/v3/__https://github.com/apache/texera/pull/5961__;!!CzAuKJ42GuquVTTmVmPViYEvSg!Mr3uz1cItQMYuQb489hbDyIxoTzsbU_m_HgLAr8-TtJg3XuT_oja4at4zks9Cf9c9dwEzegPlYuA0A4$
> ,
> > and see demo videos of the usage.
> >
> > The main purpose of this tool was to let agents manage texera services
> > easier during local development, by providing CLI for texera services.
> > Before this, I would ask my agent to launch texera services locally and
> > after 1 hour it still stucks. With this tool + a custome MCP connecting
> to
> > Claude Code, I can switch between branches and get services deployed in
> > seconds.
> >
> > I believe it can also help human developers as well. For instance, I also
> > provided `bin/local-dev.sh -i` which is an interactive interface where
> you
> > (human) can manage services, view MEM and CUP usage, and view logs. From
> > what I see, local-dev mode does not need that much of memory, and we can
> > further optimize the usage. I do believe running dev services on a 8GB
> RAM
> > machine is possible, as long as we don't do massive data processing
> during
> > dev.
> >
> > I am actively improving this tool, please feel free to try it out and
> > create issues for bug report or feature requests. I hope this can make
> > developers life easier.
> >
> >
> > Best,
> > Yicong Huang
> > yiconghuang@cs.umass.edu
> >
> > On Jun 29, 2026 at 9:58 PM -0700, Chen Li <ch...@apache.org>, wrote:
> > > Tanishq,
> > >
> > > Last time you mentioned finding a way to run the system on your laptop
> > with
> > > 8GB memory, by turning off unneeded microservices. Can you share the
> > > information and add it to our GitHub site to benefit other developers?
> > >
> > > Thanks,
> > >
> > > Chen
> >
>
Re: Running the system on a laptop with 8GB of RAM
Posted by Chen Li <ch...@apache.org>.
Thanks, Yicong. To finish this topic, let's use the tool in
https://github.com/apache/texera/pull/5961to help developers manage the
micro services efficiently.
Chen Li
On Mon, Jun 29, 2026 at 10:46 PM Yicong Huang <yi...@cs.umass.edu>
wrote:
> Hi Tanishq, Chen and all,
>
> Just wanted to add my two cents here. My local dev env has always been a
> headache. I used to use M1 macbook pro + 8GB RAM (from 2022?), and
> constantly got services OOM, build failure, and slowness. Because of this I
> had to upgrade to M4 mac mini + 24GB RAM this month. The hardware change
> solved the issue to a certain extend, where many of the issues remain on
> our dev/infra tooling. For instance, there are too many services and I/my
> agent always forget to rebuild some of them after change, and it is always
> hard to understand which version (branch/worktree/commit) of service was
> started by my agents.
>
> To solve this, I have recently introduced a new tool `bin/local-dev.sh`
> which can help developers to boot up a dev setup with one command. It can
> also have options to skip some services if not needed. It can also manage
> versions of services and use one command to refresh changed services. Here
> you can read more about it https://github.com/apache/texera/pull/5961,
> and see demo videos of the usage.
>
> The main purpose of this tool was to let agents manage texera services
> easier during local development, by providing CLI for texera services.
> Before this, I would ask my agent to launch texera services locally and
> after 1 hour it still stucks. With this tool + a custome MCP connecting to
> Claude Code, I can switch between branches and get services deployed in
> seconds.
>
> I believe it can also help human developers as well. For instance, I also
> provided `bin/local-dev.sh -i` which is an interactive interface where you
> (human) can manage services, view MEM and CUP usage, and view logs. From
> what I see, local-dev mode does not need that much of memory, and we can
> further optimize the usage. I do believe running dev services on a 8GB RAM
> machine is possible, as long as we don't do massive data processing during
> dev.
>
> I am actively improving this tool, please feel free to try it out and
> create issues for bug report or feature requests. I hope this can make
> developers life easier.
>
>
> Best,
> Yicong Huang
> yiconghuang@cs.umass.edu
>
> On Jun 29, 2026 at 9:58 PM -0700, Chen Li <ch...@apache.org>, wrote:
> > Tanishq,
> >
> > Last time you mentioned finding a way to run the system on your laptop
> with
> > 8GB memory, by turning off unneeded microservices. Can you share the
> > information and add it to our GitHub site to benefit other developers?
> >
> > Thanks,
> >
> > Chen
>
Re: Running the system on a laptop with 8GB of RAM
Posted by Yicong Huang <yi...@cs.umass.edu>.
Hi Tanishq, Chen and all,
Just wanted to add my two cents here. My local dev env has always been a headache. I used to use M1 macbook pro + 8GB RAM (from 2022?), and constantly got services OOM, build failure, and slowness. Because of this I had to upgrade to M4 mac mini + 24GB RAM this month. The hardware change solved the issue to a certain extend, where many of the issues remain on our dev/infra tooling. For instance, there are too many services and I/my agent always forget to rebuild some of them after change, and it is always hard to understand which version (branch/worktree/commit) of service was started by my agents.
To solve this, I have recently introduced a new tool `bin/local-dev.sh` which can help developers to boot up a dev setup with one command. It can also have options to skip some services if not needed. It can also manage versions of services and use one command to refresh changed services. Here you can read more about it https://github.com/apache/texera/pull/5961, and see demo videos of the usage.
The main purpose of this tool was to let agents manage texera services easier during local development, by providing CLI for texera services. Before this, I would ask my agent to launch texera services locally and after 1 hour it still stucks. With this tool + a custome MCP connecting to Claude Code, I can switch between branches and get services deployed in seconds.
I believe it can also help human developers as well. For instance, I also provided `bin/local-dev.sh -i` which is an interactive interface where you (human) can manage services, view MEM and CUP usage, and view logs. From what I see, local-dev mode does not need that much of memory, and we can further optimize the usage. I do believe running dev services on a 8GB RAM machine is possible, as long as we don't do massive data processing during dev.
I am actively improving this tool, please feel free to try it out and create issues for bug report or feature requests. I hope this can make developers life easier.
Best,
Yicong Huang
yiconghuang@cs.umass.edu
On Jun 29, 2026 at 9:58 PM -0700, Chen Li <ch...@apache.org>, wrote:
> Tanishq,
>
> Last time you mentioned finding a way to run the system on your laptop with
> 8GB memory, by turning off unneeded microservices. Can you share the
> information and add it to our GitHub site to benefit other developers?
>
> Thanks,
>
> Chen