You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Ivan Kudryavtsev <ku...@bw-sw.com> on 2017/09/10 06:37:15 UTC
KVM Host overprovisioning
Hello, community.
During the last years Linux kernel got some interesting features like KSM,
ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
extremely fast SSD drives with m.2 and PCI-E interfaces.
These facilities enable potentially interesting use of overcommited RAM for
hosts. According to IBM's investigations Zswap with LZ4/ZBUD increases
virtual RAM on 40%.
I investigated current Apache CloudStack memory overcommitment capabilities
and they mostly affect VM's RAM by utilizing ballooning and I think it's
not what necessary to open new facilities. There are many cases which can
utilize ZSWAP and fast swap devices to efficiently provision more RAM than
presents.
I suppose, CloudStack Agent for KVM can have configured parameter which
"mangles" RAM reported. From the other hand it can be done by implemented
host properties on server side. I tried manual increasing in host table:
update host set ram=ram * 1.4 where id=1;
and it seems until the next host stats update it works as expected. I think
this workaround is useful, but it's better to have the function in core
like standard.
Let me know what you think about it, it might be I don't understand
something and ACS already has it in place? I also would like to hear your
thoughts on ZSWAP usage in practice.
--
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Hi, community.
I implemented quick PR for KVM cloudstack-agent with use of
additional directive host.overcommit.mem.mb at agent.properties
https://github.com/apache/cloudstack/pull/2266
I tested it in my 4.9, works nic for me. Also, during the building I found
interesting bug in Quota plugin, practically, it fails building before
11:00AM in my GMT+6 TZ, because
https://github.com/apache/cloudstack/blob/master/plugins/database/quota/src/org/apache/cloudstack/api/response/QuotaResponseBuilderImpl.java#L513
generates incorrect date for next day causing test
https://github.com/apache/cloudstack/blob/master/plugins/database/quota/test/org/apache/cloudstack/api/response/QuotaResponseBuilderImplTest.java#L221
Fortunately while I investigated 11AM happened and code was built without
problems. Since I don't use Quota functions I stopped digging further but
PR includes small refactoring of code which just removes copy-paste.
2017-09-12 14:09 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> > Op 12 september 2017 om 9:05 schreef Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com>:
> >
> >
> > Yes, sure.
> >
> > What I want is an ability to increase host memory rather than decrease
> it.
> >
> > So the first suggestion is to add a parameter to increase amount of
> > megabytes or gigabytes, not necessary a multiplier. Manual adding via
> > agent.properties is a good way to implement it because different hosts
> can
> > have different capabilities (depending on CPU model) and manual
> "per-host"
> > configuration is better than just cluster configuration option.
> >
>
> I would use a multiply factor, but you can implement both. A PR for this
> would be welcome!
>
> Wido
>
> > 2017-09-12 13:46 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> >
> > >
> > > > Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <
> > > kudryavtsev_ia@bw-sw.com>:
> > > >
> > > >
> > > > Hi, Wido.
> > > >
> > > > Yes, you can. But it works not the way I expect because It cuts RAM
> from
> > > VM
> > > > by dividing it to overprovisioning factor like VM with 2GB of RAM
> with
> > > > Overprovisioning factor 2.0 will get 1GB displayed with "free"
> command.
> > > > That's why I finished the message with words that it might be I just
> > > don't
> > > > get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
> > > >
> > >
> > > Hmm, ok.
> > >
> > > So for the KVM Agent you can add "host reserved mb" to the
> > > agent.properties, but you are proposing a setting where you can
> multiply
> > > the memory?
> > >
> > > Eg by 1.5 if you want to and have the Agent expose that to the MGMT
> server?
> > >
> > > Wido
> > >
> > > > 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> > > >
> > > > > Hi,
> > > > >
> > > > > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > > > > kudryavtsev_ia@bw-sw.com>:
> > > > > >
> > > > > >
> > > > > > Hello, community.
> > > > > >
> > > > > > During the last years Linux kernel got some interesting features
> like
> > > > > KSM,
> > > > > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d
> xpoint,
> > > > > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > > > > >
> > > > > > These facilities enable potentially interesting use of
> overcommited
> > > RAM
> > > > > for
> > > > > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD
> > > increases
> > > > > > virtual RAM on 40%.
> > > > > >
> > > > > > I investigated current Apache CloudStack memory overcommitment
> > > > > capabilities
> > > > > > and they mostly affect VM's RAM by utilizing ballooning and I
> think
> > > it's
> > > > > > not what necessary to open new facilities. There are many cases
> > > which can
> > > > > > utilize ZSWAP and fast swap devices to efficiently provision
> more RAM
> > > > > than
> > > > > > presents.
> > > > > >
> > > > > > I suppose, CloudStack Agent for KVM can have configured parameter
> > > which
> > > > > > "mangles" RAM reported. From the other hand it can be done by
> > > implemented
> > > > > > host properties on server side. I tried manual increasing in host
> > > table:
> > > > > >
> > > > > > update host set ram=ram * 1.4 where id=1;
> > > > > >
> > > > > > and it seems until the next host stats update it works as
> expected. I
> > > > > think
> > > > > > this workaround is useful, but it's better to have the function
> in
> > > core
> > > > > > like standard.
> > > > > >
> > > > >
> > > > > Can't you set memory over provisioning on a cluster basis in the
> GUI? I
> > > > > thought you could.
> > > > >
> > > > > Wido
> > > > >
> > > > > > Let me know what you think about it, it might be I don't
> understand
> > > > > > something and ACS already has it in place? I also would like to
> hear
> > > your
> > > > > > thoughts on ZSWAP usage in practice.
> > > > > >
> > > > > > --
> > > > > > With best regards, Ivan Kudryavtsev
> > > > > > Bitworks Software, Ltd.
> > > > > > Cell: +7-923-414-1515
> > > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > With best regards, Ivan Kudryavtsev
> > > > Bitworks Software, Ltd.
> > > > Cell: +7-923-414-1515
> > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
> >
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Hi, community.
I implemented quick PR for KVM cloudstack-agent with use of
additional directive host.overcommit.mem.mb at agent.properties
https://github.com/apache/cloudstack/pull/2266
I tested it in my 4.9, works nic for me. Also, during the building I found
interesting bug in Quota plugin, practically, it fails building before
11:00AM in my GMT+6 TZ, because
https://github.com/apache/cloudstack/blob/master/plugins/database/quota/src/org/apache/cloudstack/api/response/QuotaResponseBuilderImpl.java#L513
generates incorrect date for next day causing test
https://github.com/apache/cloudstack/blob/master/plugins/database/quota/test/org/apache/cloudstack/api/response/QuotaResponseBuilderImplTest.java#L221
Fortunately while I investigated 11AM happened and code was built without
problems. Since I don't use Quota functions I stopped digging further but
PR includes small refactoring of code which just removes copy-paste.
2017-09-12 14:09 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> > Op 12 september 2017 om 9:05 schreef Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com>:
> >
> >
> > Yes, sure.
> >
> > What I want is an ability to increase host memory rather than decrease
> it.
> >
> > So the first suggestion is to add a parameter to increase amount of
> > megabytes or gigabytes, not necessary a multiplier. Manual adding via
> > agent.properties is a good way to implement it because different hosts
> can
> > have different capabilities (depending on CPU model) and manual
> "per-host"
> > configuration is better than just cluster configuration option.
> >
>
> I would use a multiply factor, but you can implement both. A PR for this
> would be welcome!
>
> Wido
>
> > 2017-09-12 13:46 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> >
> > >
> > > > Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <
> > > kudryavtsev_ia@bw-sw.com>:
> > > >
> > > >
> > > > Hi, Wido.
> > > >
> > > > Yes, you can. But it works not the way I expect because It cuts RAM
> from
> > > VM
> > > > by dividing it to overprovisioning factor like VM with 2GB of RAM
> with
> > > > Overprovisioning factor 2.0 will get 1GB displayed with "free"
> command.
> > > > That's why I finished the message with words that it might be I just
> > > don't
> > > > get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
> > > >
> > >
> > > Hmm, ok.
> > >
> > > So for the KVM Agent you can add "host reserved mb" to the
> > > agent.properties, but you are proposing a setting where you can
> multiply
> > > the memory?
> > >
> > > Eg by 1.5 if you want to and have the Agent expose that to the MGMT
> server?
> > >
> > > Wido
> > >
> > > > 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> > > >
> > > > > Hi,
> > > > >
> > > > > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > > > > kudryavtsev_ia@bw-sw.com>:
> > > > > >
> > > > > >
> > > > > > Hello, community.
> > > > > >
> > > > > > During the last years Linux kernel got some interesting features
> like
> > > > > KSM,
> > > > > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d
> xpoint,
> > > > > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > > > > >
> > > > > > These facilities enable potentially interesting use of
> overcommited
> > > RAM
> > > > > for
> > > > > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD
> > > increases
> > > > > > virtual RAM on 40%.
> > > > > >
> > > > > > I investigated current Apache CloudStack memory overcommitment
> > > > > capabilities
> > > > > > and they mostly affect VM's RAM by utilizing ballooning and I
> think
> > > it's
> > > > > > not what necessary to open new facilities. There are many cases
> > > which can
> > > > > > utilize ZSWAP and fast swap devices to efficiently provision
> more RAM
> > > > > than
> > > > > > presents.
> > > > > >
> > > > > > I suppose, CloudStack Agent for KVM can have configured parameter
> > > which
> > > > > > "mangles" RAM reported. From the other hand it can be done by
> > > implemented
> > > > > > host properties on server side. I tried manual increasing in host
> > > table:
> > > > > >
> > > > > > update host set ram=ram * 1.4 where id=1;
> > > > > >
> > > > > > and it seems until the next host stats update it works as
> expected. I
> > > > > think
> > > > > > this workaround is useful, but it's better to have the function
> in
> > > core
> > > > > > like standard.
> > > > > >
> > > > >
> > > > > Can't you set memory over provisioning on a cluster basis in the
> GUI? I
> > > > > thought you could.
> > > > >
> > > > > Wido
> > > > >
> > > > > > Let me know what you think about it, it might be I don't
> understand
> > > > > > something and ACS already has it in place? I also would like to
> hear
> > > your
> > > > > > thoughts on ZSWAP usage in practice.
> > > > > >
> > > > > > --
> > > > > > With best regards, Ivan Kudryavtsev
> > > > > > Bitworks Software, Ltd.
> > > > > > Cell: +7-923-414-1515
> > > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > With best regards, Ivan Kudryavtsev
> > > > Bitworks Software, Ltd.
> > > > Cell: +7-923-414-1515
> > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
> >
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Wido den Hollander <wi...@widodh.nl>.
> Op 12 september 2017 om 9:05 schreef Ivan Kudryavtsev <ku...@bw-sw.com>:
>
>
> Yes, sure.
>
> What I want is an ability to increase host memory rather than decrease it.
>
> So the first suggestion is to add a parameter to increase amount of
> megabytes or gigabytes, not necessary a multiplier. Manual adding via
> agent.properties is a good way to implement it because different hosts can
> have different capabilities (depending on CPU model) and manual "per-host"
> configuration is better than just cluster configuration option.
>
I would use a multiply factor, but you can implement both. A PR for this would be welcome!
Wido
> 2017-09-12 13:46 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> >
> > > Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <
> > kudryavtsev_ia@bw-sw.com>:
> > >
> > >
> > > Hi, Wido.
> > >
> > > Yes, you can. But it works not the way I expect because It cuts RAM from
> > VM
> > > by dividing it to overprovisioning factor like VM with 2GB of RAM with
> > > Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
> > > That's why I finished the message with words that it might be I just
> > don't
> > > get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
> > >
> >
> > Hmm, ok.
> >
> > So for the KVM Agent you can add "host reserved mb" to the
> > agent.properties, but you are proposing a setting where you can multiply
> > the memory?
> >
> > Eg by 1.5 if you want to and have the Agent expose that to the MGMT server?
> >
> > Wido
> >
> > > 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> > >
> > > > Hi,
> > > >
> > > > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > > > kudryavtsev_ia@bw-sw.com>:
> > > > >
> > > > >
> > > > > Hello, community.
> > > > >
> > > > > During the last years Linux kernel got some interesting features like
> > > > KSM,
> > > > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > > > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > > > >
> > > > > These facilities enable potentially interesting use of overcommited
> > RAM
> > > > for
> > > > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD
> > increases
> > > > > virtual RAM on 40%.
> > > > >
> > > > > I investigated current Apache CloudStack memory overcommitment
> > > > capabilities
> > > > > and they mostly affect VM's RAM by utilizing ballooning and I think
> > it's
> > > > > not what necessary to open new facilities. There are many cases
> > which can
> > > > > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> > > > than
> > > > > presents.
> > > > >
> > > > > I suppose, CloudStack Agent for KVM can have configured parameter
> > which
> > > > > "mangles" RAM reported. From the other hand it can be done by
> > implemented
> > > > > host properties on server side. I tried manual increasing in host
> > table:
> > > > >
> > > > > update host set ram=ram * 1.4 where id=1;
> > > > >
> > > > > and it seems until the next host stats update it works as expected. I
> > > > think
> > > > > this workaround is useful, but it's better to have the function in
> > core
> > > > > like standard.
> > > > >
> > > >
> > > > Can't you set memory over provisioning on a cluster basis in the GUI? I
> > > > thought you could.
> > > >
> > > > Wido
> > > >
> > > > > Let me know what you think about it, it might be I don't understand
> > > > > something and ACS already has it in place? I also would like to hear
> > your
> > > > > thoughts on ZSWAP usage in practice.
> > > > >
> > > > > --
> > > > > With best regards, Ivan Kudryavtsev
> > > > > Bitworks Software, Ltd.
> > > > > Cell: +7-923-414-1515
> > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > > >
> > >
> > >
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks Software, Ltd.
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> >
>
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Wido den Hollander <wi...@widodh.nl>.
> Op 12 september 2017 om 9:05 schreef Ivan Kudryavtsev <ku...@bw-sw.com>:
>
>
> Yes, sure.
>
> What I want is an ability to increase host memory rather than decrease it.
>
> So the first suggestion is to add a parameter to increase amount of
> megabytes or gigabytes, not necessary a multiplier. Manual adding via
> agent.properties is a good way to implement it because different hosts can
> have different capabilities (depending on CPU model) and manual "per-host"
> configuration is better than just cluster configuration option.
>
I would use a multiply factor, but you can implement both. A PR for this would be welcome!
Wido
> 2017-09-12 13:46 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> >
> > > Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <
> > kudryavtsev_ia@bw-sw.com>:
> > >
> > >
> > > Hi, Wido.
> > >
> > > Yes, you can. But it works not the way I expect because It cuts RAM from
> > VM
> > > by dividing it to overprovisioning factor like VM with 2GB of RAM with
> > > Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
> > > That's why I finished the message with words that it might be I just
> > don't
> > > get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
> > >
> >
> > Hmm, ok.
> >
> > So for the KVM Agent you can add "host reserved mb" to the
> > agent.properties, but you are proposing a setting where you can multiply
> > the memory?
> >
> > Eg by 1.5 if you want to and have the Agent expose that to the MGMT server?
> >
> > Wido
> >
> > > 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> > >
> > > > Hi,
> > > >
> > > > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > > > kudryavtsev_ia@bw-sw.com>:
> > > > >
> > > > >
> > > > > Hello, community.
> > > > >
> > > > > During the last years Linux kernel got some interesting features like
> > > > KSM,
> > > > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > > > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > > > >
> > > > > These facilities enable potentially interesting use of overcommited
> > RAM
> > > > for
> > > > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD
> > increases
> > > > > virtual RAM on 40%.
> > > > >
> > > > > I investigated current Apache CloudStack memory overcommitment
> > > > capabilities
> > > > > and they mostly affect VM's RAM by utilizing ballooning and I think
> > it's
> > > > > not what necessary to open new facilities. There are many cases
> > which can
> > > > > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> > > > than
> > > > > presents.
> > > > >
> > > > > I suppose, CloudStack Agent for KVM can have configured parameter
> > which
> > > > > "mangles" RAM reported. From the other hand it can be done by
> > implemented
> > > > > host properties on server side. I tried manual increasing in host
> > table:
> > > > >
> > > > > update host set ram=ram * 1.4 where id=1;
> > > > >
> > > > > and it seems until the next host stats update it works as expected. I
> > > > think
> > > > > this workaround is useful, but it's better to have the function in
> > core
> > > > > like standard.
> > > > >
> > > >
> > > > Can't you set memory over provisioning on a cluster basis in the GUI? I
> > > > thought you could.
> > > >
> > > > Wido
> > > >
> > > > > Let me know what you think about it, it might be I don't understand
> > > > > something and ACS already has it in place? I also would like to hear
> > your
> > > > > thoughts on ZSWAP usage in practice.
> > > > >
> > > > > --
> > > > > With best regards, Ivan Kudryavtsev
> > > > > Bitworks Software, Ltd.
> > > > > Cell: +7-923-414-1515
> > > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > > >
> > >
> > >
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks Software, Ltd.
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> >
>
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Yes, sure.
What I want is an ability to increase host memory rather than decrease it.
So the first suggestion is to add a parameter to increase amount of
megabytes or gigabytes, not necessary a multiplier. Manual adding via
agent.properties is a good way to implement it because different hosts can
have different capabilities (depending on CPU model) and manual "per-host"
configuration is better than just cluster configuration option.
2017-09-12 13:46 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> > Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com>:
> >
> >
> > Hi, Wido.
> >
> > Yes, you can. But it works not the way I expect because It cuts RAM from
> VM
> > by dividing it to overprovisioning factor like VM with 2GB of RAM with
> > Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
> > That's why I finished the message with words that it might be I just
> don't
> > get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
> >
>
> Hmm, ok.
>
> So for the KVM Agent you can add "host reserved mb" to the
> agent.properties, but you are proposing a setting where you can multiply
> the memory?
>
> Eg by 1.5 if you want to and have the Agent expose that to the MGMT server?
>
> Wido
>
> > 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> >
> > > Hi,
> > >
> > > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > > kudryavtsev_ia@bw-sw.com>:
> > > >
> > > >
> > > > Hello, community.
> > > >
> > > > During the last years Linux kernel got some interesting features like
> > > KSM,
> > > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > > >
> > > > These facilities enable potentially interesting use of overcommited
> RAM
> > > for
> > > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD
> increases
> > > > virtual RAM on 40%.
> > > >
> > > > I investigated current Apache CloudStack memory overcommitment
> > > capabilities
> > > > and they mostly affect VM's RAM by utilizing ballooning and I think
> it's
> > > > not what necessary to open new facilities. There are many cases
> which can
> > > > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> > > than
> > > > presents.
> > > >
> > > > I suppose, CloudStack Agent for KVM can have configured parameter
> which
> > > > "mangles" RAM reported. From the other hand it can be done by
> implemented
> > > > host properties on server side. I tried manual increasing in host
> table:
> > > >
> > > > update host set ram=ram * 1.4 where id=1;
> > > >
> > > > and it seems until the next host stats update it works as expected. I
> > > think
> > > > this workaround is useful, but it's better to have the function in
> core
> > > > like standard.
> > > >
> > >
> > > Can't you set memory over provisioning on a cluster basis in the GUI? I
> > > thought you could.
> > >
> > > Wido
> > >
> > > > Let me know what you think about it, it might be I don't understand
> > > > something and ACS already has it in place? I also would like to hear
> your
> > > > thoughts on ZSWAP usage in practice.
> > > >
> > > > --
> > > > With best regards, Ivan Kudryavtsev
> > > > Bitworks Software, Ltd.
> > > > Cell: +7-923-414-1515
> > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
> >
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Yes, sure.
What I want is an ability to increase host memory rather than decrease it.
So the first suggestion is to add a parameter to increase amount of
megabytes or gigabytes, not necessary a multiplier. Manual adding via
agent.properties is a good way to implement it because different hosts can
have different capabilities (depending on CPU model) and manual "per-host"
configuration is better than just cluster configuration option.
2017-09-12 13:46 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> > Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com>:
> >
> >
> > Hi, Wido.
> >
> > Yes, you can. But it works not the way I expect because It cuts RAM from
> VM
> > by dividing it to overprovisioning factor like VM with 2GB of RAM with
> > Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
> > That's why I finished the message with words that it might be I just
> don't
> > get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
> >
>
> Hmm, ok.
>
> So for the KVM Agent you can add "host reserved mb" to the
> agent.properties, but you are proposing a setting where you can multiply
> the memory?
>
> Eg by 1.5 if you want to and have the Agent expose that to the MGMT server?
>
> Wido
>
> > 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> >
> > > Hi,
> > >
> > > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > > kudryavtsev_ia@bw-sw.com>:
> > > >
> > > >
> > > > Hello, community.
> > > >
> > > > During the last years Linux kernel got some interesting features like
> > > KSM,
> > > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > > >
> > > > These facilities enable potentially interesting use of overcommited
> RAM
> > > for
> > > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD
> increases
> > > > virtual RAM on 40%.
> > > >
> > > > I investigated current Apache CloudStack memory overcommitment
> > > capabilities
> > > > and they mostly affect VM's RAM by utilizing ballooning and I think
> it's
> > > > not what necessary to open new facilities. There are many cases
> which can
> > > > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> > > than
> > > > presents.
> > > >
> > > > I suppose, CloudStack Agent for KVM can have configured parameter
> which
> > > > "mangles" RAM reported. From the other hand it can be done by
> implemented
> > > > host properties on server side. I tried manual increasing in host
> table:
> > > >
> > > > update host set ram=ram * 1.4 where id=1;
> > > >
> > > > and it seems until the next host stats update it works as expected. I
> > > think
> > > > this workaround is useful, but it's better to have the function in
> core
> > > > like standard.
> > > >
> > >
> > > Can't you set memory over provisioning on a cluster basis in the GUI? I
> > > thought you could.
> > >
> > > Wido
> > >
> > > > Let me know what you think about it, it might be I don't understand
> > > > something and ACS already has it in place? I also would like to hear
> your
> > > > thoughts on ZSWAP usage in practice.
> > > >
> > > > --
> > > > With best regards, Ivan Kudryavtsev
> > > > Bitworks Software, Ltd.
> > > > Cell: +7-923-414-1515
> > > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> > >
> >
> >
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Wido den Hollander <wi...@widodh.nl>.
> Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <ku...@bw-sw.com>:
>
>
> Hi, Wido.
>
> Yes, you can. But it works not the way I expect because It cuts RAM from VM
> by dividing it to overprovisioning factor like VM with 2GB of RAM with
> Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
> That's why I finished the message with words that it might be I just don't
> get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
>
Hmm, ok.
So for the KVM Agent you can add "host reserved mb" to the agent.properties, but you are proposing a setting where you can multiply the memory?
Eg by 1.5 if you want to and have the Agent expose that to the MGMT server?
Wido
> 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> > Hi,
> >
> > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > kudryavtsev_ia@bw-sw.com>:
> > >
> > >
> > > Hello, community.
> > >
> > > During the last years Linux kernel got some interesting features like
> > KSM,
> > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > >
> > > These facilities enable potentially interesting use of overcommited RAM
> > for
> > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD increases
> > > virtual RAM on 40%.
> > >
> > > I investigated current Apache CloudStack memory overcommitment
> > capabilities
> > > and they mostly affect VM's RAM by utilizing ballooning and I think it's
> > > not what necessary to open new facilities. There are many cases which can
> > > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> > than
> > > presents.
> > >
> > > I suppose, CloudStack Agent for KVM can have configured parameter which
> > > "mangles" RAM reported. From the other hand it can be done by implemented
> > > host properties on server side. I tried manual increasing in host table:
> > >
> > > update host set ram=ram * 1.4 where id=1;
> > >
> > > and it seems until the next host stats update it works as expected. I
> > think
> > > this workaround is useful, but it's better to have the function in core
> > > like standard.
> > >
> >
> > Can't you set memory over provisioning on a cluster basis in the GUI? I
> > thought you could.
> >
> > Wido
> >
> > > Let me know what you think about it, it might be I don't understand
> > > something and ACS already has it in place? I also would like to hear your
> > > thoughts on ZSWAP usage in practice.
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks Software, Ltd.
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> >
>
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Wido den Hollander <wi...@widodh.nl>.
> Op 11 september 2017 om 13:04 schreef Ivan Kudryavtsev <ku...@bw-sw.com>:
>
>
> Hi, Wido.
>
> Yes, you can. But it works not the way I expect because It cuts RAM from VM
> by dividing it to overprovisioning factor like VM with 2GB of RAM with
> Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
> That's why I finished the message with words that it might be I just don't
> get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
>
Hmm, ok.
So for the KVM Agent you can add "host reserved mb" to the agent.properties, but you are proposing a setting where you can multiply the memory?
Eg by 1.5 if you want to and have the Agent expose that to the MGMT server?
Wido
> 2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
>
> > Hi,
> >
> > > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> > kudryavtsev_ia@bw-sw.com>:
> > >
> > >
> > > Hello, community.
> > >
> > > During the last years Linux kernel got some interesting features like
> > KSM,
> > > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > > extremely fast SSD drives with m.2 and PCI-E interfaces.
> > >
> > > These facilities enable potentially interesting use of overcommited RAM
> > for
> > > hosts. According to IBM's investigations Zswap with LZ4/ZBUD increases
> > > virtual RAM on 40%.
> > >
> > > I investigated current Apache CloudStack memory overcommitment
> > capabilities
> > > and they mostly affect VM's RAM by utilizing ballooning and I think it's
> > > not what necessary to open new facilities. There are many cases which can
> > > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> > than
> > > presents.
> > >
> > > I suppose, CloudStack Agent for KVM can have configured parameter which
> > > "mangles" RAM reported. From the other hand it can be done by implemented
> > > host properties on server side. I tried manual increasing in host table:
> > >
> > > update host set ram=ram * 1.4 where id=1;
> > >
> > > and it seems until the next host stats update it works as expected. I
> > think
> > > this workaround is useful, but it's better to have the function in core
> > > like standard.
> > >
> >
> > Can't you set memory over provisioning on a cluster basis in the GUI? I
> > thought you could.
> >
> > Wido
> >
> > > Let me know what you think about it, it might be I don't understand
> > > something and ACS already has it in place? I also would like to hear your
> > > thoughts on ZSWAP usage in practice.
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks Software, Ltd.
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ <http://bw-sw.com/>
> >
>
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Hi, Wido.
Yes, you can. But it works not the way I expect because It cuts RAM from VM
by dividing it to overprovisioning factor like VM with 2GB of RAM with
Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
That's why I finished the message with words that it might be I just don't
get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> Hi,
>
> > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com>:
> >
> >
> > Hello, community.
> >
> > During the last years Linux kernel got some interesting features like
> KSM,
> > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > extremely fast SSD drives with m.2 and PCI-E interfaces.
> >
> > These facilities enable potentially interesting use of overcommited RAM
> for
> > hosts. According to IBM's investigations Zswap with LZ4/ZBUD increases
> > virtual RAM on 40%.
> >
> > I investigated current Apache CloudStack memory overcommitment
> capabilities
> > and they mostly affect VM's RAM by utilizing ballooning and I think it's
> > not what necessary to open new facilities. There are many cases which can
> > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> than
> > presents.
> >
> > I suppose, CloudStack Agent for KVM can have configured parameter which
> > "mangles" RAM reported. From the other hand it can be done by implemented
> > host properties on server side. I tried manual increasing in host table:
> >
> > update host set ram=ram * 1.4 where id=1;
> >
> > and it seems until the next host stats update it works as expected. I
> think
> > this workaround is useful, but it's better to have the function in core
> > like standard.
> >
>
> Can't you set memory over provisioning on a cluster basis in the GUI? I
> thought you could.
>
> Wido
>
> > Let me know what you think about it, it might be I don't understand
> > something and ACS already has it in place? I also would like to hear your
> > thoughts on ZSWAP usage in practice.
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Ivan Kudryavtsev <ku...@bw-sw.com>.
Hi, Wido.
Yes, you can. But it works not the way I expect because It cuts RAM from VM
by dividing it to overprovisioning factor like VM with 2GB of RAM with
Overprovisioning factor 2.0 will get 1GB displayed with "free" command.
That's why I finished the message with words that it might be I just don't
get the idea. The behaviour is the same in my prod 4.3 and new 4.9.
2017-09-11 18:00 GMT+07:00 Wido den Hollander <wi...@widodh.nl>:
> Hi,
>
> > Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <
> kudryavtsev_ia@bw-sw.com>:
> >
> >
> > Hello, community.
> >
> > During the last years Linux kernel got some interesting features like
> KSM,
> > ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> > extremely fast SSD drives with m.2 and PCI-E interfaces.
> >
> > These facilities enable potentially interesting use of overcommited RAM
> for
> > hosts. According to IBM's investigations Zswap with LZ4/ZBUD increases
> > virtual RAM on 40%.
> >
> > I investigated current Apache CloudStack memory overcommitment
> capabilities
> > and they mostly affect VM's RAM by utilizing ballooning and I think it's
> > not what necessary to open new facilities. There are many cases which can
> > utilize ZSWAP and fast swap devices to efficiently provision more RAM
> than
> > presents.
> >
> > I suppose, CloudStack Agent for KVM can have configured parameter which
> > "mangles" RAM reported. From the other hand it can be done by implemented
> > host properties on server side. I tried manual increasing in host table:
> >
> > update host set ram=ram * 1.4 where id=1;
> >
> > and it seems until the next host stats update it works as expected. I
> think
> > this workaround is useful, but it's better to have the function in core
> > like standard.
> >
>
> Can't you set memory over provisioning on a cluster basis in the GUI? I
> thought you could.
>
> Wido
>
> > Let me know what you think about it, it might be I don't understand
> > something and ACS already has it in place? I also would like to hear your
> > thoughts on ZSWAP usage in practice.
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ <http://bw-sw.com/>
>
--
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Wido den Hollander <wi...@widodh.nl>.
Hi,
> Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <ku...@bw-sw.com>:
>
>
> Hello, community.
>
> During the last years Linux kernel got some interesting features like KSM,
> ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> extremely fast SSD drives with m.2 and PCI-E interfaces.
>
> These facilities enable potentially interesting use of overcommited RAM for
> hosts. According to IBM's investigations Zswap with LZ4/ZBUD increases
> virtual RAM on 40%.
>
> I investigated current Apache CloudStack memory overcommitment capabilities
> and they mostly affect VM's RAM by utilizing ballooning and I think it's
> not what necessary to open new facilities. There are many cases which can
> utilize ZSWAP and fast swap devices to efficiently provision more RAM than
> presents.
>
> I suppose, CloudStack Agent for KVM can have configured parameter which
> "mangles" RAM reported. From the other hand it can be done by implemented
> host properties on server side. I tried manual increasing in host table:
>
> update host set ram=ram * 1.4 where id=1;
>
> and it seems until the next host stats update it works as expected. I think
> this workaround is useful, but it's better to have the function in core
> like standard.
>
Can't you set memory over provisioning on a cluster basis in the GUI? I thought you could.
Wido
> Let me know what you think about it, it might be I don't understand
> something and ACS already has it in place? I also would like to hear your
> thoughts on ZSWAP usage in practice.
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ <http://bw-sw.com/>
Re: KVM Host overprovisioning
Posted by Wido den Hollander <wi...@widodh.nl>.
Hi,
> Op 10 september 2017 om 8:37 schreef Ivan Kudryavtsev <ku...@bw-sw.com>:
>
>
> Hello, community.
>
> During the last years Linux kernel got some interesting features like KSM,
> ZSWAP, ZRAM. Hardware also steps forward and we see Intel 3d xpoint,
> extremely fast SSD drives with m.2 and PCI-E interfaces.
>
> These facilities enable potentially interesting use of overcommited RAM for
> hosts. According to IBM's investigations Zswap with LZ4/ZBUD increases
> virtual RAM on 40%.
>
> I investigated current Apache CloudStack memory overcommitment capabilities
> and they mostly affect VM's RAM by utilizing ballooning and I think it's
> not what necessary to open new facilities. There are many cases which can
> utilize ZSWAP and fast swap devices to efficiently provision more RAM than
> presents.
>
> I suppose, CloudStack Agent for KVM can have configured parameter which
> "mangles" RAM reported. From the other hand it can be done by implemented
> host properties on server side. I tried manual increasing in host table:
>
> update host set ram=ram * 1.4 where id=1;
>
> and it seems until the next host stats update it works as expected. I think
> this workaround is useful, but it's better to have the function in core
> like standard.
>
Can't you set memory over provisioning on a cluster basis in the GUI? I thought you could.
Wido
> Let me know what you think about it, it might be I don't understand
> something and ACS already has it in place? I also would like to hear your
> thoughts on ZSWAP usage in practice.
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ <http://bw-sw.com/>