You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ofbiz.apache.org by Len Shein <ls...@solveda.com> on 2015/11/24 16:48:15 UTC
VISIT and VISITOR Entities
All,
We have a LIVE application in which the VISIT AND VISITOR entities have
grown to over 19 and 18 million rows respectively.
Has anyone experienced any performance issues when these entities have grown
to such capacity?
We recently had a performance issue and decided to turn off the visit
history feature until this feature can be ruled out as the issue.
stats.persist.visit=false
stats.persist.visitor=false
Note: The application is currently using Ofbiz r.10.
Len Shein
<ma...@solveda.com> lshein@solveda.com
O: 516.742.7888 x225
O: 732.333.4303
C: 917.882.8515
RE: VISIT and VISITOR Entities
Posted by Nick Rosser <nr...@solveda.com>.
Thanks Adrian -- we'll take a look at the latest version.
Maybe a complete red herring -- but a couple of weeks ago our client requested that we change the password for the primary "admin" user. We deleted and re-added main scheduled jobs but we've had problems since ...
Nick
-----Original Message-----
From: Adrian Crum [mailto:adrian.crum@sandglass-software.com]
Sent: Tuesday, November 24, 2015 11:58 AM
To: user@ofbiz.apache.org
Subject: Re: VISIT and VISITOR Entities
Hmmm... it could be the job scheduler. It was refactored a few years ago to fix problems like that.
Try C&Ping R13 job scheduler into R10
(framework\service\src\org\ofbiz\service\job).
Adrian Crum
Sandglass Software
www.sandglass-software.com
On 11/24/2015 8:29 AM, Nick Rosser wrote:
> Adrian,
>
> Not entirely sure if it made a difference -- site was going down every now and again, initially we thought that some scheduled jobs were the culprit (since outages were in or around the scheduled job time). However, we had problems yesterday when no scheduled jobs were running (we disabled most of them for now).
>
> Site has been fine since we turned off VISIT and VISITOR tracking -- can't be 100% sure that this is the resolution factor given general problems over the last couple of weeks. I guess that if the site is fine for several days then we would have to attribute it to the serverstats change.
>
> We've heard of problems when the JOB_SANDBOX gets very large -- ever heard about instability when VISIT and VISITORS get very large? (19M and 18M rows each).
>
> Nick
>
> -----Original Message-----
> From: Adrian Crum [mailto:adrian.crum@sandglass-software.com]
> Sent: Tuesday, November 24, 2015 11:13 AM
> To: user@ofbiz.apache.org
> Subject: Re: VISIT and VISITOR Entities
>
> Did the setting change improve performance?
>
> You could write a service that deletes data older than xxx days, then make it a scheduled job.
>
> Adrian Crum
> Sandglass Software
> www.sandglass-software.com
>
> On 11/24/2015 7:48 AM, Len Shein wrote:
>> All,
>>
>> We have a LIVE application in which the VISIT AND VISITOR entities
>> have grown to over 19 and 18 million rows respectively.
>>
>> Has anyone experienced any performance issues when these entities
>> have grown to such capacity?
>>
>>
>>
>> We recently had a performance issue and decided to turn off the visit
>> history feature until this feature can be ruled out as the issue.
>>
>> stats.persist.visit=false
>>
>> stats.persist.visitor=false
>>
>>
>>
>> Note: The application is currently using Ofbiz r.10.
>>
>>
>> Len Shein
>>
>> <ma...@solveda.com> lshein@solveda.com
>>
>> O: 516.742.7888 x225
>>
>> O: 732.333.4303
>> C: 917.882.8515
>>
>>
Re: VISIT and VISITOR Entities
Posted by Adrian Crum <ad...@sandglass-software.com>.
Hmmm... it could be the job scheduler. It was refactored a few years ago
to fix problems like that.
Try C&Ping R13 job scheduler into R10
(framework\service\src\org\ofbiz\service\job).
Adrian Crum
Sandglass Software
www.sandglass-software.com
On 11/24/2015 8:29 AM, Nick Rosser wrote:
> Adrian,
>
> Not entirely sure if it made a difference -- site was going down every now and again, initially we thought that some scheduled jobs were the culprit (since outages were in or around the scheduled job time). However, we had problems yesterday when no scheduled jobs were running (we disabled most of them for now).
>
> Site has been fine since we turned off VISIT and VISITOR tracking -- can't be 100% sure that this is the resolution factor given general problems over the last couple of weeks. I guess that if the site is fine for several days then we would have to attribute it to the serverstats change.
>
> We've heard of problems when the JOB_SANDBOX gets very large -- ever heard about instability when VISIT and VISITORS get very large? (19M and 18M rows each).
>
> Nick
>
> -----Original Message-----
> From: Adrian Crum [mailto:adrian.crum@sandglass-software.com]
> Sent: Tuesday, November 24, 2015 11:13 AM
> To: user@ofbiz.apache.org
> Subject: Re: VISIT and VISITOR Entities
>
> Did the setting change improve performance?
>
> You could write a service that deletes data older than xxx days, then make it a scheduled job.
>
> Adrian Crum
> Sandglass Software
> www.sandglass-software.com
>
> On 11/24/2015 7:48 AM, Len Shein wrote:
>> All,
>>
>> We have a LIVE application in which the VISIT AND VISITOR entities
>> have grown to over 19 and 18 million rows respectively.
>>
>> Has anyone experienced any performance issues when these entities have
>> grown to such capacity?
>>
>>
>>
>> We recently had a performance issue and decided to turn off the visit
>> history feature until this feature can be ruled out as the issue.
>>
>> stats.persist.visit=false
>>
>> stats.persist.visitor=false
>>
>>
>>
>> Note: The application is currently using Ofbiz r.10.
>>
>>
>> Len Shein
>>
>> <ma...@solveda.com> lshein@solveda.com
>>
>> O: 516.742.7888 x225
>>
>> O: 732.333.4303
>> C: 917.882.8515
>>
>>
RE: VISIT and VISITOR Entities
Posted by Nick Rosser <nr...@solveda.com>.
Adrian,
Not entirely sure if it made a difference -- site was going down every now and again, initially we thought that some scheduled jobs were the culprit (since outages were in or around the scheduled job time). However, we had problems yesterday when no scheduled jobs were running (we disabled most of them for now).
Site has been fine since we turned off VISIT and VISITOR tracking -- can't be 100% sure that this is the resolution factor given general problems over the last couple of weeks. I guess that if the site is fine for several days then we would have to attribute it to the serverstats change.
We've heard of problems when the JOB_SANDBOX gets very large -- ever heard about instability when VISIT and VISITORS get very large? (19M and 18M rows each).
Nick
-----Original Message-----
From: Adrian Crum [mailto:adrian.crum@sandglass-software.com]
Sent: Tuesday, November 24, 2015 11:13 AM
To: user@ofbiz.apache.org
Subject: Re: VISIT and VISITOR Entities
Did the setting change improve performance?
You could write a service that deletes data older than xxx days, then make it a scheduled job.
Adrian Crum
Sandglass Software
www.sandglass-software.com
On 11/24/2015 7:48 AM, Len Shein wrote:
> All,
>
> We have a LIVE application in which the VISIT AND VISITOR entities
> have grown to over 19 and 18 million rows respectively.
>
> Has anyone experienced any performance issues when these entities have
> grown to such capacity?
>
>
>
> We recently had a performance issue and decided to turn off the visit
> history feature until this feature can be ruled out as the issue.
>
> stats.persist.visit=false
>
> stats.persist.visitor=false
>
>
>
> Note: The application is currently using Ofbiz r.10.
>
>
> Len Shein
>
> <ma...@solveda.com> lshein@solveda.com
>
> O: 516.742.7888 x225
>
> O: 732.333.4303
> C: 917.882.8515
>
>
Re: VISIT and VISITOR Entities
Posted by Adrian Crum <ad...@sandglass-software.com>.
Did the setting change improve performance?
You could write a service that deletes data older than xxx days, then
make it a scheduled job.
Adrian Crum
Sandglass Software
www.sandglass-software.com
On 11/24/2015 7:48 AM, Len Shein wrote:
> All,
>
> We have a LIVE application in which the VISIT AND VISITOR entities have
> grown to over 19 and 18 million rows respectively.
>
> Has anyone experienced any performance issues when these entities have grown
> to such capacity?
>
>
>
> We recently had a performance issue and decided to turn off the visit
> history feature until this feature can be ruled out as the issue.
>
> stats.persist.visit=false
>
> stats.persist.visitor=false
>
>
>
> Note: The application is currently using Ofbiz r.10.
>
>
> Len Shein
>
> <ma...@solveda.com> lshein@solveda.com
>
> O: 516.742.7888 x225
>
> O: 732.333.4303
> C: 917.882.8515
>
>
Re: VISIT and VISITOR Entities
Posted by Mike <mz...@gmail.com>.
For mysql, there is tuning_primer.sh:
https://launchpad.net/mysql-tuning-primer
For postgresql, there is pgtune: http://pgfoundry.org/projects/pgtune
Most folks just use yum/apt-get to get pgtune:
apt-get install pgtune --or--
yum install pgtune
Although neither are perfect, both utilities attempt to change the DB
settings to take advantage of hardware, or look at certain things, like the
number of tables, and adjust the DB settings for maximum performance.
The default settings for mysql are quite limiting for ofbiz. For instance,
ofbiz has over 800 tables and the default table cache for mysql is like 60
tables.
On Tue, Nov 24, 2015 at 9:22 AM, Mike <mz...@gmail.com> wrote:
> > at least not OFBiz cache since
>
> I didn't mean the ofbiz cache. I meant the database (mysql/postgres)
> cache settings.
>
> On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <nr...@solveda.com> wrote:
>
>> Thanks Mike -- don't think this is a cache issue -- at least not OFBiz
>> cache since it is not showing up in the OFBiz Cache maintenance page.
>> Nick
>>
>> -----Original Message-----
>> From: Mike [mailto:mz4wheeler@gmail.com]
>> Sent: Tuesday, November 24, 2015 11:31 AM
>> To: user <us...@ofbiz.apache.org>
>> Subject: Re: VISIT and VISITOR Entities
>>
>> Most likely this table is stealing cache memory from the rest of the
>> tables. Either increase cache memory, delete the rows, write to a separate
>> database, or keep the table trim to xxx rows (as Adrian said) via an
>> internal ofbiz job or external script (via crontab).
>>
>> On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <ls...@solveda.com> wrote:
>>
>> > All,
>> >
>> > We have a LIVE application in which the VISIT AND VISITOR entities
>> > have grown to over 19 and 18 million rows respectively.
>> >
>> > Has anyone experienced any performance issues when these entities have
>> > grown to such capacity?
>> >
>> >
>> >
>> > We recently had a performance issue and decided to turn off the visit
>> > history feature until this feature can be ruled out as the issue.
>> >
>> > stats.persist.visit=false
>> >
>> > stats.persist.visitor=false
>> >
>> >
>> >
>> > Note: The application is currently using Ofbiz r.10.
>> >
>> >
>> > Len Shein
>> >
>> > <ma...@solveda.com> lshein@solveda.com
>> >
>> > O: 516.742.7888 x225
>> >
>> > O: 732.333.4303
>> > C: 917.882.8515
>> >
>> >
>>
>
>
Re: VISIT and VISITOR Entities
Posted by Jacques Le Roux <ja...@les7arts.com>.
That's why using another DB, as suggested Mike ("write to a separate database"), could be a good test...
Jacques
Le 24/11/2015 18:23, Nick Rosser a écrit :
> Got it, thanks Mike.
> Nick
>
> -----Original Message-----
> From: Mike [mailto:mz4wheeler@gmail.com]
> Sent: Tuesday, November 24, 2015 12:23 PM
> To: user <us...@ofbiz.apache.org>
> Subject: Re: VISIT and VISITOR Entities
>
>> at least not OFBiz cache since
> I didn't mean the ofbiz cache. I meant the database (mysql/postgres) cache settings.
>
> On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <nr...@solveda.com> wrote:
>
>> Thanks Mike -- don't think this is a cache issue -- at least not OFBiz
>> cache since it is not showing up in the OFBiz Cache maintenance page.
>> Nick
>>
>> -----Original Message-----
>> From: Mike [mailto:mz4wheeler@gmail.com]
>> Sent: Tuesday, November 24, 2015 11:31 AM
>> To: user <us...@ofbiz.apache.org>
>> Subject: Re: VISIT and VISITOR Entities
>>
>> Most likely this table is stealing cache memory from the rest of the
>> tables. Either increase cache memory, delete the rows, write to a
>> separate database, or keep the table trim to xxx rows (as Adrian said)
>> via an internal ofbiz job or external script (via crontab).
>>
>> On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <ls...@solveda.com> wrote:
>>
>>> All,
>>>
>>> We have a LIVE application in which the VISIT AND VISITOR entities
>>> have grown to over 19 and 18 million rows respectively.
>>>
>>> Has anyone experienced any performance issues when these entities
>>> have grown to such capacity?
>>>
>>>
>>>
>>> We recently had a performance issue and decided to turn off the
>>> visit history feature until this feature can be ruled out as the issue.
>>>
>>> stats.persist.visit=false
>>>
>>> stats.persist.visitor=false
>>>
>>>
>>>
>>> Note: The application is currently using Ofbiz r.10.
>>>
>>>
>>> Len Shein
>>>
>>> <ma...@solveda.com> lshein@solveda.com
>>>
>>> O: 516.742.7888 x225
>>>
>>> O: 732.333.4303
>>> C: 917.882.8515
>>>
>>>
>
RE: VISIT and VISITOR Entities
Posted by Nick Rosser <nr...@solveda.com>.
Got it, thanks Mike.
Nick
-----Original Message-----
From: Mike [mailto:mz4wheeler@gmail.com]
Sent: Tuesday, November 24, 2015 12:23 PM
To: user <us...@ofbiz.apache.org>
Subject: Re: VISIT and VISITOR Entities
> at least not OFBiz cache since
I didn't mean the ofbiz cache. I meant the database (mysql/postgres) cache settings.
On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <nr...@solveda.com> wrote:
> Thanks Mike -- don't think this is a cache issue -- at least not OFBiz
> cache since it is not showing up in the OFBiz Cache maintenance page.
> Nick
>
> -----Original Message-----
> From: Mike [mailto:mz4wheeler@gmail.com]
> Sent: Tuesday, November 24, 2015 11:31 AM
> To: user <us...@ofbiz.apache.org>
> Subject: Re: VISIT and VISITOR Entities
>
> Most likely this table is stealing cache memory from the rest of the
> tables. Either increase cache memory, delete the rows, write to a
> separate database, or keep the table trim to xxx rows (as Adrian said)
> via an internal ofbiz job or external script (via crontab).
>
> On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <ls...@solveda.com> wrote:
>
> > All,
> >
> > We have a LIVE application in which the VISIT AND VISITOR entities
> > have grown to over 19 and 18 million rows respectively.
> >
> > Has anyone experienced any performance issues when these entities
> > have grown to such capacity?
> >
> >
> >
> > We recently had a performance issue and decided to turn off the
> > visit history feature until this feature can be ruled out as the issue.
> >
> > stats.persist.visit=false
> >
> > stats.persist.visitor=false
> >
> >
> >
> > Note: The application is currently using Ofbiz r.10.
> >
> >
> > Len Shein
> >
> > <ma...@solveda.com> lshein@solveda.com
> >
> > O: 516.742.7888 x225
> >
> > O: 732.333.4303
> > C: 917.882.8515
> >
> >
>
Re: VISIT and VISITOR Entities
Posted by Mike <mz...@gmail.com>.
> at least not OFBiz cache since
I didn't mean the ofbiz cache. I meant the database (mysql/postgres) cache
settings.
On Tue, Nov 24, 2015 at 8:34 AM, Nick Rosser <nr...@solveda.com> wrote:
> Thanks Mike -- don't think this is a cache issue -- at least not OFBiz
> cache since it is not showing up in the OFBiz Cache maintenance page.
> Nick
>
> -----Original Message-----
> From: Mike [mailto:mz4wheeler@gmail.com]
> Sent: Tuesday, November 24, 2015 11:31 AM
> To: user <us...@ofbiz.apache.org>
> Subject: Re: VISIT and VISITOR Entities
>
> Most likely this table is stealing cache memory from the rest of the
> tables. Either increase cache memory, delete the rows, write to a separate
> database, or keep the table trim to xxx rows (as Adrian said) via an
> internal ofbiz job or external script (via crontab).
>
> On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <ls...@solveda.com> wrote:
>
> > All,
> >
> > We have a LIVE application in which the VISIT AND VISITOR entities
> > have grown to over 19 and 18 million rows respectively.
> >
> > Has anyone experienced any performance issues when these entities have
> > grown to such capacity?
> >
> >
> >
> > We recently had a performance issue and decided to turn off the visit
> > history feature until this feature can be ruled out as the issue.
> >
> > stats.persist.visit=false
> >
> > stats.persist.visitor=false
> >
> >
> >
> > Note: The application is currently using Ofbiz r.10.
> >
> >
> > Len Shein
> >
> > <ma...@solveda.com> lshein@solveda.com
> >
> > O: 516.742.7888 x225
> >
> > O: 732.333.4303
> > C: 917.882.8515
> >
> >
>
RE: VISIT and VISITOR Entities
Posted by Nick Rosser <nr...@solveda.com>.
Thanks Mike -- don't think this is a cache issue -- at least not OFBiz cache since it is not showing up in the OFBiz Cache maintenance page.
Nick
-----Original Message-----
From: Mike [mailto:mz4wheeler@gmail.com]
Sent: Tuesday, November 24, 2015 11:31 AM
To: user <us...@ofbiz.apache.org>
Subject: Re: VISIT and VISITOR Entities
Most likely this table is stealing cache memory from the rest of the tables. Either increase cache memory, delete the rows, write to a separate database, or keep the table trim to xxx rows (as Adrian said) via an internal ofbiz job or external script (via crontab).
On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <ls...@solveda.com> wrote:
> All,
>
> We have a LIVE application in which the VISIT AND VISITOR entities
> have grown to over 19 and 18 million rows respectively.
>
> Has anyone experienced any performance issues when these entities have
> grown to such capacity?
>
>
>
> We recently had a performance issue and decided to turn off the visit
> history feature until this feature can be ruled out as the issue.
>
> stats.persist.visit=false
>
> stats.persist.visitor=false
>
>
>
> Note: The application is currently using Ofbiz r.10.
>
>
> Len Shein
>
> <ma...@solveda.com> lshein@solveda.com
>
> O: 516.742.7888 x225
>
> O: 732.333.4303
> C: 917.882.8515
>
>
Re: VISIT and VISITOR Entities
Posted by Mike <mz...@gmail.com>.
Most likely this table is stealing cache memory from the rest of the
tables. Either increase cache memory, delete the rows, write to a separate
database, or keep the table trim to xxx rows (as Adrian said) via an
internal ofbiz job or external script (via crontab).
On Tue, Nov 24, 2015 at 7:48 AM, Len Shein <ls...@solveda.com> wrote:
> All,
>
> We have a LIVE application in which the VISIT AND VISITOR entities have
> grown to over 19 and 18 million rows respectively.
>
> Has anyone experienced any performance issues when these entities have
> grown
> to such capacity?
>
>
>
> We recently had a performance issue and decided to turn off the visit
> history feature until this feature can be ruled out as the issue.
>
> stats.persist.visit=false
>
> stats.persist.visitor=false
>
>
>
> Note: The application is currently using Ofbiz r.10.
>
>
> Len Shein
>
> <ma...@solveda.com> lshein@solveda.com
>
> O: 516.742.7888 x225
>
> O: 732.333.4303
> C: 917.882.8515
>
>