You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ofbiz.apache.org by Arun Patidar <ar...@hotwaxsystems.com> on 2017/07/12 09:08:57 UTC
Deadlock on InventoryItem during load test
Hello All,
I was trying to load test OFBiz with huge numbers of order creation and
fulfilment. During test, I found a deadlock on InventoryItem enttiy.
Current system, invoke 'UpdateInventoryItemFromDetail' service to update
InventoryItem QOH total and ATP total record. There is an Eca rule on
create/update of InventoryItemDetail entity record which triggers service
'UpdateInventoryItemFromDetail'. So, with heavy load, InventoryItem record
get lock and create deadlock condition.
I know that most of you are already aware with this issue. Please let me
know if someone worked on it and have any idea to avoid this type of
deadlock.
--
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development
HotWax Systems Pvt Ltd.www.hotwaxsystems.com
Re: Deadlock on InventoryItem during load test
Posted by Arun Patidar <ar...@hotwaxsystems.com>.
Hello Taher,
IMO, this is issue of inventory data model and way of calling inventory
management services.
--
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development
HotWax Systems Pvt Ltd.www.hotwaxsystems.com
On Sat, Aug 19, 2017 at 5:53 PM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:
> Hi Arun,
>
> Ahhh very interesting. So there was nothing fundamentally wrong at the
> entity-engine level, just a case of synchronizing entity updates
> right? If yes then the JIRA is not really concerned with anything core
> the framework I assume.
>
> On Thu, Aug 17, 2017 at 12:10 PM, Arun Patidar
> <ar...@hotwaxsystems.com> wrote:
> > Hello Taher, All,
> >
> > We are now able to proceed ahead with deadlock condition. Thanks Scott
> and
> > Jacopo for discussing and suggesting solution.
> >
> > We fixed it by acquiring all InventoryItem records that are going to
> affect
> > by inventory reservation and item issuance process of order. Acquiring of
> > record means that we performed delegator.storeByCondition() operation on
> > all selected inventoryItem records before actual reservation and
> issuance.
> >
> > This solution is still an experiment. I believe that some refactoring and
> > data model changes in inventory management process will require for
> getting
> > proper fix.
> >
> >
> >
> > --
> > Thanks & Regards
> > ---
> > Arun Patidar
> > Manager, Enterprise Software Development
> >
> >
> > HotWax Systems Pvt Ltd.www.hotwaxsystems.com
> >
> >
> > On Thu, Jul 27, 2017 at 10:04 AM, Arun Patidar <
> > arun.patidar@hotwaxsystems.com> wrote:
> >
> >> Hello Taher, All,
> >>
> >> I have added details on ticket regarding my test plan. Also, attached
> >> Jmeter script which creates and fulfil sales orders. I am still working
> on
> >> this and will share more updates.
> >>
> >>
> >>
> >> --
> >> Thanks & Regards
> >> ---
> >> Arun Patidar
> >> Manager, Enterprise Software Development
> >>
> >>
> >> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
> >>
> >>
> >> On Wed, Jul 12, 2017 at 6:23 PM, Arun Patidar
> <arun.patidar@hotwaxsystems.
> >> com> wrote:
> >>
> >>> Hello Taher,
> >>>
> >>> Created a JIRA ticket OFBIZ-9491
> >>> <https://issues.apache.org/jira/browse/OFBIZ-9491> and added some more
> >>> details.
> >>>
> >>> --
> >>> Thanks & Regards
> >>> ---
> >>> Arun Patidar
> >>> Manager, Enterprise Software Development
> >>>
> >>>
> >>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
> >>>
> >>>
> >>> On Wed, Jul 12, 2017 at 5:45 PM, Arun Patidar <
> >>> arun.patidar@hotwaxsystems.com> wrote:
> >>>
> >>>> Thanks Taher for showing interest. I will create a Jira issue with
> more
> >>>> details and share it with you.
> >>>>
> >>>> --
> >>>> Thanks & Regards
> >>>> ---
> >>>> Arun Patidar
> >>>> Manager, Enterprise Software Development
> >>>>
> >>>>
> >>>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
> >>>>
> >>>>
> >>>> On Wed, Jul 12, 2017 at 4:12 PM, Taher Alkhateeb <
> >>>> slidingfilaments@gmail.com> wrote:
> >>>>
> >>>>> If there is a JIRA with exact repeat steps then I would be interested
> >>>>> in working on this issue as it sounds like a good place to do some
> >>>>> proper refactoring on many moving parts.
> >>>>>
> >>>>> On Wed, Jul 12, 2017 at 12:08 PM, Arun Patidar
> >>>>> <ar...@hotwaxsystems.com> wrote:
> >>>>> > Hello All,
> >>>>> >
> >>>>> > I was trying to load test OFBiz with huge numbers of order creation
> >>>>> and
> >>>>> > fulfilment. During test, I found a deadlock on InventoryItem
> enttiy.
> >>>>> > Current system, invoke 'UpdateInventoryItemFromDetail' service to
> >>>>> update
> >>>>> > InventoryItem QOH total and ATP total record. There is an Eca rule
> on
> >>>>> > create/update of InventoryItemDetail entity record which triggers
> >>>>> service
> >>>>> > 'UpdateInventoryItemFromDetail'. So, with heavy load,
> InventoryItem
> >>>>> record
> >>>>> > get lock and create deadlock condition.
> >>>>> >
> >>>>> > I know that most of you are already aware with this issue. Please
> let
> >>>>> me
> >>>>> > know if someone worked on it and have any idea to avoid this type
> of
> >>>>> > deadlock.
> >>>>> >
> >>>>> > --
> >>>>> > Thanks & Regards
> >>>>> > ---
> >>>>> > Arun Patidar
> >>>>> > Manager, Enterprise Software Development
> >>>>> >
> >>>>> >
> >>>>> > HotWax Systems Pvt Ltd.www.hotwaxsystems.com
> >>>>>
> >>>>
> >>>>
> >>>
> >>
>
Re: Deadlock on InventoryItem during load test
Posted by Taher Alkhateeb <sl...@gmail.com>.
Hi Arun,
Ahhh very interesting. So there was nothing fundamentally wrong at the
entity-engine level, just a case of synchronizing entity updates
right? If yes then the JIRA is not really concerned with anything core
the framework I assume.
On Thu, Aug 17, 2017 at 12:10 PM, Arun Patidar
<ar...@hotwaxsystems.com> wrote:
> Hello Taher, All,
>
> We are now able to proceed ahead with deadlock condition. Thanks Scott and
> Jacopo for discussing and suggesting solution.
>
> We fixed it by acquiring all InventoryItem records that are going to affect
> by inventory reservation and item issuance process of order. Acquiring of
> record means that we performed delegator.storeByCondition() operation on
> all selected inventoryItem records before actual reservation and issuance.
>
> This solution is still an experiment. I believe that some refactoring and
> data model changes in inventory management process will require for getting
> proper fix.
>
>
>
> --
> Thanks & Regards
> ---
> Arun Patidar
> Manager, Enterprise Software Development
>
>
> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>
>
> On Thu, Jul 27, 2017 at 10:04 AM, Arun Patidar <
> arun.patidar@hotwaxsystems.com> wrote:
>
>> Hello Taher, All,
>>
>> I have added details on ticket regarding my test plan. Also, attached
>> Jmeter script which creates and fulfil sales orders. I am still working on
>> this and will share more updates.
>>
>>
>>
>> --
>> Thanks & Regards
>> ---
>> Arun Patidar
>> Manager, Enterprise Software Development
>>
>>
>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>
>>
>> On Wed, Jul 12, 2017 at 6:23 PM, Arun Patidar <arun.patidar@hotwaxsystems.
>> com> wrote:
>>
>>> Hello Taher,
>>>
>>> Created a JIRA ticket OFBIZ-9491
>>> <https://issues.apache.org/jira/browse/OFBIZ-9491> and added some more
>>> details.
>>>
>>> --
>>> Thanks & Regards
>>> ---
>>> Arun Patidar
>>> Manager, Enterprise Software Development
>>>
>>>
>>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>>
>>>
>>> On Wed, Jul 12, 2017 at 5:45 PM, Arun Patidar <
>>> arun.patidar@hotwaxsystems.com> wrote:
>>>
>>>> Thanks Taher for showing interest. I will create a Jira issue with more
>>>> details and share it with you.
>>>>
>>>> --
>>>> Thanks & Regards
>>>> ---
>>>> Arun Patidar
>>>> Manager, Enterprise Software Development
>>>>
>>>>
>>>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>>>
>>>>
>>>> On Wed, Jul 12, 2017 at 4:12 PM, Taher Alkhateeb <
>>>> slidingfilaments@gmail.com> wrote:
>>>>
>>>>> If there is a JIRA with exact repeat steps then I would be interested
>>>>> in working on this issue as it sounds like a good place to do some
>>>>> proper refactoring on many moving parts.
>>>>>
>>>>> On Wed, Jul 12, 2017 at 12:08 PM, Arun Patidar
>>>>> <ar...@hotwaxsystems.com> wrote:
>>>>> > Hello All,
>>>>> >
>>>>> > I was trying to load test OFBiz with huge numbers of order creation
>>>>> and
>>>>> > fulfilment. During test, I found a deadlock on InventoryItem enttiy.
>>>>> > Current system, invoke 'UpdateInventoryItemFromDetail' service to
>>>>> update
>>>>> > InventoryItem QOH total and ATP total record. There is an Eca rule on
>>>>> > create/update of InventoryItemDetail entity record which triggers
>>>>> service
>>>>> > 'UpdateInventoryItemFromDetail'. So, with heavy load, InventoryItem
>>>>> record
>>>>> > get lock and create deadlock condition.
>>>>> >
>>>>> > I know that most of you are already aware with this issue. Please let
>>>>> me
>>>>> > know if someone worked on it and have any idea to avoid this type of
>>>>> > deadlock.
>>>>> >
>>>>> > --
>>>>> > Thanks & Regards
>>>>> > ---
>>>>> > Arun Patidar
>>>>> > Manager, Enterprise Software Development
>>>>> >
>>>>> >
>>>>> > HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>>>>
>>>>
>>>>
>>>
>>
Re: Deadlock on InventoryItem during load test
Posted by Arun Patidar <ar...@hotwaxsystems.com>.
Hello Taher, All,
We are now able to proceed ahead with deadlock condition. Thanks Scott and
Jacopo for discussing and suggesting solution.
We fixed it by acquiring all InventoryItem records that are going to affect
by inventory reservation and item issuance process of order. Acquiring of
record means that we performed delegator.storeByCondition() operation on
all selected inventoryItem records before actual reservation and issuance.
This solution is still an experiment. I believe that some refactoring and
data model changes in inventory management process will require for getting
proper fix.
--
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development
HotWax Systems Pvt Ltd.www.hotwaxsystems.com
On Thu, Jul 27, 2017 at 10:04 AM, Arun Patidar <
arun.patidar@hotwaxsystems.com> wrote:
> Hello Taher, All,
>
> I have added details on ticket regarding my test plan. Also, attached
> Jmeter script which creates and fulfil sales orders. I am still working on
> this and will share more updates.
>
>
>
> --
> Thanks & Regards
> ---
> Arun Patidar
> Manager, Enterprise Software Development
>
>
> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>
>
> On Wed, Jul 12, 2017 at 6:23 PM, Arun Patidar <arun.patidar@hotwaxsystems.
> com> wrote:
>
>> Hello Taher,
>>
>> Created a JIRA ticket OFBIZ-9491
>> <https://issues.apache.org/jira/browse/OFBIZ-9491> and added some more
>> details.
>>
>> --
>> Thanks & Regards
>> ---
>> Arun Patidar
>> Manager, Enterprise Software Development
>>
>>
>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>
>>
>> On Wed, Jul 12, 2017 at 5:45 PM, Arun Patidar <
>> arun.patidar@hotwaxsystems.com> wrote:
>>
>>> Thanks Taher for showing interest. I will create a Jira issue with more
>>> details and share it with you.
>>>
>>> --
>>> Thanks & Regards
>>> ---
>>> Arun Patidar
>>> Manager, Enterprise Software Development
>>>
>>>
>>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>>
>>>
>>> On Wed, Jul 12, 2017 at 4:12 PM, Taher Alkhateeb <
>>> slidingfilaments@gmail.com> wrote:
>>>
>>>> If there is a JIRA with exact repeat steps then I would be interested
>>>> in working on this issue as it sounds like a good place to do some
>>>> proper refactoring on many moving parts.
>>>>
>>>> On Wed, Jul 12, 2017 at 12:08 PM, Arun Patidar
>>>> <ar...@hotwaxsystems.com> wrote:
>>>> > Hello All,
>>>> >
>>>> > I was trying to load test OFBiz with huge numbers of order creation
>>>> and
>>>> > fulfilment. During test, I found a deadlock on InventoryItem enttiy.
>>>> > Current system, invoke 'UpdateInventoryItemFromDetail' service to
>>>> update
>>>> > InventoryItem QOH total and ATP total record. There is an Eca rule on
>>>> > create/update of InventoryItemDetail entity record which triggers
>>>> service
>>>> > 'UpdateInventoryItemFromDetail'. So, with heavy load, InventoryItem
>>>> record
>>>> > get lock and create deadlock condition.
>>>> >
>>>> > I know that most of you are already aware with this issue. Please let
>>>> me
>>>> > know if someone worked on it and have any idea to avoid this type of
>>>> > deadlock.
>>>> >
>>>> > --
>>>> > Thanks & Regards
>>>> > ---
>>>> > Arun Patidar
>>>> > Manager, Enterprise Software Development
>>>> >
>>>> >
>>>> > HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>>>
>>>
>>>
>>
>
Re: Deadlock on InventoryItem during load test
Posted by Arun Patidar <ar...@hotwaxsystems.com>.
Hello Taher, All,
I have added details on ticket regarding my test plan. Also, attached
Jmeter script which creates and fulfil sales orders. I am still working on
this and will share more updates.
--
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development
HotWax Systems Pvt Ltd.www.hotwaxsystems.com
On Wed, Jul 12, 2017 at 6:23 PM, Arun Patidar <
arun.patidar@hotwaxsystems.com> wrote:
> Hello Taher,
>
> Created a JIRA ticket OFBIZ-9491
> <https://issues.apache.org/jira/browse/OFBIZ-9491> and added some more
> details.
>
> --
> Thanks & Regards
> ---
> Arun Patidar
> Manager, Enterprise Software Development
>
>
> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>
>
> On Wed, Jul 12, 2017 at 5:45 PM, Arun Patidar <arun.patidar@hotwaxsystems.
> com> wrote:
>
>> Thanks Taher for showing interest. I will create a Jira issue with more
>> details and share it with you.
>>
>> --
>> Thanks & Regards
>> ---
>> Arun Patidar
>> Manager, Enterprise Software Development
>>
>>
>> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>
>>
>> On Wed, Jul 12, 2017 at 4:12 PM, Taher Alkhateeb <
>> slidingfilaments@gmail.com> wrote:
>>
>>> If there is a JIRA with exact repeat steps then I would be interested
>>> in working on this issue as it sounds like a good place to do some
>>> proper refactoring on many moving parts.
>>>
>>> On Wed, Jul 12, 2017 at 12:08 PM, Arun Patidar
>>> <ar...@hotwaxsystems.com> wrote:
>>> > Hello All,
>>> >
>>> > I was trying to load test OFBiz with huge numbers of order creation and
>>> > fulfilment. During test, I found a deadlock on InventoryItem enttiy.
>>> > Current system, invoke 'UpdateInventoryItemFromDetail' service to
>>> update
>>> > InventoryItem QOH total and ATP total record. There is an Eca rule on
>>> > create/update of InventoryItemDetail entity record which triggers
>>> service
>>> > 'UpdateInventoryItemFromDetail'. So, with heavy load, InventoryItem
>>> record
>>> > get lock and create deadlock condition.
>>> >
>>> > I know that most of you are already aware with this issue. Please let
>>> me
>>> > know if someone worked on it and have any idea to avoid this type of
>>> > deadlock.
>>> >
>>> > --
>>> > Thanks & Regards
>>> > ---
>>> > Arun Patidar
>>> > Manager, Enterprise Software Development
>>> >
>>> >
>>> > HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>>
>>
>>
>
Re: Deadlock on InventoryItem during load test
Posted by Arun Patidar <ar...@hotwaxsystems.com>.
Hello Taher,
Created a JIRA ticket OFBIZ-9491
<https://issues.apache.org/jira/browse/OFBIZ-9491> and added some more
details.
--
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development
HotWax Systems Pvt Ltd.www.hotwaxsystems.com
On Wed, Jul 12, 2017 at 5:45 PM, Arun Patidar <
arun.patidar@hotwaxsystems.com> wrote:
> Thanks Taher for showing interest. I will create a Jira issue with more
> details and share it with you.
>
> --
> Thanks & Regards
> ---
> Arun Patidar
> Manager, Enterprise Software Development
>
>
> HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>
>
> On Wed, Jul 12, 2017 at 4:12 PM, Taher Alkhateeb <
> slidingfilaments@gmail.com> wrote:
>
>> If there is a JIRA with exact repeat steps then I would be interested
>> in working on this issue as it sounds like a good place to do some
>> proper refactoring on many moving parts.
>>
>> On Wed, Jul 12, 2017 at 12:08 PM, Arun Patidar
>> <ar...@hotwaxsystems.com> wrote:
>> > Hello All,
>> >
>> > I was trying to load test OFBiz with huge numbers of order creation and
>> > fulfilment. During test, I found a deadlock on InventoryItem enttiy.
>> > Current system, invoke 'UpdateInventoryItemFromDetail' service to
>> update
>> > InventoryItem QOH total and ATP total record. There is an Eca rule on
>> > create/update of InventoryItemDetail entity record which triggers
>> service
>> > 'UpdateInventoryItemFromDetail'. So, with heavy load, InventoryItem
>> record
>> > get lock and create deadlock condition.
>> >
>> > I know that most of you are already aware with this issue. Please let me
>> > know if someone worked on it and have any idea to avoid this type of
>> > deadlock.
>> >
>> > --
>> > Thanks & Regards
>> > ---
>> > Arun Patidar
>> > Manager, Enterprise Software Development
>> >
>> >
>> > HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>>
>
>
Re: Deadlock on InventoryItem during load test
Posted by Arun Patidar <ar...@hotwaxsystems.com>.
Thanks Taher for showing interest. I will create a Jira issue with more
details and share it with you.
--
Thanks & Regards
---
Arun Patidar
Manager, Enterprise Software Development
HotWax Systems Pvt Ltd.www.hotwaxsystems.com
On Wed, Jul 12, 2017 at 4:12 PM, Taher Alkhateeb <slidingfilaments@gmail.com
> wrote:
> If there is a JIRA with exact repeat steps then I would be interested
> in working on this issue as it sounds like a good place to do some
> proper refactoring on many moving parts.
>
> On Wed, Jul 12, 2017 at 12:08 PM, Arun Patidar
> <ar...@hotwaxsystems.com> wrote:
> > Hello All,
> >
> > I was trying to load test OFBiz with huge numbers of order creation and
> > fulfilment. During test, I found a deadlock on InventoryItem enttiy.
> > Current system, invoke 'UpdateInventoryItemFromDetail' service to update
> > InventoryItem QOH total and ATP total record. There is an Eca rule on
> > create/update of InventoryItemDetail entity record which triggers service
> > 'UpdateInventoryItemFromDetail'. So, with heavy load, InventoryItem
> record
> > get lock and create deadlock condition.
> >
> > I know that most of you are already aware with this issue. Please let me
> > know if someone worked on it and have any idea to avoid this type of
> > deadlock.
> >
> > --
> > Thanks & Regards
> > ---
> > Arun Patidar
> > Manager, Enterprise Software Development
> >
> >
> > HotWax Systems Pvt Ltd.www.hotwaxsystems.com
>
Re: Deadlock on InventoryItem during load test
Posted by Taher Alkhateeb <sl...@gmail.com>.
If there is a JIRA with exact repeat steps then I would be interested
in working on this issue as it sounds like a good place to do some
proper refactoring on many moving parts.
On Wed, Jul 12, 2017 at 12:08 PM, Arun Patidar
<ar...@hotwaxsystems.com> wrote:
> Hello All,
>
> I was trying to load test OFBiz with huge numbers of order creation and
> fulfilment. During test, I found a deadlock on InventoryItem enttiy.
> Current system, invoke 'UpdateInventoryItemFromDetail' service to update
> InventoryItem QOH total and ATP total record. There is an Eca rule on
> create/update of InventoryItemDetail entity record which triggers service
> 'UpdateInventoryItemFromDetail'. So, with heavy load, InventoryItem record
> get lock and create deadlock condition.
>
> I know that most of you are already aware with this issue. Please let me
> know if someone worked on it and have any idea to avoid this type of
> deadlock.
>
> --
> Thanks & Regards
> ---
> Arun Patidar
> Manager, Enterprise Software Development
>
>
> HotWax Systems Pvt Ltd.www.hotwaxsystems.com