You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tephra.apache.org by Thomas D'Silva <td...@salesforce.com> on 2017/01/05 00:24:52 UTC
testing transactional deletes with compaction
Hi,
I am trying to debug a Phoenix test failure that deletes rows from a
transactional table and then runs major compaction to remove the deleted
rows. The rows are not getting removed after major compaction is run. I
think it might be related to my test setup. I have attached a patch that
modifies TransactionAwareHTableTest to demonstrate my setup.
Can anyone let me know if there is anything wrong with my test setup?
Thank you,
Thomas
Re: testing transactional deletes with compaction
Posted by Gokul Gunasekaran <go...@gmail.com>.
Hi Thomas,
I have attached a diff where the test is passing. The only changes that I
added (in addition to decreasing the snapshot interval) were to invoke
compact/flush again (so that the new snapshot is being picked). I also
increased the sleep interval. After these changes, the test is passing.
Thanks,
Gokul
On Thu, Jan 5, 2017 at 12:25 PM, Thomas D'Silva <td...@salesforce.com>
wrote:
> Thanks Gokul!
>
> On Thu, Jan 5, 2017 at 12:17 PM, Gokul Gunasekaran <go...@apache.org>
> wrote:
>
> > Hi Thomas,
> >
> > Your setup looks right. The only thing it was missing was decreasing the
> > snapshot frequency of the Tx Manager which is by default set to
> 300seconds.
> >
> > I added this line after setting the tx snapshot dir:
> >
> > conf.setLong(TxConstants.Manager.CFG_TX_SNAPSHOT_INTERVAL, 2L);
> >
> > And the test passes now. The coprocessors use the snapshot data to clean
> up
> > data.
> >
> > Thanks,
> > Gokul
> >
> > On Wed, Jan 4, 2017 at 4:24 PM, Thomas D'Silva <td...@salesforce.com>
> > wrote:
> >
> > > Hi,
> > >
> > > I am trying to debug a Phoenix test failure that deletes rows from a
> > > transactional table and then runs major compaction to remove the
> deleted
> > > rows. The rows are not getting removed after major compaction is run. I
> > > think it might be related to my test setup. I have attached a patch
> that
> > > modifies TransactionAwareHTableTest to demonstrate my setup.
> > >
> > > Can anyone let me know if there is anything wrong with my test setup?
> > >
> > > Thank you,
> > > Thomas
> > >
> >
>
Re: testing transactional deletes with compaction
Posted by Thomas D'Silva <td...@salesforce.com>.
Thanks Gokul!
On Thu, Jan 5, 2017 at 12:17 PM, Gokul Gunasekaran <go...@apache.org> wrote:
> Hi Thomas,
>
> Your setup looks right. The only thing it was missing was decreasing the
> snapshot frequency of the Tx Manager which is by default set to 300seconds.
>
> I added this line after setting the tx snapshot dir:
>
> conf.setLong(TxConstants.Manager.CFG_TX_SNAPSHOT_INTERVAL, 2L);
>
> And the test passes now. The coprocessors use the snapshot data to clean up
> data.
>
> Thanks,
> Gokul
>
> On Wed, Jan 4, 2017 at 4:24 PM, Thomas D'Silva <td...@salesforce.com>
> wrote:
>
> > Hi,
> >
> > I am trying to debug a Phoenix test failure that deletes rows from a
> > transactional table and then runs major compaction to remove the deleted
> > rows. The rows are not getting removed after major compaction is run. I
> > think it might be related to my test setup. I have attached a patch that
> > modifies TransactionAwareHTableTest to demonstrate my setup.
> >
> > Can anyone let me know if there is anything wrong with my test setup?
> >
> > Thank you,
> > Thomas
> >
>
Re: testing transactional deletes with compaction
Posted by Gokul Gunasekaran <go...@apache.org>.
Hi Thomas,
I made some mistake in my previous testing. The test is still failing with
that change I had suggested. I am looking into it. I will update this
thread soon.
Thanks,
Gokul
On Thu, Jan 5, 2017 at 12:17 PM, Gokul Gunasekaran <go...@apache.org> wrote:
> Hi Thomas,
>
> Your setup looks right. The only thing it was missing was decreasing the
> snapshot frequency of the Tx Manager which is by default set to 300seconds.
>
> I added this line after setting the tx snapshot dir:
>
> conf.setLong(TxConstants.Manager.CFG_TX_SNAPSHOT_INTERVAL, 2L);
>
> And the test passes now. The coprocessors use the snapshot data to clean
> up data.
>
> Thanks,
> Gokul
>
> On Wed, Jan 4, 2017 at 4:24 PM, Thomas D'Silva <td...@salesforce.com>
> wrote:
>
>> Hi,
>>
>> I am trying to debug a Phoenix test failure that deletes rows from a
>> transactional table and then runs major compaction to remove the deleted
>> rows. The rows are not getting removed after major compaction is run. I
>> think it might be related to my test setup. I have attached a patch that
>> modifies TransactionAwareHTableTest to demonstrate my setup.
>>
>> Can anyone let me know if there is anything wrong with my test setup?
>>
>> Thank you,
>> Thomas
>>
>
>
Re: testing transactional deletes with compaction
Posted by Gokul Gunasekaran <go...@apache.org>.
Hi Thomas,
Your setup looks right. The only thing it was missing was decreasing the
snapshot frequency of the Tx Manager which is by default set to 300seconds.
I added this line after setting the tx snapshot dir:
conf.setLong(TxConstants.Manager.CFG_TX_SNAPSHOT_INTERVAL, 2L);
And the test passes now. The coprocessors use the snapshot data to clean up
data.
Thanks,
Gokul
On Wed, Jan 4, 2017 at 4:24 PM, Thomas D'Silva <td...@salesforce.com>
wrote:
> Hi,
>
> I am trying to debug a Phoenix test failure that deletes rows from a
> transactional table and then runs major compaction to remove the deleted
> rows. The rows are not getting removed after major compaction is run. I
> think it might be related to my test setup. I have attached a patch that
> modifies TransactionAwareHTableTest to demonstrate my setup.
>
> Can anyone let me know if there is anything wrong with my test setup?
>
> Thank you,
> Thomas
>