You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "孟庆义(孟庆义)" <qi...@alibaba-inc.com> on 2014/08/11 11:01:53 UTC
inconsistent sequence number between server and client metadata cache
Hi, all :
I use phoenix 3.0 and here is what I found:
Create a table T, then metadata cache has T
Create an index I on T, then metadata cache has I, but will trigger to
update T by following code in PMetadataImpl.addTable
{{{
….
if (parentTable != null) {
List<PTable> oldIndexes = parentTable.getIndexes();
List<PTable> newIndexes =
Lists.newArrayListWithExpectedSize(oldIndexes.size() + 1);
newIndexes.addAll(oldIndexes);
if (oldTable != null) {
newIndexes.remove(oldTable);
}
newIndexes.add(table);
parentTable = PTableImpl.makePTable(parentTable,
table.getTimeStamp(), newIndexes);
tables.put(parentTable.getKey(), parentTable);
}
….
}}}
The method makePTable will increase T’s sequence number by 1.
{{{
public static PTableImpl makePTable(PTable table, long timeStamp,
List<PTable> indexes) throws SQLException {
return new PTableImpl(
table.getTenantId(), table.getSchemaName(),
table.getTableName(), table.getType(), table.getIndexState(), timeStamp,
table.getSequenceNumber() + 1, table.getPKName(),
table.getBucketNum(), getColumnsToClone(table), table.getParentTableName(),
indexes,
table.isImmutableRows(), table.getPhysicalNames(),
table.getDefaultFamilyName(), table.getViewStatement(),
table.isWALDisabled(), table.isMultiTenant(), table.getViewType(),
table.getViewIndexId());
}}}
But during my debug the server side sequence number seem unchanged, and then
I made a “alter table T drop column”, there is a
ConcurrentMutationException thrown due to inconsistent sequence number.
I can see that the code catch the Exception and update table T with sever
side metadata, and then a retry complement the drop column operation.
My question is why we increase the sequence number on client side in the
beginning? Which side is wrong, server or client?
daniel（孟庆义）
答复: inconsistent sequence number between server and client metadata cache
Posted by "孟庆义(孟庆义)" <qi...@alibaba-inc.com>.
Hi, Taylor
Thanks for your answer, but in my case it seems the server side didn't increase sequence number. And normally there will be no error because DML query doesn't use cache as they will update cache first. I found this because
I changed the code to let query use metadata cache, so I think the unit test will be hard to make. I will try to see why the server fail to increase sequence number.
-----邮件原件-----
发件人: James Taylor [mailto:jamestaylor@apache.org]
发送时间: 2014年8月12日 0:39
收件人: dev@phoenix.apache.org; 孟庆义(孟庆义)
主题: Re: inconsistent sequence number between server and client metadata cache
Hi Daniel,
The intent is that the table's sequence number increases when a structural changes is made to the table (including adding an index to it). So in this case, here's what should happen, assuming the sequence number of the tables starts at 1):
- add index I to T. The sequence number of T should increase by one on the server side. The client that makes this request updates its client-side cache for T to include the index and the increased sequence number.
- if the *same* client issues a query on T, it will have the same sequence number as the server, so no ConcurrentMutationException will occur.
- if a *different* client issues a query on T, it will have the old sequence number (assuming its in the cache), so a ConcurrentMutationException will occur. This exception is caught, the client-side cache is updated, and the second time around the query should succeed. This should be completely transparent to the client.
Are you seeing something different? Would it be possible for you to put together a small unit test that repros the unexpected behavior?
Thanks,
James
On Mon, Aug 11, 2014 at 2:01 AM, 孟庆义(孟庆义) <qi...@alibaba-inc.com> wrote:
> Hi, all :
>
>
>
> I use phoenix 3.0 and here is what I found:
>
> Create a table T, then metadata cache has T
>
> Create an index I on T, then metadata cache has I, but will trigger to
> update T by following code in PMetadataImpl.addTable
>
> {{{
>
> ….
>
> if (parentTable != null) {
>
> List<PTable> oldIndexes = parentTable.getIndexes();
>
> List<PTable> newIndexes =
> Lists.newArrayListWithExpectedSize(oldIndexes.size() + 1);
>
> newIndexes.addAll(oldIndexes);
>
> if (oldTable != null) {
>
> newIndexes.remove(oldTable);
>
> }
>
> newIndexes.add(table);
>
> parentTable = PTableImpl.makePTable(parentTable,
> table.getTimeStamp(), newIndexes);
>
> tables.put(parentTable.getKey(), parentTable);
>
> }
>
> ….
>
> }}}
>
>
>
> The method makePTable will increase T’s sequence number by 1.
>
>
>
> {{{
>
> public static PTableImpl makePTable(PTable table, long timeStamp,
> List<PTable> indexes) throws SQLException {
>
> return new PTableImpl(
>
> table.getTenantId(), table.getSchemaName(),
> table.getTableName(), table.getType(), table.getIndexState(),
> timeStamp,
>
> table.getSequenceNumber() + 1, table.getPKName(),
> table.getBucketNum(), getColumnsToClone(table),
> table.getParentTableName(), indexes,
>
> table.isImmutableRows(), table.getPhysicalNames(),
> table.getDefaultFamilyName(), table.getViewStatement(),
> table.isWALDisabled(), table.isMultiTenant(), table.getViewType(),
> table.getViewIndexId());
>
> }}}
>
>
>
> But during my debug the server side sequence number seem unchanged,
> and then I made a “alter table T drop column”, there is a
> ConcurrentMutationException thrown due to inconsistent sequence number.
>
> I can see that the code catch the Exception and update table T with
> sever side metadata, and then a retry complement the drop column operation.
>
> My question is why we increase the sequence number on client side in
> the beginning? Which side is wrong, server or client?
>
>
>
> daniel（孟庆义）
>
>
>
Re: inconsistent sequence number between server and client metadata cache
Posted by James Taylor <ja...@apache.org>.
Hi Daniel,
The intent is that the table's sequence number increases when a
structural changes is made to the table (including adding an index to
it). So in this case, here's what should happen, assuming the sequence
number of the tables starts at 1):
- add index I to T. The sequence number of T should increase by one on
the server side. The client that makes this request updates its
client-side cache for T to include the index and the increased
sequence number.
- if the *same* client issues a query on T, it will have the same
sequence number as the server, so no ConcurrentMutationException will
occur.
- if a *different* client issues a query on T, it will have the old
sequence number (assuming its in the cache), so a
ConcurrentMutationException will occur. This exception is caught, the
client-side cache is updated, and the second time around the query
should succeed. This should be completely transparent to the client.
Are you seeing something different? Would it be possible for you to
put together a small unit test that repros the unexpected behavior?
Thanks,
James
On Mon, Aug 11, 2014 at 2:01 AM, 孟庆义(孟庆义) <qi...@alibaba-inc.com> wrote:
> Hi, all :
>
>
>
> I use phoenix 3.0 and here is what I found:
>
> Create a table T, then metadata cache has T
>
> Create an index I on T, then metadata cache has I, but will trigger to
> update T by following code in PMetadataImpl.addTable
>
> {{{
>
> ….
>
> if (parentTable != null) {
>
> List<PTable> oldIndexes = parentTable.getIndexes();
>
> List<PTable> newIndexes =
> Lists.newArrayListWithExpectedSize(oldIndexes.size() + 1);
>
> newIndexes.addAll(oldIndexes);
>
> if (oldTable != null) {
>
> newIndexes.remove(oldTable);
>
> }
>
> newIndexes.add(table);
>
> parentTable = PTableImpl.makePTable(parentTable,
> table.getTimeStamp(), newIndexes);
>
> tables.put(parentTable.getKey(), parentTable);
>
> }
>
> ….
>
> }}}
>
>
>
> The method makePTable will increase T’s sequence number by 1.
>
>
>
> {{{
>
> public static PTableImpl makePTable(PTable table, long timeStamp,
> List<PTable> indexes) throws SQLException {
>
> return new PTableImpl(
>
> table.getTenantId(), table.getSchemaName(),
> table.getTableName(), table.getType(), table.getIndexState(), timeStamp,
>
> table.getSequenceNumber() + 1, table.getPKName(),
> table.getBucketNum(), getColumnsToClone(table), table.getParentTableName(),
> indexes,
>
> table.isImmutableRows(), table.getPhysicalNames(),
> table.getDefaultFamilyName(), table.getViewStatement(),
> table.isWALDisabled(), table.isMultiTenant(), table.getViewType(),
> table.getViewIndexId());
>
> }}}
>
>
>
> But during my debug the server side sequence number seem unchanged, and then
> I made a “alter table T drop column”, there is a
> ConcurrentMutationException thrown due to inconsistent sequence number.
>
> I can see that the code catch the Exception and update table T with sever
> side metadata, and then a retry complement the drop column operation.
>
> My question is why we increase the sequence number on client side in the
> beginning? Which side is wrong, server or client?
>
>
>
> daniel（孟庆义）
>
>
>