You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "James Taylor (JIRA)" <ji...@apache.org> on 2016/08/16 23:38:20 UTC
[jira] [Comment Edited] (PHOENIX-3167) CREATE TABLE on an existing
table resets the HTableMetadata
[ https://issues.apache.org/jira/browse/PHOENIX-3167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423596#comment-15423596 ]
James Taylor edited comment on PHOENIX-3167 at 8/16/16 11:37 PM:
-----------------------------------------------------------------
This seems specific to the SYSTEM.CATALOG table and KEEP_DELETED_CELLS. We should remove the code in ConnectionQueryServicesImpl that defaults the KEEP_DELETED_CELLS as that may be the root cause of this issue. We do use the existing HTableDescriptor as the basis for the modify table call we make, so existing properties won't be changed unless they're specified in the CREATE TABLE call (or we try to default them, which I think is only the case for KEEP_DELETED_CELLS). For example, the following test passes:
{code}
@Test
public void testRecreatingExistingTableMaintainsHTableProperties() throws Exception {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
String ddl = "ALTER TABLE SYSTEM.CATALOG SET REPLICATION_SCOPE='1'";
conn.createStatement().execute(ddl);
assertDeleteCellsDisabled(conn);
try {
conn.createStatement().execute(QueryConstants.CREATE_TABLE_METADATA);
} catch (TableAlreadyExistsException e) {
}
assertDeleteCellsDisabled(conn);
}
private void assertDeleteCellsDisabled(Connection conn) throws org.apache.hadoop.hbase.TableNotFoundException,
Exception {
try (HBaseAdmin admin = conn.unwrap(PhoenixConnection.class).getQueryServices().getAdmin()) {
HTableDescriptor tableDesc = admin.getTableDescriptor(Bytes.toBytes("SYSTEM.CATALOG"));
HColumnDescriptor[] columnFamilies = tableDesc.getColumnFamilies();
assertEquals("0", columnFamilies[0].getNameAsString());
assertEquals("1", columnFamilies[0].getValue("REPLICATION_SCOPE"));
}
}
{code}
The REPLICATION_SCOPE should have been set on the HTableDescriptor, though, not the column family. That's a bug - I filed PHOENIX-3186 for this.
was (Author: jamestaylor):
This seems specific to the SYSTEM.CATALOG table and KEEP_DELETED_CELLS. We should remove the code in ConnectionQueryServicesImpl that defaults the KEEP_DELETED_CELLS as that may be the root cause of this issue. We do use the existing HTableDescriptor as the basis for the modify table call we make, so existing properties won't be changed unless they're specified in the CREATE TABLE call (or we try to default them, which I think is only the case for KEEP_DELETED_CELLS). For example, the following test passes:
{code}
@Test
public void testRecreatingExistingTableMaintainsHTableProperties() throws Exception {
Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
Connection conn = DriverManager.getConnection(getUrl(), props);
String ddl = "ALTER TABLE SYSTEM.CATALOG SET REPLICATION_SCOPE='1'";
conn.createStatement().execute(ddl);
assertDeleteCellsDisabled(conn);
try {
conn.createStatement().execute(QueryConstants.CREATE_TABLE_METADATA);
} catch (TableAlreadyExistsException e) {
}
assertDeleteCellsDisabled(conn);
}
private void assertDeleteCellsDisabled(Connection conn) throws org.apache.hadoop.hbase.TableNotFoundException,
Exception {
try (HBaseAdmin admin = conn.unwrap(PhoenixConnection.class).getQueryServices().getAdmin()) {
HTableDescriptor tableDesc = admin.getTableDescriptor(Bytes.toBytes("SYSTEM.CATALOG"));
HColumnDescriptor[] columnFamilies = tableDesc.getColumnFamilies();
assertEquals("0", columnFamilies[0].getNameAsString());
assertEquals("1", columnFamilies[0].getValue("REPLICATION_SCOPE"));
}
}
{code}
The REPLICATION_SCOPE should have been set on the HTableDescriptor, though, not the column family. That's a bug.
> CREATE TABLE on an existing table resets the HTableMetadata
> -----------------------------------------------------------
>
> Key: PHOENIX-3167
> URL: https://issues.apache.org/jira/browse/PHOENIX-3167
> Project: Phoenix
> Issue Type: Bug
> Reporter: Samarth Jain
> Assignee: Samarth Jain
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3167.patch
>
>
> {code}
> @Test
> public void testRecreatingExistingTableMaintainsHTableProperties() throws Exception {
> Properties props = PropertiesUtil.deepCopy(TEST_PROPERTIES);
> Connection conn = DriverManager.getConnection(getUrl(), props);
> String ddl = "ALTER TABLE SYSTEM.CATALOG SET KEEP_DELETED_CELLS = false";
> conn.createStatement().execute(ddl);
> assertDeleteCellsDisabled(conn);
> try {
> conn.createStatement().execute(QueryConstants.CREATE_TABLE_METADATA);
> } catch (TableAlreadyExistsException e) {
> }
> assertDeleteCellsDisabled(conn);
> }
>
> private void assertDeleteCellsDisabled(Connection conn) throws org.apache.hadoop.hbase.TableNotFoundException,
> IOException, SQLException {
> try (HBaseAdmin admin = conn.unwrap(PhoenixConnection.class).getQueryServices().getAdmin()) {
> HTableDescriptor tableDesc = admin.getTableDescriptor(Bytes.toBytes("SYSTEM.CATALOG"));
> HColumnDescriptor[] columnFamilies = tableDesc.getColumnFamilies();
> assertEquals("0", columnFamilies[0].getNameAsString());
> assertEquals(KeepDeletedCells.FALSE, columnFamilies[0].getKeepDeletedCells());
> }
> }
> {code}
> [~jamestaylor], [~apurtell], [~lhofhansl], [~mujtabachohan] - this is likely why we saw that the keep deleted cells property always kept flipping to true even though we set it to false in hbase shell.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)