You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Karan Mehta (JIRA)" <ji...@apache.org> on 2017/09/24 17:49:00 UTC
[jira] [Comment Edited] (PHOENIX-4227) Row for "SYSTEM" schema is
not created during ConnectionQueryServicesImpl init
[ https://issues.apache.org/jira/browse/PHOENIX-4227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178294#comment-16178294 ]
Karan Mehta edited comment on PHOENIX-4227 at 9/24/17 5:48 PM:
---------------------------------------------------------------
I am referring to {{ConnectionQueryServicesImpl#init(String url, Properties props)}} method, which first checks if SYSTEM tables are upgraded with the method {{ensureSystemTablesUpgraded()}}, which inturn checks if SYSTEM namespace is created or not by calling the method {{ensureNamespaceCreated(String schemaName)}}. This method directly creates a HBase namespace with the HBase API, thus bypassing its entry in the {{SYSTEM.CATALOG}} table.
Here is the relevant piece of code.
{code}
try (HBaseAdmin admin = getAdmin()) {
NamespaceDescriptor namespaceDescriptor = null;
try {
namespaceDescriptor = admin.getNamespaceDescriptor(schemaName);
} catch (org.apache.hadoop.hbase.NamespaceNotFoundException e) {
}
if (namespaceDescriptor == null) {
namespaceDescriptor = NamespaceDescriptor.create(schemaName).build();
admin.createNamespace(namespaceDescriptor);
}
return;
} catch (IOException e) {
sqlE = ServerUtil.parseServerException(e);
} finally {
if (sqlE != null) { throw sqlE; }
}
{code}
I added the method in {{SystemTablePermissionsIT#testNamespaceMappedSystemTables()}} test to read SYSCAT. However I found that row missing. After executing the statement {{CREATE SCHEMA IF NOT EXISTS SYSTEM}}, the row was created in SYSCAT.
{code}
private void readTable(String tableName) throws SQLException, IOException {
System.out.println("Reading table: " + tableName);
try (Connection conn = DriverManager.getConnection(getJdbcUrl(), clientProperties);) {
HTable table = (HTable) conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName.getBytes());
Scan scan = new Scan();
ResultScanner results = table.getScanner(scan);
Result result = results.next();
while(result != null) {
System.out.println("TableName = " + Bytes.toString(result.getRow()));
NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> map = result.getMap();
for(Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> entry : map.entrySet()) {
for(Map.Entry<byte[], NavigableMap<Long, byte[]>> entry1 : entry.getValue().entrySet()) {
System.out.println("\tentry1.getKey() = " + Bytes.toString(entry1.getKey()));
for(Map.Entry<Long, byte[]> entry2 : entry1.getValue().entrySet()) {
System.out.println("\t\tentry2.getValue() = " + Bytes.toString(entry2.getValue()));
}
}
}
result = results.next();
}
}
System.out.println("End of reading");
}
{code}
This init method is used by PhoenixDriver and PhoenixEmbeddedDriver, connect() method.
was (Author: karanmehta93):
I am referring to {{ConnectionQueryServicesImpl#init(String url, Properties props)}} method, which first checks if SYSTEM tables are upgraded with the method {{ensureSystemTablesUpgraded()}}, which inturn checks if SYSTEM namespace is created or not by calling the method {{ensureNamespaceCreated(String schemaName)}}. This method directly creates a HBase namespace with the HBase API, thus bypassing its entry in the {{SYSTEM.CATALOG}} table.
Here is the relevant piece of code.
{code}
try (HBaseAdmin admin = getAdmin()) {
NamespaceDescriptor namespaceDescriptor = null;
try {
namespaceDescriptor = admin.getNamespaceDescriptor(schemaName);
} catch (org.apache.hadoop.hbase.NamespaceNotFoundException e) {
}
if (namespaceDescriptor == null) {
namespaceDescriptor = NamespaceDescriptor.create(schemaName).build();
admin.createNamespace(namespaceDescriptor);
}
return;
} catch (IOException e) {
sqlE = ServerUtil.parseServerException(e);
} finally {
if (sqlE != null) { throw sqlE; }
}
{code}
I added the method in {{SystemTablePermissionsIT#testNamespaceMappedSystemTables() }} test to read SYSCAT. However I found that row missing. After executing the statement {{CREATE SCHEMA IF NOT EXISTS SYSTEM}}, the row was created in SYSCAT.
{code}
private void readTable(String tableName) throws SQLException, IOException {
System.out.println("Reading table: " + tableName);
try (Connection conn = DriverManager.getConnection(getJdbcUrl(), clientProperties);) {
HTable table = (HTable) conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(tableName.getBytes());
Scan scan = new Scan();
ResultScanner results = table.getScanner(scan);
Result result = results.next();
while(result != null) {
System.out.println("TableName = " + Bytes.toString(result.getRow()));
NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> map = result.getMap();
for(Map.Entry<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> entry : map.entrySet()) {
for(Map.Entry<byte[], NavigableMap<Long, byte[]>> entry1 : entry.getValue().entrySet()) {
System.out.println("\tentry1.getKey() = " + Bytes.toString(entry1.getKey()));
for(Map.Entry<Long, byte[]> entry2 : entry1.getValue().entrySet()) {
System.out.println("\t\tentry2.getValue() = " + Bytes.toString(entry2.getValue()));
}
}
}
result = results.next();
}
}
System.out.println("End of reading");
}
{code}
This init method is used by PhoenixDriver and PhoenixEmbeddedDriver, connect() method.
> Row for "SYSTEM" schema is not created during ConnectionQueryServicesImpl init
> -------------------------------------------------------------------------------
>
> Key: PHOENIX-4227
> URL: https://issues.apache.org/jira/browse/PHOENIX-4227
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.12.0
> Reporter: Karan Mehta
>
> For every new schema created through Phoenix, a row containing the schema name (with a 0x0 at the beginning and the end) is added to SYSTEM.CATALOG table. This is used for by other statements such as "USE SCHEMA" or "CREATE SCHEMA" to determine if the schema exists or not.
> However when we turn on namespaces, SYSTEM tables are automatically migrated to SYSTEM namespace in HBase. However an SYSTEM.CATALOG entry with the row (\x0SYSTEM\x0) is not created. Thus other statements trying to use it might receive a SCHEMA_NOT_FOUND_EXCEPTION even though the schema exists in the file.
> A "CREATE SCHEMA SYSTEM" statement can create the corresponding row entry in the SYSCAT table. This JIRA is to fix this behaviour and make it consistent for all schemas across Phoenix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)