You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Nixon Rodrigues (Jira)" <ji...@apache.org> on 2019/12/06 12:25:00 UTC
[jira] [Updated] (ATLAS-3551) Atlas unable to import entities if an
HBase column family is deleted
[ https://issues.apache.org/jira/browse/ATLAS-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Nixon Rodrigues updated ATLAS-3551:
-----------------------------------
Description:
*Description*
Importing Hbase entities fails if an Hbase column family was previously deleted.
*Steps to reproduce:*
kinit as user hbase
create a table:
{noformat}
$ echo "create 'default:hbase_table_csaba2' ,{NAME=>'col_fam_csaba2_1'},{NAME=>'col_fam_csaba2_2'}" | /usr/hdp/current/hbase-client/bin/hbase shell -n
Created table default:hbase_table_csaba2
Took 1.2510 seconds
Hbase::Table - hbase_table_csaba2
{noformat}
import the table:
{noformat}
$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
HBase Data Model imported successfully!!!
{noformat}
delete one of the column families:
{noformat}
$ echo "alter 'default:hbase_table_csaba2', NAME => 'col_fam_csaba2_1', METHOD=>'delete'" | /usr/hdp/current/hbase-client/bin/hbase shell -n
Updating all regions with the new schema...
1/1 regions updated.
Done.
Took 2.5881 seconds
{noformat}
import the table again:
{noformat}
$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
ImportHBaseEntities failed. Please check the log file for the detailed error message
Failed to import HBase Data Model!!!
Log file content: (/grid/0/log/import-hbase.log):
2019-10-03 12:41:28,391 INFO - [main:] ~ Table already present in Atlas. Updating it..: default:hbase_table_csaba2@cl1 (HBaseBridge:403)
2019-10-03 12:41:28,416 INFO - [main:] ~ method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=404 (AtlasBaseClient:387)
2019-10-03 12:41:28,418 ERROR - [main:] ~ ImportHBaseEntities failed (HBaseBridge:196)
org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@2a389173 failed with status 404 (Not Found) Response Body (
{"errorCode":"ATLAS-404-00-007","errorMessage":"Invalid instance creation/updation parameters passed : hbase_column_family.table: mandatory attribute value missing in type hbase_column_family"}
)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:427)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
at org.apache.atlas.AtlasClientV2.updateEntity(AtlasClientV2.java:324)
at org.apache.atlas.hbase.bridge.HBaseBridge.updateEntityInAtlas(HBaseBridge.java:615)
at org.apache.atlas.hbase.bridge.HBaseBridge.createOrUpdateTable(HBaseBridge.java:409)
at org.apache.atlas.hbase.bridge.HBaseBridge.importTable(HBaseBridge.java:282)
at org.apache.atlas.hbase.bridge.HBaseBridge.importHBaseEntities(HBaseBridge.java:235)
at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:186)
This causes the atlas split 2 failure: test_import_hbase_without_args
{noformat}
was:
Importing Hbase entities fails if an Hbase column family was previously deleted.
Steps to reproduce:
kinit as user hbase
create a table:
{{$ echo "create 'default:hbase_table_csaba2' ,\{NAME=>'col_fam_csaba2_1'},\{NAME=>'col_fam_csaba2_2'}" | /usr/hdp/current/hbase-client/bin/hbase shell -n
Created table default:hbase_table_csaba2
Took 1.2510 seconds
Hbase::Table - hbase_table_csaba2}}
import the table:
{{$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
HBase Data Model imported successfully!!!}}
delete one of the column families:
{{$ echo "alter 'default:hbase_table_csaba2', NAME => 'col_fam_csaba2_1', METHOD=>'delete'" | /usr/hdp/current/hbase-client/bin/hbase shell -n
Updating all regions with the new schema...
1/1 regions updated.
Done.
Took 2.5881 seconds}}
import the table again:
{{$ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
ImportHBaseEntities failed. Please check the log file for the detailed error message
Failed to import HBase Data Model!!!}}
Log file content: (/grid/0/log/import-hbase.log):
{{2019-10-03 12:41:28,391 INFO - [main:] ~ Table already present in Atlas. Updating it..: default:hbase_table_csaba2@cl1 (HBaseBridge:403)
2019-10-03 12:41:28,416 INFO - [main:] ~ method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=404 (AtlasBaseClient:387)
2019-10-03 12:41:28,418 ERROR - [main:] ~ ImportHBaseEntities failed (HBaseBridge:196)
org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@2a389173 failed with status 404 (Not Found) Response Body (\{"errorCode":"ATLAS-404-00-007","errorMessage":"Invalid instance creation/updation parameters passed : hbase_column_family.table: mandatory attribute value missing in type hbase_column_family"})
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:427)
at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
at org.apache.atlas.AtlasClientV2.updateEntity(AtlasClientV2.java:324)
at org.apache.atlas.hbase.bridge.HBaseBridge.updateEntityInAtlas(HBaseBridge.java:615)
at org.apache.atlas.hbase.bridge.HBaseBridge.createOrUpdateTable(HBaseBridge.java:409)
at org.apache.atlas.hbase.bridge.HBaseBridge.importTable(HBaseBridge.java:282)
at org.apache.atlas.hbase.bridge.HBaseBridge.importHBaseEntities(HBaseBridge.java:235)
at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:186)}}
# This causes the atlas split 2 failure: test_import_hbase_without_args
> Atlas unable to import entities if an HBase column family is deleted
> --------------------------------------------------------------------
>
> Key: ATLAS-3551
> URL: https://issues.apache.org/jira/browse/ATLAS-3551
> Project: Atlas
> Issue Type: Bug
> Affects Versions: 2.0.0
> Reporter: Nixon Rodrigues
> Priority: Minor
>
> *Description*
> Importing Hbase entities fails if an Hbase column family was previously deleted.
> *Steps to reproduce:*
> kinit as user hbase
> create a table:
>
> {noformat}
> $ echo "create 'default:hbase_table_csaba2' ,{NAME=>'col_fam_csaba2_1'},{NAME=>'col_fam_csaba2_2'}" | /usr/hdp/current/hbase-client/bin/hbase shell -n
> Created table default:hbase_table_csaba2
> Took 1.2510 seconds
> Hbase::Table - hbase_table_csaba2
> {noformat}
> import the table:
> {noformat}
> $ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
> HBase Data Model imported successfully!!!
>
> {noformat}
> delete one of the column families:
> {noformat}
> $ echo "alter 'default:hbase_table_csaba2', NAME => 'col_fam_csaba2_1', METHOD=>'delete'" | /usr/hdp/current/hbase-client/bin/hbase shell -n
> Updating all regions with the new schema...
> 1/1 regions updated.
> Done.
> Took 2.5881 seconds
>
> {noformat}
> import the table again:
> {noformat}
> $ /usr/hdp/current/atlas-server/hook-bin/import-hbase.sh -t hbase_table_csaba2 -Datlas.conf=/etc/hbase/conf
> ImportHBaseEntities failed. Please check the log file for the detailed error message
> Failed to import HBase Data Model!!!
> Log file content: (/grid/0/log/import-hbase.log):
> 2019-10-03 12:41:28,391 INFO - [main:] ~ Table already present in Atlas. Updating it..: default:hbase_table_csaba2@cl1 (HBaseBridge:403)
> 2019-10-03 12:41:28,416 INFO - [main:] ~ method=POST path=api/atlas/v2/entity/ contentType=application/json; charset=UTF-8 accept=application/json status=404 (AtlasBaseClient:387)
> 2019-10-03 12:41:28,418 ERROR - [main:] ~ ImportHBaseEntities failed (HBaseBridge:196)
> org.apache.atlas.AtlasServiceException: Metadata service API org.apache.atlas.AtlasClientV2$API_V2@2a389173 failed with status 404 (Not Found) Response Body (
> {"errorCode":"ATLAS-404-00-007","errorMessage":"Invalid instance creation/updation parameters passed : hbase_column_family.table: mandatory attribute value missing in type hbase_column_family"}
> )
> at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:427)
> at org.apache.atlas.AtlasBaseClient.callAPIWithResource(AtlasBaseClient.java:353)
> at org.apache.atlas.AtlasBaseClient.callAPI(AtlasBaseClient.java:229)
> at org.apache.atlas.AtlasClientV2.updateEntity(AtlasClientV2.java:324)
> at org.apache.atlas.hbase.bridge.HBaseBridge.updateEntityInAtlas(HBaseBridge.java:615)
> at org.apache.atlas.hbase.bridge.HBaseBridge.createOrUpdateTable(HBaseBridge.java:409)
> at org.apache.atlas.hbase.bridge.HBaseBridge.importTable(HBaseBridge.java:282)
> at org.apache.atlas.hbase.bridge.HBaseBridge.importHBaseEntities(HBaseBridge.java:235)
> at org.apache.atlas.hbase.bridge.HBaseBridge.main(HBaseBridge.java:186)
> This causes the atlas split 2 failure: test_import_hbase_without_args
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)