[jira] [Comment Edited] (PHOENIX-6976) Add empty column discovery to ScanUtil via HRegion

["Ujjawal Kumar (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17731986#comment-17731986 ] 

Ujjawal Kumar edited comment on PHOENIX-6976 at 6/13/23 8:35 AM:
-----------------------------------------------------------------

{quote}If we really want to avoid accessing Sysycat, we could just add this information as a table property{quote}

This would require syncing table properties via table descriptor to be same as SYSTEM.CATALOG (there are ways to add arbitrary properties to HBase table descriptor via Table Options as part of DDL (https://phoenix.apache.org/language/index.html#options)) 

Currently, phoenix checks it via encoding scheme (which is kept in SYSTEM.CATALOG) which itself can't detect if the table has multiple kinds of empty columns.

The idea to do it via HBase Scan would be something similar (use the first encountered type of empty column)

RegionScanner scanner = hRegion.getScanner(new Scan());
    List<Cell> row = new ArrayList<>();
    scanner.next(row);
    for(Cell cell : row) {
      if(emptyCF == null){
        emptyCF = CellUtil.cloneFamily(cell);
      }
      if(ScanUtil.isEmptyColumn(cell, emptyCF, QueryConstants.EMPTY_COLUMN_BYTES)) {
        emptyCQ = QueryConstants.EMPTY_COLUMN_BYTES;
        break;
      } //Empty column is always encoded in FOUR_BYTE format, since it's a reserved qualifier. See EncodedColumnsUtil#isReservedColumnQualifier
      else if(ScanUtil.isEmptyColumn(cell, emptyCF, QueryConstants.ENCODED_EMPTY_COLUMN_BYTES)) {
        emptyCQ = QueryConstants.ENCODED_EMPTY_COLUMN_BYTES;
        break;
      }
    }


was (Author: ukumar):
{quote}If we really want to avoid accessing Sysycat, we could just add this information as a table property{quote}

This would require syncing table properties via table descriptor (there are ways to add arbitrary properties to HBase table descriptor via Table Options as part of DDL (https://phoenix.apache.org/language/index.html#options)) 

Currently, phoenix checks it via encoding scheme (which is kept in SYSTEM.CATALOG) which itself can't detect if the table has multiple kinds of empty columns.

The idea to do it via HBase Scan would be something similar (use the first encountered type of empty column)

RegionScanner scanner = hRegion.getScanner(new Scan());
    List<Cell> row = new ArrayList<>();
    scanner.next(row);
    for(Cell cell : row) {
      if(emptyCF == null){
        emptyCF = CellUtil.cloneFamily(cell);
      }
      if(ScanUtil.isEmptyColumn(cell, emptyCF, QueryConstants.EMPTY_COLUMN_BYTES)) {
        emptyCQ = QueryConstants.EMPTY_COLUMN_BYTES;
        break;
      } //Empty column is always encoded in FOUR_BYTE format, since it's a reserved qualifier. See EncodedColumnsUtil#isReservedColumnQualifier
      else if(ScanUtil.isEmptyColumn(cell, emptyCF, QueryConstants.ENCODED_EMPTY_COLUMN_BYTES)) {
        emptyCQ = QueryConstants.ENCODED_EMPTY_COLUMN_BYTES;
        break;
      }
    }

> Add empty column discovery to ScanUtil via HRegion
> --------------------------------------------------
>
>                 Key: PHOENIX-6976
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-6976
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 5.1.3
>            Reporter: Ujjawal Kumar
>            Priority: Minor
>
> Features like PHOENIX-6888 depend upon empty column discovery to decide compaction behaviour, we can add a util method to ScanUtil so that empty column can be discovered without going through SYSTEM.CATALOG by scanning the HRegion (e.g. in case of HBase level coprocessors) and searching for empty column 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)