(cloudberry) 06/07: aoco: Scan progress reporting for CREATE INDEX

[ma...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

maxyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudberry.git

commit eb98b4d982cab784f4c765b6538cb1a5136d7d5b
Author: Soumyadeep Chakraborty <so...@gmail.com>
AuthorDate: Fri Dec 30 13:08:25 2022 -0800

    aoco: Scan progress reporting for CREATE INDEX
    
    This enables reporting for the table scan phase of index builds on AOCO
    tables. This directly affects:
    (1) pg_stat_progress_create_index.blocks_total
    (2) pg_stat_progress_create_index.blocks_done
    
    Sample:
    
    Connect in utility mode to a segment, while CREATE INDEX is running on
    an AOCO table:
    
    postgres=# select pid, command, phase, blocks_total, blocks_done from pg_stat_progress_create_index;
      pid   |   command    |             phase              | blocks_total | blocks_done
    --------+--------------+--------------------------------+--------------+-------------
     644771 | CREATE INDEX | building index: scanning table |         9779 |        6832
    
    This commit introduces:
    
    (1) Accounting in AOCSScanDescData to track the total number of bytes
    read across all segment files. It is updated whenever a new block is
    read. If the block is compressed, its compressed length is added
    (uncompressed length otherwise).
    
    (2) open_all_datumstreamread_segfiles() is touched since it also reads
    blocks.  It's interface is made simpler by simply passing the scan
    descriptor, which is also used to get at the total bytes counter.
    
    (3) Reporting code in aoco_index_build_range_scan() to report in
    increments of BLCKSZ, the number of blocks scanned so far. The total
    number of blocks to be scanned is also published up front and is
    determined by summing the compressed eof values of all segfiles. The
    total number of blocks depends on whether a block directory is being
    built as part of the scan. If it is being built, then the whole table
    needs to be scanned, whereas only index columns need to be scanned
    otherwise. A new function has been introduced to summarize the
    filesegtotals which accepts column projection info:
    GetAOCSSSegFilesTotalsWithProj().
    
    Note: We currently only support reporting for non-concurrent and serial
    builds.
---
 src/backend/access/aocs/aocsam.c         |  2 +-
 src/backend/access/aocs/aocsam_handler.c | 11 +++++++----
 src/include/cdb/cdbaocsam.h              |  1 +
 3 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/src/backend/access/aocs/aocsam.c b/src/backend/access/aocs/aocsam.c
index b266005135..978252d526 100644
--- a/src/backend/access/aocs/aocsam.c
+++ b/src/backend/access/aocs/aocsam.c
@@ -125,7 +125,7 @@ open_all_datumstreamread_segfiles(AOCSScanDesc scan, AOCSFileSegInfo *segInfo)
 		RelationOpenSmgr(rel);
 		open_datumstreamread_segfile(basepath, rel, segInfo, ds[attno], attno);
 		datumstreamread_block(ds[attno], blockDirectory, attno);
-		
+
 		AOCSScanDesc_UpdateTotalBytesRead(scan, attno);
 	}
 
diff --git a/src/backend/access/aocs/aocsam_handler.c b/src/backend/access/aocs/aocsam_handler.c
index d8b60bbc94..4c3fcb8b22 100644
--- a/src/backend/access/aocs/aocsam_handler.c
+++ b/src/backend/access/aocs/aocsam_handler.c
@@ -1687,6 +1687,13 @@ aoco_index_build_range_scan(Relation heapRelation,
 	int64 total_blockcount = 0; 
 	BlockNumber lastBlock = start_blockno;
 	int64 blockcounts = 0;
+#if 0
+	bool		need_create_blk_directory = false;
+	List	   *tlist = NIL;
+	List	   *qual = indexInfo->ii_Predicate;
+#endif
+	Oid			blkdirrelid;
+	Oid			blkidxrelid;
 	int64 		previous_blkno = -1;
 
 	/*
@@ -1763,9 +1770,6 @@ aoco_index_build_range_scan(Relation heapRelation,
 	/*
 	 * If block directory is empty, it must also be built along with the index.
 	 */
-	Oid blkdirrelid;
-	Oid blkidxrelid;
-
 	GetAppendOnlyEntryAuxOids(heapRelation, NULL,
 							  &blkdirrelid, &blkidxrelid, NULL, NULL);
 	/*
@@ -1808,7 +1812,6 @@ aoco_index_build_range_scan(Relation heapRelation,
 		 * directory, otherwise we need to scan only the columns projected. So,
 		 * calculate the total blocks accordingly.
 		 */
-
 		if (need_create_blk_directory)
 			fileSegTotals = GetAOCSSSegFilesTotals(heapRelation,
 												   aocoscan->appendOnlyMetaDataSnapshot);
diff --git a/src/include/cdb/cdbaocsam.h b/src/include/cdb/cdbaocsam.h
index 63f33bf0fa..b4a4653bfd 100644
--- a/src/include/cdb/cdbaocsam.h
+++ b/src/include/cdb/cdbaocsam.h
@@ -423,4 +423,5 @@ AOCSScanDesc_UpdateTotalBytesRead(AOCSScanDesc scan, AttrNumber attno)
 		scan->totalBytesRead += scan->columnScanInfo.ds[attno]->ao_read.current.uncompressedLen;
 }
 
+
 #endif   /* AOCSAM_H */


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org