[GitHub] [airflow] potiuk commented on a diff in pull request #25860: Improve logging of output for Breeze commands in CI

[GitBox <gi...@apache.org>]

potiuk commented on code in PR #25860:
URL: https://github.com/apache/airflow/pull/25860#discussion_r951395647


##########
dev/breeze/src/airflow_breeze/utils/parallel.py:
##########
@@ -88,71 +83,142 @@ def get_last_lines_of_file(file_name: str, num_lines: int = 2) -> Tuple[List[str
     return last_lines, last_lines_no_colors
 
 
-DOCKER_BUILDX_PROGRESS_MATCHER = re.compile(r'\s*#(\d*) ')
-last_docker_build_lines: Dict[Output, str] = {}
+class ProgressLines(NamedTuple):
+    lines: List[str]
+    skip_truncation: List[bool]
+
+
+class AbstractProgressInfoMatcher(metaclass=ABCMeta):
+    @abstractmethod
+    def get_best_matching_lines(self, output: Output) -> Optional[ProgressLines]:
+        """
+        Return best matching lines of the output. It also indicates if the lines potentially need truncation
+        :param output: file that should be analysed for the output
+        :return: tuple of array of lines to print and boolean indications whether the lines need truncation
+        """
+
+
+class DockerBuildxProgressMatcher(AbstractProgressInfoMatcher):
+    DOCKER_BUILDX_PROGRESS_MATCHER = re.compile(r'\s*#(\d*) ')
+
+    def __init__(self):
+        self.last_docker_build_line: Optional[str] = None
+
+    def get_best_matching_lines(self, output) -> Optional[ProgressLines]:
+        last_lines, last_lines_no_colors = get_last_lines_of_file(output.file_name, num_lines=5)
+        best_progress: int = 0
+        best_line: Optional[str] = None
+        for index, line in enumerate(last_lines_no_colors):
+            match = DockerBuildxProgressMatcher.DOCKER_BUILDX_PROGRESS_MATCHER.match(line)
+            if match:
+                docker_progress = int(match.group(1))
+                if docker_progress > best_progress:
+                    best_progress = docker_progress
+                    best_line = last_lines[index]
+        if best_line is None:
+            best_line = self.last_docker_build_line
+        else:
+            self.last_docker_build_line = best_line
+        if best_line is None:
+            return None
+        return ProgressLines(lines=[best_line], skip_truncation=[False])
 
 
-def progress_method_docker_buildx(output: Output) -> Optional[str]:
-    last_lines, last_lines_no_colors = get_last_lines_of_file(output.file_name, num_lines=5)
-    best_progress: int = 0
-    best_line: Optional[str] = None
-    for index, line in enumerate(last_lines_no_colors):
-        match = DOCKER_BUILDX_PROGRESS_MATCHER.match(line)
-        if match:
-            docker_progress = int(match.group(1))
-            if docker_progress > best_progress:
-                best_progress = docker_progress
+class GenericRegexpProgressMatcher(AbstractProgressInfoMatcher):
+    def __init__(
+        self,
+        regexp: str,
+        lines_to_search: int,
+        regexp_for_joined_line: Optional[str] = None,
+        regexp_to_skip_truncation: Optional[str] = None,
+    ):
+        self.last_good_match: Optional[str] = None
+        self.matcher = re.compile(regexp)
+        self.lines_to_search = lines_to_search
+        self.matcher_for_joined_line = re.compile(regexp_for_joined_line) if regexp_for_joined_line else None

Review Comment:
   This one allows to see two last important lines of the output (for example in test output we will see):
   
   ![image](https://user-images.githubusercontent.com/595491/185924331-469d8d2a-f0b8-44c6-bef7-68feacfa3e69.png)
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org