You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@buildstream.apache.org by GitBox <gi...@apache.org> on 2020/12/30 10:23:15 UTC
[GitHub] [buildstream] BuildStream-Migration-Bot opened a new issue #18: progress monitoring and elapsed time in logs...
BuildStream-Migration-Bot opened a new issue #18:
URL: https://github.com/apache/buildstream/issues/18
[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/18)
In GitLab by [[Gitlab user @devcurmudgeon]](https://gitlab.com/devcurmudgeon) on Apr 28, 2017, 11:44
as discussed on [irc](https://irclogs.baserock.org/buildstream/%23buildstream.2017-04-26.log.html#t2017-04-26T11:17:58) it's important for a user to monitor progress of a build run. Based on experience with morph + ybd i recommend:
- some ongoing indicator of [current task/tasks required for this run/total tasks for target]
- ongoing indicator of total elapsed time since run started
- record elapsed time for each component task completion
Note that
- users may want to glance at current run status while it's running (so want to see current elapsed time)
- users may want to look at the timeline profile of a run later, which means having elapsed time in every line of the log is useful
- users may want to compare specific task times between machines/runs (which AWS machine gives me best time vs cost result?) and between components (which builds are taking most time, to optimise that build or maybe try to adjust dependencies etc)
IMO the current realtime clocks per task are shiny but not really useful, and the zeroes+blanks are taking up space that would be better used as snapshot of elapsed time when that logline was generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] BuildStream-Migration-Bot commented on issue #18: progress monitoring and elapsed time in logs...
Posted by GitBox <gi...@apache.org>.
BuildStream-Migration-Bot commented on issue #18:
URL: https://github.com/apache/buildstream/issues/18#issuecomment-752406204
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] BuildStream-Migration-Bot commented on issue #18: progress monitoring and elapsed time in logs...
Posted by GitBox <gi...@apache.org>.
BuildStream-Migration-Bot commented on issue #18:
URL: https://github.com/apache/buildstream/issues/18#issuecomment-752406153
In GitLab by [[Gitlab user @tristanvb]](https://gitlab.com/tristanvb) on May 1, 2017, 12:27
So this is partly addressed, I'm not sure I'm happy with the layout but it's an improvement.
Now the separator line between the rolling log and the task list shows the following centered in the `====` separator:
* The main bst invocation target
* The total elapsed time of the session
Other information fit on that line, but the result was too cluttered to easily read at a glance, so the status area now has 3 lines as a header... example:
```
... rolling log lines ...
===================================== 00:00:12 gedit.bst =====================================
[Session:4] [Total:6] [Fetched:4] [Built:0]
~~~~~ Active Tasks ~~~~~
... active tasks displayed here ...
```
Again here, the extra centered separator takes a precious additional line, but without it the additional information looks cluttered and difficult to read.
The additional information is element counts of:
* `Total` The total number of elements in the pipeline
* `Session` The number of elements which were queued in this session / invocation
* Number of elements which have passed each scheduler queue, in this case:
* Number of elements which have passed through the `fetch` queue
* Number of elements which have passed through the `build` queue
For the Queue monitors, they will display different information depending on the `bst` invocation, e.g. only `fetch` queue is used for `bst fetch`. After implementing the artifact cache sharing, a `bst build` invocation will also have a third `upload` queue for a client configured and authenticated for artifact uploads.
This should mostly take care of ongoing status information but maybe could use some polish...
## Regarding Timestamps
So here I'm not entirely sure that I agree with *"having elapsed time in every line of the log is useful"*, as in, I'm mostly interested to know how much time it took for a given task to complete, while the information of *"the offset time in the session whence the task began"* seems not to be of much value or relevance.
Now, it is also a goal to have presentable log lines, which means either:
* Making a best effort to ensure the lines fit within 80 characters *(what we currently do)*
* Perhaps intelligently wrapping the last (dynamic length) portion of the log lines so that they do not wrap all the way around the terminal, but instead wrap at the column where the dynamic length "short message" began, would be a nice improvement.
This is a bit difficult to reason around though since it's also a problem that elements themselves can have overly long names which dont fit into the 30 or 40 char padding we currently try to align them into, it would be best to have reliably columnized log lines at least when viewed in the terminal for better readability, maybe this can be done with an ellipsis (`...`) of the *leading* chars for overly long element names (filenames).
With that added emphasis on how important I think presentation of output should be out of the way, this is the conundrum I have about adding *"session offset timestamps"* to each and every line:
* I care a lot about knowing how long it took a given task to complete
* I'm not sure what the relevance is for offsets in time, except perhaps in a case that we did not have a status area at all, in which case it tells the user generally how long this has been taking so far
* If we were to have both, *even* if we only showed task/activity elapsed time in lines where a task completes, it *still* takes an extra column for those lines; which means we bust the 80 chars target in log lines more often, which is undesirable
If the *"session offset timestamps"* are that desirable, an alternative would be only output those to the target log file (i.e. `bst --log-file <filename> ...`), or to additionally only output those when the terminal width is at a given threshold, maybe at 100 or 120 chars wide.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [buildstream] BuildStream-Migration-Bot closed issue #18: progress monitoring and elapsed time in logs...
Posted by GitBox <gi...@apache.org>.
BuildStream-Migration-Bot closed issue #18:
URL: https://github.com/apache/buildstream/issues/18
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org