You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "alamb (via GitHub)" <gi...@apache.org> on 2026/08/24 19:34:16 UTC
[PR] docs: Fix rendering of `get_record_batch_memory_size` buffer diagram [datafusion]
alamb opened a new pull request, #24637:
URL: https://github.com/apache/datafusion/pull/24637
## Which issue does this PR close?
- N/A (minor doc fix). Follow on to #24592, which I noticed while reviewing that PR.
## Rationale for this change
The docs for [`get_record_batch_memory_size`](https://docs.rs/datafusion/latest/datafusion/common/utils/memory/fn.get_record_batch_memory_size.html) render poorly on docs.rs:
1. The ASCII-art buffer diagram is not in a code block, so rustdoc collapses it into a single unreadable line
2. A stray backtick in ```Current `RecordBatch`.get_array_memory_size()` will ...``` causes the rest of the paragraph to render as inline code
<img width="600" alt="Screenshot of broken rendering" src="https://docs.rs/datafusion/latest/datafusion/common/utils/memory/fn.get_record_batch_memory_size.html" />
## What changes are included in this PR?
- Wrap the diagram in a `text` code fence so it renders as drawn
- Replace the mis-quoted method reference with an intra-doc link to `RecordBatch::get_array_memory_size`
## Are these changes tested?
Yes: `RUSTDOCFLAGS="-D warnings" cargo doc -p datafusion-common --no-deps` passes (this also verifies the new intra-doc link resolves).
## Are there any user-facing changes?
Documentation only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Fix rendering of `get_record_batch_memory_size` buffer diagram [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #24637:
URL: https://github.com/apache/datafusion/pull/24637#issuecomment-5400572964
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/24637?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 81.43%. Comparing base ([`26b40dd`](https://app.codecov.io/gh/apache/datafusion/commit/26b40dd781d2eeab1acc38370a7069a8b9ebe450?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`df47c1e`](https://app.codecov.io/gh/apache/datafusion/commit/df47c1efbaf9dbdbcbe988bce2904e32113e1725?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #24637 +/- ##
==========================================
- Coverage 81.43% 81.43% -0.01%
==========================================
Files 1118 1118
Lines 399414 399414
Branches 399414 399414
==========================================
- Hits 325278 325260 -18
- Misses 55145 55155 +10
- Partials 18991 18999 +8
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/24637?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Fix rendering of `get_record_batch_memory_size` buffer diagram [datafusion]
Posted by "nuno-faria (via GitHub)" <gi...@apache.org>.
nuno-faria commented on code in PR #24637:
URL: https://github.com/apache/datafusion/pull/24637#discussion_r3856334525
##########
datafusion/common/src/utils/memory.rs:
##########
@@ -131,19 +131,21 @@ pub fn estimate_memory_size<T>(num_elements: usize, fixed_size: usize) -> Result
/// For a `RecordBatch` with two columns: `col1` and `col2`, two columns are pointing
/// to a sub-region of the same buffer.
///
+/// ```text
/// {xxxxxxxxxxxxxxxxxxx} <--- buffer
/// ^ ^ ^ ^
/// | | | |
/// col1->{ } | |
/// col2--------->{ }
+/// ```
///
/// In the above case, `get_record_batch_memory_size` will return the size of
/// the buffer, instead of the sum of `col1` and `col2`'s actual memory size.
///
-/// Note: Current `RecordBatch`.get_array_memory_size()` will double count the
-/// buffer memory size if multiple arrays within the batch are sharing the same
-/// `Buffer`. This method provides temporary fix until the issue is resolved:
-/// <https://github.com/apache/arrow-rs/issues/6439>
+/// Note: The current [`RecordBatch::get_array_memory_size`] will double count
+/// the buffer memory size if multiple arrays within the batch are sharing the
+/// same `Buffer`. This method provides a temporary fix until the issue is
+/// resolved: <https://github.com/apache/arrow-rs/issues/6439>
Review Comment:
I wonder if this is still needed, since the issue has been completed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Fix rendering of `get_record_batch_memory_size` buffer diagram [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb merged PR #24637:
URL: https://github.com/apache/datafusion/pull/24637
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Fix rendering of `get_record_batch_memory_size` buffer diagram [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #24637:
URL: https://github.com/apache/datafusion/pull/24637#discussion_r3856817631
##########
datafusion/common/src/utils/memory.rs:
##########
@@ -131,19 +131,21 @@ pub fn estimate_memory_size<T>(num_elements: usize, fixed_size: usize) -> Result
/// For a `RecordBatch` with two columns: `col1` and `col2`, two columns are pointing
/// to a sub-region of the same buffer.
///
+/// ```text
/// {xxxxxxxxxxxxxxxxxxx} <--- buffer
/// ^ ^ ^ ^
/// | | | |
/// col1->{ } | |
/// col2--------->{ }
+/// ```
///
/// In the above case, `get_record_batch_memory_size` will return the size of
/// the buffer, instead of the sum of `col1` and `col2`'s actual memory size.
///
-/// Note: Current `RecordBatch`.get_array_memory_size()` will double count the
-/// buffer memory size if multiple arrays within the batch are sharing the same
-/// `Buffer`. This method provides temporary fix until the issue is resolved:
-/// <https://github.com/apache/arrow-rs/issues/6439>
+/// Note: The current [`RecordBatch::get_array_memory_size`] will double count
+/// the buffer memory size if multiple arrays within the batch are sharing the
+/// same `Buffer`. This method provides a temporary fix until the issue is
+/// resolved: <https://github.com/apache/arrow-rs/issues/6439>
Review Comment:
That is an excellent point -- I will remove it in a follow on PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] docs: Fix rendering of `get_record_batch_memory_size` buffer diagram [datafusion]
Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #24637:
URL: https://github.com/apache/datafusion/pull/24637#discussion_r3856929248
##########
datafusion/common/src/utils/memory.rs:
##########
@@ -131,19 +131,21 @@ pub fn estimate_memory_size<T>(num_elements: usize, fixed_size: usize) -> Result
/// For a `RecordBatch` with two columns: `col1` and `col2`, two columns are pointing
/// to a sub-region of the same buffer.
///
+/// ```text
/// {xxxxxxxxxxxxxxxxxxx} <--- buffer
/// ^ ^ ^ ^
/// | | | |
/// col1->{ } | |
/// col2--------->{ }
+/// ```
///
/// In the above case, `get_record_batch_memory_size` will return the size of
/// the buffer, instead of the sum of `col1` and `col2`'s actual memory size.
///
-/// Note: Current `RecordBatch`.get_array_memory_size()` will double count the
-/// buffer memory size if multiple arrays within the batch are sharing the same
-/// `Buffer`. This method provides temporary fix until the issue is resolved:
-/// <https://github.com/apache/arrow-rs/issues/6439>
+/// Note: The current [`RecordBatch::get_array_memory_size`] will double count
+/// the buffer memory size if multiple arrays within the batch are sharing the
+/// same `Buffer`. This method provides a temporary fix until the issue is
+/// resolved: <https://github.com/apache/arrow-rs/issues/6439>
Review Comment:
- Follow on PR https://github.com/apache/datafusion/pull/24674
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org