You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@arrow.apache.org by GitBox <gi...@apache.org> on 2022/05/31 08:30:54 UTC
[GitHub] [arrow-datafusion] dependabot[bot] opened a new pull request, #2664: Update parquet requirement from 14.0.0 to 15.0.0
dependabot[bot] opened a new pull request, #2664:
URL: https://github.com/apache/arrow-datafusion/pull/2664
Updates the requirements on [parquet](https://github.com/apache/arrow-rs) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md">parquet's changelog</a>.</em></p>
<blockquote>
<h2><a href="https://github.com/apache/arrow-rs/tree/15.0.0">15.0.0</a> (2022-05-27)</h2>
<p><a href="https://github.com/apache/arrow-rs/compare/14.0.0...15.0.0">Full Changelog</a></p>
<p><strong>Breaking changes:</strong></p>
<ul>
<li>Change <code>ArrayDataBuilder::null_bit_buffer</code> to accept <code>Option<Buffer></code> rather than <code>Buffer</code> <a href="https://github-redirect.dependabot.com/apache/arrow-rs/pull/1739">#1739</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/HaoYang670">HaoYang670</a>)</li>
<li>Remove <code>null_count</code> from <code>ArrayData::try_new()</code> <a href="https://github-redirect.dependabot.com/apache/arrow-rs/pull/1721">#1721</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/HaoYang670">HaoYang670</a>)</li>
<li>Change parquet writers to use standard <code>std:io::Write</code> rather custom <code>ParquetWriter</code> trait (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1717">#1717</a>) (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1163">#1163</a>) <a href="https://github-redirect.dependabot.com/apache/arrow-rs/pull/1719">#1719</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>] (<a href="https://github.com/tustvold">tustvold</a>)</li>
<li>Add explicit column mask for selection in parquet: <code>ProjectionMask</code> (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1701">#1701</a>) <a href="https://github-redirect.dependabot.com/apache/arrow-rs/pull/1716">#1716</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>] (<a href="https://github.com/tustvold">tustvold</a>)</li>
<li>Add type_ids in Union datatype <a href="https://github-redirect.dependabot.com/apache/arrow-rs/pull/1703">#1703</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>] [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/viirya">viirya</a>)</li>
<li>Fix Parquet Reader's Arrow Schema Inference <a href="https://github-redirect.dependabot.com/apache/arrow-rs/pull/1682">#1682</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>] [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/tustvold">tustvold</a>)</li>
</ul>
<p><strong>Implemented enhancements:</strong></p>
<ul>
<li>Rename the <code>string</code> kernel to <code>concatenate_elements</code> <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1747">#1747</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li><code>ArrayDataBuilder::null_bit_buffer</code> should accept <code>Option<Buffer></code> as input type <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1737">#1737</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Fix schema comparison for non_canonical_map when running flight test <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1730">#1730</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Add support in aggregate kernel for <code>BinaryArray</code> <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1724">#1724</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Fix incorrect null_count in <code>generate_unions_case</code> integration test <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1712">#1712</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Keep type ids in Union datatype to follow Arrow spec and integrate with other implementations <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1690">#1690</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Support Reading Alternative List Representations to Arrow From Parquet <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1680">#1680</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Speed up the offsets checking <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1675">#1675</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Separate Parquet -> Arrow Schema Conversion From ArrayBuilder <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1655">#1655</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Add <code>leaf_columns</code> argument to <code>ArrowReader::get_record_reader_by_columns</code> <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1653">#1653</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Implement <code>string_concat</code> kernel <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1540">#1540</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Improve Unit Test Coverage of ArrayReaderBuilder <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1484">#1484</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
</ul>
<p><strong>Fixed bugs:</strong></p>
<ul>
<li>Parquet write failure (from record batches) when data is nested two levels deep <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1744">#1744</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>IPC reader may break on projection <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1735">#1735</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Latest nightly fails to build with feature simd <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1734">#1734</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Trying to write parquet file in parallel results in corrupt file <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1717">#1717</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Roundtrip failure when using DELTA_BINARY_PACKED <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1708">#1708</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li><code>ArrayData::try_new</code> cannot always return expected error. <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1707">#1707</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>"out of order projection is not supported" after Fix Parquet Arrow Schema Inference <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1701">#1701</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Rust is not interoperability with C++ for IPC schemas with dictionaries <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1694">#1694</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
<li>Incorrect Repeated Field Schema Inference <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1681">#1681</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Parquet Treats Embedded Arrow Schema as Authoritative <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1663">#1663</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>parquet_to_arrow_schema_by_columns Incorrectly Handles Nested Types <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1654">#1654</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Inconsistent Arrow Schema When Projecting Nested Parquet File <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1652">#1652</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>StructArrayReader Cannot Handle Nested Lists <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1651">#1651</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Bug (<code>substring</code> kernel): The null buffer is not aligned when <code>offset != 0</code> <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1639">#1639</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>]</li>
</ul>
<p><strong>Documentation updates:</strong></p>
<ul>
<li>Parquet command line tool does not install "globally" <a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1710">#1710</a> [<a href="https://github.com/apache/arrow-rs/labels/parquet">parquet</a>]</li>
<li>Improve integration test document to follow Arrow C++ repo CI <a href="https://github-redirect.dependabot.com/apache/arrow-rs/pull/1742">#1742</a> [<a href="https://github.com/apache/arrow-rs/labels/arrow">arrow</a>] (<a href="https://github.com/viirya">viirya</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/apache/arrow-rs/commit/6e6a9e148c0f50ed1e0eeebd667245e67d7826b6"><code>6e6a9e1</code></a> Prepare for arrow 15.0.0 release (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1751">#1751</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/8e1666a8206f2eea4dd4e55c9365859c6a32a3f0"><code>8e1666a</code></a> Support writing nested lists to parquet (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1746">#1746</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/9722f062c2ebedeeaeccd789e9cb4808a5ad9e58"><code>9722f06</code></a> Rewrite <code>ArrayDataBuilder::null_bit_buffer</code> (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1739">#1739</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/b2657801b7381d88df70dde03fe5c5384906b2ae"><code>b265780</code></a> Rename <code>string_concat</code> to <code>concat_elements_utf8</code> (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1754">#1754</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/7e8fbf95dbb2f25830c20cd01691874e025f9527"><code>7e8fbf9</code></a> Test for list array equality with different offsets (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1756">#1756</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/3e9e9df7fb10f68607e34f1612b799d0bee3e3d6"><code>3e9e9df</code></a> support min max binary (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1725">#1725</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/949ad4396e206c857117922be9aaabe0490ddb63"><code>949ad43</code></a> rename (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1752">#1752</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/7391710c12b9876242c6321e9667af0b9eb10118"><code>7391710</code></a> Fix projection in IPC reader (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1736">#1736</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/2ba1ef417837e994e4ed820a7d339708e6db7580"><code>2ba1ef4</code></a> Implementation string concat (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1720">#1720</a>)</li>
<li><a href="https://github.com/apache/arrow-rs/commit/105be5a22184aa9a606deb5937be4a334e7ba355"><code>105be5a</code></a> Update integration test readme (<a href="https://github-redirect.dependabot.com/apache/arrow-rs/issues/1742">#1742</a>)</li>
<li>Additional commits viewable in <a href="https://github.com/apache/arrow-rs/compare/14.0.0...15.0.0">compare view</a></li>
</ul>
</details>
<br />
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] dependabot[bot] commented on pull request #2664: Update parquet requirement from 14.0.0 to 15.0.0
Posted by GitBox <gi...@apache.org>.
dependabot[bot] commented on PR #2664:
URL: https://github.com/apache/arrow-datafusion/pull/2664#issuecomment-1141837419
OK, I won't notify you again about this release, but will get in touch when a new version is available. If you'd rather skip all updates until the next major or minor version, let me know by commenting `@dependabot ignore this major version` or `@dependabot ignore this minor version`. You can also ignore all major, minor, or patch releases for a dependency by adding an [`ignore` condition](https://docs.github.com/en/code-security/supply-chain-security/configuration-options-for-dependency-updates#ignore) with the desired `update_types` to your config file.
If you change your mind, just re-open this PR and I'll resolve any conflicts on it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] alamb commented on pull request #2664: Update parquet requirement from 14.0.0 to 15.0.0
Posted by GitBox <gi...@apache.org>.
alamb commented on PR #2664:
URL: https://github.com/apache/arrow-datafusion/pull/2664#issuecomment-1142059759
Closed by https://github.com/apache/arrow-datafusion/pull/2631
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [arrow-datafusion] tustvold closed pull request #2664: Update parquet requirement from 14.0.0 to 15.0.0
Posted by GitBox <gi...@apache.org>.
tustvold closed pull request #2664: Update parquet requirement from 14.0.0 to 15.0.0
URL: https://github.com/apache/arrow-datafusion/pull/2664
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org