You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@arrow.apache.org by al...@apache.org on 2022/02/14 12:06:17 UTC
[arrow-datafusion] branch master updated: Update CHANGELOG.md, update release scripts (#1807)
This is an automated email from the ASF dual-hosted git repository.
alamb pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow-datafusion.git
The following commit(s) were added to refs/heads/master by this push:
new 81e76ed Update CHANGELOG.md, update release scripts (#1807)
81e76ed is described below
commit 81e76edc330067332ac9a3972525ecbe92953267
Author: Andrew Lamb <an...@nerdnetworks.org>
AuthorDate: Mon Feb 14 07:06:10 2022 -0500
Update CHANGELOG.md, update release scripts (#1807)
---
CHANGELOG.md | 1 -
datafusion/CHANGELOG.md | 287 ++++++++++++++++++++++++++++++++
dev/release/README.md | 4 +-
dev/release/update_change_log-all.sh | 1 -
dev/release/update_change_log-python.sh | 32 ----
dev/release/update_change_log.sh | 8 +-
6 files changed, 293 insertions(+), 40 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 467cddc..ed77f4b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -21,7 +21,6 @@ Changelogs are maintained separately for each subproject. Please check out the
changelog file within each subproject folder for more details:
* [Datafusion CHANGELOG](./datafusion/CHANGELOG.md)
-* [Datafusion Python Binding CHANGELOG](./python/CHANGELOG.md)
* [Ballista CHANGELOG](./ballista/CHANGELOG.md)
For older versions, see [apache/arrow/CHANGELOG.md](https://github.com/apache/arrow/blob/master/CHANGELOG.md).
diff --git a/datafusion/CHANGELOG.md b/datafusion/CHANGELOG.md
index c22b055..8e04f75 100644
--- a/datafusion/CHANGELOG.md
+++ b/datafusion/CHANGELOG.md
@@ -19,6 +19,293 @@
# Changelog
+## [7.0.0](https://github.com/apache/arrow-datafusion/tree/7.0.0) (2022-02-10)
+
+[Full Changelog](https://github.com/apache/arrow-datafusion/compare/6.0.0...7.0.0)
+
+**Breaking changes:**
+
+- Consolidate various configurations options, remove unrelated `batch_size` [\#1565](https://github.com/apache/arrow-datafusion/issues/1565)
+- Extract logical plans in LogicalPlan as independent struct [\#1228](https://github.com/apache/arrow-datafusion/issues/1228)
+- Update `ExecutionPlan` to know about sortedness and repartitioning optimizer pass respect the invariants [\#1776](https://github.com/apache/arrow-datafusion/pull/1776) ([alamb](https://github.com/alamb))
+- Update to `arrow 8.0.0` [\#1673](https://github.com/apache/arrow-datafusion/pull/1673) ([alamb](https://github.com/alamb))
+- Remove non idiomatic `DataFusionError::into_arrow_external_error` in favor of From conversion [\#1645](https://github.com/apache/arrow-datafusion/pull/1645) ([alamb](https://github.com/alamb))
+- Remove `Accumulator::update` and `Accumulator::merge` [\#1582](https://github.com/apache/arrow-datafusion/pull/1582) ([Jimexist](https://github.com/Jimexist))
+- implement `Hash` for various types and replace `PartialOrd` [\#1580](https://github.com/apache/arrow-datafusion/pull/1580) ([Jimexist](https://github.com/Jimexist))
+- Replace `DatafusionError` with `GenericError` in `ObjectStore` interface [\#1541](https://github.com/apache/arrow-datafusion/pull/1541) ([matthewmturner](https://github.com/matthewmturner))
+- Make `FLOAT` SQL type map to `Float32` rather than `Float64` [\#1423](https://github.com/apache/arrow-datafusion/pull/1423) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515))
+- Map `REAL` SQL type to `Float32` rather than `Float64` to be consistent with pg [\#1390](https://github.com/apache/arrow-datafusion/pull/1390) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([hntd187](https://github.com/hntd187))
+
+**Implemented enhancements:**
+
+- Create new `datafusion_expr` crate [\#1753](https://github.com/apache/arrow-datafusion/issues/1753)
+- Create new `datafusion_common` crate [\#1752](https://github.com/apache/arrow-datafusion/issues/1752)
+- API to get Expr's type and nullability without a `DFSchema` [\#1725](https://github.com/apache/arrow-datafusion/issues/1725)
+- Cleaner API to create `Expr::ScalarFunction` programatically [\#1718](https://github.com/apache/arrow-datafusion/issues/1718)
+- Introduce a `Vec<u8>` based row-wise representation for DataFusion [\#1708](https://github.com/apache/arrow-datafusion/issues/1708)
+- Simplify creating new `ListingTable` [\#1705](https://github.com/apache/arrow-datafusion/issues/1705)
+- Implement TableProvider for DataFrameImpl to allow registration of logical plans [\#1698](https://github.com/apache/arrow-datafusion/issues/1698)
+- Public Expr simplification API [\#1694](https://github.com/apache/arrow-datafusion/issues/1694)
+- Query Optimizer: Add OUTER --\> INNER join conversion [\#1670](https://github.com/apache/arrow-datafusion/issues/1670)
+- Support reading from CSV, Avro and Json files that have mergeable/compatible, but not identical schemas [\#1669](https://github.com/apache/arrow-datafusion/issues/1669)
+- Remove `DataFusionError::into_arrow_external_error` in favor of `From` conversion [\#1644](https://github.com/apache/arrow-datafusion/issues/1644)
+- Include join type in display implementation for logical plan [\#1620](https://github.com/apache/arrow-datafusion/issues/1620)
+- Switch datafusion to using `eq_dyn_scalar`, etc kernels [\#1610](https://github.com/apache/arrow-datafusion/issues/1610)
+- Proposal: Remove `Accumulator::update` and `Accumulator::merge` [\#1549](https://github.com/apache/arrow-datafusion/issues/1549)
+- Replace DataFusionError/Result with impl Error for ObjectStore and Reader [\#1540](https://github.com/apache/arrow-datafusion/issues/1540)
+- Add `approx_quantile` support [\#1538](https://github.com/apache/arrow-datafusion/issues/1538)
+- support sorting decimal data type [\#1522](https://github.com/apache/arrow-datafusion/issues/1522)
+- Keep all datafusion's packages up to date with Dependabot [\#1472](https://github.com/apache/arrow-datafusion/issues/1472)
+- ExecutionContext support init ExecutionContextState with `new(state: Arc<Mutex<ExecutionContextState>>)` method [\#1439](https://github.com/apache/arrow-datafusion/issues/1439)
+- support the decimal scalar value [\#1393](https://github.com/apache/arrow-datafusion/issues/1393)
+- Documentation for using scalar functions with the the DataFrame API [\#1364](https://github.com/apache/arrow-datafusion/issues/1364)
+- Support `boolean == boolean` and `boolean != boolean` operators [\#1159](https://github.com/apache/arrow-datafusion/issues/1159)
+- Support DataType::Decimal\(15, 2\) in TPC-H benchmark [\#174](https://github.com/apache/arrow-datafusion/issues/174)
+- Make `MemoryStream` public [\#150](https://github.com/apache/arrow-datafusion/issues/150)
+- Add support for Parquet schema merging [\#132](https://github.com/apache/arrow-datafusion/issues/132)
+- Add SQL support for IN expression [\#118](https://github.com/apache/arrow-datafusion/issues/118)
+- Add logging to datafusion-cli [\#1789](https://github.com/apache/arrow-datafusion/pull/1789) ([alamb](https://github.com/alamb))
+- Add `approx_median()` aggregate function [\#1729](https://github.com/apache/arrow-datafusion/pull/1729) ([realno](https://github.com/realno))
+- Add join type for logical plan display [\#1674](https://github.com/apache/arrow-datafusion/pull/1674) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- Fix null comparison for Parquet pruning predicate [\#1595](https://github.com/apache/arrow-datafusion/pull/1595) ([viirya](https://github.com/viirya))
+- Add `corr` aggregate function [\#1561](https://github.com/apache/arrow-datafusion/pull/1561) ([realno](https://github.com/realno))
+- Add `covar`, `covar_pop` and `covar_samp` aggregate functions [\#1551](https://github.com/apache/arrow-datafusion/pull/1551) ([realno](https://github.com/realno))
+- Add `approx_quantile()` aggregation function [\#1539](https://github.com/apache/arrow-datafusion/pull/1539) ([domodwyer](https://github.com/domodwyer))
+- Initial MemoryManager and DiskManager APIs for query execution + External Sort implementation [\#1526](https://github.com/apache/arrow-datafusion/pull/1526) ([yjshen](https://github.com/yjshen))
+- Add `stddev` and `variance` [\#1525](https://github.com/apache/arrow-datafusion/pull/1525) ([realno](https://github.com/realno))
+- Add `rem` operation for Expr [\#1467](https://github.com/apache/arrow-datafusion/pull/1467) ([liukun4515](https://github.com/liukun4515))
+- support decimal data type in create table [\#1431](https://github.com/apache/arrow-datafusion/pull/1431) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515))
+- Ordering by index in select expression [\#1419](https://github.com/apache/arrow-datafusion/pull/1419) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([hntd187](https://github.com/hntd187))
+- Add support for `ORDER BY` on unprojected columns [\#1415](https://github.com/apache/arrow-datafusion/pull/1415) ([viirya](https://github.com/viirya))
+- Support decimal for `min` and `max` aggregate [\#1407](https://github.com/apache/arrow-datafusion/pull/1407) ([liukun4515](https://github.com/liukun4515))
+- Consolidate `ConstantFolding` and `SimplifyExpression` [\#1375](https://github.com/apache/arrow-datafusion/pull/1375) ([alamb](https://github.com/alamb))
+- Datafusion cli quiet mode command to contain option bool [\#1345](https://github.com/apache/arrow-datafusion/pull/1345) ([Jimexist](https://github.com/Jimexist))
+- Implement `array_agg` aggregate function [\#1300](https://github.com/apache/arrow-datafusion/pull/1300) ([viirya](https://github.com/viirya))
+- Add a command to switch output format in cli [\#1284](https://github.com/apache/arrow-datafusion/pull/1284) ([capkurmagati](https://github.com/capkurmagati))
+- Support `=`, `<`, `<=`, `>`, `>=`, `!=`, `is distinct from`, `is not distinct from` for `BooleanArray` [\#1163](https://github.com/apache/arrow-datafusion/pull/1163) ([alamb](https://github.com/alamb))
+
+**Fixed bugs:**
+
+- Unsupported data type in hasher: Timestamp\(Second, None\) [\#1768](https://github.com/apache/arrow-datafusion/issues/1768)
+- SQL column identifiers should be converted to lowercase when unquoted [\#1746](https://github.com/apache/arrow-datafusion/issues/1746)
+- Data type Dictionary\(Int32, Utf8\) not supported for binary operation 'eq' on dyn arrays [\#1605](https://github.com/apache/arrow-datafusion/issues/1605)
+- datafusion doesn't process predicate pushdown correctly when there is outer join [\#1586](https://github.com/apache/arrow-datafusion/issues/1586)
+- casting `Int64` to `Float64` unsuccessfully caused tpch8 to fail [\#1576](https://github.com/apache/arrow-datafusion/issues/1576)
+- CTE/WITH .. UNION ALL confuses name resolution in WHERE [\#1509](https://github.com/apache/arrow-datafusion/issues/1509)
+- ORDER BY min\(x\) results in error `Plan("No field named 'foo.x'. Valid fields are 'MIN(foo.x)'.")` [\#1479](https://github.com/apache/arrow-datafusion/issues/1479)
+- Sort discards field metadata on the output schema [\#1476](https://github.com/apache/arrow-datafusion/issues/1476)
+- Datafusion should not strip out timezone information from existing types [\#1454](https://github.com/apache/arrow-datafusion/issues/1454)
+- Error on some queries: "column types must match schema types, expected XXX but found YYY" [\#1447](https://github.com/apache/arrow-datafusion/issues/1447)
+- Query failing to return any results when filter is an equality check on strings \(bad statistics in parquet\) [\#1433](https://github.com/apache/arrow-datafusion/issues/1433)
+- Field names containing period such as `f.c1` cannot be named in SQL query [\#1432](https://github.com/apache/arrow-datafusion/issues/1432)
+- `Select *` returns an unexpected result [\#1412](https://github.com/apache/arrow-datafusion/issues/1412)
+- Turn off unused default features of chrono and ahash [\#1398](https://github.com/apache/arrow-datafusion/issues/1398)
+- real data type is float32 in PG database, but in the datafusion it is as float64 [\#1380](https://github.com/apache/arrow-datafusion/issues/1380)
+- TPC-H q10 performance regression \(expression for filter with added alias is not pushed down\) [\#1367](https://github.com/apache/arrow-datafusion/issues/1367)
+- ProjectionExec Loses Field Metadata [\#1361](https://github.com/apache/arrow-datafusion/issues/1361)
+- Support Filter on unprojected columns [\#1351](https://github.com/apache/arrow-datafusion/issues/1351)
+- NULLS ORDER is inconsistent with postgres [\#1343](https://github.com/apache/arrow-datafusion/issues/1343)
+- Fix bug while merging `RecordBatch`, add `SortPreservingMerge` fuzz tester [\#1678](https://github.com/apache/arrow-datafusion/pull/1678) ([alamb](https://github.com/alamb))
+- fix a cte block with same name for many times [\#1639](https://github.com/apache/arrow-datafusion/pull/1639) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- fix: casting Int64 to Float64 unsuccessfully caused tpch8 to fail [\#1601](https://github.com/apache/arrow-datafusion/pull/1601) ([xudong963](https://github.com/xudong963))
+- Fix single\_distinct\_to\_groupby for arbitrary expressions [\#1519](https://github.com/apache/arrow-datafusion/pull/1519) ([james727](https://github.com/james727))
+- Fix SortExec discards field metadata on the output schema [\#1477](https://github.com/apache/arrow-datafusion/pull/1477) ([alamb](https://github.com/alamb))
+- fix calculate in many\_to\_many\_hash\_partition test. [\#1463](https://github.com/apache/arrow-datafusion/pull/1463) ([Ted-Jiang](https://github.com/Ted-Jiang))
+- Add Timezone to Scalar::Time\* types, and better timezone awareness to Datafusion's time types [\#1455](https://github.com/apache/arrow-datafusion/pull/1455) ([maxburke](https://github.com/maxburke))
+- Support identifiers with `.` in them [\#1449](https://github.com/apache/arrow-datafusion/pull/1449) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([alamb](https://github.com/alamb))
+- Fixes for working with functions in dataframes, additional documentation [\#1430](https://github.com/apache/arrow-datafusion/pull/1430) ([tobyhede](https://github.com/tobyhede))
+- \[Minor\] Fix `send_time` metric for hash-repartition [\#1421](https://github.com/apache/arrow-datafusion/pull/1421) ([Dandandan](https://github.com/Dandandan))
+- fix: Select \* returns an unexpected result [\#1413](https://github.com/apache/arrow-datafusion/pull/1413) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- Make cli handle multiple whitespaces [\#1388](https://github.com/apache/arrow-datafusion/pull/1388) ([capkurmagati](https://github.com/capkurmagati))
+- Metadata is kept in projections for non-derived columns [\#1378](https://github.com/apache/arrow-datafusion/pull/1378) ([hntd187](https://github.com/hntd187))
+- Fix Predicate Pushdown: split\_members should be able to split aliased predicate [\#1368](https://github.com/apache/arrow-datafusion/pull/1368) ([viirya](https://github.com/viirya))
+- Change the arg names and make parameters more meaningful [\#1357](https://github.com/apache/arrow-datafusion/pull/1357) ([liukun4515](https://github.com/liukun4515))
+- collect table stats by default for listing table [\#1347](https://github.com/apache/arrow-datafusion/pull/1347) ([houqp](https://github.com/houqp))
+- fix: make nulls-order consistent with postgres [\#1344](https://github.com/apache/arrow-datafusion/pull/1344) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- Avoid changing expression names during constant folding [\#1319](https://github.com/apache/arrow-datafusion/pull/1319) ([viirya](https://github.com/viirya))
+- improve error message for invalid create table statement [\#1294](https://github.com/apache/arrow-datafusion/pull/1294) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([houqp](https://github.com/houqp))
+- Forbid creating the table with the same name [\#1288](https://github.com/apache/arrow-datafusion/pull/1288) ([liukun4515](https://github.com/liukun4515))
+
+**Documentation updates:**
+
+- Clarify docs about `Accumulator::update` and `Accumulator::update_batch` [\#1542](https://github.com/apache/arrow-datafusion/pull/1542) ([alamb](https://github.com/alamb))
+- Fix duplicated `cargo run --example parquet_sql` [\#1482](https://github.com/apache/arrow-datafusion/pull/1482) ([sergey-melnychuk](https://github.com/sergey-melnychuk))
+- add documentation to Datafusion cli's new commands [\#1348](https://github.com/apache/arrow-datafusion/pull/1348) ([liukun4515](https://github.com/liukun4515))
+- fix some clippy warnings from nightly channel [\#1277](https://github.com/apache/arrow-datafusion/pull/1277) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([Jimexist](https://github.com/Jimexist))
+
+**Performance improvements:**
+
+- Parquet pruning predicate for `IS NULL` [\#1591](https://github.com/apache/arrow-datafusion/issues/1591)
+- Fix predicate pushdown for outer joins [\#1618](https://github.com/apache/arrow-datafusion/pull/1618) ([james727](https://github.com/james727))
+- fix: sql planner creates cross join instead of inner join from select predicates [\#1566](https://github.com/apache/arrow-datafusion/pull/1566) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- Split fetch\_metadata into fetch\_statistics and fetch\_schema [\#1365](https://github.com/apache/arrow-datafusion/pull/1365) ([Dandandan](https://github.com/Dandandan))
+- Optimize the performance queries with a single distinct aggregate [\#1315](https://github.com/apache/arrow-datafusion/pull/1315) ([ic4y](https://github.com/ic4y))
+- Left join could use bitmap for left join instead of Vec\<bool\> [\#1291](https://github.com/apache/arrow-datafusion/pull/1291) ([boazberman](https://github.com/boazberman))
+
+**Closed issues:**
+
+- Add `release compile` to CI [\#1728](https://github.com/apache/arrow-datafusion/issues/1728)
+- DiskManager and TempFiles getting created several times per query [\#1690](https://github.com/apache/arrow-datafusion/issues/1690)
+- Add a test for the `pyarrow` feature in CI [\#1635](https://github.com/apache/arrow-datafusion/issues/1635)
+- SQL tests for when sorting exceeded available memory and had to spill to disk [\#1573](https://github.com/apache/arrow-datafusion/issues/1573)
+- Consolidate the N-way merging code and `SortPreservingMergeStream` \(which has quite good tests of what is often quite tricky code, and it will be performance critical\) [\#1572](https://github.com/apache/arrow-datafusion/issues/1572)
+- Consolidate the `SortExec` code \(so there is only a single sort operator that does in memory sorting if it has enough memory budget but then spills to disk if needed\). [\#1571](https://github.com/apache/arrow-datafusion/issues/1571)
+- Track memory usage in Non Limited Operators [\#1569](https://github.com/apache/arrow-datafusion/issues/1569)
+- \[Question\] Why does ballista store tables in the client instead of in the SchedulerServer [\#1473](https://github.com/apache/arrow-datafusion/issues/1473)
+- Consolidate Projection for Schema and RecordBatch [\#1425](https://github.com/apache/arrow-datafusion/issues/1425)
+- Support Sort on unprojected columns [\#1372](https://github.com/apache/arrow-datafusion/issues/1372)
+- Unused code in hash\_aggregate [\#1362](https://github.com/apache/arrow-datafusion/issues/1362)
+- Why use the expr types before coercion to get the result type? [\#1358](https://github.com/apache/arrow-datafusion/issues/1358)
+- A problem about the projection\_push\_down optimizer gathers valid columns [\#1312](https://github.com/apache/arrow-datafusion/issues/1312)
+- apply constant folding to `LogicalPlan::Values` [\#1170](https://github.com/apache/arrow-datafusion/issues/1170)
+- reduce usage of `IntoIterator<Item = Expr>` in logical plan builder window fn [\#372](https://github.com/apache/arrow-datafusion/issues/372)
+- Why does DataFusion throw a Tokio 0.2 runtime error? [\#176](https://github.com/apache/arrow-datafusion/issues/176)
+- TPC-H Query 14 [\#165](https://github.com/apache/arrow-datafusion/issues/165)
+- Length kernel returns bytes not character length [\#156](https://github.com/apache/arrow-datafusion/issues/156)
+- Split the logical operators out into separate source files [\#115](https://github.com/apache/arrow-datafusion/issues/115)
+
+**Merged pull requests:**
+
+- Update versions for split crates [\#1803](https://github.com/apache/arrow-datafusion/pull/1803) ([matthewmturner](https://github.com/matthewmturner))
+- Improve the error message and UX of tpch benchmark program [\#1800](https://github.com/apache/arrow-datafusion/pull/1800) ([alamb](https://github.com/alamb))
+- rename references of expr in logical plan module after datafusion-expr split [\#1797](https://github.com/apache/arrow-datafusion/pull/1797) ([Jimexist](https://github.com/Jimexist))
+- \[split/13\] move rest of expr to expr\_fn in datafusion-expr module [\#1794](https://github.com/apache/arrow-datafusion/pull/1794) ([Jimexist](https://github.com/Jimexist))
+- Update datafusion versions [\#1793](https://github.com/apache/arrow-datafusion/pull/1793) ([matthewmturner](https://github.com/matthewmturner))
+- Less verbose plans in debug logging [\#1787](https://github.com/apache/arrow-datafusion/pull/1787) ([alamb](https://github.com/alamb))
+- \[split/11\] split expr type and null info to be expr-schemable [\#1784](https://github.com/apache/arrow-datafusion/pull/1784) ([Jimexist](https://github.com/Jimexist))
+- Introduce `Row` format backed by raw bytes [\#1782](https://github.com/apache/arrow-datafusion/pull/1782) ([yjshen](https://github.com/yjshen))
+- rewrite predicates before pushing to union inputs [\#1781](https://github.com/apache/arrow-datafusion/pull/1781) ([korowa](https://github.com/korowa))
+- \[split/10\] split up expr for rewriting, visiting, and simplification traits [\#1774](https://github.com/apache/arrow-datafusion/pull/1774) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([Jimexist](https://github.com/Jimexist))
+- \#1768 Support TimeUnit::Second in hasher [\#1769](https://github.com/apache/arrow-datafusion/pull/1769) ([jychen7](https://github.com/jychen7))
+- TPC-H benchmark can optionally write JSON output file with benchmark summary [\#1766](https://github.com/apache/arrow-datafusion/pull/1766) ([andygrove](https://github.com/andygrove))
+- \[split/8\] move `Accumulator` and `ColumnarValue` to datafusion-expr [\#1765](https://github.com/apache/arrow-datafusion/pull/1765) ([Jimexist](https://github.com/Jimexist))
+- \[split/7\] move built-in scalar function to datafusion-expr [\#1764](https://github.com/apache/arrow-datafusion/pull/1764) ([Jimexist](https://github.com/Jimexist))
+- \[split/6\] move signature, type signature, volatility to datafusion-expr [\#1763](https://github.com/apache/arrow-datafusion/pull/1763) ([Jimexist](https://github.com/Jimexist))
+- \[split/9+12\] move udf, udaf, `Expr` to datafusion-expr module [\#1762](https://github.com/apache/arrow-datafusion/pull/1762) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([Jimexist](https://github.com/Jimexist))
+- \[split/5\] move window frame and operator to datafusion-expr module [\#1761](https://github.com/apache/arrow-datafusion/pull/1761) ([Jimexist](https://github.com/Jimexist))
+- \[split/4\] move scalar value to datafusion-common [\#1760](https://github.com/apache/arrow-datafusion/pull/1760) ([Jimexist](https://github.com/Jimexist))
+- \[split/3\] split datafusion expr module and move aggregate and window function expr [\#1759](https://github.com/apache/arrow-datafusion/pull/1759) ([Jimexist](https://github.com/Jimexist))
+- \[split/2\] move column and dfschema to datafusion-common module [\#1758](https://github.com/apache/arrow-datafusion/pull/1758) ([Jimexist](https://github.com/Jimexist))
+- Use ordered-float 2.10 [\#1756](https://github.com/apache/arrow-datafusion/pull/1756) ([andygrove](https://github.com/andygrove))
+- \[split/1\] split datafusion-common module [\#1751](https://github.com/apache/arrow-datafusion/pull/1751) ([Jimexist](https://github.com/Jimexist))
+- use clap 3 style args parsing for datafusion cli [\#1749](https://github.com/apache/arrow-datafusion/pull/1749) ([Jimexist](https://github.com/Jimexist))
+- fix: Case insensitive unquoted identifiers in SQL [\#1747](https://github.com/apache/arrow-datafusion/pull/1747) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([mkmik](https://github.com/mkmik))
+- Move more tests out of context.rs [\#1743](https://github.com/apache/arrow-datafusion/pull/1743) ([alamb](https://github.com/alamb))
+- Move optimize test out of context.rs [\#1742](https://github.com/apache/arrow-datafusion/pull/1742) ([alamb](https://github.com/alamb))
+- Fix typos in crate documentation [\#1739](https://github.com/apache/arrow-datafusion/pull/1739) ([r4ntix](https://github.com/r4ntix))
+- add `cargo check --release` to ci [\#1737](https://github.com/apache/arrow-datafusion/pull/1737) ([xudong963](https://github.com/xudong963))
+- Update parking\_lot requirement from 0.11 to 0.12 [\#1735](https://github.com/apache/arrow-datafusion/pull/1735) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Create built-in scalar functions programmatically [\#1734](https://github.com/apache/arrow-datafusion/pull/1734) ([HaoYang670](https://github.com/HaoYang670))
+- Prevent repartitioning of certain operator's direct children \(\#1731\) [\#1732](https://github.com/apache/arrow-datafusion/pull/1732) ([tustvold](https://github.com/tustvold))
+- API to get Expr's type and nullability without a `DFSchema` [\#1726](https://github.com/apache/arrow-datafusion/pull/1726) ([alamb](https://github.com/alamb))
+- minor: fix `cargo run --release` error [\#1723](https://github.com/apache/arrow-datafusion/pull/1723) ([xudong963](https://github.com/xudong963))
+- substitute `parking_lot::Mutex` for `std::sync::Mutex` [\#1720](https://github.com/apache/arrow-datafusion/pull/1720) ([xudong963](https://github.com/xudong963))
+- Convert boolean case expressions to boolean logic [\#1719](https://github.com/apache/arrow-datafusion/pull/1719) ([tustvold](https://github.com/tustvold))
+- Add Expression Simplification API [\#1717](https://github.com/apache/arrow-datafusion/pull/1717) ([alamb](https://github.com/alamb))
+- Create ListingTableConfig which includes file format and schema inference [\#1715](https://github.com/apache/arrow-datafusion/pull/1715) ([matthewmturner](https://github.com/matthewmturner))
+- make `select_to_plan` clearer [\#1714](https://github.com/apache/arrow-datafusion/pull/1714) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- Add upper bound for public function `signature` [\#1713](https://github.com/apache/arrow-datafusion/pull/1713) ([HaoYang670](https://github.com/HaoYang670))
+- Add tests and CI for optional pyarrow module [\#1711](https://github.com/apache/arrow-datafusion/pull/1711) ([wjones127](https://github.com/wjones127))
+- Create SchemaAdapter trait to map table schema to file schemas [\#1709](https://github.com/apache/arrow-datafusion/pull/1709) ([thinkharderdev](https://github.com/thinkharderdev))
+- refine test in repartition.rs & coalesce\_batches.rs [\#1707](https://github.com/apache/arrow-datafusion/pull/1707) ([xudong963](https://github.com/xudong963))
+- Fuzz test for spillable sort [\#1706](https://github.com/apache/arrow-datafusion/pull/1706) ([yjshen](https://github.com/yjshen))
+- Support `create_physical_expr` and `ExecutionContextState` or `DefaultPhysicalPlanner` for faster speed [\#1700](https://github.com/apache/arrow-datafusion/pull/1700) ([alamb](https://github.com/alamb))
+- Implement TableProvider for DataFrameImpl [\#1699](https://github.com/apache/arrow-datafusion/pull/1699) ([cpcloud](https://github.com/cpcloud))
+- Move timestamp related tests out of context.rs and into sql integration test [\#1696](https://github.com/apache/arrow-datafusion/pull/1696) ([alamb](https://github.com/alamb))
+- Lazy TempDir creation in DiskManager [\#1695](https://github.com/apache/arrow-datafusion/pull/1695) ([alamb](https://github.com/alamb))
+- Add `MemTrackingMetrics` to ease memory tracking for non-limited memory consumers [\#1691](https://github.com/apache/arrow-datafusion/pull/1691) ([yjshen](https://github.com/yjshen))
+- \(minor\) Reduce memory manager and disk manager logs from `info!` to `debug!` [\#1689](https://github.com/apache/arrow-datafusion/pull/1689) ([alamb](https://github.com/alamb))
+- Make `SortPreservingMergeStream` stable on input stream order [\#1687](https://github.com/apache/arrow-datafusion/pull/1687) ([alamb](https://github.com/alamb))
+- Incorporate dyn scalar kernels [\#1685](https://github.com/apache/arrow-datafusion/pull/1685) ([matthewmturner](https://github.com/matthewmturner))
+- Move `information_schema` tests out of execution/context.rs to `sql_integration` tests [\#1684](https://github.com/apache/arrow-datafusion/pull/1684) ([alamb](https://github.com/alamb))
+- Add a new metric type: `Gauge` + `CurrentMemoryUsage` to metrics [\#1682](https://github.com/apache/arrow-datafusion/pull/1682) ([yjshen](https://github.com/yjshen))
+- refactor array\_agg to not to have `update` and `merge` [\#1681](https://github.com/apache/arrow-datafusion/pull/1681) ([Jimexist](https://github.com/Jimexist))
+- Use NamedTempFile rather than `String` in DiskManager [\#1680](https://github.com/apache/arrow-datafusion/pull/1680) ([alamb](https://github.com/alamb))
+- upgrade clap to version 3 [\#1672](https://github.com/apache/arrow-datafusion/pull/1672) ([Jimexist](https://github.com/Jimexist))
+- Improve configuration and resource use of `MemoryManager` and `DiskManager` [\#1668](https://github.com/apache/arrow-datafusion/pull/1668) ([alamb](https://github.com/alamb))
+- feat: Support quarter granularity in date\_trunc function [\#1667](https://github.com/apache/arrow-datafusion/pull/1667) ([ovr](https://github.com/ovr))
+- Fix can not load parquet table form spark in datafusion-cli. [\#1665](https://github.com/apache/arrow-datafusion/pull/1665) ([Ted-Jiang](https://github.com/Ted-Jiang))
+- Make `MemoryManager` and `MemoryStream` public [\#1664](https://github.com/apache/arrow-datafusion/pull/1664) ([yjshen](https://github.com/yjshen))
+- \[Cleanup\] Move `AggregatedMetricsSet` to `metrics` for further reuse [\#1663](https://github.com/apache/arrow-datafusion/pull/1663) ([yjshen](https://github.com/yjshen))
+- fix: substr - correct behaivour with negative start pos [\#1660](https://github.com/apache/arrow-datafusion/pull/1660) ([ovr](https://github.com/ovr))
+- suppport bitwise and as an example [\#1653](https://github.com/apache/arrow-datafusion/pull/1653) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515))
+- refine match pattern related code [\#1650](https://github.com/apache/arrow-datafusion/pull/1650) ([xudong963](https://github.com/xudong963))
+- update md-5, sha2, blake2 [\#1647](https://github.com/apache/arrow-datafusion/pull/1647) ([xudong963](https://github.com/xudong963))
+- Add `DataFusionError` -\> `ArrowError` conversion [\#1643](https://github.com/apache/arrow-datafusion/pull/1643) ([alamb](https://github.com/alamb))
+- Add `spill_count` and `spilled_bytes` to `BaselineMetrics`, test sort with spill [\#1641](https://github.com/apache/arrow-datafusion/pull/1641) ([yjshen](https://github.com/yjshen))
+- support hash decimal array and group by [\#1640](https://github.com/apache/arrow-datafusion/pull/1640) ([liukun4515](https://github.com/liukun4515))
+- Consolidate Schema and RecordBatch projection [\#1638](https://github.com/apache/arrow-datafusion/pull/1638) ([alamb](https://github.com/alamb))
+- Update hashbrown requirement from 0.11 to 0.12 [\#1631](https://github.com/apache/arrow-datafusion/pull/1631) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Update pyo3 requirement from 0.14 to 0.15 [\#1627](https://github.com/apache/arrow-datafusion/pull/1627) ([dependabot[bot]](https://github.com/apps/dependabot))
+- Optimize `SortPreservingMergeStream` to avoid `SortKeyCursor` sharing [\#1624](https://github.com/apache/arrow-datafusion/pull/1624) ([yjshen](https://github.com/yjshen))
+- Handle merging of evolved schemas in ParquetExec [\#1622](https://github.com/apache/arrow-datafusion/pull/1622) ([thinkharderdev](https://github.com/thinkharderdev))
+- feat: Support Substring\(str \[from int\] \[for int\]\) [\#1621](https://github.com/apache/arrow-datafusion/pull/1621) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([ovr](https://github.com/ovr))
+- feat: Support complex interval via IntervalMonthDayNano [\#1615](https://github.com/apache/arrow-datafusion/pull/1615) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([ovr](https://github.com/ovr))
+- consolidate binary\_expr coercion rule code into `binary_rule.rs` module [\#1607](https://github.com/apache/arrow-datafusion/pull/1607) ([alamb](https://github.com/alamb))
+- Fix comparison of dictionary arrays [\#1606](https://github.com/apache/arrow-datafusion/pull/1606) ([alamb](https://github.com/alamb))
+- add test for decimal to decimal [\#1603](https://github.com/apache/arrow-datafusion/pull/1603) ([liukun4515](https://github.com/liukun4515))
+- update nightly version [\#1597](https://github.com/apache/arrow-datafusion/pull/1597) ([Jimexist](https://github.com/Jimexist))
+- Consolidate sort and external\_sort [\#1596](https://github.com/apache/arrow-datafusion/pull/1596) ([yjshen](https://github.com/yjshen))
+- support from\_slice for binary, string, and boolean array types [\#1589](https://github.com/apache/arrow-datafusion/pull/1589) ([Jimexist](https://github.com/Jimexist))
+- add from\_slice trait to ease arrow2 migration [\#1588](https://github.com/apache/arrow-datafusion/pull/1588) ([Jimexist](https://github.com/Jimexist))
+- Implement ARRAY\_AGG\(DISTINCT ...\) [\#1579](https://github.com/apache/arrow-datafusion/pull/1579) ([james727](https://github.com/james727))
+- Rename sql integration tests from `mod` to `sql_integration` [\#1575](https://github.com/apache/arrow-datafusion/pull/1575) ([alamb](https://github.com/alamb))
+- minor: improve the benchmark readme [\#1567](https://github.com/apache/arrow-datafusion/pull/1567) ([xudong963](https://github.com/xudong963))
+- Consolidate `batch_size` configuration in `ExecutionConfig`, `RuntimeConfig` and `PhysicalPlanConfig` [\#1562](https://github.com/apache/arrow-datafusion/pull/1562) ([yjshen](https://github.com/yjshen))
+- Update to rust 1.58 [\#1557](https://github.com/apache/arrow-datafusion/pull/1557) ([xudong963](https://github.com/xudong963))
+- support mathematics operation for decimal data type [\#1554](https://github.com/apache/arrow-datafusion/pull/1554) ([liukun4515](https://github.com/liukun4515))
+- Address clippy warnings [\#1553](https://github.com/apache/arrow-datafusion/pull/1553) ([sergey-melnychuk](https://github.com/sergey-melnychuk))
+- enhance arithmetic operation for array with scalar [\#1552](https://github.com/apache/arrow-datafusion/pull/1552) ([liukun4515](https://github.com/liukun4515))
+- Remove unused `update` and `merge` implementations from Aggregates and supporting `ScalarValue` arithmetic [\#1550](https://github.com/apache/arrow-datafusion/pull/1550) ([alamb](https://github.com/alamb))
+- Add batch operations to stddev [\#1547](https://github.com/apache/arrow-datafusion/pull/1547) ([realno](https://github.com/realno))
+- Mark ARRAY\_AGG\(DISTINCT ...\) not implemented [\#1534](https://github.com/apache/arrow-datafusion/pull/1534) ([james727](https://github.com/james727))
+- Update to arrow-7.0.0 [\#1523](https://github.com/apache/arrow-datafusion/pull/1523) ([alamb](https://github.com/alamb))
+- Fix ORDER BY on aggregate [\#1506](https://github.com/apache/arrow-datafusion/pull/1506) ([viirya](https://github.com/viirya))
+- Add example on how to query multiple parquet files [\#1497](https://github.com/apache/arrow-datafusion/pull/1497) ([nitisht](https://github.com/nitisht))
+- Refactor testing modules [\#1491](https://github.com/apache/arrow-datafusion/pull/1491) ([hntd187](https://github.com/hntd187))
+- add rfcs for datafusion [\#1490](https://github.com/apache/arrow-datafusion/pull/1490) ([xudong963](https://github.com/xudong963))
+- support comparison for decimal data type and refactor the binary coercion rule [\#1483](https://github.com/apache/arrow-datafusion/pull/1483) ([liukun4515](https://github.com/liukun4515))
+- Minor: Rename `predicate_builder` --\> `pruning_predicate` for consistency [\#1481](https://github.com/apache/arrow-datafusion/pull/1481) ([alamb](https://github.com/alamb))
+- Tests for support try\_cast/cast decimal to numeric [\#1465](https://github.com/apache/arrow-datafusion/pull/1465) ([liukun4515](https://github.com/liukun4515))
+- Avoid send empty batches for Hash partitioning. [\#1459](https://github.com/apache/arrow-datafusion/pull/1459) ([Ted-Jiang](https://github.com/Ted-Jiang))
+- Planner code cleanup [\#1450](https://github.com/apache/arrow-datafusion/pull/1450) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([alamb](https://github.com/alamb))
+- Fix bug in projection: "column types must match schema types, expected XXX but found YYY" [\#1448](https://github.com/apache/arrow-datafusion/pull/1448) ([alamb](https://github.com/alamb))
+- Update arrow-rs to 6.4.0 and replace boolean comparison in datafusion with arrow compute kernel [\#1446](https://github.com/apache/arrow-datafusion/pull/1446) ([xudong963](https://github.com/xudong963))
+- support cast/try\_cast for decimal: signed numeric to decimal [\#1442](https://github.com/apache/arrow-datafusion/pull/1442) ([liukun4515](https://github.com/liukun4515))
+- Consolidate decimal error checking and improve error messages [\#1438](https://github.com/apache/arrow-datafusion/pull/1438) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([alamb](https://github.com/alamb))
+- use 0.13 sql parser [\#1435](https://github.com/apache/arrow-datafusion/pull/1435) ([Jimexist](https://github.com/Jimexist))
+- Minor Code cleanups [\#1428](https://github.com/apache/arrow-datafusion/pull/1428) ([alamb](https://github.com/alamb))
+- Clarify communication on bi-weekly sync [\#1427](https://github.com/apache/arrow-datafusion/pull/1427) ([alamb](https://github.com/alamb))
+- support sum/avg agg for decimal, change sum\(float32\) --\> float64 [\#1408](https://github.com/apache/arrow-datafusion/pull/1408) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515))
+- Fix bugs with nullability during rewrites: Combine `simplify` and `Simplifier` [\#1401](https://github.com/apache/arrow-datafusion/pull/1401) ([alamb](https://github.com/alamb))
+- Minimize features [\#1399](https://github.com/apache/arrow-datafusion/pull/1399) ([carols10cents](https://github.com/carols10cents))
+- Update rust vesion to 1.57 [\#1395](https://github.com/apache/arrow-datafusion/pull/1395) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- support decimal scalar value [\#1394](https://github.com/apache/arrow-datafusion/pull/1394) ([liukun4515](https://github.com/liukun4515))
+- Add coercion rules for AggregateFunctions [\#1387](https://github.com/apache/arrow-datafusion/pull/1387) ([liukun4515](https://github.com/liukun4515))
+- upgrade the arrow-rs version [\#1385](https://github.com/apache/arrow-datafusion/pull/1385) ([liukun4515](https://github.com/liukun4515))
+- add array agg name [\#1382](https://github.com/apache/arrow-datafusion/pull/1382) ([liukun4515](https://github.com/liukun4515))
+- Make tests for `simplify` and `Simplifer` consistent [\#1376](https://github.com/apache/arrow-datafusion/pull/1376) ([alamb](https://github.com/alamb))
+- Refactor: Consolidate expression simplification code in `simplify_expression.rs` [\#1374](https://github.com/apache/arrow-datafusion/pull/1374) ([alamb](https://github.com/alamb))
+- remove unused code in hash\_aggregate [\#1370](https://github.com/apache/arrow-datafusion/pull/1370) ([ic4y](https://github.com/ic4y))
+- Use `BufReader` for LocalFileReader to revert performance regression in parquet reading [\#1366](https://github.com/apache/arrow-datafusion/pull/1366) ([Dandandan](https://github.com/Dandandan))
+- Add unit test for constant folding on values [\#1355](https://github.com/apache/arrow-datafusion/pull/1355) ([viirya](https://github.com/viirya))
+- Extract logical plan: rename the plan name \(follow up\) [\#1354](https://github.com/apache/arrow-datafusion/pull/1354) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515))
+- Moved aggr\_test\_schema to test\_utils [\#1338](https://github.com/apache/arrow-datafusion/pull/1338) ([rdettai](https://github.com/rdettai))
+- upgrade arrow-rs to 6.2.0 [\#1334](https://github.com/apache/arrow-datafusion/pull/1334) ([liukun4515](https://github.com/liukun4515))
+- Update release instructions [\#1331](https://github.com/apache/arrow-datafusion/pull/1331) ([alamb](https://github.com/alamb))
+- \#1268: allow datafusion-cli to toggle quiet flag within CLI [\#1330](https://github.com/apache/arrow-datafusion/pull/1330) ([jgoday](https://github.com/jgoday))
+- Extract Aggregate, Sort, and Join to struct from AggregatePlan [\#1326](https://github.com/apache/arrow-datafusion/pull/1326) ([matthewmturner](https://github.com/matthewmturner))
+- Extract `EmptyRelation`, `Limit`, `Values` from `LogicalPlan` [\#1325](https://github.com/apache/arrow-datafusion/pull/1325) ([liukun4515](https://github.com/liukun4515))
+- Extract CrossJoin, Repartition, Union in LogicalPlan [\#1322](https://github.com/apache/arrow-datafusion/pull/1322) ([liukun4515](https://github.com/liukun4515))
+- Fifth batch of updating sql tests to use assert\_batches\_eq [\#1318](https://github.com/apache/arrow-datafusion/pull/1318) ([matthewmturner](https://github.com/matthewmturner))
+- Extract Explain, Analyze, Extension in LogicalPlan as independent struct [\#1317](https://github.com/apache/arrow-datafusion/pull/1317) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
+- Extract CreateMemoryTable, DropTable, CreateExternalTable in LogicalPlan as independent struct [\#1311](https://github.com/apache/arrow-datafusion/pull/1311) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515))
+- Extract Projection, Filter, Window in LogicalPlan as independent struct [\#1309](https://github.com/apache/arrow-datafusion/pull/1309) ([ic4y](https://github.com/ic4y))
+- Add PSQL comparison tests for except, intersect [\#1292](https://github.com/apache/arrow-datafusion/pull/1292) ([mrob95](https://github.com/mrob95))
+- Extract logical plans in LogicalPlan as independent struct: TableScan [\#1290](https://github.com/apache/arrow-datafusion/pull/1290) ([xudong963](https://github.com/xudong963))
+- Add statement helper command to cli [\#1285](https://github.com/apache/arrow-datafusion/pull/1285) ([matthewmturner](https://github.com/matthewmturner))
+- Python bindings for window functions [\#819](https://github.com/apache/arrow-datafusion/pull/819) [[sql](https://github.com/apache/arrow-datafusion/labels/sql)] ([jgoday](https://github.com/jgoday))
+
## [6.0.0](https://github.com/apache/arrow-datafusion/tree/6.0.0) (2021-11-13)
[Full Changelog](https://github.com/apache/arrow-datafusion/compare/5.0.0...6.0.0)
diff --git a/dev/release/README.md b/dev/release/README.md
index 7930e2b..86d36a0 100644
--- a/dev/release/README.md
+++ b/dev/release/README.md
@@ -21,7 +21,7 @@
## Sub-projects
-The Datafusion repo contains 3 different releasable sub-projects: Datafusion, Ballista and Datafusion python binding.
+The Datafusion repo contains 2 different releasable sub-projects: Datafusion, Ballista
We use Datafusion release to drive the release for the other sub-projects. As a
result, Datafusion version bump is required for every release while version
@@ -64,7 +64,7 @@ The CHANGELOG is managed automatically using
This script creates a changelog using github PRs and issues based on the labels
associated with them.
-## Prepare release comimts and PR
+## Prepare release commits and PR
Prepare a PR to update `CHANGELOG.md` and versions to reflect the planned
release.
diff --git a/dev/release/update_change_log-all.sh b/dev/release/update_change_log-all.sh
index c5639cc..0d4cf00 100755
--- a/dev/release/update_change_log-all.sh
+++ b/dev/release/update_change_log-all.sh
@@ -28,4 +28,3 @@ SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
${SOURCE_DIR}/update_change_log-datafusion.sh
${SOURCE_DIR}/update_change_log-ballista.sh
-${SOURCE_DIR}/update_change_log-python.sh
diff --git a/dev/release/update_change_log-python.sh b/dev/release/update_change_log-python.sh
deleted file mode 100755
index 6d428e8..0000000
--- a/dev/release/update_change_log-python.sh
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/bin/bash
-#
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-#
-
-# Usage:
-# CHANGELOG_GITHUB_TOKEN=<TOKEN> ./update_change_log-python.sh
-
-SOURCE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-SOURCE_TOP_DIR="$(cd "${SOURCE_DIR}/../../" && pwd)"
-
-CURRENT_VER=$(grep version "${SOURCE_TOP_DIR}/python/Cargo.toml" | head -n 1 | awk '{print $3}' | tr -d '"')
-${SOURCE_DIR}/update_change_log.sh \
- python \
- python-0.3.0 \
- --exclude-tags-regex "ballista-.+" \
- --future-release "python-${CURRENT_VER}"
diff --git a/dev/release/update_change_log.sh b/dev/release/update_change_log.sh
index 1d1570d..07ba6f9 100755
--- a/dev/release/update_change_log.sh
+++ b/dev/release/update_change_log.sh
@@ -50,14 +50,14 @@ OUTPUT_PATH="${PROJECT}/CHANGELOG.md"
pushd ${SOURCE_TOP_DIR}
# reset content in changelog
-git co "${SINCE_TAG}" "${OUTPUT_PATH}"
+git checkout "${SINCE_TAG}" "${OUTPUT_PATH}"
# remove license header so github-changelog-generator has a clean base to append
-sed -i '1,18d' "${OUTPUT_PATH}"
+sed -i.bak '1,18d' "${OUTPUT_PATH}"
docker run -it --rm \
-e CHANGELOG_GITHUB_TOKEN=$CHANGELOG_GITHUB_TOKEN \
-v "$(pwd)":/usr/local/src/your-app \
- githubchangeloggenerator/github-changelog-generator:1.16.2 \
+ githubchangeloggenerator/github-changelog-generator \
--user apache \
--project arrow-datafusion \
--since-tag "${SINCE_TAG}" \
@@ -66,7 +66,7 @@ docker run -it --rm \
--output "${OUTPUT_PATH}" \
"$@"
-sed -i "s/\\\n/\n\n/" "${OUTPUT_PATH}"
+sed -i.bak "s/\\\n/\n\n/" "${OUTPUT_PATH}"
echo '<!---
Licensed to the Apache Software Foundation (ASF) under one