You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "dariocurr (via GitHub)" <gi...@apache.org> on 2026/07/22 10:40:27 UTC
[PR] feat: allow configuring datafusion-cli history file location [datafusion]
dariocurr opened a new pull request, #23797:
URL: https://github.com/apache/datafusion/pull/23797
## Which issue does this PR close?
- Closes #23796.
## Rationale for this change
`datafusion-cli`'s interactive REPL hardcodes its rustyline history file as
the relative path `.history`. Because the path is relative, the history file
is created (and looked up) in whatever directory the CLI happens to be
launched from, so every working directory silently accumulates its own
hidden `.history` file with no way to point it somewhere else.
## What changes are included in this PR?
- Add a `--history-file <PATH>` CLI flag to `datafusion-cli`, validated at
parse time (rejects a path that is an existing directory, or whose parent
directory doesn't exist).
- Add a `DATAFUSION_HISTORY_FILE` environment variable as an alternative way
to set it (e.g. for scripted/containerized setups). The `--history-file`
flag takes precedence over the environment variable.
- The existing default behavior (`.history`, relative to the current
directory) is unchanged when neither is set, so this is non-breaking.
- `exec_from_repl` now takes `history_file: Option<&Path>` instead of
hardcoding `.history` twice; the `.history` default now lives in one place
inside `exec_from_repl` itself.
- Updated `datafusion-cli/examples/cli-session-context.rs` (the only other
caller of `exec_from_repl`) and `docs/source/user-guide/cli/usage.md`
accordingly.
## Are these changes tested?
This is a small, mostly mechanical CLI argument-plumbing change with no new
branching logic worth unit-testing in isolation (the added validator mirrors
the existing `parse_valid_file`/`parse_valid_data_dir` helpers, which aren't
separately unit-tested either). I manually verified end-to-end via the built
binary:
- Default (`.history` in cwd, unchanged behavior)
- `--history-file <path>` override
- `DATAFUSION_HISTORY_FILE=<path>` override
- `--history-file` taking precedence over `DATAFUSION_HISTORY_FILE` when
both are set
- `--history-file` pointing at a non-existent parent directory fails fast
with a clear error instead of a late rustyline error
Existing `datafusion-cli` unit tests pass (`cargo test -p datafusion-cli --lib --bins`).
## Are there any user-facing changes?
Yes: a new `--history-file` CLI flag and `DATAFUSION_HISTORY_FILE`
environment variable, documented in `docs/source/user-guide/cli/usage.md`.
No breaking changes; default behavior is unchanged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "dariocurr (via GitHub)" <gi...@apache.org>.
dariocurr commented on PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#issuecomment-5131142810
Is there any news on that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "dariocurr (via GitHub)" <gi...@apache.org>.
dariocurr commented on code in PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#discussion_r3650045330
##########
datafusion-cli/src/exec.rs:
##########
@@ -129,13 +130,15 @@ pub async fn exec_from_files(
pub async fn exec_from_repl(
ctx: &dyn CliSessionContext,
print_options: &mut PrintOptions,
+ history_file: Option<&Path>,
Review Comment:
Took this suggestion and pushed an update: `exec_from_repl` now takes a `ReplOptions` struct (in its own new module, `datafusion-cli/src/repl_options.rs`, mirroring how `PrintOptions` gets its own module) instead of the raw `Option<&Path>`.
I kept it as a plain, exhaustively-constructed struct rather than adding `#[non_exhaustive]` + partial-construction-via-`Default`, so it stays consistent with `PrintOptions`'s existing convention in this same crate (plain struct, named-field construction at each call site) rather than introducing a second, different pattern for "options" structs. So this doesn't fully close the door on future signature churn the way your suggestion would, but it does give us a dedicated place to add REPL-specific options later without touching `PrintOptions`, and it let me centralize both the `.history` default and the `DATAFUSION_HISTORY_FILE` env var resolution into `ReplOptions::default()` instead of duplicating that logic across callers. Thanks for the pointer to the pattern either way!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#issuecomment-5045410843
<!-- semver-check-comment -->
Thank you for opening this pull request!
Reviewer note: [cargo-semver-checks](https://github.com/obi1kenobi/cargo-semver-checks) reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).
<details>
<summary>Details</summary>
```
Cloning apache/main
Building datafusion-cli v54.1.0 (current)
Built [ 198.130s] (current)
Parsing datafusion-cli v54.1.0 (current)
Parsed [ 0.035s] (current)
Building datafusion-cli v54.1.0 (baseline)
Built [ 190.643s] (baseline)
Parsing datafusion-cli v54.1.0 (baseline)
Parsed [ 0.033s] (baseline)
Checking datafusion-cli v54.1.0 -> v54.1.0 (no change; assume patch)
Checked [ 0.120s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip
--- failure function_parameter_count_changed: pub fn parameter count changed ---
Description:
A publicly-visible function now takes a different number of parameters.
ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/function_parameter_count_changed.ron
Failed in:
datafusion_cli::exec::exec_from_repl now takes 3 parameters instead of 2, in /home/runner/work/datafusion/datafusion/datafusion-cli/src/exec.rs:130
Summary semver requires new major version: 1 major and 0 minor checks failed
Finished [ 392.162s] datafusion-cli
```
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "ratmice (via GitHub)" <gi...@apache.org>.
ratmice commented on code in PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#discussion_r3646501978
##########
datafusion-cli/src/exec.rs:
##########
@@ -129,13 +130,15 @@ pub async fn exec_from_files(
pub async fn exec_from_repl(
ctx: &dyn CliSessionContext,
print_options: &mut PrintOptions,
+ history_file: Option<&Path>,
Review Comment:
I should clarify that I'm just a drive-by reviewer, not really a maintainer or anything so take my suggestions for what their worth, perhaps the maintainers will have conflicting ideas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "dariocurr (via GitHub)" <gi...@apache.org>.
dariocurr commented on PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#issuecomment-5426317381
any update on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#issuecomment-5078267873
## [Codecov](https://app.codecov.io/gh/apache/datafusion/pull/23797?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 80.63%. Comparing base ([`dfe0f26`](https://app.codecov.io/gh/apache/datafusion/commit/dfe0f26b5f944c2ca7dfa748313f8eac00232775?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`55c1045`](https://app.codecov.io/gh/apache/datafusion/commit/55c1045b039a838405c341cae7517c1825b848e0?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 58 commits behind head on main.
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## main #23797 +/- ##
==========================================
- Coverage 80.71% 80.63% -0.09%
==========================================
Files 1089 1091 +2
Lines 368760 370720 +1960
Branches 368760 370720 +1960
==========================================
+ Hits 297636 298918 +1282
- Misses 53377 53959 +582
- Partials 17747 17843 +96
```
</details>
[:umbrella: View full report in Codecov by Harness](https://app.codecov.io/gh/apache/datafusion/pull/23797?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "dariocurr (via GitHub)" <gi...@apache.org>.
dariocurr commented on PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#issuecomment-5848604683
any update on this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "ratmice (via GitHub)" <gi...@apache.org>.
ratmice commented on PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#issuecomment-6253394001
Don't really have anything to add, as you've addressed the comments in my initial review.
I'm just a drive-by community member who also has some datafusion-cli PRs open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "dariocurr (via GitHub)" <gi...@apache.org>.
dariocurr commented on PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#issuecomment-5045514511
Re the cargo-semver-checks note above: this is an expected, intentional minor breaking change. `exec_from_repl` gained a required `history_file: Option<&Path>` parameter so the interactive shell's history location can be configured (this PR's whole purpose); the only in-tree caller besides `main.rs` is `datafusion-cli/examples/cli-session-context.rs`, which has been updated accordingly.
Per the [API health policy](https://datafusion.apache.org/contributor-guide/api-health.html), could a committer add the `api change` label to this PR? I don't have push access to add it myself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "ratmice (via GitHub)" <gi...@apache.org>.
ratmice commented on code in PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#discussion_r3646429473
##########
datafusion-cli/src/exec.rs:
##########
@@ -129,13 +130,15 @@ pub async fn exec_from_files(
pub async fn exec_from_repl(
ctx: &dyn CliSessionContext,
print_options: &mut PrintOptions,
+ history_file: Option<&Path>,
Review Comment:
I think it might be better instead of just adding an `Option<&Path>`,
to declare some struct like perhaps
```
#[non_exhaustive]
#[derive(Default)]
struct OtherOptions {
history_path: Option<PathBuf>,
}
```
Or manually implement default to default to ".history" here in the struct than in the function.
Then if any new options are needed which have default values, we don't have to change the abi again to add them, because they should just get picked up by the non_exhaustive/default combo.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "ratmice (via GitHub)" <gi...@apache.org>.
ratmice commented on code in PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#discussion_r3646501978
##########
datafusion-cli/src/exec.rs:
##########
@@ -129,13 +130,15 @@ pub async fn exec_from_files(
pub async fn exec_from_repl(
ctx: &dyn CliSessionContext,
print_options: &mut PrintOptions,
+ history_file: Option<&Path>,
Review Comment:
I should clarify that I'm just a drive-by reviewer, not really a maintainer or anything so take my suggestions for what it's worth.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org
Re: [PR] feat: allow configuring datafusion-cli history file location [datafusion]
Posted by "ratmice (via GitHub)" <gi...@apache.org>.
ratmice commented on code in PR #23797:
URL: https://github.com/apache/datafusion/pull/23797#discussion_r3650143910
##########
datafusion-cli/src/exec.rs:
##########
@@ -129,13 +130,15 @@ pub async fn exec_from_files(
pub async fn exec_from_repl(
ctx: &dyn CliSessionContext,
print_options: &mut PrintOptions,
+ history_file: Option<&Path>,
Review Comment:
Good thinking to keep it consistent, we could always add `#[non_exhaustive]` to both in follow up patch if that is desired.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org