You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@daffodil.apache.org by "Interrante, John A (GE Research, US)" <Jo...@ge.com> on 2023/02/14 20:10:17 UTC
How to run scalafmt manually
I wondered how to run scalafmt manually after merging Mike's scalafmt PR. I found out that I can run scalafmt from sbt using the scalafmt plugin's task keys:
sbt:daffodil> scalafmtCheck
[info] scalafmt: Checking 5 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-tdml-lib)...
...
[info] scalafmt: Checking 3 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
[success] Total time: 12 s, completed Feb 14, 2023, 11:20:55 AM
sbt:daffodil> scalafmt
[success] Total time: 0 s, completed Feb 14, 2023, 11:20:59 AM
sbt:daffodil>
I also tried some other task keys from this more complete list (https://scalameta.org/scalafmt/docs/installation.html#task-keys):
. myproject/scalafmt: Format main sources of myproject project
. myproject/test:scalafmt: Format test sources of myproject project
. scalafmtCheck: Check if the scala sources under the project have been formatted.
. scalafmtSbt: Format *.sbt and project/*.scala files.
. scalafmtSbtCheck: Check if the files have been formatted by scalafmtSbt.
. scalafmtOnly <file>...: Format specified files listed.
. scalafmtAll or scalafmtCheckAll: Execute the scalafmt or scalafmtCheck task for all configurations in which it is enabled (since v2.0.0-RC5)
Both "daffodil-tdml-processor/scalafmt" and "scalafmt" format just one project and format all projects respectively, but note that neither will format test files. To do that, you will need "daffodil-tdml-processor/test:scalafmt" or "Test/scalafmt". It turns out that the PR did not format any test files, as I confirmed by running "Test/scalafmtCheck" and " IntegrationTest/scalafmtCheck", both of which reported that the test files still needed to be formatted. I can format them by running Test/scalafmt" and "IntegrationTest/scalafmt" respectively:
sbt:daffodil> Test/scalafmt
[info] scalafmt: Formatting 1 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-propgen)...
...
[info] scalafmt: Reformatted 138 Scala sources
[success] Total time: 3 s, completed Feb 14, 2023, 11:37:31 AM
sbt:daffodil> IntegrationTest/scalafmt
[info] scalafmt: Formatting 1 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
[info] scalafmt: Reformatted 1 Scala sources
[success] Total time: 0 s, completed Feb 14, 2023, 11:37:39 AM
sbt:daffodil>
I also confirmed that the PR did not format any *.sbt or project/*.scala files by running their task keys as well:
sbt:daffodil> scalafmtSbtCheck
[info] scalafmt: Checking 4 Scala sources (/home/interran/apache/daffodil-tuxji)...
...
[error] (Compile / scalafmtSbtCheck) scalafmt: 4 files must be formatted (/home/interran/apache/daffodil-tuxji)
[error] Total time: 0 s, completed Feb 14, 2023, 11:41:28 AM
sbt:daffodil> scalafmtSbt
[info] scalafmt: Formatting 4 Scala sources /home/interran/apache/daffodil-tuxji)...
...
[success] Total time: 0 s, completed Feb 14, 2023, 11:41:36 AM
[warn] build source files have changed
[warn] modified files:
[warn] /home/interran/apache/daffodil-tuxji/build.sbt
[warn] /home/interran/apache/daffodil-tuxji/daffodil-cli/build.sbt
[warn] /home/interran/apache/daffodil-tuxji/project/ForkCaptureLogger.scala
[warn] /home/interran/apache/daffodil-tuxji/project/Dependencies.scala
[warn] /home/interran/apache/daffodil-tuxji/project/OsgiCheck.scala
[warn] /home/interran/apache/daffodil-tuxji/project/Rat.scala
[warn] Apply these changes by running `reload`.
[warn] Automatically reload the build when source changes are detected by setting `Global / onChangedBuildSource := ReloadOnSourceChanges`.
[warn] Disable this warning by setting `Global / onChangedBuildSource := IgnoreSourceChanges`.
sbt:daffodil>
You can save yourself some time by using the "scalafmtCheckAll" and "scalafmtAll" task keys. They will check or format both source and test/integrationtest files so you won't have to type so many commands yourself:
sbt:daffodil> scalafmtCheckAll
[success] Total time: 0 s, completed Feb 14, 2023, 11:43:26 AM
sbt:daffodil> scalafmtAll
[success] Total time: 0 s, completed Feb 14, 2023, 11:43:42 AM
sbt:daffodil>
However, the All tasks won't check or format the *.sbt and project/*.scala files, so you'll still need to run the "scalafmtSbtCheck" and "scalafmtSbt" tasks for completeness:
sbt:daffodil> scalafmtSbtCheck
[info] scalafmt: Checking 4 Scala sources (/home/interran/apache/daffodil-tuxji)...
[info] scalafmt: Checking 1 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-japi)...
[info] scalafmt: Checking 2 Scala sources (/home/interran/apache/daffodil-tuxji)...
[info] scalafmt: Checking 1 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
[success] Total time: 0 s, completed Feb 14, 2023, 11:49:29 AM
sbt:daffodil> scalafmtSbt
[info] scalafmt: Formatting 4 Scala sources (/home/interran/apache/daffodil-tuxji)...
[info] scalafmt: Formatting 1 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-japi)...
[info] scalafmt: Formatting 2 Scala sources (/home/interran/apache/daffodil-tuxji)...
[info] scalafmt: Formatting 1 Scala sources (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
[success] Total time: 0 s, completed Feb 14, 2023, 11:49:32 AM
sbt:daffodil>
That's a total of 4 tasks we will need to document in DEVELOP.md for developers to remember how to run:
. scalafmtAll or scalafmtCheckAll: Execute the scalafmt or scalafmtCheck task for all src/, test/, and it/ *.scala files.
. scalafmtSbt or scalafmtSbtCheck: Execute the scalafmt or scalafmtCheck task for all *.sbt and project/*.scala files.
Mike, will you please reopen https://issues.apache.org/jira/browse/DAFFODIL-2133, send another PR which formats the rest of the scala and sbt files, documents these tasks in DEVELOP.md, and executes the scalafmtCheckAll and scalafmtSbtCheck tasks instead of only the scalafmtCheck task in the CI workflow?
Thanks,
John
Re: How to run scalafmt manually
Posted by Matthew Benedict de Detrich <ma...@aiven.io.INVALID>.
I am not that familiar with Daffodil project itself but I am quite familiar
with Scala/SBT/scalafmt but I would point out in case its helpful that
scalafmt also provides native binaries (see assets under
https://github.com/scalameta/scalafmt/releases) which can check that the
entire project is formatted outside of sbt and its shell. scalafmt also
provides packages for these native binaries to various operating systems, I
believe Windows is the exception here.
I maintain Pekko which is an Apache Incubator project that also uses sbt
and scalafmt, and what we ended up doing is using a github action that uses
these native binaries to check the entire codebase is formatted correctly
(see
https://github.com/apache/incubator-pekko/blob/main/.github/workflows/format.yml).
Along with the usage of diff-ref (see
https://github.com/apache/incubator-pekko/blob/main/.github/workflows/format.yml#L24)
so we only format files which have been changed with git, this allows us to
quickly check that the entire codebase is formatted concurrently to any
other build steps on the PR.
We also added a forced github check which actually makes it impossible to
merge a PR into main if the code is not formatted correctly, see
https://github.com/apache/incubator-pekko-sbt-paradox/blob/main/.asf.yaml#L28-L35,
of course this only works if main is protected.
This is how we enforce that our codebase is always correctly
formatted/clean which might be helpful for Daffodil. I can open up a PR if
the community is open to this.
On Tue, Feb 14, 2023 at 9:11 PM Interrante, John A (GE Research, US) <
John.Interrante@ge.com> wrote:
> I wondered how to run scalafmt manually after merging Mike's scalafmt PR.
> I found out that I can run scalafmt from sbt using the scalafmt plugin's
> task keys:
>
> sbt:daffodil> scalafmtCheck
> [info] scalafmt: Checking 5 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-tdml-lib)...
> ...
> [info] scalafmt: Checking 3 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
> [success] Total time: 12 s, completed Feb 14, 2023, 11:20:55 AM
> sbt:daffodil> scalafmt
> [success] Total time: 0 s, completed Feb 14, 2023, 11:20:59 AM
> sbt:daffodil>
>
> I also tried some other task keys from this more complete list (
> https://scalameta.org/scalafmt/docs/installation.html#task-keys):
>
> . myproject/scalafmt: Format main sources of myproject project
> . myproject/test:scalafmt: Format test sources of myproject project
> . scalafmtCheck: Check if the scala sources under the project have been
> formatted.
> . scalafmtSbt: Format *.sbt and project/*.scala files.
> . scalafmtSbtCheck: Check if the files have been formatted by scalafmtSbt.
> . scalafmtOnly <file>...: Format specified files listed.
> . scalafmtAll or scalafmtCheckAll: Execute
> the scalafmt or scalafmtCheck task for all configurations in which it is
> enabled (since v2.0.0-RC5)
>
> Both "daffodil-tdml-processor/scalafmt" and "scalafmt" format just one
> project and format all projects respectively, but note that neither will
> format test files. To do that, you will need
> "daffodil-tdml-processor/test:scalafmt" or "Test/scalafmt". It turns out
> that the PR did not format any test files, as I confirmed by running
> "Test/scalafmtCheck" and " IntegrationTest/scalafmtCheck", both of which
> reported that the test files still needed to be formatted. I can format
> them by running Test/scalafmt" and "IntegrationTest/scalafmt" respectively:
>
> sbt:daffodil> Test/scalafmt
> [info] scalafmt: Formatting 1 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-propgen)...
> ...
> [info] scalafmt: Reformatted 138 Scala sources
> [success] Total time: 3 s, completed Feb 14, 2023, 11:37:31 AM
> sbt:daffodil> IntegrationTest/scalafmt
> [info] scalafmt: Formatting 1 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
> [info] scalafmt: Reformatted 1 Scala sources
> [success] Total time: 0 s, completed Feb 14, 2023, 11:37:39 AM
> sbt:daffodil>
>
> I also confirmed that the PR did not format any *.sbt or project/*.scala
> files by running their task keys as well:
>
> sbt:daffodil> scalafmtSbtCheck
> [info] scalafmt: Checking 4 Scala sources
> (/home/interran/apache/daffodil-tuxji)...
> ...
> [error] (Compile / scalafmtSbtCheck) scalafmt: 4 files must be formatted
> (/home/interran/apache/daffodil-tuxji)
> [error] Total time: 0 s, completed Feb 14, 2023, 11:41:28 AM
> sbt:daffodil> scalafmtSbt
> [info] scalafmt: Formatting 4 Scala sources
> /home/interran/apache/daffodil-tuxji)...
> ...
> [success] Total time: 0 s, completed Feb 14, 2023, 11:41:36 AM
> [warn] build source files have changed
> [warn] modified files:
> [warn] /home/interran/apache/daffodil-tuxji/build.sbt
> [warn] /home/interran/apache/daffodil-tuxji/daffodil-cli/build.sbt
> [warn]
> /home/interran/apache/daffodil-tuxji/project/ForkCaptureLogger.scala
> [warn] /home/interran/apache/daffodil-tuxji/project/Dependencies.scala
> [warn] /home/interran/apache/daffodil-tuxji/project/OsgiCheck.scala
> [warn] /home/interran/apache/daffodil-tuxji/project/Rat.scala
> [warn] Apply these changes by running `reload`.
> [warn] Automatically reload the build when source changes are detected by
> setting `Global / onChangedBuildSource := ReloadOnSourceChanges`.
> [warn] Disable this warning by setting `Global / onChangedBuildSource :=
> IgnoreSourceChanges`.
> sbt:daffodil>
>
> You can save yourself some time by using the "scalafmtCheckAll" and
> "scalafmtAll" task keys. They will check or format both source and
> test/integrationtest files so you won't have to type so many commands
> yourself:
>
> sbt:daffodil> scalafmtCheckAll
> [success] Total time: 0 s, completed Feb 14, 2023, 11:43:26 AM
> sbt:daffodil> scalafmtAll
> [success] Total time: 0 s, completed Feb 14, 2023, 11:43:42 AM
> sbt:daffodil>
>
> However, the All tasks won't check or format the *.sbt and project/*.scala
> files, so you'll still need to run the "scalafmtSbtCheck" and "scalafmtSbt"
> tasks for completeness:
>
> sbt:daffodil> scalafmtSbtCheck
> [info] scalafmt: Checking 4 Scala sources
> (/home/interran/apache/daffodil-tuxji)...
> [info] scalafmt: Checking 1 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-japi)...
> [info] scalafmt: Checking 2 Scala sources
> (/home/interran/apache/daffodil-tuxji)...
> [info] scalafmt: Checking 1 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
> [success] Total time: 0 s, completed Feb 14, 2023, 11:49:29 AM
> sbt:daffodil> scalafmtSbt
> [info] scalafmt: Formatting 4 Scala sources
> (/home/interran/apache/daffodil-tuxji)...
> [info] scalafmt: Formatting 1 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-japi)...
> [info] scalafmt: Formatting 2 Scala sources
> (/home/interran/apache/daffodil-tuxji)...
> [info] scalafmt: Formatting 1 Scala sources
> (/home/interran/apache/daffodil-tuxji/daffodil-cli)...
> [success] Total time: 0 s, completed Feb 14, 2023, 11:49:32 AM
> sbt:daffodil>
>
> That's a total of 4 tasks we will need to document in DEVELOP.md for
> developers to remember how to run:
>
> . scalafmtAll or scalafmtCheckAll: Execute the scalafmt or scalafmtCheck
> task for all src/, test/, and it/ *.scala files.
> . scalafmtSbt or scalafmtSbtCheck: Execute the scalafmt or scalafmtCheck
> task for all *.sbt and project/*.scala files.
>
> Mike, will you please reopen
> https://issues.apache.org/jira/browse/DAFFODIL-2133, send another PR
> which formats the rest of the scala and sbt files, documents these tasks in
> DEVELOP.md, and executes the scalafmtCheckAll and scalafmtSbtCheck tasks
> instead of only the scalafmtCheck task in the CI workflow?
>
> Thanks,
> John
>
--
Matthew de Detrich
*Aiven Deutschland GmbH*
Immanuelkirchstraße 26, 10405 Berlin
Amtsgericht Charlottenburg, HRB 209739 B
Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen
*m:* +491603708037
*w:* aiven.io *e:* matthew.dedetrich@aiven.io