You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Udi Meiri <eh...@google.com> on 2020/01/08 03:01:52 UTC
[DISCUSS] Python static type checkers
Hi,
We recently added mypy to the Jenkins Lint job for PRs (currently ignores
errors). Mypy is a static type checker.
There's a JIRA for adding another static type checker named pytype
https://issues.apache.org/jira/browse/BEAM-9064
I wanted to ask the community their thoughts on this. (see JIRA issue
comments as well)
- Should PRs have to pass more than 1 static type checker? (in pre-commit
tests)
- If not, should the remaining type checkers be run as a post-commit tests?
- How much effort should be put into supporting more than 1 type checker?
(i.e. making sure that they all pass)
Re: [DISCUSS] Python static type checkers
Posted by Robert Bradshaw <ro...@google.com>.
I am fine with adding this as a linter. I would not want to block either
(let alone both) until we have some experience with them. Hopefully, if our
code is clean and correctly typed, it should pass both. Where it doesn't,
I'm hopeful that the looseness provided by gradual typing will allow us to
have reasonable workarounds. But I think all we can do is speculate until
we have more data.
On Tue, Jan 7, 2020 at 7:02 PM Udi Meiri <eh...@google.com> wrote:
> Hi,
> We recently added mypy to the Jenkins Lint job for PRs (currently ignores
> errors). Mypy is a static type checker.
>
> There's a JIRA for adding another static type checker named pytype
> https://issues.apache.org/jira/browse/BEAM-9064
>
> I wanted to ask the community their thoughts on this. (see JIRA issue
> comments as well)
>
> - Should PRs have to pass more than 1 static type checker? (in pre-commit
> tests)
> - If not, should the remaining type checkers be run as a post-commit tests?
> - How much effort should be put into supporting more than 1 type checker?
> (i.e. making sure that they all pass)
>
>
>
>
Re: [DISCUSS] Python static type checkers
Posted by Luke Cwik <lc...@google.com>.
I would rather we focus on doing well with one type checker and it seems
that mypy is significantly more popular than pytype so its more natural for
users. I would support pytype if it covered more PEPs and was the newer and
upcoming thing but that doesn't seem to be the case.
On Sun, Jan 12, 2020 at 7:48 PM Chad Dombrova <ch...@gmail.com> wrote:
> Hi folks,
> I agree with Robert that we need to wait and see before making any
> decisions, but I do have some opinions about the probable/desired outcome.
>
> I haven't used pytype, but my experience working with mypy over the past
> few years -- and following various issues and peps related to it and typing
> in general -- has taught me there's still a lot of room for interpretation
> and thus variation between type checkers.
>
> Here's a simple example: ignoring errors. Both tools support ignoring
> errors using a `type: ignore` comment, but only mypy (to my knowledge)
> supports specifying an error type so that only that error is suppressed,
> e.g. `type: ignore[error-code-here]`. There's even room for differences
> with regard to the line number where the error is emitted and thus where
> the ignore comment must be placed (end of statement, site of open paren,
> site of close paren, etc). I know this because mypy has actually made
> adjustments to this once or twice over the years, which necessitated moving
> existing ignore comments. So just imagine having to ignore the same error
> separately for each type checker. It's not the end of the world, but it's
> ugly and frustrating.
>
> As a user, it can be quite challenging to solve certain typing issues, and
> there's a fairly steep learning curve – I wouldn't want to burden users
> with *two* type checker, each with its own idiosyncrasies. That said, a
> linter that doesn't actually prevent merges when an error occurs will be
> ignored by users and quickly become less-than-useful. Post-commit would
> not be a good idea for all the reasons that a post-commit lint check would
> be annoying (user's will trip it often and feel surprised/blind-sided).
>
> In the little exposure that I've had with pytype it seems to lag behind
> mypy in terms of features, especially wrt typing-related peps (it never
> fully supported pep484 multi-line type comments and it still doesn't
> support pep561, I see no mention of pep589/TypedDict in the docs, but then
> again they are *incredibly* light). I've gotten mypy completely passing,
> and I know it very well, so I'm pretty biased towards making it the one and
> only type checker that generates pre-commit errors. I see little advantage
> to most end users in supporting pytype, except y'know, Google has kind of
> an important presence in Apache Beam project :)
>
> Some quick pypi download figures to back that up:
>
> Downloads last month:
> pytype: 24,864
> mypy: 1,502,582
>
> So to sum up this email in a sentence: running mypy in pre-commit checks
> would be beneficial, but making pytype also pass would be a burden with
> little benefit for the majority of users.
>
> But as I said at the outset, before we make any decisions we should get at
> least one type checker in place and start getting feedback, because we're
> still in the land of conjecture.
>
> Hopefully I'll have all my typing changes merged in the next month or two,
> at which point we can discuss enabling it as part of the pre-commit lint
> job.
>
> -chad
>
>
>
>
> On Tue, Jan 7, 2020 at 7:02 PM Udi Meiri <eh...@google.com> wrote:
>
>> Hi,
>> We recently added mypy to the Jenkins Lint job for PRs (currently ignores
>> errors). Mypy is a static type checker.
>>
>> There's a JIRA for adding another static type checker named pytype
>> https://issues.apache.org/jira/browse/BEAM-9064
>>
>> I wanted to ask the community their thoughts on this. (see JIRA issue
>> comments as well)
>>
>> - Should PRs have to pass more than 1 static type checker? (in pre-commit
>> tests)
>> - If not, should the remaining type checkers be run as a post-commit
>> tests?
>> - How much effort should be put into supporting more than 1 type checker?
>> (i.e. making sure that they all pass)
>>
>>
>>
>>
Re: [DISCUSS] Python static type checkers
Posted by Ahmet Altay <al...@google.com>.
On Mon, Jan 13, 2020 at 6:43 PM Chad Dombrova <ch...@gmail.com> wrote:
> > I agree with focusing one mypy for now, but I would propose soon after,
>> or in parallel if it will be different folks, to work on pytype and enable
>> it as a first class citizen similar to mypy. If there will be a large delta
>> between the two then we can decide on what to do next.
>>
>> If there is a large delta, I wonder if what is needed to provide
>> sufficient typing on the public API of beam (needed for users) could
>> be a much smaller subset than that required fully
>> documenting/typechecking the internals (which is also likely to be
>> where more of the tricky bits are).
>>
>
> Let's separate our two use cases for type checking:
>
> 1) developers working on the Beam source
> 2) developers using Beam
>
> For #1, there's little to gain from running a second type checker, and as
> discussed, it's a burden. so mypy is the clear winner here.
>
> For #2, there's actually no need to expose all of the internals of Beam to
> pytype, and as Robert points out, this is where the "tricky bits" reside.
> We could use mpy's stubgen tool to create .pyi files of just the API
> interface -- no internals -- as a separate apache_beam_stubs package. Then
> pytype users (or users of other type checkers) could optionally install
> this and point pytype at it. This would be a bit more automatic for end
> users if pytype supported pep561[1], which is all about the packaging and
> discovery of typed packages, including stub-only packages like I'm
> proposing.
>
> https://github.com/google/pytype/issues/151
>
This proposal sounds reasonable to me.
Udi, would it be possible to share this thread with pytype developers or
invite them to comment here. It would be good if they can take Beam as a
use case and see the feedback shared here.
>
>
> -chad
>
>
Re: [DISCUSS] Python static type checkers
Posted by Chad Dombrova <ch...@gmail.com>.
>
> > I agree with focusing one mypy for now, but I would propose soon after,
> or in parallel if it will be different folks, to work on pytype and enable
> it as a first class citizen similar to mypy. If there will be a large delta
> between the two then we can decide on what to do next.
>
> If there is a large delta, I wonder if what is needed to provide
> sufficient typing on the public API of beam (needed for users) could
> be a much smaller subset than that required fully
> documenting/typechecking the internals (which is also likely to be
> where more of the tricky bits are).
>
Let's separate our two use cases for type checking:
1) developers working on the Beam source
2) developers using Beam
For #1, there's little to gain from running a second type checker, and as
discussed, it's a burden. so mypy is the clear winner here.
For #2, there's actually no need to expose all of the internals of Beam to
pytype, and as Robert points out, this is where the "tricky bits" reside.
We could use mpy's stubgen tool to create .pyi files of just the API
interface -- no internals -- as a separate apache_beam_stubs package. Then
pytype users (or users of other type checkers) could optionally install
this and point pytype at it. This would be a bit more automatic for end
users if pytype supported pep561[1], which is all about the packaging and
discovery of typed packages, including stub-only packages like I'm
proposing.
https://github.com/google/pytype/issues/151
-chad
Re: [DISCUSS] Python static type checkers
Posted by Robert Bradshaw <ro...@google.com>.
On Mon, Jan 13, 2020 at 6:04 PM Ahmet Altay <al...@google.com> wrote:
>
> > The most important gain would be compatibility with Google internal code.
> I would like to clarify this. This refers to users of Beam who by default are using pytype as part of the toolchain. Even though they are internal to a one single company and not vocal on Beam, they still represent a large group of distinct teams using Beam.
>
> On Mon, Jan 13, 2020 at 5:51 PM Robert Bradshaw <ro...@google.com> wrote:
>>
>> On Mon, Jan 13, 2020 at 5:34 PM Chad Dombrova <ch...@gmail.com> wrote:
>> >>
>> >> Pytype seems to detect attribute errors that mypy has not, so it acts as a kind-of linter in this case.
>> >> Examples:
>> >> https://github.com/apache/beam/pull/10528/files#diff-0cb34b4622b0b7d7256d28b1ee1d52fc
>> >> https://github.com/apache/beam/pull/10528/files#diff-7e4ad8c086414399957cdbea711ebd36
>> >> https://github.com/apache/beam/pull/10528/files#diff-d5c3f4f603204c5c5917d89e90dba53d
>> >> (it also makes pytype more strict in a sense)
>> >
>> > Note that mypy is still not fully passing on master so it's unclear from those diffs exactly how the two tools differ. Many of the fixes you've made for pytype look familiar to me from mypy, but my fixes may not be merged yet. For example, mypy also does not support @total_ordering, but my fix for that is still pending.
>>
>> As it seems we have a workaround to ignore pytype for now, it seems
>> to make the most sense to focus on getting mypy working completely
>> before focusing on that. In the long term, we could look into making
>> pytype a post-commit which would at least be a useful signal for those
>> caring about it, and only make it a precommit if the diff between what
>> it requires and mypy requires is quite small.
>
> The workaround to ignore pytype is making the experience for type users worse than before. Previously, they could have type checked their pipelines with pytype and pytype would have attempted to fill in with best effort type information. Now, pytype is instructed to not provide this information.
Thanks for the clarification--I had assumed that it only suppressed
warnings from this codebase, not that it would disable providing what
it could. (Which, of course, is still less if it can't understand the
code.)
> I agree with focusing one mypy for now, but I would propose soon after, or in parallel if it will be different folks, to work on pytype and enable it as a first class citizen similar to mypy. If there will be a large delta between the two then we can decide on what to do next.
If there is a large delta, I wonder if what is needed to provide
sufficient typing on the public API of beam (needed for users) could
be a much smaller subset than that required fully
documenting/typechecking the internals (which is also likely to be
where more of the tricky bits are).
Re: [DISCUSS] Python static type checkers
Posted by Ahmet Altay <al...@google.com>.
> The most important gain would be compatibility with Google internal code.
I would like to clarify this. This refers to users of Beam who by default
are using pytype as part of the toolchain. Even though they are internal to
a one single company and not vocal on Beam, they still represent a large
group of distinct teams using Beam.
On Mon, Jan 13, 2020 at 5:51 PM Robert Bradshaw <ro...@google.com> wrote:
> On Mon, Jan 13, 2020 at 5:34 PM Chad Dombrova <ch...@gmail.com> wrote:
> >>
> >> Pytype seems to detect attribute errors that mypy has not, so it acts
> as a kind-of linter in this case.
> >> Examples:
> >>
> https://github.com/apache/beam/pull/10528/files#diff-0cb34b4622b0b7d7256d28b1ee1d52fc
> >>
> https://github.com/apache/beam/pull/10528/files#diff-7e4ad8c086414399957cdbea711ebd36
> >>
> https://github.com/apache/beam/pull/10528/files#diff-d5c3f4f603204c5c5917d89e90dba53d
> >> (it also makes pytype more strict in a sense)
> >
> > Note that mypy is still not fully passing on master so it's unclear from
> those diffs exactly how the two tools differ. Many of the fixes you've
> made for pytype look familiar to me from mypy, but my fixes may not be
> merged yet. For example, mypy also does not support @total_ordering, but
> my fix for that is still pending.
>
> As it seems we have a workaround to ignore pytype for now, it seems
> to make the most sense to focus on getting mypy working completely
> before focusing on that. In the long term, we could look into making
> pytype a post-commit which would at least be a useful signal for those
> caring about it, and only make it a precommit if the diff between what
> it requires and mypy requires is quite small.
>
The workaround to ignore pytype is making the experience for type users
worse than before. Previously, they could have type checked their pipelines
with pytype and pytype would have attempted to fill in with best effort
type information. Now, pytype is instructed to not provide this information.
I agree with focusing one mypy for now, but I would propose soon after, or
in parallel if it will be different folks, to work on pytype and enable it
as a first class citizen similar to mypy. If there will be a large delta
between the two then we can decide on what to do next.
Re: [DISCUSS] Python static type checkers
Posted by Robert Bradshaw <ro...@google.com>.
On Mon, Jan 13, 2020 at 5:34 PM Chad Dombrova <ch...@gmail.com> wrote:
>>
>> Pytype seems to detect attribute errors that mypy has not, so it acts as a kind-of linter in this case.
>> Examples:
>> https://github.com/apache/beam/pull/10528/files#diff-0cb34b4622b0b7d7256d28b1ee1d52fc
>> https://github.com/apache/beam/pull/10528/files#diff-7e4ad8c086414399957cdbea711ebd36
>> https://github.com/apache/beam/pull/10528/files#diff-d5c3f4f603204c5c5917d89e90dba53d
>> (it also makes pytype more strict in a sense)
>
> Note that mypy is still not fully passing on master so it's unclear from those diffs exactly how the two tools differ. Many of the fixes you've made for pytype look familiar to me from mypy, but my fixes may not be merged yet. For example, mypy also does not support @total_ordering, but my fix for that is still pending.
As it seems we have a workaround to ignore pytype for now, it seems
to make the most sense to focus on getting mypy working completely
before focusing on that. In the long term, we could look into making
pytype a post-commit which would at least be a useful signal for those
caring about it, and only make it a precommit if the diff between what
it requires and mypy requires is quite small.
Re: [DISCUSS] Python static type checkers
Posted by Chad Dombrova <ch...@gmail.com>.
>
> Pytype seems to detect attribute errors that mypy has not, so it acts as a
> kind-of linter in this case.
> Examples:
>
> https://github.com/apache/beam/pull/10528/files#diff-0cb34b4622b0b7d7256d28b1ee1d52fc
>
> https://github.com/apache/beam/pull/10528/files#diff-7e4ad8c086414399957cdbea711ebd36
>
> https://github.com/apache/beam/pull/10528/files#diff-d5c3f4f603204c5c5917d89e90dba53d
> (it also makes pytype more strict in a sense)
>
Note that mypy is still not fully passing on master so it's unclear from
those diffs exactly how the two tools differ. Many of the fixes you've
made for pytype look familiar to me from mypy, but my fixes may not be
merged yet. For example, mypy also does not support @total_ordering, but
my fix for that is still pending.
Re: [DISCUSS] Python static type checkers
Posted by Udi Meiri <eh...@google.com>.
The most important gain would be compatibility with Google internal code.
TLDR: I don't expect non-Googlers to fix pytype issues in Beam, nor would
they have access to internal code that is validated against pytype with
Beam.
Pytype seems to detect attribute errors that mypy has not, so it acts as a
kind-of linter in this case.
Examples:
https://github.com/apache/beam/pull/10528/files#diff-0cb34b4622b0b7d7256d28b1ee1d52fc
https://github.com/apache/beam/pull/10528/files#diff-7e4ad8c086414399957cdbea711ebd36
https://github.com/apache/beam/pull/10528/files#diff-d5c3f4f603204c5c5917d89e90dba53d
(it also makes pytype more strict in a sense)
Pytype has bugs. I've filed/found these so far:
https://github.com/google/pytype/issues/491 (isinstance bug)
https://github.com/google/pytype/issues/445 (@overload bug)
https://github.com/google/pytype/issues/488 (builtins.object != object bug)
https://github.com/google/pytype/issues/485 (# type: ignore[...] mypy
compatibility issue)
https://github.com/google/pytype/issues/480 (alternate comment syntax -
wontfix)
On Mon, Jan 13, 2020 at 3:49 PM Kyle Weaver <kc...@google.com> wrote:
> Udi, what would we gain by using pytype?
>
> Also, has anyone tried running pytype against Beam? If it's not too much
> trouble, it might be helpful to diff the pytype and mypy results to get a
> feel for exactly how big the discrepancy is.
>
> On Mon, Jan 13, 2020 at 3:26 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> Looking at this from the outside, it seems like mypy is the obvious
>> choice. Also running pytype could potentially be informative in some cases
>> but only if there is a specific gap. What about maintenance/governance of
>> the two projects?
>>
>> Kenn
>>
>> On Sun, Jan 12, 2020 at 7:48 PM Chad Dombrova <ch...@gmail.com> wrote:
>>
>>> Hi folks,
>>> I agree with Robert that we need to wait and see before making any
>>> decisions, but I do have some opinions about the probable/desired outcome.
>>>
>>> I haven't used pytype, but my experience working with mypy over the past
>>> few years -- and following various issues and peps related to it and typing
>>> in general -- has taught me there's still a lot of room for interpretation
>>> and thus variation between type checkers.
>>>
>>> Here's a simple example: ignoring errors. Both tools support ignoring
>>> errors using a `type: ignore` comment, but only mypy (to my knowledge)
>>> supports specifying an error type so that only that error is suppressed,
>>> e.g. `type: ignore[error-code-here]`. There's even room for differences
>>> with regard to the line number where the error is emitted and thus where
>>> the ignore comment must be placed (end of statement, site of open paren,
>>> site of close paren, etc). I know this because mypy has actually made
>>> adjustments to this once or twice over the years, which necessitated moving
>>> existing ignore comments. So just imagine having to ignore the same error
>>> separately for each type checker. It's not the end of the world, but it's
>>> ugly and frustrating.
>>>
>>> As a user, it can be quite challenging to solve certain typing issues,
>>> and there's a fairly steep learning curve – I wouldn't want to burden
>>> users with *two* type checker, each with its own idiosyncrasies. That
>>> said, a linter that doesn't actually prevent merges when an error occurs
>>> will be ignored by users and quickly become less-than-useful. Post-commit
>>> would not be a good idea for all the reasons that a post-commit lint check
>>> would be annoying (user's will trip it often and feel
>>> surprised/blind-sided).
>>>
>>> In the little exposure that I've had with pytype it seems to lag behind
>>> mypy in terms of features, especially wrt typing-related peps (it never
>>> fully supported pep484 multi-line type comments and it still doesn't
>>> support pep561, I see no mention of pep589/TypedDict in the docs, but then
>>> again they are *incredibly* light). I've gotten mypy completely
>>> passing, and I know it very well, so I'm pretty biased towards making it
>>> the one and only type checker that generates pre-commit errors. I see
>>> little advantage to most end users in supporting pytype, except y'know,
>>> Google has kind of an important presence in Apache Beam project :)
>>>
>>> Some quick pypi download figures to back that up:
>>>
>>> Downloads last month:
>>> pytype: 24,864
>>> mypy: 1,502,582
>>>
>>> So to sum up this email in a sentence: running mypy in pre-commit checks
>>> would be beneficial, but making pytype also pass would be a burden with
>>> little benefit for the majority of users.
>>>
>>> But as I said at the outset, before we make any decisions we should get
>>> at least one type checker in place and start getting feedback, because
>>> we're still in the land of conjecture.
>>>
>>> Hopefully I'll have all my typing changes merged in the next month or
>>> two, at which point we can discuss enabling it as part of the pre-commit
>>> lint job.
>>>
>>> -chad
>>>
>>>
>>>
>>>
>>> On Tue, Jan 7, 2020 at 7:02 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Hi,
>>>> We recently added mypy to the Jenkins Lint job for PRs (currently
>>>> ignores errors). Mypy is a static type checker.
>>>>
>>>> There's a JIRA for adding another static type checker named pytype
>>>> https://issues.apache.org/jira/browse/BEAM-9064
>>>>
>>>> I wanted to ask the community their thoughts on this. (see JIRA issue
>>>> comments as well)
>>>>
>>>> - Should PRs have to pass more than 1 static type checker? (in
>>>> pre-commit tests)
>>>> - If not, should the remaining type checkers be run as a post-commit
>>>> tests?
>>>> - How much effort should be put into supporting more than 1 type
>>>> checker? (i.e. making sure that they all pass)
>>>>
>>>>
>>>>
>>>>
Re: [DISCUSS] Python static type checkers
Posted by Kyle Weaver <kc...@google.com>.
Udi, what would we gain by using pytype?
Also, has anyone tried running pytype against Beam? If it's not too much
trouble, it might be helpful to diff the pytype and mypy results to get a
feel for exactly how big the discrepancy is.
On Mon, Jan 13, 2020 at 3:26 PM Kenneth Knowles <ke...@apache.org> wrote:
> Looking at this from the outside, it seems like mypy is the obvious
> choice. Also running pytype could potentially be informative in some cases
> but only if there is a specific gap. What about maintenance/governance of
> the two projects?
>
> Kenn
>
> On Sun, Jan 12, 2020 at 7:48 PM Chad Dombrova <ch...@gmail.com> wrote:
>
>> Hi folks,
>> I agree with Robert that we need to wait and see before making any
>> decisions, but I do have some opinions about the probable/desired outcome.
>>
>> I haven't used pytype, but my experience working with mypy over the past
>> few years -- and following various issues and peps related to it and typing
>> in general -- has taught me there's still a lot of room for interpretation
>> and thus variation between type checkers.
>>
>> Here's a simple example: ignoring errors. Both tools support ignoring
>> errors using a `type: ignore` comment, but only mypy (to my knowledge)
>> supports specifying an error type so that only that error is suppressed,
>> e.g. `type: ignore[error-code-here]`. There's even room for differences
>> with regard to the line number where the error is emitted and thus where
>> the ignore comment must be placed (end of statement, site of open paren,
>> site of close paren, etc). I know this because mypy has actually made
>> adjustments to this once or twice over the years, which necessitated moving
>> existing ignore comments. So just imagine having to ignore the same error
>> separately for each type checker. It's not the end of the world, but it's
>> ugly and frustrating.
>>
>> As a user, it can be quite challenging to solve certain typing issues,
>> and there's a fairly steep learning curve – I wouldn't want to burden
>> users with *two* type checker, each with its own idiosyncrasies. That
>> said, a linter that doesn't actually prevent merges when an error occurs
>> will be ignored by users and quickly become less-than-useful. Post-commit
>> would not be a good idea for all the reasons that a post-commit lint check
>> would be annoying (user's will trip it often and feel
>> surprised/blind-sided).
>>
>> In the little exposure that I've had with pytype it seems to lag behind
>> mypy in terms of features, especially wrt typing-related peps (it never
>> fully supported pep484 multi-line type comments and it still doesn't
>> support pep561, I see no mention of pep589/TypedDict in the docs, but then
>> again they are *incredibly* light). I've gotten mypy completely
>> passing, and I know it very well, so I'm pretty biased towards making it
>> the one and only type checker that generates pre-commit errors. I see
>> little advantage to most end users in supporting pytype, except y'know,
>> Google has kind of an important presence in Apache Beam project :)
>>
>> Some quick pypi download figures to back that up:
>>
>> Downloads last month:
>> pytype: 24,864
>> mypy: 1,502,582
>>
>> So to sum up this email in a sentence: running mypy in pre-commit checks
>> would be beneficial, but making pytype also pass would be a burden with
>> little benefit for the majority of users.
>>
>> But as I said at the outset, before we make any decisions we should get
>> at least one type checker in place and start getting feedback, because
>> we're still in the land of conjecture.
>>
>> Hopefully I'll have all my typing changes merged in the next month or
>> two, at which point we can discuss enabling it as part of the pre-commit
>> lint job.
>>
>> -chad
>>
>>
>>
>>
>> On Tue, Jan 7, 2020 at 7:02 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Hi,
>>> We recently added mypy to the Jenkins Lint job for PRs (currently
>>> ignores errors). Mypy is a static type checker.
>>>
>>> There's a JIRA for adding another static type checker named pytype
>>> https://issues.apache.org/jira/browse/BEAM-9064
>>>
>>> I wanted to ask the community their thoughts on this. (see JIRA issue
>>> comments as well)
>>>
>>> - Should PRs have to pass more than 1 static type checker? (in
>>> pre-commit tests)
>>> - If not, should the remaining type checkers be run as a post-commit
>>> tests?
>>> - How much effort should be put into supporting more than 1 type
>>> checker? (i.e. making sure that they all pass)
>>>
>>>
>>>
>>>
Re: [DISCUSS] Python static type checkers
Posted by Kenneth Knowles <ke...@apache.org>.
Looking at this from the outside, it seems like mypy is the obvious choice.
Also running pytype could potentially be informative in some cases but only
if there is a specific gap. What about maintenance/governance of the two
projects?
Kenn
On Sun, Jan 12, 2020 at 7:48 PM Chad Dombrova <ch...@gmail.com> wrote:
> Hi folks,
> I agree with Robert that we need to wait and see before making any
> decisions, but I do have some opinions about the probable/desired outcome.
>
> I haven't used pytype, but my experience working with mypy over the past
> few years -- and following various issues and peps related to it and typing
> in general -- has taught me there's still a lot of room for interpretation
> and thus variation between type checkers.
>
> Here's a simple example: ignoring errors. Both tools support ignoring
> errors using a `type: ignore` comment, but only mypy (to my knowledge)
> supports specifying an error type so that only that error is suppressed,
> e.g. `type: ignore[error-code-here]`. There's even room for differences
> with regard to the line number where the error is emitted and thus where
> the ignore comment must be placed (end of statement, site of open paren,
> site of close paren, etc). I know this because mypy has actually made
> adjustments to this once or twice over the years, which necessitated moving
> existing ignore comments. So just imagine having to ignore the same error
> separately for each type checker. It's not the end of the world, but it's
> ugly and frustrating.
>
> As a user, it can be quite challenging to solve certain typing issues, and
> there's a fairly steep learning curve – I wouldn't want to burden users
> with *two* type checker, each with its own idiosyncrasies. That said, a
> linter that doesn't actually prevent merges when an error occurs will be
> ignored by users and quickly become less-than-useful. Post-commit would
> not be a good idea for all the reasons that a post-commit lint check would
> be annoying (user's will trip it often and feel surprised/blind-sided).
>
> In the little exposure that I've had with pytype it seems to lag behind
> mypy in terms of features, especially wrt typing-related peps (it never
> fully supported pep484 multi-line type comments and it still doesn't
> support pep561, I see no mention of pep589/TypedDict in the docs, but then
> again they are *incredibly* light). I've gotten mypy completely passing,
> and I know it very well, so I'm pretty biased towards making it the one and
> only type checker that generates pre-commit errors. I see little advantage
> to most end users in supporting pytype, except y'know, Google has kind of
> an important presence in Apache Beam project :)
>
> Some quick pypi download figures to back that up:
>
> Downloads last month:
> pytype: 24,864
> mypy: 1,502,582
>
> So to sum up this email in a sentence: running mypy in pre-commit checks
> would be beneficial, but making pytype also pass would be a burden with
> little benefit for the majority of users.
>
> But as I said at the outset, before we make any decisions we should get at
> least one type checker in place and start getting feedback, because we're
> still in the land of conjecture.
>
> Hopefully I'll have all my typing changes merged in the next month or two,
> at which point we can discuss enabling it as part of the pre-commit lint
> job.
>
> -chad
>
>
>
>
> On Tue, Jan 7, 2020 at 7:02 PM Udi Meiri <eh...@google.com> wrote:
>
>> Hi,
>> We recently added mypy to the Jenkins Lint job for PRs (currently ignores
>> errors). Mypy is a static type checker.
>>
>> There's a JIRA for adding another static type checker named pytype
>> https://issues.apache.org/jira/browse/BEAM-9064
>>
>> I wanted to ask the community their thoughts on this. (see JIRA issue
>> comments as well)
>>
>> - Should PRs have to pass more than 1 static type checker? (in pre-commit
>> tests)
>> - If not, should the remaining type checkers be run as a post-commit
>> tests?
>> - How much effort should be put into supporting more than 1 type checker?
>> (i.e. making sure that they all pass)
>>
>>
>>
>>
Re: [DISCUSS] Python static type checkers
Posted by Chad Dombrova <ch...@gmail.com>.
Hi folks,
I agree with Robert that we need to wait and see before making any
decisions, but I do have some opinions about the probable/desired outcome.
I haven't used pytype, but my experience working with mypy over the past
few years -- and following various issues and peps related to it and typing
in general -- has taught me there's still a lot of room for interpretation
and thus variation between type checkers.
Here's a simple example: ignoring errors. Both tools support ignoring
errors using a `type: ignore` comment, but only mypy (to my knowledge)
supports specifying an error type so that only that error is suppressed,
e.g. `type: ignore[error-code-here]`. There's even room for differences
with regard to the line number where the error is emitted and thus where
the ignore comment must be placed (end of statement, site of open paren,
site of close paren, etc). I know this because mypy has actually made
adjustments to this once or twice over the years, which necessitated moving
existing ignore comments. So just imagine having to ignore the same error
separately for each type checker. It's not the end of the world, but it's
ugly and frustrating.
As a user, it can be quite challenging to solve certain typing issues, and
there's a fairly steep learning curve – I wouldn't want to burden users
with *two* type checker, each with its own idiosyncrasies. That said, a
linter that doesn't actually prevent merges when an error occurs will be
ignored by users and quickly become less-than-useful. Post-commit would
not be a good idea for all the reasons that a post-commit lint check would
be annoying (user's will trip it often and feel surprised/blind-sided).
In the little exposure that I've had with pytype it seems to lag behind
mypy in terms of features, especially wrt typing-related peps (it never
fully supported pep484 multi-line type comments and it still doesn't
support pep561, I see no mention of pep589/TypedDict in the docs, but then
again they are *incredibly* light). I've gotten mypy completely passing,
and I know it very well, so I'm pretty biased towards making it the one and
only type checker that generates pre-commit errors. I see little advantage
to most end users in supporting pytype, except y'know, Google has kind of
an important presence in Apache Beam project :)
Some quick pypi download figures to back that up:
Downloads last month:
pytype: 24,864
mypy: 1,502,582
So to sum up this email in a sentence: running mypy in pre-commit checks
would be beneficial, but making pytype also pass would be a burden with
little benefit for the majority of users.
But as I said at the outset, before we make any decisions we should get at
least one type checker in place and start getting feedback, because we're
still in the land of conjecture.
Hopefully I'll have all my typing changes merged in the next month or two,
at which point we can discuss enabling it as part of the pre-commit lint
job.
-chad
On Tue, Jan 7, 2020 at 7:02 PM Udi Meiri <eh...@google.com> wrote:
> Hi,
> We recently added mypy to the Jenkins Lint job for PRs (currently ignores
> errors). Mypy is a static type checker.
>
> There's a JIRA for adding another static type checker named pytype
> https://issues.apache.org/jira/browse/BEAM-9064
>
> I wanted to ask the community their thoughts on this. (see JIRA issue
> comments as well)
>
> - Should PRs have to pass more than 1 static type checker? (in pre-commit
> tests)
> - If not, should the remaining type checkers be run as a post-commit tests?
> - How much effort should be put into supporting more than 1 type checker?
> (i.e. making sure that they all pass)
>
>
>
>