You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@datafusion.apache.org by "2010YOUY01 (via GitHub)" <gi...@apache.org> on 2025/10/23 03:32:11 UTC

[PR] doc: Contributor guide for AI-generated PRs [datafusion]

2010YOUY01 opened a new pull request, #18237:
URL: https://github.com/apache/datafusion/pull/18237

   ## Which issue does this PR close?
   
   <!--
   We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes #123` indicates that this PR will close issue #123.
   -->
   
   Part of https://github.com/apache/datafusion/issues/18095
   
   ## Rationale for this change
   
   <!--
    Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed.
    Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes.  
   -->
   
   ## What changes are included in this PR?
   
   <!--
   There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
   -->
   
   In the contributor guide documentation, better explain why large AI-generated PRs without understanding should be rejected. See more in the original issue discussion.
   
   ## Are these changes tested?
   
   <!--
   We typically require tests for all PRs in order to:
   1. Prevent the code from being accidentally broken by subsequent changes
   2. Serve as another way to document the expected behavior of the code
   
   If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
   -->
   
   ## Are there any user-facing changes?
   
   <!--
   If there are user-facing changes then we may require documentation to be updated before approving the PR.
   -->
   
   <!--
   If there are any breaking changes to public APIs, please add the `api change` label.
   -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on code in PR #18237:
URL: https://github.com/apache/datafusion/pull/18237#discussion_r2453925841


##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+For AI-assisted PRs, it is recommended to follow these policies:

Review Comment:
   ```suggestion
   DataFusion has the following policy for AI-assisted PRs:
   ```



##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+For AI-assisted PRs, it is recommended to follow these policies:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.
+- **Calls out unknowns and assumptions**. Missing some corners in a AI-generated solution is fine (e.g. not sure API call X is working for edge case Y), pointing it out can help the review focus)

Review Comment:
   ```suggestion
   - **Calls out unknowns and assumptions**. It's okay to not fully understand some bits of AI generated code. You should comment on these cases and point them out to reviewers so that they can use their knowledge of the codebase to clear up any concerns. For example, you might comment "calling this function here seems to work but I'm not familiar with how it works internally, I wonder if there's a race condition if it is called concurrently".
   ```
   
   Just  a suggestion...



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on PR #18237:
URL: https://github.com/apache/datafusion/pull/18237#issuecomment-3436468712

   > Generally this is great! I think we should get some more feedback but move to merge this quickly and iterate on it if we have to since it's just documentation.
   > 
   > My personal small suggestion would be to prohibit AI generated comments or responses to review. AI generated summaries in the PR description are probably good, humans myself included are pretty lazy about writing detailed descriptions, AI is great at it and it can end up in the commit message as well which is good for browsing git history. Other than that I expect to be conversing with a human when I ask a question. I don't see any scenario where AI generated comments are helpful, other than for translation / language reasons. I don't feel strongly about this, if there's any opposition let's leave it out.
   
   Thanks for the review!
   
   +1 for no AI responses to review.
   
   However, sometimes I feel AI can't really generate a good PR summary, it's mostly repeating code details, but not to capture the important insights behind it to make the PR more approachable to reviewers. I think it would be an interesting discussion around "How to write a good PR description", like the discussion in https://github.com/apache/datafusion/issues/18095


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "2010YOUY01 (via GitHub)" <gi...@apache.org>.
2010YOUY01 commented on code in PR #18237:
URL: https://github.com/apache/datafusion/pull/18237#discussion_r2458497049


##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+DataFusion has the following policy for AI-assisted PRs:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and be able to defend the design and code during review.

Review Comment:
   ```suggestion
   - The PR author should **understand the core ideas** behind the implementation **end-to-end**, and be able to justify the design and code during review.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "adriangb (via GitHub)" <gi...@apache.org>.
adriangb commented on code in PR #18237:
URL: https://github.com/apache/datafusion/pull/18237#discussion_r2453928400


##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+For AI-assisted PRs, it is recommended to follow these policies:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.
+- **Calls out unknowns and assumptions**. Missing some corners in a AI-generated solution is fine (e.g. not sure API call X is working for edge case Y), pointing it out can help the review focus)

Review Comment:
   ```suggestion
   - **Calls out unknowns and assumptions**. It's okay to not fully understand some bits of AI generated code. You should comment on these cases and point them out to reviewers so that they can use their knowledge of the codebase to clear up any concerns. For example, you might comment "calling this function here seems to work but I'm not familiar with how it works internally, I wonder if there's a race condition if it is called concurrently".
   ```
   
   I mainly wanted to make the example more concrete.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on code in PR #18237:
URL: https://github.com/apache/datafusion/pull/18237#discussion_r2457346517


##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+DataFusion has the following policy for AI-assisted PRs:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.
+- **Calls out unknowns and assumptions**. It's okay to not fully understand some bits of AI generated code. You should comment on these cases and point them out to reviewers so that they can use their knowledge of the codebase to clear up any concerns. For example, you might comment "calling this function here seems to work but I'm not familiar with how it works internally, I wonder if there's a race condition if it is called concurrently".
+
+### Why fully AI-generated PRs without understanding are not helpful
+
+Today, AI tools cannot reliably make complex changes to DataFusion on their own, which is why we rely on pull requests and code review.
+
+The purposes of code review are:
+
+1. Finish the intended task.
+2. Share knowledge between authors and reviewers, as a long-term investment in the project. For this reason, even if someone familiar with the codebase can finish a task quickly, we're still happy to help a new contributor work on it even if it takes longer.
+
+An AI dump for an issue doesn’t meet these purposes. Maintainers could finish the task faster by using AI directly, and little knowledge is shared if the author does not intend to understand the solution.

Review Comment:
   I really liked how you described such PRs on https://github.com/apache/datafusion/issues/18095#issuecomment-3434939345. Specifically that code reviews where the submitter is acting as a human proxy for an AI adds no value for either the submitter or reviewer
   
   > This doesn’t work for fully AI-generated PRs without understanding, because reviewers are mostly prompting through a pass-through human proxy. 
   
   Maybe we can work that in too
   
   ```suggestion
   An AI dump for an issue doesn’t meet these purposes. Maintainers could finish the task faster by using AI directly, and the submitters gain little knowledge if they act only as a pass through AI proxy without understanding.
   ```



##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+DataFusion has the following policy for AI-assisted PRs:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.
+- **Calls out unknowns and assumptions**. It's okay to not fully understand some bits of AI generated code. You should comment on these cases and point them out to reviewers so that they can use their knowledge of the codebase to clear up any concerns. For example, you might comment "calling this function here seems to work but I'm not familiar with how it works internally, I wonder if there's a race condition if it is called concurrently".
+
+### Why fully AI-generated PRs without understanding are not helpful
+
+Today, AI tools cannot reliably make complex changes to DataFusion on their own, which is why we rely on pull requests and code review.
+
+The purposes of code review are:
+
+1. Finish the intended task.
+2. Share knowledge between authors and reviewers, as a long-term investment in the project. For this reason, even if someone familiar with the codebase can finish a task quickly, we're still happy to help a new contributor work on it even if it takes longer.
+
+An AI dump for an issue doesn’t meet these purposes. Maintainers could finish the task faster by using AI directly, and little knowledge is shared if the author does not intend to understand the solution.
+
+Please understand the reviewing capacity is **very limited** for the project, so such PRs might be closed or get redirected.

Review Comment:
   Since we don't actually close PRs very often, I predict what will happen is the PRs will just stall b/c too much review capacity is needed
   
   ```suggestion
   Please understand the reviewing capacity is **very limited** for the project,  so large PRs which appear to not have the requisite understanding might not get reviewed, and eventually closed or redirected.
   ```



##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+DataFusion has the following policy for AI-assisted PRs:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.

Review Comment:
   nit: Maybe we could also use the phrase "**justify** the design and code during review" (defend makes it sound to me like we will be attacking the contributor, but I think that is a fairly nuanced opinion)



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "pepijnve (via GitHub)" <gi...@apache.org>.
pepijnve commented on code in PR #18237:
URL: https://github.com/apache/datafusion/pull/18237#discussion_r2454383809


##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+For AI-assisted PRs, it is recommended to follow these policies:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.
+- **Calls out unknowns and assumptions**. Missing some corners in a AI-generated solution is fine (e.g. not sure API call X is working for edge case Y), pointing it out can help the review focus)

Review Comment:
   I like how both of you worded this. It's ok to not understand everything, but please say so up front so people know where to review in more detail.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "Weijun-H (via GitHub)" <gi...@apache.org>.
Weijun-H commented on code in PR #18237:
URL: https://github.com/apache/datafusion/pull/18237#discussion_r2456631895


##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+DataFusion has the following policy for AI-assisted PRs:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.

Review Comment:
   nit
   ```suggestion
   - The PR author should **understand the core ideas** behind the implementation **end-to-end**, and be able to defend the design and code during review.
   ```



##########
docs/source/contributor-guide/index.md:
##########
@@ -175,3 +175,27 @@ The good thing about open code and open development is that any issues in one ch
 
 Pull requests will be marked with a `stale` label after 60 days of inactivity and then closed 7 days after that.
 Commenting on the PR will remove the `stale` label.
+
+## AI-Assisted contributions
+
+DataFusion has the following policy for AI-assisted PRs:
+
+- The PR author should **understand the core ideas** behind the implementation **end-to-end**, and is able to defend the design and code during review.
+- **Calls out unknowns and assumptions**. It's okay to not fully understand some bits of AI generated code. You should comment on these cases and point them out to reviewers so that they can use their knowledge of the codebase to clear up any concerns. For example, you might comment "calling this function here seems to work but I'm not familiar with how it works internally, I wonder if there's a race condition if it is called concurrently".
+
+### Why fully AI-generated PRs without understanding are not helpful
+
+Today, AI tools cannot reliably make complex changes to DataFusion on their own, which is why we rely on pull requests and code review.
+
+The purposes of code review are:
+
+1. Finish the intended task.
+2. Share knowledge between authors and reviewers, as a long-term investment in the project. For this reason, even if someone familiar with the codebase can finish a task quickly, we're still happy to help a new contributor work on it even if it takes longer.
+
+An AI dump for an issue doesn’t meet these purposes. Maintainers could finish the task faster by using AI directly, and little knowledge is shared if the author does not intend to understand the solution.
+
+Please understand the reviewing capacity is **very limited** for the project, so such PRs might be closed or get redirected.
+
+### Better ways to contribute than an “AI dump”
+
+It's recommended to write a high-quality issue with a clear problem statement and a minimal, reproducible example. This can make it easier for others to contribute.

Review Comment:
   👍 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org


Re: [PR] doc: Contributor guide for AI-generated PRs [datafusion]

Posted by "comphead (via GitHub)" <gi...@apache.org>.
comphead merged PR #18237:
URL: https://github.com/apache/datafusion/pull/18237


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscribe@datafusion.apache.org
For additional commands, e-mail: github-help@datafusion.apache.org