You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "kou (via GitHub)" <gi...@apache.org> on 2024/09/11 08:02:58 UTC

[I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

kou opened a new issue, #44062:
URL: https://github.com/apache/arrow/issues/44062

   ### Describe the enhancement requested
   
   Archery generates product of all testers: https://github.com/apache/arrow/blob/8556001e6a8b4c7f35d4e18c28704d7811005904/dev/archery/archery/integration/runner.py#L88-L90
   
   If we enable C++, Java and Rust, we use the following patterns:
   
   | Producer | Consumer |
   |--------|--------|
   | C++ | C++ |
   | C++ | Java |
   | C++ | Rust | 
   | Java | C++ |
   | Java | Java |
   | Java | Rust | 
   | Rust | C++ |
   | Rust | Java |
   | Rust | Rust | 
   
   In apache/arrow, the following patterns are redundant because they should be done in apache/arrow-rs:
   
   | Producer | Consumer |
   |--------|--------|
   | Rust | Rust | 
   
   In apache/arror-rs, the following patterns are redundant because they should be done in apache/arrow:
   
   | Producer | Consumer |
   |--------|--------|
   | C++ | C++ |
   | C++ | Java |
   | Java | C++ |
   | Java | Java |
   
   Can we remove redundant patterns?
   
   ### Component(s)
   
   Developer Tools, Integration


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2354229440

   Issue resolved by pull request 44099
   https://github.com/apache/arrow/pull/44099


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "zeroshade (via GitHub)" <gi...@apache.org>.
zeroshade commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2343836656

   Also seems like a good idea to me too, thanks @kou 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2351601884

   Thank you @kou 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2350266865

   Yes. The following change is needed with the current #44099 implementation:
   
   ```diff
   diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml
   index 41edc1bb19..dcaf03c44a 100644
   --- a/.github/workflows/integration.yml
   +++ b/.github/workflows/integration.yml
   @@ -64,6 +64,7 @@ jobs:
          ARROW_INTEGRATION_GO: ON
          ARROW_INTEGRATION_JAVA: ON
          ARROW_INTEGRATION_JS: ON
   +      ARROW_INTEGRATION_TARGET_LANGUAGES: "rust"
          ARCHERY_INTEGRATION_WITH_NANOARROW: "1"
          # https://github.com/apache/arrow/pull/38403/files#r1371281630
          ARCHERY_INTEGRATION_WITH_RUST: "1"
   ```
   
   But this may be changed.
   
   I'll open a PR in apache/arrow-rs after I complete this. Please wait for a while.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2342942512

   @alamb @tustvold @paleolimbot  @zeroshade @pitrou What do you think about this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou closed issue #44062: [Dev][Archery][Integration] Reduce needless test matrix
URL: https://github.com/apache/arrow/issues/44062


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2343668002

   Seems like a good idea to me. Thanks @kou 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "paleolimbot (via GitHub)" <gi...@apache.org>.
paleolimbot commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2343884554

   I would also love this for the integration run in nanoarrow (i.e., I would rather only run the nanoarrow-related scenarios!).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "pitrou (via GitHub)" <gi...@apache.org>.
pitrou commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2342978619

   No opinion from me. I suppose it will make the test a bit shorter, but not by much since it's still `O(n**2)` where `n` is the number of implementations.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "alamb (via GitHub)" <gi...@apache.org>.
alamb commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2350133490

   > In apache/arror-rs, the following patterns are redundant because they should be done in apache/arrow:
   
   Can you give me some hint about how to reduce the test matrix in arrow-rs to remove the redundancies?
   
   Here is how the job is run
   
   https://github.com/apache/arrow-rs/blob/889a90436ffd69e94975049b047c413c53b2dfe3/.github/workflows/integration.yml#L56-L67
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2354265584

   https://github.com/apache/arrow-rs/issues/6406


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [Dev][Archery][Integration] Reduce needless test matrix [arrow]

Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #44062:
URL: https://github.com/apache/arrow/issues/44062#issuecomment-2345033343

   Thanks for sharing your opinions.
   I'll implement this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org