You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "laozhoubuluo (via GitHub)" <gi...@apache.org> on 2024/03/23 09:32:35 UTC
[I] Provide more ways to publish Binary Artifacts [arrow]
laozhoubuluo opened a new issue, #40760:
URL: https://github.com/apache/arrow/issues/40760
### Describe the enhancement requested
Currently, `apache.jfrog.io` is the only release channel for various Binary Artifacts of Apache Arrow. Therefore, if `apache.jfrog.io` stops serving, there will be no other channel to obtain Binary Artifacts. However, `apache.jfrog.io` does not provide stable service. The following is a list of multiple service outages in recent years.
https://github.com/apache/arrow/issues/12686
https://github.com/apache/arrow/issues/34675
https://github.com/apache/arrow/issues/40744
https://github.com/apache/arrow/issues/40759
Considering that this problem has occurred many times, it has seriously affected various downstream software that relies on Apache Arrow. Whether to consider adding at least one other release channel for the Binary Artifacts of Apache Arrow, such as GitHub Releases or `downloads.apache.org` to avoid `apache.jfrog.io` becoming the only release channel for Binary Artifacts. And provide alternatives for obtaining Binary Artifacts in the installation instructions, even if it may require more troublesome methods such as manual installation of DPKG.
The implementation of this measure can effectively reduce risks in the downstream software supply chain and avoid the risk that a single component cannot be installed, resulting in the entire software being unable to be installed, or even the software being unable to continue to work normally.
### Component(s)
Release
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@arrow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2024715428
Thanks.
If we use repository.apache.org for .deb/.rpm, we use https://repo1.maven.org/maven2/org/apache/arrow/debian/ and so on for APT/Yum repositories, right? Hmm. Can we use apache.org domain instead of maven.org domain?
FYI: Our upload script for Java: https://github.com/apache/arrow/blob/main/dev/release/06-java-upload.sh
It uses `mvn deploy:deploy-file`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017508996
@raulcd Thanks ! Much appreciated ! And no worries at all 😄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017435106
I've removed old Releases from dist.apache.org
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2016504762
+1
I talked about this with @jbonofre yesterday and he mentioned that while repository.apache.org currently only hosts java binaries the underlying nexus software can host a number of package repos (rpm, deb, python,...).
GitHub releases could certainly also be a fallback for some things but they don't over repo functionality we need for at the minimum the Linux packages.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2025121713
`maven.org` is an alias to https://repository.apache.org/content/groups/public/ so yeah, we can use repository.apache.org name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017143341
> I think that we can use GitHub Releases for some binaries (which don't require metadata for package repository) but we can't use GitHub Releases for others (which require metadata for package repository, e.g. RPM/deb/wheel).
I was wrong. We can use GitHub Releases for wheel because we publish the voted wheels to https://pypi.org/.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017332006
At Karaf (like most of other Apache projects) we are using both:
- as required by ASF policy, we upload the source distributions to dist.apache.org (that's the only requirement from ASF), as you can see on https://dist.apache.org/repos/dist/release/karaf/
- for convenience, other "binary" files are uploaded to Nexus, as you can see on https://repo1.maven.org/maven2/org/apache/karaf/ (we also duplicate the source distributions here for convenience even if it's not strictly required
We do almost the same in Arrow: the source distributions are on dist (https://dist.apache.org/repos/dist/release/arrow/).
By the way, as dist.apache.org artifacts are automatically copy to archives.apache.org, dist.apache.org should only content only the latest releases (for instance 15.0.0 and 15.0.1, etc should be deleted from dist.apache.org).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "assignUser (via GitHub)" <gi...@apache.org>.
assignUser commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017045806
I assume we can use the [API](https://help.sonatype.com/en/rest-and-integration-api.html) toupload binaries once a matching repository is created, but I haven't looked into it in detail/spoken with infra
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017276439
No need to upgrade to Nexus 3: even with Nexus 2 we can upload any kind of files, via HTTPs for instance. No need to use the Nexus API. Maven release plugin is "just" HTTPs client (via aether).
For instance, in Apache Karaf, I publish features XML, tar.gz, zip, etc.
Manually, it's possible to use `mvn deploy:deploy-file` providing the artifact type, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017281728
Could you tell us which files are uploaded to repository.apache.org?
It seems that files listed in https://karaf.apache.org/download.html use dist.apache.org not repository.apache.org.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "jbonofre (via GitHub)" <gi...@apache.org>.
jbonofre commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2016539677
I think we have different options: nexus, dist, gh.
Let me investigate a bit what could be the easiest one to integrate in our build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017142618
Thanks.
It seems that we can use deb with Nexus Repository Manager 3 or later: https://help.sonatype.com/en/repository-manager-feature-matrix.html
It seems that https://repository.apache.org/ used Nexus Repository Manager 2:
> Nexus Repository Manager 2.14.20-02
Could you ask INFRA whether there is a plan to upgrade repository.apache.org or not?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "kou (via GitHub)" <gi...@apache.org>.
kou commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2016954073
> while repository.apache.org currently only hosts java binaries the underlying nexus software can host a number of package repos (rpm, deb, python,...).
Could you share a document URL how to use repository.apache.org for RPM/deb/wheel?
> I think we have different options: nexus, dist, gh.
I think that we can't use dist.apache.org because our binaries are large to use dist.apache.org.
I think that we can use GitHub Releases for some binaries (which don't require metadata for package repository) but we can't use GitHub Releases for others (which require metadata for package repository, e.g. RPM/deb/wheel).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Provide more ways to publish Binary Artifacts [arrow]
Posted by "raulcd (via GitHub)" <gi...@apache.org>.
raulcd commented on issue #40760:
URL: https://github.com/apache/arrow/issues/40760#issuecomment-2017372936
Yes, sorry, I have to run the remove artifacts task from the post release tasks. I'll do it today.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@arrow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org