You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by Tadayoshi Sato <sa...@gmail.com> on 2021/05/17 10:11:17 UTC
GitHub API limits hit when building camel-website locally
Hi folks,
When trying to build camel-website locally, quite oftentimes I hit GitHub
API limits for unauthenticated requests and get the errors like this:
$ yarn build:hugo
Building sites … ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
https://api.github.com/repos/apache/camel-k-runtime/issues?state=closed&milestone=10":
Failed to retrieve remote file: Forbidden
ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=18":
Failed to retrieve remote file: Forbidden
ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=11":
Failed to retrieve remote file: Forbidden
...
Is there any good way to eschew this issue when developing locally?
Thank you,
--
Tadayoshi Sato
Re: GitHub API limits hit when building camel-website locally
Posted by Zoran Regvart <zo...@regvart.com>.
Hi Tada,
I'm glad that helped :)
zoran
On Tue, May 18, 2021 at 5:24 AM Tadayoshi Sato <sa...@gmail.com> wrote:
>
> Thanks Karen for your info! It's very helpful.
>
> And today I noticed that Zoran already updated README with a workaround
> solution to the issue:
> https://github.com/apache/camel-website#checks-publishing-the-website
> So far the workaround seems to work fine for me.
>
> On Mon, May 17, 2021 at 10:20 PM Karen Lease <ka...@gmail.com>
> wrote:
>
> > Hi Tadayoshi,
> >
> > I had this same issue very recently land Zoran Regvart gave me this
> > information:
> >
> > > Yes, this is a known limitation, GitHub removed support for providing
> > > authentication tokens via URL parameters and the HTTP calls from Hugo
> > > can't include HTTP headers (https://github.com/gohugoio/hugo/issues/5617).
> > We get around that with caches, i.e.
> > > for Hugo `HUGO_CACHEDIR` can be set to a filesystem path that stores
> > > the caches. On ASF Jenkins we keep the cache, and for the checks
> > > running on GitHub I think we get higher API limits (or so it seems).
> > >
> > > And yes, it's definitely worth mentioning that in the README, I was
> > > hoping that custom headers in HTTP calls would land soon in Hugo so I
> > > didn't invest time in an alternative to getJSON. I think a pragmatic
> > > course of action would be to wait for this to get supported in Hugo
> > > until we see issues in publishing the website.
> >
> > I found that if I ran the build again after a short time (about 15
> > minutes) it passed because the API limit was reset, and then it was
> > cached, at least until I ran clean.
> > So far, I didn't add this information to the documentation about
> > building the website, so I'll put that on my "to do" list.
> >
> > -Karen Lease
> >
> > On 17/05/2021 12:11, Tadayoshi Sato wrote:
> > > Hi folks,
> > >
> > > When trying to build camel-website locally, quite oftentimes I hit GitHub
> > > API limits for unauthenticated requests and get the errors like this:
> > >
> > > $ yarn build:hugo
> > > Building sites … ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> > >
> > https://api.github.com/repos/apache/camel-k-runtime/issues?state=closed&milestone=10
> > ":
> > > Failed to retrieve remote file: Forbidden
> > > ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> > >
> > https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=18
> > ":
> > > Failed to retrieve remote file: Forbidden
> > > ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> > >
> > https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=11
> > ":
> > > Failed to retrieve remote file: Forbidden
> > > ...
> > >
> > >
> > > Is there any good way to eschew this issue when developing locally?
> > >
> > > Thank you,
> > >
> >
>
>
> --
> Tadayoshi Sato
--
Zoran Regvart
Re: GitHub API limits hit when building camel-website locally
Posted by Tadayoshi Sato <sa...@gmail.com>.
Thanks Karen for your info! It's very helpful.
And today I noticed that Zoran already updated README with a workaround
solution to the issue:
https://github.com/apache/camel-website#checks-publishing-the-website
So far the workaround seems to work fine for me.
On Mon, May 17, 2021 at 10:20 PM Karen Lease <ka...@gmail.com>
wrote:
> Hi Tadayoshi,
>
> I had this same issue very recently land Zoran Regvart gave me this
> information:
>
> > Yes, this is a known limitation, GitHub removed support for providing
> > authentication tokens via URL parameters and the HTTP calls from Hugo
> > can't include HTTP headers (https://github.com/gohugoio/hugo/issues/5617).
> We get around that with caches, i.e.
> > for Hugo `HUGO_CACHEDIR` can be set to a filesystem path that stores
> > the caches. On ASF Jenkins we keep the cache, and for the checks
> > running on GitHub I think we get higher API limits (or so it seems).
> >
> > And yes, it's definitely worth mentioning that in the README, I was
> > hoping that custom headers in HTTP calls would land soon in Hugo so I
> > didn't invest time in an alternative to getJSON. I think a pragmatic
> > course of action would be to wait for this to get supported in Hugo
> > until we see issues in publishing the website.
>
> I found that if I ran the build again after a short time (about 15
> minutes) it passed because the API limit was reset, and then it was
> cached, at least until I ran clean.
> So far, I didn't add this information to the documentation about
> building the website, so I'll put that on my "to do" list.
>
> -Karen Lease
>
> On 17/05/2021 12:11, Tadayoshi Sato wrote:
> > Hi folks,
> >
> > When trying to build camel-website locally, quite oftentimes I hit GitHub
> > API limits for unauthenticated requests and get the errors like this:
> >
> > $ yarn build:hugo
> > Building sites … ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> >
> https://api.github.com/repos/apache/camel-k-runtime/issues?state=closed&milestone=10
> ":
> > Failed to retrieve remote file: Forbidden
> > ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> >
> https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=18
> ":
> > Failed to retrieve remote file: Forbidden
> > ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> >
> https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=11
> ":
> > Failed to retrieve remote file: Forbidden
> > ...
> >
> >
> > Is there any good way to eschew this issue when developing locally?
> >
> > Thank you,
> >
>
--
Tadayoshi Sato
Re: GitHub API limits hit when building camel-website locally
Posted by Karen Lease <ka...@gmail.com>.
Hi Tadayoshi,
I had this same issue very recently land Zoran Regvart gave me this
information:
> Yes, this is a known limitation, GitHub removed support for providing
> authentication tokens via URL parameters and the HTTP calls from Hugo
> can't include HTTP headers (https://github.com/gohugoio/hugo/issues/5617). We get around that with caches, i.e.
> for Hugo `HUGO_CACHEDIR` can be set to a filesystem path that stores
> the caches. On ASF Jenkins we keep the cache, and for the checks
> running on GitHub I think we get higher API limits (or so it seems).
>
> And yes, it's definitely worth mentioning that in the README, I was
> hoping that custom headers in HTTP calls would land soon in Hugo so I
> didn't invest time in an alternative to getJSON. I think a pragmatic
> course of action would be to wait for this to get supported in Hugo
> until we see issues in publishing the website.
I found that if I ran the build again after a short time (about 15
minutes) it passed because the API limit was reset, and then it was
cached, at least until I ran clean.
So far, I didn't add this information to the documentation about
building the website, so I'll put that on my "to do" list.
-Karen Lease
On 17/05/2021 12:11, Tadayoshi Sato wrote:
> Hi folks,
>
> When trying to build camel-website locally, quite oftentimes I hit GitHub
> API limits for unauthenticated requests and get the errors like this:
>
> $ yarn build:hugo
> Building sites … ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> https://api.github.com/repos/apache/camel-k-runtime/issues?state=closed&milestone=10":
> Failed to retrieve remote file: Forbidden
> ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=18":
> Failed to retrieve remote file: Forbidden
> ERROR 2021/05/17 19:04:33 Failed to get JSON resource "
> https://api.github.com/repos/apache/camel-k/issues?state=closed&milestone=11":
> Failed to retrieve remote file: Forbidden
> ...
>
>
> Is there any good way to eschew this issue when developing locally?
>
> Thank you,
>