You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kibble.apache.org by Robert Munteanu <ro...@apache.org> on 2018/09/13 14:38:26 UTC
API access available?
Hi,
I am trying to simplify my job of keeping the list of sources up-to-
date for the Kibble demo instance. Basically we add git repositories
periodically and I want to add them to Kibble as well.
I was thinking of using the Kibble API to retrieve the list of sources
and checking which of our repositories are not there, so I can easily
paste them in Kibble.
So I tried to access the Kibble API
$ curl -v --netrc https://demo.kibble.apache.org/api/sources
(snip)
* Server auth using Basic with user 'rombert@pache.org'
> GET /api/sources HTTP/1.1
> Host: demo.kibble.apache.org
> Authorization: Basic REDACTED
> User-Agent: curl/7.61.1
> Accept: */*
>
< HTTP/1.1 403 Authentication failed
< Date: Thu, 13 Sep 2018 14:33:51 GMT
< Server: gunicorn/19.6.0
< Content-Type: application/json
< Transfer-Encoding: chunked
<
{
"reason": "You must be logged in to use this API endpoint! %s",
"code": 403
}
Is it possible to get read-only API access to the list of sources? If
not I'll look for something else, but would simplify things for me :-)
Thanks,
Robert
Re: API access available?
Posted by Robert Munteanu <ro...@apache.org>.
On Thu, 2018-09-13 at 16:42 +0200, Daniel Gruno wrote:
> On 09/13/2018 04:38 PM, Robert Munteanu wrote:
> > Hi,
> >
> > I am trying to simplify my job of keeping the list of sources up-
> > to-
> > date for the Kibble demo instance. Basically we add git
> > repositories
> > periodically and I want to add them to Kibble as well.
> >
> > I was thinking of using the Kibble API to retrieve the list of
> > sources
> > and checking which of our repositories are not there, so I can
> > easily
> > paste them in Kibble.
>
> It's a bit convoluted at the moment, apologies.
> First, you have to obtain a session cookie by logging in;
> https://demo.kibble.apache.org/apidoc.html#put-api-session
> when you have that cookie, pass it to the API end point you wish to
> use,
> and it should work just fine.
>
> I'll be working later next week on API tokens for use here, it's one
> of
> the things that are still missing.
I'm trying to set this up but not managing - perhaps there is a flaw in
my code? I'm using something like
$ curl --cookie-jar .cookies -v -X PUT -H "Content-Type: application/json" https://demo.kibble.apache.org/api/session -d "{ \"email\": \"${1}\", \"password\": \"${2}\" }"
$ curl -v --cookie .cookies https://demo.kibble.apache.org/api/session
$ rm .cookies
While the first call succeeds, for the second one I get
< HTTP/1.1 403 Authentication failed
< Date: Mon, 17 Sep 2018 16:24:42 GMT
< Server: gunicorn/19.6.0
< Content-Type: application/json
< Transfer-Encoding: chunked
<
{
"code": 403,
"reason": "You must be logged in to use this API endpoint! %s"
}
I can see the cookie being sent and then returned. But If I got my
timezone calculations correctly, the cookie immediately expires
Just now I got
< Set-Cookie: kibble_session=f7dc532d-dcb4-4fa2-ad80-3e8c83a4b0e2; expires=Tue, 17 Sep 2019 16:27:14 GMT
which looks to be 'now'.
Any hints on how to solve this?
Thanks,
Robert
Re: API access available?
Posted by Daniel Gruno <hu...@apache.org>.
On 09/18/2018 05:52 PM, Robert Munteanu wrote:
> One things which threw me off a bit is that even existing repositories
> now have a status of "Source hasn't been processed yet..." . That looks
> mostly cosmetic though since the old data is still there.
Aaah, yeah, I think the logic here is "this source object was changed,
so we haven't technically scanned it _as is_ yet". Just caught that
myself :)
With regards,
Daniel.
>
> Thanks,
>
> Robert
>
Re: API access available?
Posted by Daniel Gruno <hu...@apache.org>.
On 09/18/2018 05:58 PM, Robert Munteanu wrote:
> On Tue, 2018-09-18 at 17:55 +0200, Daniel Gruno wrote:
>> One thing to note; For GitHub repos, you will need to specify a
>> user/pass if you want github issues/PRs to be looked at - this can be
>> an
>> anonymous account if you like. I'll see if I can't work our regular
>> kibble user into the repos you added, assuming you did not specify
>> any user.
>
> Interesting, I did not know that (maybe add it as a hint on the page?).
> I did not specify any user as I saw no need :-) - GitHub repos are
> public but it would be interesting to see PR information as well.
>
> Thanks,
>
> Robert
>
It should say that the user/pass is required, but perhaps we need to
tweak the wording there.
WTR API access, there's now a WIP token system in place.
Go to https://demo.kibble.apache.org/api/session in a browser, and
you'll see a 'token' element if you're logged in. You can pass that to
curl as a 'Kibble-Token' header to gain access to things.
curl -H 'Kibble-Token: abcdef1234' https://....
With regards,
Daniel.
Re: API access available?
Posted by Daniel Gruno <hu...@apache.org>.
On 09/18/2018 05:58 PM, Robert Munteanu wrote:
> On Tue, 2018-09-18 at 17:55 +0200, Daniel Gruno wrote:
>> One thing to note; For GitHub repos, you will need to specify a
>> user/pass if you want github issues/PRs to be looked at - this can be
>> an
>> anonymous account if you like. I'll see if I can't work our regular
>> kibble user into the repos you added, assuming you did not specify
>> any user.
>
> Interesting, I did not know that (maybe add it as a hint on the page?).
> I did not specify any user as I saw no need :-) - GitHub repos are
> public but it would be interesting to see PR information as well.
>
> Thanks,
>
> Robert
>
I've updated all the sling repos with the kibble default user/pass for
github :)
Re: API access available?
Posted by Robert Munteanu <ro...@apache.org>.
On Tue, 2018-09-18 at 17:55 +0200, Daniel Gruno wrote:
> One thing to note; For GitHub repos, you will need to specify a
> user/pass if you want github issues/PRs to be looked at - this can be
> an
> anonymous account if you like. I'll see if I can't work our regular
> kibble user into the repos you added, assuming you did not specify
> any user.
Interesting, I did not know that (maybe add it as a hint on the page?).
I did not specify any user as I saw no need :-) - GitHub repos are
public but it would be interesting to see PR information as well.
Thanks,
Robert
Re: API access available?
Posted by Daniel Gruno <hu...@apache.org>.
On 09/18/2018 05:52 PM, Robert Munteanu wrote:
> On Thu, 2018-09-13 at 16:45 +0200, Daniel Gruno wrote:
>> On 09/13/2018 04:42 PM, Daniel Gruno wrote:
>>> On 09/13/2018 04:38 PM, Robert Munteanu wrote:
>>>> Hi,
>>>>
>>>> I am trying to simplify my job of keeping the list of sources up-
>>>> to-
>>>> date for the Kibble demo instance. Basically we add git
>>>> repositories
>>>> periodically and I want to add them to Kibble as well.
>>>>
>>>> I was thinking of using the Kibble API to retrieve the list of
>>>> sources
>>>> and checking which of our repositories are not there, so I can
>>>> easily
>>>> paste them in Kibble.
>>>
>>> It's a bit convoluted at the moment, apologies.
>>> First, you have to obtain a session cookie by logging in;
>>> https://demo.kibble.apache.org/apidoc.html#put-api-session
>>> when you have that cookie, pass it to the API end point you wish to
>>> use,
>>> and it should work just fine.
>>>
>>> I'll be working later next week on API tokens for use here, it's
>>> one of
>>> the things that are still missing.
>>>
>>> With regards,
>>> Daniel.
>>>
>>
>> I should add one important/easy thing here:
>>
>> If you want to keep the list of sources up to date with e.g. github,
>> you
>> can just PUT a request with all the sources you have, and it will
>> both
>> update existing ones and add the ones it doesn't have, there are
>> checks
>> against adding duplicates, so only sources that aren't in the DB
>> would
>> get added even if you post old sources to the API.
>>
>
> That is nice! Since I'm having some issues with the API (see earlier
> emails) I used the web form to paste in all Sling repos again. It seems
> to have worked - the repos are added (though I see not yet processed )
> and the old date is still there.
I'll look into the API issue - I think it might be an issue with the
session maker, which MAY need a cookie before you PUT to session...which
obviously shouldn't be a thing :D maybe I should just look into proper
API tokens instead.
>
> One things which threw me off a bit is that even existing repositories
> now have a status of "Source hasn't been processed yet..." . That looks
> mostly cosmetic though since the old data is still there.
I think some of it is a remnant of me copying over a bunch of it
directly from the old database, not through the API.
One thing to note; For GitHub repos, you will need to specify a
user/pass if you want github issues/PRs to be looked at - this can be an
anonymous account if you like. I'll see if I can't work our regular
kibble user into the repos you added, assuming you did not specify any user.
>
> Thanks,
>
> Robert
>
Re: API access available?
Posted by Robert Munteanu <ro...@apache.org>.
On Thu, 2018-09-13 at 16:45 +0200, Daniel Gruno wrote:
> On 09/13/2018 04:42 PM, Daniel Gruno wrote:
> > On 09/13/2018 04:38 PM, Robert Munteanu wrote:
> > > Hi,
> > >
> > > I am trying to simplify my job of keeping the list of sources up-
> > > to-
> > > date for the Kibble demo instance. Basically we add git
> > > repositories
> > > periodically and I want to add them to Kibble as well.
> > >
> > > I was thinking of using the Kibble API to retrieve the list of
> > > sources
> > > and checking which of our repositories are not there, so I can
> > > easily
> > > paste them in Kibble.
> >
> > It's a bit convoluted at the moment, apologies.
> > First, you have to obtain a session cookie by logging in;
> > https://demo.kibble.apache.org/apidoc.html#put-api-session
> > when you have that cookie, pass it to the API end point you wish to
> > use,
> > and it should work just fine.
> >
> > I'll be working later next week on API tokens for use here, it's
> > one of
> > the things that are still missing.
> >
> > With regards,
> > Daniel.
> >
>
> I should add one important/easy thing here:
>
> If you want to keep the list of sources up to date with e.g. github,
> you
> can just PUT a request with all the sources you have, and it will
> both
> update existing ones and add the ones it doesn't have, there are
> checks
> against adding duplicates, so only sources that aren't in the DB
> would
> get added even if you post old sources to the API.
>
That is nice! Since I'm having some issues with the API (see earlier
emails) I used the web form to paste in all Sling repos again. It seems
to have worked - the repos are added (though I see not yet processed )
and the old date is still there.
One things which threw me off a bit is that even existing repositories
now have a status of "Source hasn't been processed yet..." . That looks
mostly cosmetic though since the old data is still there.
Thanks,
Robert
Re: API access available?
Posted by Daniel Gruno <hu...@apache.org>.
On 09/13/2018 04:42 PM, Daniel Gruno wrote:
> On 09/13/2018 04:38 PM, Robert Munteanu wrote:
>> Hi,
>>
>> I am trying to simplify my job of keeping the list of sources up-to-
>> date for the Kibble demo instance. Basically we add git repositories
>> periodically and I want to add them to Kibble as well.
>>
>> I was thinking of using the Kibble API to retrieve the list of sources
>> and checking which of our repositories are not there, so I can easily
>> paste them in Kibble.
>
> It's a bit convoluted at the moment, apologies.
> First, you have to obtain a session cookie by logging in;
> https://demo.kibble.apache.org/apidoc.html#put-api-session
> when you have that cookie, pass it to the API end point you wish to use,
> and it should work just fine.
>
> I'll be working later next week on API tokens for use here, it's one of
> the things that are still missing.
>
> With regards,
> Daniel.
>
I should add one important/easy thing here:
If you want to keep the list of sources up to date with e.g. github, you
can just PUT a request with all the sources you have, and it will both
update existing ones and add the ones it doesn't have, there are checks
against adding duplicates, so only sources that aren't in the DB would
get added even if you post old sources to the API.
Re: API access available?
Posted by Daniel Gruno <hu...@apache.org>.
On 09/13/2018 04:38 PM, Robert Munteanu wrote:
> Hi,
>
> I am trying to simplify my job of keeping the list of sources up-to-
> date for the Kibble demo instance. Basically we add git repositories
> periodically and I want to add them to Kibble as well.
>
> I was thinking of using the Kibble API to retrieve the list of sources
> and checking which of our repositories are not there, so I can easily
> paste them in Kibble.
It's a bit convoluted at the moment, apologies.
First, you have to obtain a session cookie by logging in;
https://demo.kibble.apache.org/apidoc.html#put-api-session
when you have that cookie, pass it to the API end point you wish to use,
and it should work just fine.
I'll be working later next week on API tokens for use here, it's one of
the things that are still missing.
With regards,
Daniel.