You are viewing a plain text version of this content. The canonical link for it is here.
Posted to distributedlog-dev@bookkeeper.apache.org by Asko Kauppi <as...@zalando.fi> on 2016/12/21 12:29:06 UTC
Is there a REST API for Write and Read proxies?
I’m reading http://distributedlog.incubator.apache.org/docs/latest/user_guide/api/proxy.html <http://distributedlog.incubator.apache.org/docs/latest/user_guide/api/proxy.html>
Ideally, I wouldn’t need to use a library to talk to a proxy service, right? Is there documentation on how to access the proxies as REST endpoints / are they such?
My preferred environment is Scala and akka-http.
Asko Kauppi
Zalando Tech Helsinki
Re: Is there a REST API for Write and Read proxies?
Posted by Asko Kauppi <as...@zalando.fi>.
I appreciate that, Sijie, but in practise I think the tooling would collide.
I am most productive in Scala and akka-http, and I think those are a good platform for the future. Doing something HTTP and Websocket based in Java is outside of my experience, and willingness.
While merging the two together is practically possible (both are on JVM) I don’t really see the point. But anyways, the low hanging fruit is planning the interface and I already did some drafts on that, as a discussion starter with Jay.
Asko Kauppi
Zalando Tech Helsinki
> On 22 Dec 2016, at 21.43, Sijie Guo <si...@apache.org> wrote:
>
> One thought about the REST/websockets proxy - it would be good that it can
> be part of current proxy (so people doesn't have to deploy additional
> services) or another module for this project **eventually**. You can choose
> the best way for you guys to collaborate on the development. If you guys
> are willing to instantiate this within this project, a new module might be
> a good starting point.
Re: Is there a REST API for Write and Read proxies?
Posted by Sijie Guo <si...@apache.org>.
On Thu, Dec 22, 2016 at 12:58 AM, Asko Kauppi <as...@zalando.fi>
wrote:
> Hi Jay,
>
> > One comment about "a REST / websockets proxy", are you thinking of
> running
> > a separate service or part of current proxy (e.g. listening on a separate
> > port to handle REST/websockets request)?
>
>
> I was thinking of a separate service, and a separate project, at least
> first. I work in Scala and akka-http. The project would contain two parts /
> subprojects:
>
> - Akka Streams interface to the DistributedLog proxy APIs. I have already
> started work on this.
> - the REST API / websocket proxy
> It’s really nice to be collaborating with this, once we have time. Do you
> know what’s the normal procedure within the project, or should we simply do
> it outside since also the development work would be elsewhere?
>
One thought about the REST/websockets proxy - it would be good that it can
be part of current proxy (so people doesn't have to deploy additional
services) or another module for this project **eventually**. You can choose
the best way for you guys to collaborate on the development. If you guys
are willing to instantiate this within this project, a new module might be
a good starting point.
>
> Asko Kauppi
> Zalando Tech Helsinki
>
>
>
> > On 22 Dec 2016, at 6.13, Jay Juma <ja...@gmail.com> wrote:
> >
> > Asko,
> >
> > That's something that I am also looking for. I am interested in that.
> >
> > One comment about "a REST / websockets proxy", are you thinking of
> running
> > a separate service or part of current proxy (e.g. listening on a separate
> > port to handle REST/websockets request)?
> >
> > - Jay
> >
> > On Wed, Dec 21, 2016 at 7:40 PM, Asko Kauppi <as...@zalando.fi>
> wrote:
> >
> >> Thanks for clarification. The question was not meant as criticism, just
> >> trying to understand the architecture.
> >>
> >> Once/if we make a REST API / websockets proxy (of the current proxy), it
> >> could also take care of access control? I’m thinking of a system where
> >> access to data is close to the data, not at the services handling the
> data.
> >>
> >> Anyone interested in working on something like that?
> >>
> >> Asko Kauppi
> >> Zalando Tech Helsinki
> >>
> >>
> >>> On 21 Dec 2016, at 19.09, Leigh Stewart <ls...@twitter.com.INVALID>
> >> wrote:
> >>>
> >>> The challenge is the protocol uses a redirection mechanism so theres a
> >>> protocol beyond thrift/transport.
> >>>
> >>> As Jay says we plan to make improvements in this area.
> >>>
> >>> On Wed, Dec 21, 2016 at 8:53 AM, Jay Juma <ja...@gmail.com> wrote:
> >>>
> >>>> Hi Asko,
> >>>>
> >>>> I don't think there is a REST api available in the proxy service. The
> >> API
> >>>> seems to be thrift-rpc based. I found there is a JIRA to support gPRC
> >> wire
> >>>> protocol. It should not be difficult to add a REST api.
> >>>>
> >>>> - Jay
> >>>>
> >>>> On Wed, Dec 21, 2016 at 4:29 AM, Asko Kauppi <as...@zalando.fi>
> >>>> wrote:
> >>>>
> >>>>> I’m reading http://distributedlog.incubator.apache.org/docs/
> >>>>> latest/user_guide/api/proxy.html <http://distributedlog.
> >>>>> incubator.apache.org/docs/latest/user_guide/api/proxy.html>
> >>>>>
> >>>>> Ideally, I wouldn’t need to use a library to talk to a proxy service,
> >>>>> right? Is there documentation on how to access the proxies as REST
> >>>>> endpoints / are they such?
> >>>>>
> >>>>> My preferred environment is Scala and akka-http.
> >>>>>
> >>>>> Asko Kauppi
> >>>>> Zalando Tech Helsinki
> >>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: Is there a REST API for Write and Read proxies?
Posted by Asko Kauppi <as...@zalando.fi>.
Hi Jay,
> One comment about "a REST / websockets proxy", are you thinking of running
> a separate service or part of current proxy (e.g. listening on a separate
> port to handle REST/websockets request)?
I was thinking of a separate service, and a separate project, at least first. I work in Scala and akka-http. The project would contain two parts / subprojects:
- Akka Streams interface to the DistributedLog proxy APIs. I have already started work on this.
- the REST API / websocket proxy
It’s really nice to be collaborating with this, once we have time. Do you know what’s the normal procedure within the project, or should we simply do it outside since also the development work would be elsewhere?
Asko Kauppi
Zalando Tech Helsinki
> On 22 Dec 2016, at 6.13, Jay Juma <ja...@gmail.com> wrote:
>
> Asko,
>
> That's something that I am also looking for. I am interested in that.
>
> One comment about "a REST / websockets proxy", are you thinking of running
> a separate service or part of current proxy (e.g. listening on a separate
> port to handle REST/websockets request)?
>
> - Jay
>
> On Wed, Dec 21, 2016 at 7:40 PM, Asko Kauppi <as...@zalando.fi> wrote:
>
>> Thanks for clarification. The question was not meant as criticism, just
>> trying to understand the architecture.
>>
>> Once/if we make a REST API / websockets proxy (of the current proxy), it
>> could also take care of access control? I’m thinking of a system where
>> access to data is close to the data, not at the services handling the data.
>>
>> Anyone interested in working on something like that?
>>
>> Asko Kauppi
>> Zalando Tech Helsinki
>>
>>
>>> On 21 Dec 2016, at 19.09, Leigh Stewart <ls...@twitter.com.INVALID>
>> wrote:
>>>
>>> The challenge is the protocol uses a redirection mechanism so theres a
>>> protocol beyond thrift/transport.
>>>
>>> As Jay says we plan to make improvements in this area.
>>>
>>> On Wed, Dec 21, 2016 at 8:53 AM, Jay Juma <ja...@gmail.com> wrote:
>>>
>>>> Hi Asko,
>>>>
>>>> I don't think there is a REST api available in the proxy service. The
>> API
>>>> seems to be thrift-rpc based. I found there is a JIRA to support gPRC
>> wire
>>>> protocol. It should not be difficult to add a REST api.
>>>>
>>>> - Jay
>>>>
>>>> On Wed, Dec 21, 2016 at 4:29 AM, Asko Kauppi <as...@zalando.fi>
>>>> wrote:
>>>>
>>>>> I’m reading http://distributedlog.incubator.apache.org/docs/
>>>>> latest/user_guide/api/proxy.html <http://distributedlog.
>>>>> incubator.apache.org/docs/latest/user_guide/api/proxy.html>
>>>>>
>>>>> Ideally, I wouldn’t need to use a library to talk to a proxy service,
>>>>> right? Is there documentation on how to access the proxies as REST
>>>>> endpoints / are they such?
>>>>>
>>>>> My preferred environment is Scala and akka-http.
>>>>>
>>>>> Asko Kauppi
>>>>> Zalando Tech Helsinki
>>>>>
>>>>>
>>>>
>>
>>
Re: Is there a REST API for Write and Read proxies?
Posted by Jay Juma <ja...@gmail.com>.
Asko,
That's something that I am also looking for. I am interested in that.
One comment about "a REST / websockets proxy", are you thinking of running
a separate service or part of current proxy (e.g. listening on a separate
port to handle REST/websockets request)?
- Jay
On Wed, Dec 21, 2016 at 7:40 PM, Asko Kauppi <as...@zalando.fi> wrote:
> Thanks for clarification. The question was not meant as criticism, just
> trying to understand the architecture.
>
> Once/if we make a REST API / websockets proxy (of the current proxy), it
> could also take care of access control? I’m thinking of a system where
> access to data is close to the data, not at the services handling the data.
>
> Anyone interested in working on something like that?
>
> Asko Kauppi
> Zalando Tech Helsinki
>
>
> > On 21 Dec 2016, at 19.09, Leigh Stewart <ls...@twitter.com.INVALID>
> wrote:
> >
> > The challenge is the protocol uses a redirection mechanism so theres a
> > protocol beyond thrift/transport.
> >
> > As Jay says we plan to make improvements in this area.
> >
> > On Wed, Dec 21, 2016 at 8:53 AM, Jay Juma <ja...@gmail.com> wrote:
> >
> >> Hi Asko,
> >>
> >> I don't think there is a REST api available in the proxy service. The
> API
> >> seems to be thrift-rpc based. I found there is a JIRA to support gPRC
> wire
> >> protocol. It should not be difficult to add a REST api.
> >>
> >> - Jay
> >>
> >> On Wed, Dec 21, 2016 at 4:29 AM, Asko Kauppi <as...@zalando.fi>
> >> wrote:
> >>
> >>> I’m reading http://distributedlog.incubator.apache.org/docs/
> >>> latest/user_guide/api/proxy.html <http://distributedlog.
> >>> incubator.apache.org/docs/latest/user_guide/api/proxy.html>
> >>>
> >>> Ideally, I wouldn’t need to use a library to talk to a proxy service,
> >>> right? Is there documentation on how to access the proxies as REST
> >>> endpoints / are they such?
> >>>
> >>> My preferred environment is Scala and akka-http.
> >>>
> >>> Asko Kauppi
> >>> Zalando Tech Helsinki
> >>>
> >>>
> >>
>
>
Re: Is there a REST API for Write and Read proxies?
Posted by Asko Kauppi <as...@zalando.fi>.
Thanks for clarification. The question was not meant as criticism, just trying to understand the architecture.
Once/if we make a REST API / websockets proxy (of the current proxy), it could also take care of access control? I’m thinking of a system where access to data is close to the data, not at the services handling the data.
Anyone interested in working on something like that?
Asko Kauppi
Zalando Tech Helsinki
> On 21 Dec 2016, at 19.09, Leigh Stewart <ls...@twitter.com.INVALID> wrote:
>
> The challenge is the protocol uses a redirection mechanism so theres a
> protocol beyond thrift/transport.
>
> As Jay says we plan to make improvements in this area.
>
> On Wed, Dec 21, 2016 at 8:53 AM, Jay Juma <ja...@gmail.com> wrote:
>
>> Hi Asko,
>>
>> I don't think there is a REST api available in the proxy service. The API
>> seems to be thrift-rpc based. I found there is a JIRA to support gPRC wire
>> protocol. It should not be difficult to add a REST api.
>>
>> - Jay
>>
>> On Wed, Dec 21, 2016 at 4:29 AM, Asko Kauppi <as...@zalando.fi>
>> wrote:
>>
>>> I’m reading http://distributedlog.incubator.apache.org/docs/
>>> latest/user_guide/api/proxy.html <http://distributedlog.
>>> incubator.apache.org/docs/latest/user_guide/api/proxy.html>
>>>
>>> Ideally, I wouldn’t need to use a library to talk to a proxy service,
>>> right? Is there documentation on how to access the proxies as REST
>>> endpoints / are they such?
>>>
>>> My preferred environment is Scala and akka-http.
>>>
>>> Asko Kauppi
>>> Zalando Tech Helsinki
>>>
>>>
>>
Re: Is there a REST API for Write and Read proxies?
Posted by Leigh Stewart <ls...@twitter.com.INVALID>.
The challenge is the protocol uses a redirection mechanism so theres a
protocol beyond thrift/transport.
As Jay says we plan to make improvements in this area.
On Wed, Dec 21, 2016 at 8:53 AM, Jay Juma <ja...@gmail.com> wrote:
> Hi Asko,
>
> I don't think there is a REST api available in the proxy service. The API
> seems to be thrift-rpc based. I found there is a JIRA to support gPRC wire
> protocol. It should not be difficult to add a REST api.
>
> - Jay
>
> On Wed, Dec 21, 2016 at 4:29 AM, Asko Kauppi <as...@zalando.fi>
> wrote:
>
> > I’m reading http://distributedlog.incubator.apache.org/docs/
> > latest/user_guide/api/proxy.html <http://distributedlog.
> > incubator.apache.org/docs/latest/user_guide/api/proxy.html>
> >
> > Ideally, I wouldn’t need to use a library to talk to a proxy service,
> > right? Is there documentation on how to access the proxies as REST
> > endpoints / are they such?
> >
> > My preferred environment is Scala and akka-http.
> >
> > Asko Kauppi
> > Zalando Tech Helsinki
> >
> >
>
Re: Is there a REST API for Write and Read proxies?
Posted by Jay Juma <ja...@gmail.com>.
Hi Asko,
I don't think there is a REST api available in the proxy service. The API
seems to be thrift-rpc based. I found there is a JIRA to support gPRC wire
protocol. It should not be difficult to add a REST api.
- Jay
On Wed, Dec 21, 2016 at 4:29 AM, Asko Kauppi <as...@zalando.fi> wrote:
> I’m reading http://distributedlog.incubator.apache.org/docs/
> latest/user_guide/api/proxy.html <http://distributedlog.
> incubator.apache.org/docs/latest/user_guide/api/proxy.html>
>
> Ideally, I wouldn’t need to use a library to talk to a proxy service,
> right? Is there documentation on how to access the proxies as REST
> endpoints / are they such?
>
> My preferred environment is Scala and akka-http.
>
> Asko Kauppi
> Zalando Tech Helsinki
>
>