Re: RFC: new routing functionality for messenger

[Rajith Attapattu <ra...@gmail.com>]

For starters I would copy this email to the user list.
(In the future we should post things like this to a more wider
audience, especially if we are looking for feedback based on real
world use cases.)

I actually like the minimalistic approach you've taken here. It works
well in an embedded context.
Support for regex will be a welcome addition.
I don't think we should bother with a config file for proton. To me
it's neither here nor there. Let the application developer or the
deployment folks who use the proton based application worry about it.
As both you and Ted pointed out, a more sophisticated routing
capability is best achieved outside the scope of Proton.

Regards,

Rajith

On Mon, Mar 25, 2013 at 9:30 AM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Mon, Mar 25, 2013 at 8:36 AM, Ted Ross <tr...@redhat.com> wrote:
>
>> I generally like this translation facility, though I'll need to think
>> through some use cases to make sure there's sufficient capability in the
>> matching and substitution.
>>
>
> I would definitely appreciate some more feedback on this area. I think some
> real world usage will help us out. I struggled with various options here. I
> didn't want to add a dependency onto a regular expression library without
> at least some kind of fallback as I think this functionality is going to
> end up being too useful to be optional.
>
> There are a number of public domain regular expression libraries out there
> (e.g. t-rex) that are basically designed to be small enough and simple
> enough to be sucked into a codebase wholesale for when you can't afford an
> external dependency. I considered grabbing one of those, but I also wasn't
> 100% sure that full on regex was the best/only option here. There is
> something nice and simple about being able to say route("*", "my-uplink")
> as opposed to writing a fullon regex.
>
> I can see possibly even wanting to support multiple syntax options
> eventually, but for starters I decided to keep it simple. I wrote a very
> small glob style matcher with substitution. It's roughly 100 lines of code
> and should work on any platform. I'm entirely open to replacing this with
> something a real regex library if we want or something else entirely, I
> just wanted something simple/portable to start with so we could actually
> play with the functionality and learn a bit more about what is appropriate.
>
>
>> I question the value of having this as part of the API.  It seems to me
>> that the Messenger routing table should be part of the configuration of a
>> system (i.e. in a place like /etc/qpid/messenger/, or the Windows Registry,
>> etc.).  Eventually, it might be desirable to have an address translation
>> service that can be accessed remotely.  I guess I see this translation as
>> being not the concern of the application, but more of the deployment of the
>> application.
>>
>>
> Initially I was actually going to add the functionality via some kind of
> configuration file as you say. I have a similar conceptual view of the
> functionality as you describe, i.e. it really should only be used as part
> of the deployment/configuration of the messenger, not part of the
> "application work" portion of the API. I ended up going this route for two
> reasons (no pun intended). For starters it was a bit more expedient as I
> didn't have to write some kind of file format parser, but the second, and
> perhaps more important reason, is that I wasn't sure of the implications of
> adding dependencies onto files/file systems. I want messenger to be usable
> from embedded devices that might not even have a file system, so I figured
> it would be safer to offer as a pure API. I do think it would be
> appropriate to layer some kind of system integration on top of this API
> just as you describe.
>
> --Rafael

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

Re: RFC: new routing functionality for messenger

[Rajith Attapattu <ra...@gmail.com>]

On Mon, Mar 25, 2013 at 11:42 AM, Rafael Schloming <rh...@alum.mit.edu> wrote:
> On Mon, Mar 25, 2013 at 10:34 AM, Rajith Attapattu <ra...@gmail.com>wrote:
>
>> For starters I would copy this email to the user list.
>> (In the future we should post things like this to a more wider
>> audience, especially if we are looking for feedback based on real
>> world use cases.)
>>
>
> I posted it where I intended it to go. I don't mind a broader audience
> looking at it, but at this stage I'm really looking for feedback from
> proton developers and people more keyed into messenger's distributed
> messaging model. This is for two reasons: (1) I don't want to push out
> stuff and have people start using it before it is sufficiently reviewed by
> other proton developers, and (2) a lot of the people on the users list
> (perhaps most of them) are quite firmly entrenched in the traditional hub
> and spoke broker model, and from their perspective the primary use for this
> functionality would simply be to route("*", "my-giant-hub-broker/$1") and
> thereby make messenger simulate your traditional hub and spoke client,
> however this functionality is about way more than that.
>
> I think after a round of discussion and documentation (not just API docs)
> it would probably be more digestable to a broader audience. Feel free to
> kick off wider discussion if you wish though, I've no objection, I'm just
> explaining my thinking at this stage.

Thanks for your explanation, and you do have a point about people
starting to use the feature a bit prematurely and perhaps in a
narrower scope than what we would have liked.
Unfortunately the same holds true if we publish it even after we deem
it's ready for general consumption.
Changing the mindset especially after years of being conditioned to a
particular approach is going to be difficult.
The challenge for Proton is to educate our users and the general
development community that we can do much more than the hub-spoke
model.

My desire to bring the discussion to a more wider audience was
motivated by recent comments/observations by Fraser and Gordon.
Within our user base, we do seem to have a few early adopters who
seems keen to be a bit more involved in the development process than a
regular user would.
Perhaps a more reasonable approach would be to copy dev@ instead of
user@ hoping the above crowd is on @dev.

>
>> I actually like the minimalistic approach you've taken here. It works
>> well in an embedded context.
>> Support for regex will be a welcome addition.
>> I don't think we should bother with a config file for proton. To me
>> it's neither here nor there. Let the application developer or the
>> deployment folks who use the proton based application worry about it.
>> As both you and Ted pointed out, a more sophisticated routing
>> capability is best achieved outside the scope of Proton.
>>
>
> I agree it's not the job of a library to provide system level
> integration/configuration, however we should do what we can to make it easy
> for system integrators to do so themselves and, to the extent that we can,
> encourage them to provide as consistent an experience as possible across
> systems, e.g. I considered plugging into system provided regex libraries
> for providing regular expression support, but that could result in slightly
> different styles of regex depending on which system you're on, so I decided
> against that option. (Not saying I don't want to provide regex support,
> just that if we do we should find a way that works the same regardless of
> where you are.)

Thats a fair point, I agree with you.

> Regarding your statement on sophisticated routing capabilities being kept
> outside of proton, I think it's worth pointing out that unless you're
> considering just the hub and spoke topology, routing isn't something that
> is contained to a single component or even a single class of component in
> your network. It's something that everything, including the endpoints in
> your network need to be aware of and cooperatively participate in so that
> you can achieve a given overall effect.

This is indeed what I meant, perhaps I should have elaborated a bit more.
As you said, for complicated networks, the routing aspect will be
de-centralized and achieved by the collaboration/cooperation of all
kinds of nodes/services within the network that may span
organizations.
Therefore it's impossible for us to provide a comprehensive solution
through proton, or other components (routers, bridges, brokers ..etc)
we offer through the larger QPid umbrella.
What we can provide is some simple basic building blocks/helper
utilities that will help the end users to compose more larger complex
routing strategies.

In that regard what you've offered seems quite good. Anything more
will be neither here nor there.

Rajith

> --Rafael

Re: RFC: new routing functionality for messenger

[Rafael Schloming <rh...@alum.mit.edu>]

On Mon, Mar 25, 2013 at 10:34 AM, Rajith Attapattu <ra...@gmail.com>wrote:

> For starters I would copy this email to the user list.
> (In the future we should post things like this to a more wider
> audience, especially if we are looking for feedback based on real
> world use cases.)
>

I posted it where I intended it to go. I don't mind a broader audience
looking at it, but at this stage I'm really looking for feedback from
proton developers and people more keyed into messenger's distributed
messaging model. This is for two reasons: (1) I don't want to push out
stuff and have people start using it before it is sufficiently reviewed by
other proton developers, and (2) a lot of the people on the users list
(perhaps most of them) are quite firmly entrenched in the traditional hub
and spoke broker model, and from their perspective the primary use for this
functionality would simply be to route("*", "my-giant-hub-broker/$1") and
thereby make messenger simulate your traditional hub and spoke client,
however this functionality is about way more than that.

I think after a round of discussion and documentation (not just API docs)
it would probably be more digestable to a broader audience. Feel free to
kick off wider discussion if you wish though, I've no objection, I'm just
explaining my thinking at this stage.


> I actually like the minimalistic approach you've taken here. It works
> well in an embedded context.
> Support for regex will be a welcome addition.
> I don't think we should bother with a config file for proton. To me
> it's neither here nor there. Let the application developer or the
> deployment folks who use the proton based application worry about it.
> As both you and Ted pointed out, a more sophisticated routing
> capability is best achieved outside the scope of Proton.
>

I agree it's not the job of a library to provide system level
integration/configuration, however we should do what we can to make it easy
for system integrators to do so themselves and, to the extent that we can,
encourage them to provide as consistent an experience as possible across
systems, e.g. I considered plugging into system provided regex libraries
for providing regular expression support, but that could result in slightly
different styles of regex depending on which system you're on, so I decided
against that option. (Not saying I don't want to provide regex support,
just that if we do we should find a way that works the same regardless of
where you are.)

Regarding your statement on sophisticated routing capabilities being kept
outside of proton, I think it's worth pointing out that unless you're
considering just the hub and spoke topology, routing isn't something that
is contained to a single component or even a single class of component in
your network. It's something that everything, including the endpoints in
your network need to be aware of and cooperatively participate in so that
you can achieve a given overall effect.

--Rafael