[vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

Hi everyone,

Been doing some research into XMPP compliant servers and happily stumbled
upon Vysper and MINA which I've had great success with in the past.  So, a
few questions:

1.  is it possible to deploy multiple instances of vysper to multiple
servers and they are aware of each other?  so that i can introduce a
solution that has no points of failure...
2.  i want to leverage apache cassandra as a backend repository for data.
to do this, i would create a storage provider that handles the cassandra
interaction?

thanks in advance
-sasha

-- 
Sasha Dolgy
sasha.dolgy@gmail.com

Re: [vysper] questions

[Paul <pa...@sonar.me>]

Definitely. We have already started the implementation, 
so I will get back to the group as soon as it's ready 
to be integrated.

Thanks!

--Paul
Paul Tepper Fisher
CTO, Sonar
http://www.sonar.me

Re: [vysper] questions

[Niklas Gustavsson <ni...@protocol7.com>]

On Thu, Mar 1, 2012 at 2:42 AM, Paul <pa...@sonar.me> wrote:
> Very coincidentally, we happen to be working on a
> clustering approach using Cassandra. So if there is
> any way we can collaborate on this effort, I think
> that would be great. I saw the patch that converts
> ResourceRegistry into an interface, and we can
>  build upon that contribution.

Sounds awesome. If this is something you would like to provide back to
the community, I'm sure we would be interested.

/niklas

Re: [vysper] questions

[Paul <pa...@sonar.me>]

Very coincidentally, we happen to be working on a 
clustering approach using Cassandra. So if there is 
any way we can collaborate on this effort, I think 
that would be great. I saw the patch that converts 
ResourceRegistry into an interface, and we can
 build upon that contribution. 

Thanks!

Paul

Re: [vysper] questions

[Paul <pa...@sonar.me>]

Can you give me some guidance on how to 
address the server-to-server relaying? I am close to being 
complete with a Cassandra implementation of the ResourceRegistry
 and other StorageProviders. But I am a bit stuck on how to go about
 relaying a request from one server to another. I imagine we could 
use the DNS SRV stuff and S2S to get this to work, but I'm not as 
 familiar with that part of the codebase. Or is there something 
that we can use from Mina? I also thought about leveraging 
Akka, but that seems like the wrong direction.

Anyway, please let me know what you think.

I appreciate any help you can provide.

Thanks,

Paul

Niklas Gustavsson <ni...@...> writes:

Re: [vysper] questions

[Niklas Gustavsson <ni...@protocol7.com>]

On Fri, Feb 18, 2011 at 11:51 AM, Ashish <pa...@gmail.com> wrote:
> Would like to hear more on what all we need to cluster :)
> I might be able to help out here a bit, as I am working on something similar.

I haven't really thought too much about this, so take this as me
thinking our loud.

XMPP uses DNS SRV to provide for fallback between multiple running
servers. But, as Bernd points out we would need to share the resource
registry among servers and also provide a way for servers to relay
messages in-between each other (user1@vysper.org on server1.vysper.org
sends a message to user2@vysper.org who happens to be on
server2.vysper.org).

I was kind of planning to suggest clustering support as a Google
Summer of Code project. Does that make sense?

/niklas

Re: [vysper] questions

[Ashish <pa...@gmail.com>]

On Fri, Feb 18, 2011 at 3:30 PM, Niklas Gustavsson <ni...@protocol7.com> wrote:
> On Fri, Feb 18, 2011 at 10:51 AM, Sasha Dolgy <sd...@gmail.com> wrote:
>> Been doing some research into XMPP compliant servers and happily stumbled
>> upon Vysper and MINA which I've had great success with in the past.
>
> That's good to hear :-)
>
>> 1.  is it possible to deploy multiple instances of vysper to multiple
>> servers and they are aware of each other?  so that i can introduce a
>> solution that has no points of failure...
>
> No, we do not currently support clustering of Vysper. However, this is
> an area that I'm personally is interested in looking into so I would
> be happy to collaborate on this if you like.

Would like to hear more on what all we need to cluster :)
I might be able to help out here a bit, as I am working on something similar.

>
>> 2.  i want to leverage apache cassandra as a backend repository for data.
>> to do this, i would create a storage provider that handles the cassandra
>> interaction?

cheers
ashish

Re: [vysper] questions

[Bernd Fondermann <be...@googlemail.com>]

On Fri, Feb 18, 2011 at 14:22, Sasha Dolgy <sd...@gmail.com> wrote:
> One more clarification.  The JCR work in Vysper
> (org.apache.vysper.storage.jcr) is to use JCR as a storage provider for user
> management, roster management, etc.  ?

yep. but I'm not very happy with them.

  Bernd

>
> On Fri, Feb 18, 2011 at 1:29 PM, Bernd Fondermann <
> bernd.fondermann@googlemail.com> wrote:
>
>> On Fri, Feb 18, 2011 at 13:15, Sasha Dolgy <sd...@gmail.com> wrote:
>> > Hi Bernd,
>> >
>> > I had a look at
>> > org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the
>> way
>> > Vysper currently works today is that all sessions are stored in memory?
>>  If
>> > vysper instance crashes, the resource id : session relationships aren't
>> > persisted anywhere?
>>
>> No they aren't. But since XMPP uses long-living TCP/IP connections,
>> recovery from a server crash is equivalent to a
>> client/server-reconnect anyway.
>> This does not impose practical problems. (Leaving a reverse proxy setup
>> aside.)
>> However, a cluster of XMPP servers for the exact same domain(!) would
>> need to share the ResourceRegistry datastructure.
>> (You could also have a federation of XMPP servers, but that'd mean
>> different users might have different account names like
>> you@first.vysper.org, me@second.vysper.org.)
>>
>

Re: [vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

One more clarification.  The JCR work in Vysper
(org.apache.vysper.storage.jcr) is to use JCR as a storage provider for user
management, roster management, etc.  ?

On Fri, Feb 18, 2011 at 1:29 PM, Bernd Fondermann <
bernd.fondermann@googlemail.com> wrote:

> On Fri, Feb 18, 2011 at 13:15, Sasha Dolgy <sd...@gmail.com> wrote:
> > Hi Bernd,
> >
> > I had a look at
> > org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the
> way
> > Vysper currently works today is that all sessions are stored in memory?
>  If
> > vysper instance crashes, the resource id : session relationships aren't
> > persisted anywhere?
>
> No they aren't. But since XMPP uses long-living TCP/IP connections,
> recovery from a server crash is equivalent to a
> client/server-reconnect anyway.
> This does not impose practical problems. (Leaving a reverse proxy setup
> aside.)
> However, a cluster of XMPP servers for the exact same domain(!) would
> need to share the ResourceRegistry datastructure.
> (You could also have a federation of XMPP servers, but that'd mean
> different users might have different account names like
> you@first.vysper.org, me@second.vysper.org.)
>

Re: [vysper] questions

[Bernd Fondermann <be...@googlemail.com>]

On Fri, Feb 18, 2011 at 13:15, Sasha Dolgy <sd...@gmail.com> wrote:
> Hi Bernd,
>
> I had a look at
> org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the way
> Vysper currently works today is that all sessions are stored in memory?  If
> vysper instance crashes, the resource id : session relationships aren't
> persisted anywhere?

No they aren't. But since XMPP uses long-living TCP/IP connections,
recovery from a server crash is equivalent to a
client/server-reconnect anyway.
This does not impose practical problems. (Leaving a reverse proxy setup aside.)
However, a cluster of XMPP servers for the exact same domain(!) would
need to share the ResourceRegistry datastructure.
(You could also have a federation of XMPP servers, but that'd mean
different users might have different account names like
you@first.vysper.org, me@second.vysper.org.)

> It's been years since I properly programmed .. Have only gotten involved in
> Cassandra in the past four weeks and have made some significant mistakes and
> successes along the way.  For me, I am curious to deploy a compliant XMPP
> solution that potentially will leverage the user information i persist in
> cassandra.  i started work on writing a module in my application for user
> interactions (online / offline messages) only to stop suddenly late last
> night and ask myself why i'm re-inventing a wheel ....

Vysper can easily be embedded into other Java apps.

  Bernd

>
> -sd
> On Fri, Feb 18, 2011 at 12:06 PM, Bernd Fondermann <
> bernd.fondermann@googlemail.com> wrote:
>
>> On Fri, Feb 18, 2011 at 11:12, Sasha Dolgy <sd...@gmail.com> wrote:
>> > Hi Niklas,
>> >
>> > The more I think about it, and the more I read, the vysper instances may
>> not
>> > need to be aware of each other ... this can be accomplished by all vysper
>> > instances using a common storage provider.  Similar to how we cluster
>> http
>> > servers ontop of a shared storage pool.
>>
>> To achieve this we would need to push even more state to storage
>> providers than we do now.
>> Hint: org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry
>> must be shared on all instances.
>>
>> > It shifts the complexity down a level and makes mass deployment of new
>> > instances much easier and doesn't overly complicate the current
>> > implementation.
>> >
>> > If I make headway creating a storage provider interfacing with cassandra
>> > i'll drop a note here.
>>
>> By all means, please do that! I think cassandra is the perfect backend
>> for Vysper.
>> If you'd contribute code, that would be awesome (I don't remember
>> having used this word for some time.).
>>
>>  Bernd
>>
>
>
>
> --
> Sasha Dolgy
> sasha.dolgy@gmail.com
>

Re: [vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

Hi Bernd,

I had a look at
org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry ... so the way
Vysper currently works today is that all sessions are stored in memory?  If
vysper instance crashes, the resource id : session relationships aren't
persisted anywhere?

It's been years since I properly programmed .. Have only gotten involved in
Cassandra in the past four weeks and have made some significant mistakes and
successes along the way.  For me, I am curious to deploy a compliant XMPP
solution that potentially will leverage the user information i persist in
cassandra.  i started work on writing a module in my application for user
interactions (online / offline messages) only to stop suddenly late last
night and ask myself why i'm re-inventing a wheel ....

-sd
On Fri, Feb 18, 2011 at 12:06 PM, Bernd Fondermann <
bernd.fondermann@googlemail.com> wrote:

> On Fri, Feb 18, 2011 at 11:12, Sasha Dolgy <sd...@gmail.com> wrote:
> > Hi Niklas,
> >
> > The more I think about it, and the more I read, the vysper instances may
> not
> > need to be aware of each other ... this can be accomplished by all vysper
> > instances using a common storage provider.  Similar to how we cluster
> http
> > servers ontop of a shared storage pool.
>
> To achieve this we would need to push even more state to storage
> providers than we do now.
> Hint: org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry
> must be shared on all instances.
>
> > It shifts the complexity down a level and makes mass deployment of new
> > instances much easier and doesn't overly complicate the current
> > implementation.
> >
> > If I make headway creating a storage provider interfacing with cassandra
> > i'll drop a note here.
>
> By all means, please do that! I think cassandra is the perfect backend
> for Vysper.
> If you'd contribute code, that would be awesome (I don't remember
> having used this word for some time.).
>
>  Bernd
>



-- 
Sasha Dolgy
sasha.dolgy@gmail.com

Re: [vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

> https://issues.apache.org/jira/browse/VYSPER-266
>


uploaded a patch for this.
https://issues.apache.org/jira/secure/attachment/12471431/ResourceRegistry-to-Interface.patch

No doubt the patch could be cleaned up.  Hoping a few others may see some
value in this.  I for one would like the ability to create an alternate
method for the ResourceRegistry like I can with Storage providers which is
why i've done this without affecting the default approach.

Thanks for the mini-tutorial Bernd about getting up to speed and helping
out.   Appreciate it.

-sd

Re: [vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

On Fri, Feb 18, 2011 at 4:43 PM, Bernd Fondermann <
bernd.fondermann@googlemail.com> wrote:

>
> > I started taking a look at the ResourceRegistry as you suggested.
>  Wouldn't
> > a good first step be to abstract this and make it an interface and move
> the
> > existing code into something that then implements it?
>
> +1
>
> > This way people could
> > write different resource registry implementations so that we could see
> just
> > how badly having it back onto Cassandra would be ; )
>
> right :-)
>

https://issues.apache.org/jira/browse/VYSPER-266


>
>
> > Also, what I find a little odd, the password for an entity is in clear
> > text....?  Is this covered in the XMPP protocols anywhere?
>
> Ah. Next JIRA, please. We shouldn't do that, and no storage backend is
> required to store plain text.
> XMPP is agnostic about how passwords are stored.
>
https://issues.apache.org/jira/browse/VYSPER-267

Re: [vysper] questions

[Bernd Fondermann <be...@googlemail.com>]

On Fri, Feb 18, 2011 at 16:35, Sasha Dolgy <sd...@gmail.com> wrote:
> Hi,
>
>  I have a working prototype using cassandra as a storage provider for
> [ org.apache.vysper.xmpp.authorization.AccountManagement/UserAuthorization
> ]
>
> verifyAccountExists(Entity jid)
> addUser(String username, String password)
>
> I started taking a look at the ResourceRegistry as you suggested.  Wouldn't
> a good first step be to abstract this and make it an interface and move the
> existing code into something that then implements it?

+1

> This way people could
> write different resource registry implementations so that we could see just
> how badly having it back onto Cassandra would be ; )

right :-)

> My code isn't pretty.  Having said that, what or where is the best way to
> share it?  Admittedly, it has been years since i've contributed something
> anywhere.

Please open a JIRA ticket (
https://issues.apache.org/jira/browse/VYSPER , registration required)
create a diff (IDEs, Unix, SVN tools etc support that) and attach it
to the JIRA issue.
Note: Don't forget to check the ASF grant tick box.
Thanks for contributing!

> Also, what I find a little odd, the password for an entity is in clear
> text....?  Is this covered in the XMPP protocols anywhere?

Ah. Next JIRA, please. We shouldn't do that, and no storage backend is
required to store plain text.
XMPP is agnostic about how passwords are stored.

Thanks,

  Bernd

>
> -sd
>
>
> On Fri, Feb 18, 2011 at 12:06 PM, Bernd Fondermann <
> bernd.fondermann@googlemail.com> wrote:
>
>
>>  > It shifts the complexity down a level and makes mass deployment of new
>> > instances much easier and doesn't overly complicate the current
>> > implementation.
>> >
>> > If I make headway creating a storage provider interfacing with cassandra
>> > i'll drop a note here.
>>
>>
>> By all means, please do that! I think cassandra is the perfect backend
>> for Vysper.
>> If you'd contribute code, that would be awesome (I don't remember
>> having used this word for some time.).
>>
>>  Bernd
>>
>

Re: [vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

Hi,

 I have a working prototype using cassandra as a storage provider for
[ org.apache.vysper.xmpp.authorization.AccountManagement/UserAuthorization
]

verifyAccountExists(Entity jid)
addUser(String username, String password)

I started taking a look at the ResourceRegistry as you suggested.  Wouldn't
a good first step be to abstract this and make it an interface and move the
existing code into something that then implements it?  This way people could
write different resource registry implementations so that we could see just
how badly having it back onto Cassandra would be ; )

My code isn't pretty.  Having said that, what or where is the best way to
share it?  Admittedly, it has been years since i've contributed something
anywhere.

Also, what I find a little odd, the password for an entity is in clear
text....?  Is this covered in the XMPP protocols anywhere?

-sd


On Fri, Feb 18, 2011 at 12:06 PM, Bernd Fondermann <
bernd.fondermann@googlemail.com> wrote:


>  > It shifts the complexity down a level and makes mass deployment of new
> > instances much easier and doesn't overly complicate the current
> > implementation.
> >
> > If I make headway creating a storage provider interfacing with cassandra
> > i'll drop a note here.
>
>
> By all means, please do that! I think cassandra is the perfect backend
> for Vysper.
> If you'd contribute code, that would be awesome (I don't remember
> having used this word for some time.).
>
>  Bernd
>

Re: [vysper] questions

[Bernd Fondermann <be...@googlemail.com>]

On Fri, Feb 18, 2011 at 11:12, Sasha Dolgy <sd...@gmail.com> wrote:
> Hi Niklas,
>
> The more I think about it, and the more I read, the vysper instances may not
> need to be aware of each other ... this can be accomplished by all vysper
> instances using a common storage provider.  Similar to how we cluster http
> servers ontop of a shared storage pool.

To achieve this we would need to push even more state to storage
providers than we do now.
Hint: org.apache.vysper.xmpp.state.resourcebinding.ResourceRegistry
must be shared on all instances.

> It shifts the complexity down a level and makes mass deployment of new
> instances much easier and doesn't overly complicate the current
> implementation.
>
> If I make headway creating a storage provider interfacing with cassandra
> i'll drop a note here.

By all means, please do that! I think cassandra is the perfect backend
for Vysper.
If you'd contribute code, that would be awesome (I don't remember
having used this word for some time.).

  Bernd

Re: [vysper] questions

[Niklas Gustavsson <ni...@protocol7.com>]

On Fri, Feb 18, 2011 at 4:40 PM, Sasha Dolgy <sd...@gmail.com> wrote:
> Although vysper doesn't currently support it, do the XMPP standards support
> it?

The XMPP RFCs specifies the hostname lookup, using DNS SRV records.
But beyond that, I don't think there is any active spec for how to
cluster-enable XMPP servers.

/niklas

Re: [vysper] questions

[Bernd Fondermann <be...@googlemail.com>]

On Fri, Feb 18, 2011 at 16:40, Sasha Dolgy <sd...@gmail.com> wrote:
> Hi Niklas,
>
> Although vysper doesn't currently support it, do the XMPP standards support
> it?
> -sd

I assume you're referring to the clustering...
No, XMPP does not provide an extension yet. There is one[1], but it is
"deferred" and not recommended for implementation.
The XSF ( xmpp.org ) is a very open-minded community. If you'd want to
go that far you could start a new spec effort there.

   Bernd

[1] http://xmpp.org/extensions/xep-0051.html

>
>
> On Fri, Feb 18, 2011 at 3:48 PM, Niklas Gustavsson <ni...@protocol7.com>wrote:
>
>> I don't think this is sufficient since we do not currently support
>> relaying messages between users connected to the same domain but
>> different servers.
>>
>> /niklas
>>
>

Re: [vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

Hi Niklas,

Although vysper doesn't currently support it, do the XMPP standards support
it?
-sd


On Fri, Feb 18, 2011 at 3:48 PM, Niklas Gustavsson <ni...@protocol7.com>wrote:

> I don't think this is sufficient since we do not currently support
> relaying messages between users connected to the same domain but
> different servers.
>
> /niklas
>

Re: [vysper] questions

[Niklas Gustavsson <ni...@protocol7.com>]

On Fri, Feb 18, 2011 at 11:12 AM, Sasha Dolgy <sd...@gmail.com> wrote:
> The more I think about it, and the more I read, the vysper instances may not
> need to be aware of each other ... this can be accomplished by all vysper
> instances using a common storage provider.  Similar to how we cluster http
> servers ontop of a shared storage pool.

I don't think this is sufficient since we do not currently support
relaying messages between users connected to the same domain but
different servers.

/niklas

Re: [vysper] questions

[Sasha Dolgy <sd...@gmail.com>]

Hi Niklas,

The more I think about it, and the more I read, the vysper instances may not
need to be aware of each other ... this can be accomplished by all vysper
instances using a common storage provider.  Similar to how we cluster http
servers ontop of a shared storage pool.

It shifts the complexity down a level and makes mass deployment of new
instances much easier and doesn't overly complicate the current
implementation.

If I make headway creating a storage provider interfacing with cassandra
i'll drop a note here.

-sd

On Fri, Feb 18, 2011 at 11:00 AM, Niklas Gustavsson <ni...@protocol7.com>wrote:

>
> > 1.  is it possible to deploy multiple instances of vysper to multiple
> > servers and they are aware of each other?  so that i can introduce a
> > solution that has no points of failure...
>
> No, we do not currently support clustering of Vysper. However, this is
> an area that I'm personally is interested in looking into so I would
> be happy to collaborate on this if you like.
>
> > 2.  i want to leverage apache cassandra as a backend repository for data.
> > to do this, i would create a storage provider that handles the cassandra
> > interaction?
>
> Yes.
>
> /niklas
>

Re: [vysper] questions

[Niklas Gustavsson <ni...@protocol7.com>]

On Fri, Feb 18, 2011 at 10:51 AM, Sasha Dolgy <sd...@gmail.com> wrote:
> Been doing some research into XMPP compliant servers and happily stumbled
> upon Vysper and MINA which I've had great success with in the past.

That's good to hear :-)

> 1.  is it possible to deploy multiple instances of vysper to multiple
> servers and they are aware of each other?  so that i can introduce a
> solution that has no points of failure...

No, we do not currently support clustering of Vysper. However, this is
an area that I'm personally is interested in looking into so I would
be happy to collaborate on this if you like.

> 2.  i want to leverage apache cassandra as a backend repository for data.
> to do this, i would create a storage provider that handles the cassandra
> interaction?

Yes.

/niklas