You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Patricia Shanahan <pa...@acm.org> on 2015/04/01 16:40:09 UTC
River future direction
I would like to start a discussion of future direction for the River
project, and whether we have the right resources to get there.
Recently, I have been trying to push improving the "Getting Started"
experience in the hope of attracting and keeping new users who may turn
into committers. I am not sure what to do next to make that happen. I
feel we may have a problem of not having enough committer time to get
more committers.
I do have some free time, but not quite the right skills. When I got
involved in River I expected to be working on Java programming for
refactoring, performance enhancement, and multithread/multiprocessor
issues. Writing good "Getting Started" materials calls for actual River
expertise and experience, which I lack.
I have about five weeks before I need to file my next board report. I
would like to use that time to build a consensus on where we are going,
and whether we need help from outside the project.
Patricia
Re: River future direction
Posted by Dawid Loubser <da...@travellinck.com>.
This is an interesting topic, probably with very widely-ranging views
from the community members.
River enables strongly-typed, ad-hoc distributed networks of services.
Strong typing, and real, compiler-checked services contracts are superb,
and offer distinct advantages over River's competing technologies.
Come to think of it - I'd be interested in a discussion over what
exactly River's competitors are?
Both Java as a whole, and Jini/River, has received a bad rap for "bloat"
and "complexity" - and if we can change that perception (and realite,
where needed), all that remains to be asked, is this: What is the
compelling use-case for this technology?
I think we all scoff at this "internet of things" bandwagon, knowing
that Jini provided an infrastructure for this 16+ years ago. In all that
time, however - what has happened? Is now the time?
River has to convince the world about the strengths of this model. It
has to illustrate why code downloading is desirable, and make it easy to
do, and appreciate.
I have to feel that the real use-case is that which it always has been -
small, embedded devices, and smart phones. True peer-to-peer discovery
and interaction, something being tackled from another angle by projects
like ZeroMQ's Project Hydra <https://github.com/innotech/hydra>.
I feel that I would enjoy something like programming my Raspberry Pi
much more, if I could interact with an ever-growing set of clean service
interfaces representing the capabilities of my own device, and other
devices around it on the network, without manually having to wire things
together using all sorts of painful low-level APIs and mix-and-match
technologies.
River should make the "internet of things" come alive, demonstrating a
higher-level understanding of how to describe, and interact with,
services. It should be fun, and obvious.
/
//Ironically, we use River in the enterprise space (not in the embedded
space), but all the River technicalities are largely hidden by the Rio
Project for us. We are using it for the purposes of JavaSpaces, upon
which I've built a flow-based programming -like framework, for
long-running, reliable, flexible processes./
kind regards,
Dawid
On 01/04/2015 16:40, Patricia Shanahan wrote:
> I would like to start a discussion of future direction for the River
> project, and whether we have the right resources to get there.
>
> Recently, I have been trying to push improving the "Getting Started"
> experience in the hope of attracting and keeping new users who may
> turn into committers. I am not sure what to do next to make that
> happen. I feel we may have a problem of not having enough committer
> time to get more committers.
>
> I do have some free time, but not quite the right skills. When I got
> involved in River I expected to be working on Java programming for
> refactoring, performance enhancement, and multithread/multiprocessor
> issues. Writing good "Getting Started" materials calls for actual
> River expertise and experience, which I lack.
>
> I have about five weeks before I need to file my next board report. I
> would like to use that time to build a consensus on where we are
> going, and whether we need help from outside the project.
>
> Patricia
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
I think ultimately we need to get it into a conveniently editable format
- one of the current problems has been a lack of web site integration
and maintenance. Looking ahead, there may be additions or changes to
e.g. extend it to other devices that need doing at a time when you are
not available to do them.
In the short term, dev@river.apache.org does allow attachments. If you
send it in any convenient format we can sort out getting it into the web
site.
Thanks very much,
Patricia
On 4/5/2015 11:25 PM, Bishnu Gautam wrote:
> Thanks Patricia !
> I will work on that. I will update you about the material within a month. Between, let me know if I can upload (PDF file) it in the River website.
> RegardsBishnu
>
>
> Bishnu Prasad Gautam
>
>
>> Date: Fri, 3 Apr 2015 18:04:32 -0700
>> From: pats@acm.org
>> To: dev@river.apache.org; user@river.apache.org
>> Subject: Re: River future direction
>>
>> I think a tutorial on using River with Raspberry Pi would be a valuable
>> asset for the web site.
>>
>> I am getting the feeling that the real strength of River is going to be
>> in what I think of as the "household area network" which includes a
>> house's computers, printers, and IoT devices, as well as the cars and
>> personal devices that travel with household members even when they are
>> away from home.
>>
>> Patricia
>>
>> On 4/2/2015 6:01 PM, Bishnu Gautam wrote:
>>> Hello AllJust a quick note !! I can contribute for "Getting Started"
>>> and tutorials of River. We are working on how to implement services
>>> developed on River by using Raspberry Pi. So, let me know if you
>>> guys are interested about it. I can prepare it and send you the
>>> tutorial of using River on Raspberry Pi within a month. I think once
>>> we are able to provide some tutorial on it, we will be able to bring
>>> River for "Fog Computing" technology. Personally, I propose to bring
>>> River on Internet so that we will have more users and developers and
>>> there will be better future for the people who are involved in
>>> River/Jini technologies. RegardsBishnu
>>>
>>> Bishnu Prasad Gautam
>>>
>>>
>>>> Date: Wed, 1 Apr 2015 08:57:34 -0700 From: pats@acm.org To:
>>>> dev@river.apache.org; user@river.apache.org Subject: Re: River
>>>> future direction
>>>>
>>>> I realized I made a mistake initially posting this only to the
>>>> "dev" mailing list. I definitely want the opinions of any River
>>>> user.
>>>>
>>>> Additionally, a River user who is not interested in River internals
>>>> may be the perfect person to contribute to "Getting Started"
>>>> examples and tutorials.
>>>>
>>>> Patricia (Chair, River PMC)
>>>>
>>>> On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
>>>>> I would like to start a discussion of future direction for the
>>>>> River project, and whether we have the right resources to get
>>>>> there.
>>>>>
>>>>> Recently, I have been trying to push improving the "Getting
>>>>> Started" experience in the hope of attracting and keeping new
>>>>> users who may turn into committers. I am not sure what to do next
>>>>> to make that happen. I feel we may have a problem of not having
>>>>> enough committer time to get more committers.
>>>>>
>>>>> I do have some free time, but not quite the right skills. When I
>>>>> got involved in River I expected to be working on Java
>>>>> programming for refactoring, performance enhancement, and
>>>>> multithread/multiprocessor issues. Writing good "Getting Started"
>>>>> materials calls for actual River expertise and experience, which
>>>>> I lack.
>>>>>
>>>>> I have about five weeks before I need to file my next board
>>>>> report. I would like to use that time to build a consensus on
>>>>> where we are going, and whether we need help from outside the
>>>>> project.
>>>>>
>>>>> Patricia
>>>
>>>
>
>
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
I think ultimately we need to get it into a conveniently editable format
- one of the current problems has been a lack of web site integration
and maintenance. Looking ahead, there may be additions or changes to
e.g. extend it to other devices that need doing at a time when you are
not available to do them.
In the short term, dev@river.apache.org does allow attachments. If you
send it in any convenient format we can sort out getting it into the web
site.
Thanks very much,
Patricia
On 4/5/2015 11:25 PM, Bishnu Gautam wrote:
> Thanks Patricia !
> I will work on that. I will update you about the material within a month. Between, let me know if I can upload (PDF file) it in the River website.
> RegardsBishnu
>
>
> Bishnu Prasad Gautam
>
>
>> Date: Fri, 3 Apr 2015 18:04:32 -0700
>> From: pats@acm.org
>> To: dev@river.apache.org; user@river.apache.org
>> Subject: Re: River future direction
>>
>> I think a tutorial on using River with Raspberry Pi would be a valuable
>> asset for the web site.
>>
>> I am getting the feeling that the real strength of River is going to be
>> in what I think of as the "household area network" which includes a
>> house's computers, printers, and IoT devices, as well as the cars and
>> personal devices that travel with household members even when they are
>> away from home.
>>
>> Patricia
>>
>> On 4/2/2015 6:01 PM, Bishnu Gautam wrote:
>>> Hello AllJust a quick note !! I can contribute for "Getting Started"
>>> and tutorials of River. We are working on how to implement services
>>> developed on River by using Raspberry Pi. So, let me know if you
>>> guys are interested about it. I can prepare it and send you the
>>> tutorial of using River on Raspberry Pi within a month. I think once
>>> we are able to provide some tutorial on it, we will be able to bring
>>> River for "Fog Computing" technology. Personally, I propose to bring
>>> River on Internet so that we will have more users and developers and
>>> there will be better future for the people who are involved in
>>> River/Jini technologies. RegardsBishnu
>>>
>>> Bishnu Prasad Gautam
>>>
>>>
>>>> Date: Wed, 1 Apr 2015 08:57:34 -0700 From: pats@acm.org To:
>>>> dev@river.apache.org; user@river.apache.org Subject: Re: River
>>>> future direction
>>>>
>>>> I realized I made a mistake initially posting this only to the
>>>> "dev" mailing list. I definitely want the opinions of any River
>>>> user.
>>>>
>>>> Additionally, a River user who is not interested in River internals
>>>> may be the perfect person to contribute to "Getting Started"
>>>> examples and tutorials.
>>>>
>>>> Patricia (Chair, River PMC)
>>>>
>>>> On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
>>>>> I would like to start a discussion of future direction for the
>>>>> River project, and whether we have the right resources to get
>>>>> there.
>>>>>
>>>>> Recently, I have been trying to push improving the "Getting
>>>>> Started" experience in the hope of attracting and keeping new
>>>>> users who may turn into committers. I am not sure what to do next
>>>>> to make that happen. I feel we may have a problem of not having
>>>>> enough committer time to get more committers.
>>>>>
>>>>> I do have some free time, but not quite the right skills. When I
>>>>> got involved in River I expected to be working on Java
>>>>> programming for refactoring, performance enhancement, and
>>>>> multithread/multiprocessor issues. Writing good "Getting Started"
>>>>> materials calls for actual River expertise and experience, which
>>>>> I lack.
>>>>>
>>>>> I have about five weeks before I need to file my next board
>>>>> report. I would like to use that time to build a consensus on
>>>>> where we are going, and whether we need help from outside the
>>>>> project.
>>>>>
>>>>> Patricia
>>>
>>>
>
>
RE: River future direction
Posted by Bishnu Gautam <bi...@hotmail.com>.
Thanks Patricia !
I will work on that. I will update you about the material within a month. Between, let me know if I can upload (PDF file) it in the River website.
RegardsBishnu
Bishnu Prasad Gautam
> Date: Fri, 3 Apr 2015 18:04:32 -0700
> From: pats@acm.org
> To: dev@river.apache.org; user@river.apache.org
> Subject: Re: River future direction
>
> I think a tutorial on using River with Raspberry Pi would be a valuable
> asset for the web site.
>
> I am getting the feeling that the real strength of River is going to be
> in what I think of as the "household area network" which includes a
> house's computers, printers, and IoT devices, as well as the cars and
> personal devices that travel with household members even when they are
> away from home.
>
> Patricia
>
> On 4/2/2015 6:01 PM, Bishnu Gautam wrote:
> > Hello AllJust a quick note !! I can contribute for "Getting Started"
> > and tutorials of River. We are working on how to implement services
> > developed on River by using Raspberry Pi. So, let me know if you
> > guys are interested about it. I can prepare it and send you the
> > tutorial of using River on Raspberry Pi within a month. I think once
> > we are able to provide some tutorial on it, we will be able to bring
> > River for "Fog Computing" technology. Personally, I propose to bring
> > River on Internet so that we will have more users and developers and
> > there will be better future for the people who are involved in
> > River/Jini technologies. RegardsBishnu
> >
> > Bishnu Prasad Gautam
> >
> >
> >> Date: Wed, 1 Apr 2015 08:57:34 -0700 From: pats@acm.org To:
> >> dev@river.apache.org; user@river.apache.org Subject: Re: River
> >> future direction
> >>
> >> I realized I made a mistake initially posting this only to the
> >> "dev" mailing list. I definitely want the opinions of any River
> >> user.
> >>
> >> Additionally, a River user who is not interested in River internals
> >> may be the perfect person to contribute to "Getting Started"
> >> examples and tutorials.
> >>
> >> Patricia (Chair, River PMC)
> >>
> >> On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
> >>> I would like to start a discussion of future direction for the
> >>> River project, and whether we have the right resources to get
> >>> there.
> >>>
> >>> Recently, I have been trying to push improving the "Getting
> >>> Started" experience in the hope of attracting and keeping new
> >>> users who may turn into committers. I am not sure what to do next
> >>> to make that happen. I feel we may have a problem of not having
> >>> enough committer time to get more committers.
> >>>
> >>> I do have some free time, but not quite the right skills. When I
> >>> got involved in River I expected to be working on Java
> >>> programming for refactoring, performance enhancement, and
> >>> multithread/multiprocessor issues. Writing good "Getting Started"
> >>> materials calls for actual River expertise and experience, which
> >>> I lack.
> >>>
> >>> I have about five weeks before I need to file my next board
> >>> report. I would like to use that time to build a consensus on
> >>> where we are going, and whether we need help from outside the
> >>> project.
> >>>
> >>> Patricia
> >
> >
Re: River future direction
Posted by Peter <ji...@zeus.net.au>.
Fair comment, let's try to understand the problem domain better first.
Regards,
Peter.
----- Original message -----
> Peter:
>
> Could you expand on “dynamic multiple IP endpoints for clients”? Are
> you thinking of load balancing, failover, or what?
>
> In the case of failover, I’d tend to say “Just let the endpoint fail,
> and then the application can lookup a new one that works” - I’ve tried
> implementing magical behaviour in smart proxies, and never been happy
> with the results. I would tend to say the same with leader election -
> just have the elected leader register the service in Reggie. In the
> case of a new leader, let the old one fail, then the clients can lookup
> the new one.
>
> I’d like to understand better what you’re suggesting.
>
>
> Cheers,
>
> Greg Trasuk
> On Apr 10, 2015, at 9:06 AM, Peter <ji...@zeus.net.au> wrote:
>
> > And if not, do you have any thoughts on leader election and how JERI
> > server side endpoints might communicate state?
> >
> > JERI could use dynamic multiple IP endpoints for clients.
> >
> > Regards,
> >
> > Peter.
> >
> > ----- Original message -----
> > > Thanks Brian, very interesting.
> > >
> > > Regarding high availability and self healing, would you reccommend
> > > River support using zookeeper?
> > >
> > > If so, how are you using it?
> > >
> > > Regards,
> > >
> > > Peter.
> > >
> > >
> > > ----- Original message -----
> > > > We've built a scalable distributed graph database using river (
> > > > www.blazegraph.com, formally www.bigdata.com). There are
> > > > actually two versions that use river.
> > > >
> > > > 1. High availability and self-healing based on low write
> > > > replication. This uses zookeeper for the leader election. It would
> > > > be nice if river supported these semantics in the base
> > > > distribution.
> > > >
> > > > 2. Horizontally scaled using dynamic sharding. This implementation
> > > > uses river to expose the shards on the data services, to
> > > > distribute the evaluation of high level queries over the shards
> > > > and services, etc.
> > > >
> > > > River has made it relatively easy to have services export RMI
> > > > interfaces.
> > > >
> > > > Some possible issues to consider:
> > > >
> > > > a. better communications about the project (i.e., more marketing).
> > > > I think that most of the technical documentation for the river
> > > > ecosystem (other than javadoc) is pretty old. It might still be
> > > > valid, but there is a perception of stagnation. Maybe a weekly
> > > > blog post, project of the month, etc.
> > > > b. bindings for other languages (yes, this is not that easy).
> > > > c. leader election semantics and similar interesting patterns in
> > > > the base platform.
> > > > d. design patterns for building scalable applications on river (it
> > > > is really not that easy to get started if I remember back to when
> > > > I first used the platform in the mid 2000s.)
> > > > e. good tutorials for non-multicast environments (EC2).
> > > > f. good tutorials for security, including living with firewalls and
> > > > constraining ports.
> > > > g. maybe a json interface to reggie so applications can easily see
> > > > what's in the various registrars?
> > > >
> > > > Thanks,
> > > > Bryan
> > > >
> > > >
> > > >
> > > > ----
> > > > Bryan Thompson
> > > > Chief Scientist & Founder
> > > > SYSTAP, LLC
> > > > 4501 Tower Road
> > > > Greensboro, NC 27410
> > > > bryan@systap.com
> > > > http://blazegraph.com
> > > > http://blog.bigdata.com <http://bigdata.com>
> > > > http://mapgraph.io
> > > >
> > > > Blazegraph™ <http://www.blazegraph.com/> is our ultra
> > > > high-performance graph database that supports both RDF/SPARQL and
> > > > Tinkerpop/Blueprints APIs. MapGraph™
> > > > <http://www.systap.com/mapgraph> is our disruptive new technology
> > > > to use GPUs to accelerate data-parallel graph analytics.
> > > >
> > > > CONFIDENTIALITY NOTICE: This email and its contents and
> > > > attachments are for the sole use of the intended recipient(s) and
> > > > are confidential or proprietary to SYSTAP. Any unauthorized
> > > > review, use, disclosure, dissemination or copying of this email or
> > > > its contents or attachments is prohibited. If you have received
> > > > this communication in error, please notify the sender by reply
> > > > email and permanently delete all copies of the email and its
> > > > contents and attachments.
> > > >
> > > > On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk
> > > > <tr...@stratuscom.com> wrote:
> > > >
> > > > >
> > > > > Well, here’s what I’ve used Jini for:
> > > > >
> > > > > Desktop Java applications communicating with a back-end
> > > > > implemented as a set of Jini services in “LAN” scope. In this
> > > > > scenario, Jini has the following valuable features:
> > > > > - Zero-configuration on the client side. The client is able
> > > > > to use multicast discovery to find the lookup service(s) and
> > > > > then lookup the service it wants to use. There is no need for
> > > > > the client app to have a configured url to reach the back-end.
> > > > > - If the backend needs to move to a different host, the clients
> > > > > don’t need to be reconfigured.
> > > > > - The client can export a service endpoint, even without
> > > > > publishing it to the service registrar. It can use that
> > > > > endpoint to receive event notifications. So the user
> > > > > interface can be completely asynchronous. This is not easy to
> > > > > do using a request/response protocol like http (i.e. SOAP or
> > > > > RESTful services).
> > > > >
> > > > > Shared Data Store on a LAN. For example, one service scanned a
> > > > > programmable logic controller for process data. Other clients
> > > > > would display selected portions of that data. For instance, a
> > > > > daemon would display the aforementioned process data on overhead
> > > > > displays. Another daemon took the process data and logged it to
> > > > > a database every eight hours or so. Apart from the
> > > > > zero-configuration aspects mentioned above, Jini lets us avoid
> > > > > setting up a central data store. You want to find a tag called
> > > > > “/cell-006/smt1/placement-defects”? Simply query the lookup
> > > > > service for services implementing the “SharedDataStore”
> > > > > interface, then select the one that lists a service attribute
> > > > > that matches the prefix of the tag you want, then subscribe to
> > > > > change events for it. The entire idea of a distributed
> > > > > key-value store, along with the Paxos-based consistency
> > > > > algorithm and leader-elections that go along with it ——is simply
> > > > > not necessary ! —. You just get the data from the source.
> > > > >
> > > > > Distributed computing framework, roughly based on data flow
> > > > > architecture, using JavaSpaces in a leader/follower pattern.
> > > > > Like any tuple-space system, the architecture has flexible
> > > > > scaling of resources (just add more followers as required).
> > > > > As well, you get automatically optimal dynamic load balancing
> > > > > (since faster processors finish a packet of work faster, they
> > > > > simply pull work out of the JavaSpace in proportion to their
> > > > > speed. Being JavaSpaces, the strong typing and dynamic class
> > > > > loading turns out to be useful in a number of ways, mainly that
> > > > > you can distribute code to the followers without any extra
> > > > > effort.
> > > > >
> > > > > All of the above had aspects of what has recently come to be
> > > > > known as “micro-services architecture” (I’ve been doing this
> > > > > kind of architecture for 30 years, but that’s a different story,
> > > > > and the kids today won’t believe it anyway). In all cases,
> > > > > the ability of Jini services to easily discover and coordinate
> > > > > with other services, with very little static configuration makes
> > > > > the system very flexible.
> > > > >
> > > > > Which leads to my thoughts for the future of River - service
> > > > > integration in the cloud/data centre. I look at the
> > > > > convolutions that people are going through to get service
> > > > > discovery working in a Docker environment (e.g.
> > > > > https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores),
> > > > > and I think that Jini has solved this problem already. The
> > > > > dynamic discovery and zero-configuration nature of Jini, not to
> > > > > mention the inherent fault-tolerance that goes along with
> > > > > leasing, etc, makes Jini perfectly suited to a
> > > > > dynamically-scalable environment. We just haven’t made it
> > > > > easy to get started. Also, in the past, people were often
> > > > > left with the impression that Jini was too complex. I think
> > > > > that people have come around to the idea that the problem-space
> > > > > for distributed computing is complex, so the solution-space is
> > > > > necessarily complex as well.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Greg Trasuk.
> > >
> >
>
Re: River future direction
Posted by Greg Trasuk <tr...@stratuscom.com>.
Peter:
Could you expand on “dynamic multiple IP endpoints for clients”? Are you thinking of load balancing, failover, or what?
In the case of failover, I’d tend to say “Just let the endpoint fail, and then the application can lookup a new one that works” - I’ve tried implementing magical behaviour in smart proxies, and never been happy with the results. I would tend to say the same with leader election - just have the elected leader register the service in Reggie. In the case of a new leader, let the old one fail, then the clients can lookup the new one.
I’d like to understand better what you’re suggesting.
Cheers,
Greg Trasuk
On Apr 10, 2015, at 9:06 AM, Peter <ji...@zeus.net.au> wrote:
> And if not, do you have any thoughts on leader election and how JERI server side endpoints might communicate state?
>
> JERI could use dynamic multiple IP endpoints for clients.
>
> Regards,
>
> Peter.
>
> ----- Original message -----
>> Thanks Brian, very interesting.
>>
>> Regarding high availability and self healing, would you reccommend River
>> support using zookeeper?
>>
>> If so, how are you using it?
>>
>> Regards,
>>
>> Peter.
>>
>>
>> ----- Original message -----
>>> We've built a scalable distributed graph database using river (
>>> www.blazegraph.com, formally www.bigdata.com). There are actually two
>>> versions that use river.
>>>
>>> 1. High availability and self-healing based on low write replication.
>>> This uses zookeeper for the leader election. It would be nice if river
>>> supported these semantics in the base distribution.
>>>
>>> 2. Horizontally scaled using dynamic sharding. This implementation uses
>>> river to expose the shards on the data services, to distribute the
>>> evaluation of high level queries over the shards and services, etc.
>>>
>>> River has made it relatively easy to have services export RMI
>>> interfaces.
>>>
>>> Some possible issues to consider:
>>>
>>> a. better communications about the project (i.e., more marketing). I
>>> think that most of the technical documentation for the river ecosystem
>>> (other than javadoc) is pretty old. It might still be valid, but
>>> there is a perception of stagnation. Maybe a weekly blog post,
>>> project of the month, etc.
>>> b. bindings for other languages (yes, this is not that easy).
>>> c. leader election semantics and similar interesting patterns in the
>>> base platform.
>>> d. design patterns for building scalable applications on river (it is
>>> really not that easy to get started if I remember back to when I first
>>> used the platform in the mid 2000s.)
>>> e. good tutorials for non-multicast environments (EC2).
>>> f. good tutorials for security, including living with firewalls and
>>> constraining ports.
>>> g. maybe a json interface to reggie so applications can easily see
>>> what's in the various registrars?
>>>
>>> Thanks,
>>> Bryan
>>>
>>>
>>>
>>> ----
>>> Bryan Thompson
>>> Chief Scientist & Founder
>>> SYSTAP, LLC
>>> 4501 Tower Road
>>> Greensboro, NC 27410
>>> bryan@systap.com
>>> http://blazegraph.com
>>> http://blog.bigdata.com <http://bigdata.com>
>>> http://mapgraph.io
>>>
>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive
>>> new technology to use GPUs to accelerate data-parallel graph analytics.
>>>
>>> CONFIDENTIALITY NOTICE: This email and its contents and attachments
>>> are for the sole use of the intended recipient(s) and are confidential
>>> or proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>>> dissemination or copying of this email or its contents or attachments
>>> is prohibited. If you have received this communication in error, please
>>> notify the sender by reply email and permanently delete all copies of
>>> the email and its contents and attachments.
>>>
>>> On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com>
>>> wrote:
>>>
>>>>
>>>> Well, here’s what I’ve used Jini for:
>>>>
>>>> Desktop Java applications communicating with a back-end implemented
>>>> as a set of Jini services in “LAN” scope. In this scenario, Jini
>>>> has the following valuable features:
>>>> - Zero-configuration on the client side. The client is able to use
>>>> multicast discovery to find the lookup service(s) and then lookup the
>>>> service it wants to use. There is no need for the client app to
>>>> have a configured url to reach the back-end.
>>>> - If the backend needs to move to a different host, the clients don’t
>>>> need to be reconfigured.
>>>> - The client can export a service endpoint, even without publishing
>>>> it to the service registrar. It can use that endpoint to receive
>>>> event notifications. So the user interface can be completely
>>>> asynchronous. This is not easy to do using a request/response
>>>> protocol like http (i.e. SOAP or RESTful services).
>>>>
>>>> Shared Data Store on a LAN. For example, one service scanned a
>>>> programmable logic controller for process data. Other clients would
>>>> display selected portions of that data. For instance, a daemon
>>>> would display the aforementioned process data on overhead displays.
>>>> Another daemon took the process data and logged it to a database
>>>> every eight hours or so. Apart from the zero-configuration aspects
>>>> mentioned above, Jini lets us avoid setting up a central data store.
>>>> You want to find a tag called “/cell-006/smt1/placement-defects”?
>>>> Simply query the lookup service for services implementing the
>>>> “SharedDataStore” interface, then select the one that lists a
>>>> service attribute that matches the prefix of the tag you want, then
>>>> subscribe to change events for it. The entire idea of a
>>>> distributed key-value store, along with the Paxos-based consistency
>>>> algorithm and leader-elections that go along with it ——is simply not
>>>> necessary ! —. You just get the data from the source.
>>>>
>>>> Distributed computing framework, roughly based on data flow
>>>> architecture, using JavaSpaces in a leader/follower pattern. Like
>>>> any tuple-space system, the architecture has flexible scaling of
>>>> resources (just add more followers as required). As well, you get
>>>> automatically optimal dynamic load balancing (since faster
>>>> processors finish a packet of work faster, they simply pull work out
>>>> of the JavaSpace in proportion to their speed. Being JavaSpaces,
>>>> the strong typing and dynamic class loading turns out to be useful
>>>> in a number of ways, mainly that you can distribute code to the
>>>> followers without any extra effort.
>>>>
>>>> All of the above had aspects of what has recently come to be known as
>>>> “micro-services architecture” (I’ve been doing this kind of
>>>> architecture for 30 years, but that’s a different story, and the kids
>>>> today won’t believe it anyway). In all cases, the ability of Jini
>>>> services to easily discover and coordinate with other services, with
>>>> very little static configuration makes the system very flexible.
>>>>
>>>> Which leads to my thoughts for the future of River - service
>>>> integration in the cloud/data centre. I look at the convolutions
>>>> that people are going through to get service discovery working in a
>>>> Docker environment (e.g.
>>>> https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores),
>>>> and I think that Jini has solved this problem already. The dynamic
>>>> discovery and zero-configuration nature of Jini, not to mention the
>>>> inherent fault-tolerance that goes along with leasing, etc, makes
>>>> Jini perfectly suited to a dynamically-scalable environment. We
>>>> just haven’t made it easy to get started. Also, in the past,
>>>> people were often left with the impression that Jini was too
>>>> complex. I think that people have come around to the idea that the
>>>> problem-space for distributed computing is complex, so the
>>>> solution-space is necessarily complex as well.
>>>>
>>>> Cheers,
>>>>
>>>> Greg Trasuk.
>>
>
Re: River future direction
Posted by Bryan Thompson <br...@systap.com>.
Peter,
My apologies for not responding sooner. There is a fairly in depth
description of how we use zookeeper at [1], which is a white paper that
covers the system architecture for embedded, replicated (HA), and
horizontally scaled deployments. There is a high level presentation on the
HA architecture at [2]. Configuration is described at [3]. The gory
details of the zookeeper integration with the database to support
self-healing in HA are at [4].
Our primary use for zookeeper is:
- Atomic decision making about the order of physical services within a
logical service. I.e., leader election.
We also use zookeeper for maintaining a little bit of additional state
about the quorum (current token, target replication count, etc).
All in all, zookeeper is not easy to use. Perhaps this is in common with
other distributed systems. I know that a commercial version of jini
(gigaspaces) does support leader election patterns (atomic queue
operations). This might be something to fold into river. Another option
would be a paxos library, but at the time that we did this design there was
no suitable library.
Thanks,
Bryan
- [1] Blazegraph White Paper
<http://www.blazegraph.com/whitepapers/bigdata_architecture_whitepaper.pdf>
-
http://www.blazegraph.com/whitepapers/bigdata_architecture_whitepaper.pdf
- [2] HA Replication Cluster
<http://www.bigdata.com/whitepapers/HA-Replication-Cluster.pdf> -
http://www.bigdata.com/whitepapers/HA-Replication-Cluster.pdf
- [3] http://wiki.blazegraph.com/wiki/index.php/HAJournalServer
- [4]
https://docs.google.com/presentation/d/1IdKQaBouV-a3Bjblk8PtXk-L29Rf_8VjInw7xWKwMDA/edit?usp=sharing
(Run state transitions for the HA architecture).
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
bryan@systap.com
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io
Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.
On Fri, Apr 10, 2015 at 9:06 AM, Peter <ji...@zeus.net.au> wrote:
> And if not, do you have any thoughts on leader election and how JERI
> server side endpoints might communicate state?
>
> JERI could use dynamic multiple IP endpoints for clients.
>
> Regards,
>
> Peter.
>
> ----- Original message -----
> > Thanks Brian, very interesting.
> >
> > Regarding high availability and self healing, would you reccommend River
> > support using zookeeper?
> >
> > If so, how are you using it?
> >
> > Regards,
> >
> > Peter.
> >
> >
> > ----- Original message -----
> > > We've built a scalable distributed graph database using river (
> > > www.blazegraph.com, formally www.bigdata.com). There are actually
> two
> > > versions that use river.
> > >
> > > 1. High availability and self-healing based on low write replication.
> > > This uses zookeeper for the leader election. It would be nice if river
> > > supported these semantics in the base distribution.
> > >
> > > 2. Horizontally scaled using dynamic sharding. This implementation
> uses
> > > river to expose the shards on the data services, to distribute the
> > > evaluation of high level queries over the shards and services, etc.
> > >
> > > River has made it relatively easy to have services export RMI
> > > interfaces.
> > >
> > > Some possible issues to consider:
> > >
> > > a. better communications about the project (i.e., more marketing). I
> > > think that most of the technical documentation for the river ecosystem
> > > (other than javadoc) is pretty old. It might still be valid, but
> > > there is a perception of stagnation. Maybe a weekly blog post,
> > > project of the month, etc.
> > > b. bindings for other languages (yes, this is not that easy).
> > > c. leader election semantics and similar interesting patterns in the
> > > base platform.
> > > d. design patterns for building scalable applications on river (it is
> > > really not that easy to get started if I remember back to when I first
> > > used the platform in the mid 2000s.)
> > > e. good tutorials for non-multicast environments (EC2).
> > > f. good tutorials for security, including living with firewalls and
> > > constraining ports.
> > > g. maybe a json interface to reggie so applications can easily see
> > > what's in the various registrars?
> > >
> > > Thanks,
> > > Bryan
> > >
> > >
> > >
> > > ----
> > > Bryan Thompson
> > > Chief Scientist & Founder
> > > SYSTAP, LLC
> > > 4501 Tower Road
> > > Greensboro, NC 27410
> > > bryan@systap.com
> > > http://blazegraph.com
> > > http://blog.bigdata.com <http://bigdata.com>
> > > http://mapgraph.io
> > >
> > > Blazegraph™ <http://www.blazegraph.com/> is our ultra
> high-performance
> > > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> > > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our
> disruptive
> > > new technology to use GPUs to accelerate data-parallel graph
> analytics.
> > >
> > > CONFIDENTIALITY NOTICE: This email and its contents and attachments
> > > are for the sole use of the intended recipient(s) and are confidential
> > > or proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> > > dissemination or copying of this email or its contents or attachments
> > > is prohibited. If you have received this communication in error,
> please
> > > notify the sender by reply email and permanently delete all copies of
> > > the email and its contents and attachments.
> > >
> > > On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com>
> > > wrote:
> > >
> > > >
> > > > Well, here’s what I’ve used Jini for:
> > > >
> > > > Desktop Java applications communicating with a back-end implemented
> > > > as a set of Jini services in “LAN” scope. In this scenario, Jini
> > > > has the following valuable features:
> > > > - Zero-configuration on the client side. The client is able to use
> > > > multicast discovery to find the lookup service(s) and then lookup
> the
> > > > service it wants to use. There is no need for the client app to
> > > > have a configured url to reach the back-end.
> > > > - If the backend needs to move to a different host, the clients
> don’t
> > > > need to be reconfigured.
> > > > - The client can export a service endpoint, even without publishing
> > > > it to the service registrar. It can use that endpoint to receive
> > > > event notifications. So the user interface can be completely
> > > > asynchronous. This is not easy to do using a request/response
> > > > protocol like http (i.e. SOAP or RESTful services).
> > > >
> > > > Shared Data Store on a LAN. For example, one service scanned a
> > > > programmable logic controller for process data. Other clients
> would
> > > > display selected portions of that data. For instance, a daemon
> > > > would display the aforementioned process data on overhead displays.
> > > > Another daemon took the process data and logged it to a database
> > > > every eight hours or so. Apart from the zero-configuration aspects
> > > > mentioned above, Jini lets us avoid setting up a central data store.
> > > > You want to find a tag called “/cell-006/smt1/placement-defects”?
> > > > Simply query the lookup service for services implementing the
> > > > “SharedDataStore” interface, then select the one that lists a
> > > > service attribute that matches the prefix of the tag you want, then
> > > > subscribe to change events for it. The entire idea of a
> > > > distributed key-value store, along with the Paxos-based consistency
> > > > algorithm and leader-elections that go along with it ——is simply not
> > > > necessary ! —. You just get the data from the source.
> > > >
> > > > Distributed computing framework, roughly based on data flow
> > > > architecture, using JavaSpaces in a leader/follower pattern. Like
> > > > any tuple-space system, the architecture has flexible scaling of
> > > > resources (just add more followers as required). As well, you get
> > > > automatically optimal dynamic load balancing (since faster
> > > > processors finish a packet of work faster, they simply pull work out
> > > > of the JavaSpace in proportion to their speed. Being JavaSpaces,
> > > > the strong typing and dynamic class loading turns out to be useful
> > > > in a number of ways, mainly that you can distribute code to the
> > > > followers without any extra effort.
> > > >
> > > > All of the above had aspects of what has recently come to be known
> as
> > > > “micro-services architecture” (I’ve been doing this kind of
> > > > architecture for 30 years, but that’s a different story, and the
> kids
> > > > today won’t believe it anyway). In all cases, the ability of Jini
> > > > services to easily discover and coordinate with other services, with
> > > > very little static configuration makes the system very flexible.
> > > >
> > > > Which leads to my thoughts for the future of River - service
> > > > integration in the cloud/data centre. I look at the convolutions
> > > > that people are going through to get service discovery working in a
> > > > Docker environment (e.g.
> > > >
> https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores),
>
> > > > and I think that Jini has solved this problem already. The dynamic
> > > > discovery and zero-configuration nature of Jini, not to mention the
> > > > inherent fault-tolerance that goes along with leasing, etc, makes
> > > > Jini perfectly suited to a dynamically-scalable environment. We
> > > > just haven’t made it easy to get started. Also, in the past,
> > > > people were often left with the impression that Jini was too
> > > > complex. I think that people have come around to the idea that the
> > > > problem-space for distributed computing is complex, so the
> > > > solution-space is necessarily complex as well.
> > > >
> > > > Cheers,
> > > >
> > > > Greg Trasuk.
> >
>
>
Re: River future direction
Posted by Peter <ji...@zeus.net.au>.
And if not, do you have any thoughts on leader election and how JERI server side endpoints might communicate state?
JERI could use dynamic multiple IP endpoints for clients.
Regards,
Peter.
----- Original message -----
> Thanks Brian, very interesting.
>
> Regarding high availability and self healing, would you reccommend River
> support using zookeeper?
>
> If so, how are you using it?
>
> Regards,
>
> Peter.
>
>
> ----- Original message -----
> > We've built a scalable distributed graph database using river (
> > www.blazegraph.com, formally www.bigdata.com). There are actually two
> > versions that use river.
> >
> > 1. High availability and self-healing based on low write replication.
> > This uses zookeeper for the leader election. It would be nice if river
> > supported these semantics in the base distribution.
> >
> > 2. Horizontally scaled using dynamic sharding. This implementation uses
> > river to expose the shards on the data services, to distribute the
> > evaluation of high level queries over the shards and services, etc.
> >
> > River has made it relatively easy to have services export RMI
> > interfaces.
> >
> > Some possible issues to consider:
> >
> > a. better communications about the project (i.e., more marketing). I
> > think that most of the technical documentation for the river ecosystem
> > (other than javadoc) is pretty old. It might still be valid, but
> > there is a perception of stagnation. Maybe a weekly blog post,
> > project of the month, etc.
> > b. bindings for other languages (yes, this is not that easy).
> > c. leader election semantics and similar interesting patterns in the
> > base platform.
> > d. design patterns for building scalable applications on river (it is
> > really not that easy to get started if I remember back to when I first
> > used the platform in the mid 2000s.)
> > e. good tutorials for non-multicast environments (EC2).
> > f. good tutorials for security, including living with firewalls and
> > constraining ports.
> > g. maybe a json interface to reggie so applications can easily see
> > what's in the various registrars?
> >
> > Thanks,
> > Bryan
> >
> >
> >
> > ----
> > Bryan Thompson
> > Chief Scientist & Founder
> > SYSTAP, LLC
> > 4501 Tower Road
> > Greensboro, NC 27410
> > bryan@systap.com
> > http://blazegraph.com
> > http://blog.bigdata.com <http://bigdata.com>
> > http://mapgraph.io
> >
> > Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> > graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> > APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive
> > new technology to use GPUs to accelerate data-parallel graph analytics.
> >
> > CONFIDENTIALITY NOTICE: This email and its contents and attachments
> > are for the sole use of the intended recipient(s) and are confidential
> > or proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> > dissemination or copying of this email or its contents or attachments
> > is prohibited. If you have received this communication in error, please
> > notify the sender by reply email and permanently delete all copies of
> > the email and its contents and attachments.
> >
> > On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com>
> > wrote:
> >
> > >
> > > Well, here’s what I’ve used Jini for:
> > >
> > > Desktop Java applications communicating with a back-end implemented
> > > as a set of Jini services in “LAN” scope. In this scenario, Jini
> > > has the following valuable features:
> > > - Zero-configuration on the client side. The client is able to use
> > > multicast discovery to find the lookup service(s) and then lookup the
> > > service it wants to use. There is no need for the client app to
> > > have a configured url to reach the back-end.
> > > - If the backend needs to move to a different host, the clients don’t
> > > need to be reconfigured.
> > > - The client can export a service endpoint, even without publishing
> > > it to the service registrar. It can use that endpoint to receive
> > > event notifications. So the user interface can be completely
> > > asynchronous. This is not easy to do using a request/response
> > > protocol like http (i.e. SOAP or RESTful services).
> > >
> > > Shared Data Store on a LAN. For example, one service scanned a
> > > programmable logic controller for process data. Other clients would
> > > display selected portions of that data. For instance, a daemon
> > > would display the aforementioned process data on overhead displays.
> > > Another daemon took the process data and logged it to a database
> > > every eight hours or so. Apart from the zero-configuration aspects
> > > mentioned above, Jini lets us avoid setting up a central data store.
> > > You want to find a tag called “/cell-006/smt1/placement-defects”?
> > > Simply query the lookup service for services implementing the
> > > “SharedDataStore” interface, then select the one that lists a
> > > service attribute that matches the prefix of the tag you want, then
> > > subscribe to change events for it. The entire idea of a
> > > distributed key-value store, along with the Paxos-based consistency
> > > algorithm and leader-elections that go along with it ——is simply not
> > > necessary ! —. You just get the data from the source.
> > >
> > > Distributed computing framework, roughly based on data flow
> > > architecture, using JavaSpaces in a leader/follower pattern. Like
> > > any tuple-space system, the architecture has flexible scaling of
> > > resources (just add more followers as required). As well, you get
> > > automatically optimal dynamic load balancing (since faster
> > > processors finish a packet of work faster, they simply pull work out
> > > of the JavaSpace in proportion to their speed. Being JavaSpaces,
> > > the strong typing and dynamic class loading turns out to be useful
> > > in a number of ways, mainly that you can distribute code to the
> > > followers without any extra effort.
> > >
> > > All of the above had aspects of what has recently come to be known as
> > > “micro-services architecture” (I’ve been doing this kind of
> > > architecture for 30 years, but that’s a different story, and the kids
> > > today won’t believe it anyway). In all cases, the ability of Jini
> > > services to easily discover and coordinate with other services, with
> > > very little static configuration makes the system very flexible.
> > >
> > > Which leads to my thoughts for the future of River - service
> > > integration in the cloud/data centre. I look at the convolutions
> > > that people are going through to get service discovery working in a
> > > Docker environment (e.g.
> > > https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores),
> > > and I think that Jini has solved this problem already. The dynamic
> > > discovery and zero-configuration nature of Jini, not to mention the
> > > inherent fault-tolerance that goes along with leasing, etc, makes
> > > Jini perfectly suited to a dynamically-scalable environment. We
> > > just haven’t made it easy to get started. Also, in the past,
> > > people were often left with the impression that Jini was too
> > > complex. I think that people have come around to the idea that the
> > > problem-space for distributed computing is complex, so the
> > > solution-space is necessarily complex as well.
> > >
> > > Cheers,
> > >
> > > Greg Trasuk.
>
Re: River future direction
Posted by Peter <ji...@zeus.net.au>.
Thanks Brian, very interesting.
Regarding high availability and self healing, would you reccommend River support using zookeeper?
If so, how are you using it?
Regards,
Peter.
----- Original message -----
> We've built a scalable distributed graph database using river (
> www.blazegraph.com, formally www.bigdata.com). There are actually two
> versions that use river.
>
> 1. High availability and self-healing based on low write replication.
> This uses zookeeper for the leader election. It would be nice if river
> supported these semantics in the base distribution.
>
> 2. Horizontally scaled using dynamic sharding. This implementation uses
> river to expose the shards on the data services, to distribute the
> evaluation of high level queries over the shards and services, etc.
>
> River has made it relatively easy to have services export RMI interfaces.
>
> Some possible issues to consider:
>
> a. better communications about the project (i.e., more marketing). I
> think that most of the technical documentation for the river ecosystem
> (other than javadoc) is pretty old. It might still be valid, but there
> is a perception of stagnation. Maybe a weekly blog post, project of the
> month, etc.
> b. bindings for other languages (yes, this is not that easy).
> c. leader election semantics and similar interesting patterns in the base
> platform.
> d. design patterns for building scalable applications on river (it is
> really not that easy to get started if I remember back to when I first
> used the platform in the mid 2000s.)
> e. good tutorials for non-multicast environments (EC2).
> f. good tutorials for security, including living with firewalls and
> constraining ports.
> g. maybe a json interface to reggie so applications can easily see what's
> in the various registrars?
>
> Thanks,
> Bryan
>
>
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> bryan@systap.com
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please
> notify the sender by reply email and permanently delete all copies of
> the email and its contents and attachments.
>
> On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com>
> wrote:
>
> >
> > Well, here’s what I’ve used Jini for:
> >
> > Desktop Java applications communicating with a back-end implemented as
> > a set of Jini services in “LAN” scope. In this scenario, Jini has the
> > following valuable features:
> > - Zero-configuration on the client side. The client is able to use
> > multicast discovery to find the lookup service(s) and then lookup the
> > service it wants to use. There is no need for the client app to have a
> > configured url to reach the back-end.
> > - If the backend needs to move to a different host, the clients don’t
> > need to be reconfigured.
> > - The client can export a service endpoint, even without publishing it
> > to the service registrar. It can use that endpoint to receive event
> > notifications. So the user interface can be completely asynchronous.
> > This is not easy to do using a request/response protocol like http
> > (i.e. SOAP or RESTful services).
> >
> > Shared Data Store on a LAN. For example, one service scanned a
> > programmable logic controller for process data. Other clients would
> > display selected portions of that data. For instance, a daemon would
> > display the aforementioned process data on overhead displays. Another
> > daemon took the process data and logged it to a database every eight
> > hours or so. Apart from the zero-configuration aspects mentioned
> > above, Jini lets us avoid setting up a central data store. You want
> > to find a tag called “/cell-006/smt1/placement-defects”? Simply query
> > the lookup service for services implementing the “SharedDataStore”
> > interface, then select the one that lists a service attribute that
> > matches the prefix of the tag you want, then subscribe to change
> > events for it. The entire idea of a distributed key-value store,
> > along with the Paxos-based consistency algorithm and leader-elections
> > that go along with it ——is simply not necessary ! —. You just get the
> > data from the source.
> >
> > Distributed computing framework, roughly based on data flow
> > architecture, using JavaSpaces in a leader/follower pattern. Like any
> > tuple-space system, the architecture has flexible scaling of resources
> > (just add more followers as required). As well, you get automatically
> > optimal dynamic load balancing (since faster processors finish a
> > packet of work faster, they simply pull work out of the JavaSpace in
> > proportion to their speed. Being JavaSpaces, the strong typing and
> > dynamic class loading turns out to be useful in a number of ways,
> > mainly that you can distribute code to the followers without any extra
> > effort.
> >
> > All of the above had aspects of what has recently come to be known as
> > “micro-services architecture” (I’ve been doing this kind of
> > architecture for 30 years, but that’s a different story, and the kids
> > today won’t believe it anyway). In all cases, the ability of Jini
> > services to easily discover and coordinate with other services, with
> > very little static configuration makes the system very flexible.
> >
> > Which leads to my thoughts for the future of River - service
> > integration in the cloud/data centre. I look at the convolutions that
> > people are going through to get service discovery working in a Docker
> > environment (e.g.
> > https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores),
> > and I think that Jini has solved this problem already. The dynamic
> > discovery and zero-configuration nature of Jini, not to mention the
> > inherent fault-tolerance that goes along with leasing, etc, makes Jini
> > perfectly suited to a dynamically-scalable environment. We just
> > haven’t made it easy to get started. Also, in the past, people were
> > often left with the impression that Jini was too complex. I think
> > that people have come around to the idea that the problem-space for
> > distributed computing is complex, so the solution-space is necessarily
> > complex as well.
> >
> > Cheers,
> >
> > Greg Trasuk.
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
Good idea. I've copied the user list.
On 4/6/2015 5:53 AM, Bryan Thompson wrote:
> Sure. Maybe we could draw together 10-20 success stories and publish them
> out one a week for a few months? We could certainly point our user base at
> the success story posts for river. It would be nice to use this as an
> opportunity to convince users that jini/river was a good idea as a
> distributed systems platform. This is purely a marketing issue, so having a
> flurry of good press would be nice.
>
> Thanks,
> Bryan
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> bryan@systap.com
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Mon, Apr 6, 2015 at 8:48 AM, Patricia Shanahan <pa...@acm.org> wrote:
>
>> On 4/6/2015 4:55 AM, Bryan Thompson wrote:
>>
>>> We've built a scalable distributed graph database using river (
>>> www.blazegraph.com, formally www.bigdata.com). There are actually two
>>> versions that use river.
>>>
>>> 1. High availability and self-healing based on low write replication. This
>>> uses zookeeper for the leader election. It would be nice if river
>>> supported
>>> these semantics in the base distribution.
>>>
>>> 2. Horizontally scaled using dynamic sharding. This implementation uses
>>> river to expose the shards on the data services, to distribute the
>>> evaluation of high level queries over the shards and services, etc.
>>>
>>> River has made it relatively easy to have services export RMI interfaces.
>>>
>>
>> Perhaps a "success" story we could post on the web site?
>>
>>
>>> Some possible issues to consider:
>>>
>>> a. better communications about the project (i.e., more marketing). I think
>>> that most of the technical documentation for the river ecosystem (other
>>> than javadoc) is pretty old. It might still be valid, but there is a
>>> perception of stagnation. Maybe a weekly blog post, project of the month,
>>> etc.
>>>
>>
>> I don't know whether I am right about this, and it is one of the topics on
>> which I am really looking for feedback, but I feel we have to fix our
>> "Getting started" experience first. Once that is done, we have a better
>> chance of converting marketing leads into active users.
>>
>> b. bindings for other languages (yes, this is not that easy).
>>> c. leader election semantics and similar interesting patterns in the base
>>> platform.
>>> d. design patterns for building scalable applications on river (it is
>>> really not that easy to get started if I remember back to when I first
>>> used
>>> the platform in the mid 2000s.)
>>> e. good tutorials for non-multicast environments (EC2).
>>> f. good tutorials for security, including living with firewalls and
>>> constraining ports.
>>> g. maybe a json interface to reggie so applications can easily see what's
>>> in the various registrars?
>>>
>>> Thanks,
>>> Bryan
>>>
>>>
>>>
>>> ----
>>> Bryan Thompson
>>> Chief Scientist & Founder
>>> SYSTAP, LLC
>>> 4501 Tower Road
>>> Greensboro, NC 27410
>>> bryan@systap.com
>>> http://blazegraph.com
>>> http://blog.bigdata.com <http://bigdata.com>
>>> http://mapgraph.io
>>>
>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
>>> technology to use GPUs to accelerate data-parallel graph analytics.
>>>
>>> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
>>> for the sole use of the intended recipient(s) and are confidential or
>>> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>>> dissemination or copying of this email or its contents or attachments is
>>> prohibited. If you have received this communication in error, please
>>> notify
>>> the sender by reply email and permanently delete all copies of the email
>>> and its contents and attachments.
>>>
>>> On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com>
>>> wrote:
>>>
>>>
>>>> Well, here’s what I’ve used Jini for:
>>>>
>>>> Desktop Java applications communicating with a back-end implemented as a
>>>> set of Jini services in “LAN” scope. In this scenario, Jini has the
>>>> following valuable features:
>>>> - Zero-configuration on the client side. The client is able to use
>>>> multicast discovery to find the lookup service(s) and then lookup the
>>>> service it wants to use. There is no need for the client app to have a
>>>> configured url to reach the back-end.
>>>> - If the backend needs to move to a different host, the clients don’t
>>>> need
>>>> to be reconfigured.
>>>> - The client can export a service endpoint, even without publishing it to
>>>> the service registrar. It can use that endpoint to receive event
>>>> notifications. So the user interface can be completely asynchronous.
>>>> This
>>>> is not easy to do using a request/response protocol like http (i.e. SOAP
>>>> or
>>>> RESTful services).
>>>>
>>>> Shared Data Store on a LAN. For example, one service scanned a
>>>> programmable logic controller for process data. Other clients would
>>>> display selected portions of that data. For instance, a daemon would
>>>> display the aforementioned process data on overhead displays. Another
>>>> daemon took the process data and logged it to a database every eight
>>>> hours
>>>> or so. Apart from the zero-configuration aspects mentioned above, Jini
>>>> lets us avoid setting up a central data store. You want to find a tag
>>>> called “/cell-006/smt1/placement-defects”? Simply query the lookup
>>>> service
>>>> for services implementing the “SharedDataStore” interface, then select
>>>> the
>>>> one that lists a service attribute that matches the prefix of the tag you
>>>> want, then subscribe to change events for it. The entire idea of a
>>>> distributed key-value store, along with the Paxos-based consistency
>>>> algorithm and leader-elections that go along with it ——is simply not
>>>> necessary ! —. You just get the data from the source.
>>>>
>>>> Distributed computing framework, roughly based on data flow architecture,
>>>> using JavaSpaces in a leader/follower pattern. Like any tuple-space
>>>> system, the architecture has flexible scaling of resources (just add more
>>>> followers as required). As well, you get automatically optimal dynamic
>>>> load balancing (since faster processors finish a packet of work faster,
>>>> they simply pull work out of the JavaSpace in proportion to their speed.
>>>> Being JavaSpaces, the strong typing and dynamic class loading turns out
>>>> to
>>>> be useful in a number of ways, mainly that you can distribute code to the
>>>> followers without any extra effort.
>>>>
>>>> All of the above had aspects of what has recently come to be known as
>>>> “micro-services architecture” (I’ve been doing this kind of architecture
>>>> for 30 years, but that’s a different story, and the kids today won’t
>>>> believe it anyway). In all cases, the ability of Jini services to easily
>>>> discover and coordinate with other services, with very little static
>>>> configuration makes the system very flexible.
>>>>
>>>> Which leads to my thoughts for the future of River - service integration
>>>> in the cloud/data centre. I look at the convolutions that people are
>>>> going
>>>> through to get service discovery working in a Docker environment (e.g.
>>>> https://www.digitalocean.com/community/tutorials/the-
>>>> docker-ecosystem-service-discovery-and-distributed-configuration-stores
>>>> ),
>>>> and I think that Jini has solved this problem already. The dynamic
>>>> discovery and zero-configuration nature of Jini, not to mention the
>>>> inherent fault-tolerance that goes along with leasing, etc, makes Jini
>>>> perfectly suited to a dynamically-scalable environment. We just haven’t
>>>> made it easy to get started. Also, in the past, people were often left
>>>> with the impression that Jini was too complex. I think that people have
>>>> come around to the idea that the problem-space for distributed computing
>>>> is
>>>> complex, so the solution-space is necessarily complex as well.
>>>>
>>>> Cheers,
>>>>
>>>> Greg Trasuk.
>>>>
>>>
>>>
>
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
Good idea. I've copied the user list.
On 4/6/2015 5:53 AM, Bryan Thompson wrote:
> Sure. Maybe we could draw together 10-20 success stories and publish them
> out one a week for a few months? We could certainly point our user base at
> the success story posts for river. It would be nice to use this as an
> opportunity to convince users that jini/river was a good idea as a
> distributed systems platform. This is purely a marketing issue, so having a
> flurry of good press would be nice.
>
> Thanks,
> Bryan
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> bryan@systap.com
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Mon, Apr 6, 2015 at 8:48 AM, Patricia Shanahan <pa...@acm.org> wrote:
>
>> On 4/6/2015 4:55 AM, Bryan Thompson wrote:
>>
>>> We've built a scalable distributed graph database using river (
>>> www.blazegraph.com, formally www.bigdata.com). There are actually two
>>> versions that use river.
>>>
>>> 1. High availability and self-healing based on low write replication. This
>>> uses zookeeper for the leader election. It would be nice if river
>>> supported
>>> these semantics in the base distribution.
>>>
>>> 2. Horizontally scaled using dynamic sharding. This implementation uses
>>> river to expose the shards on the data services, to distribute the
>>> evaluation of high level queries over the shards and services, etc.
>>>
>>> River has made it relatively easy to have services export RMI interfaces.
>>>
>>
>> Perhaps a "success" story we could post on the web site?
>>
>>
>>> Some possible issues to consider:
>>>
>>> a. better communications about the project (i.e., more marketing). I think
>>> that most of the technical documentation for the river ecosystem (other
>>> than javadoc) is pretty old. It might still be valid, but there is a
>>> perception of stagnation. Maybe a weekly blog post, project of the month,
>>> etc.
>>>
>>
>> I don't know whether I am right about this, and it is one of the topics on
>> which I am really looking for feedback, but I feel we have to fix our
>> "Getting started" experience first. Once that is done, we have a better
>> chance of converting marketing leads into active users.
>>
>> b. bindings for other languages (yes, this is not that easy).
>>> c. leader election semantics and similar interesting patterns in the base
>>> platform.
>>> d. design patterns for building scalable applications on river (it is
>>> really not that easy to get started if I remember back to when I first
>>> used
>>> the platform in the mid 2000s.)
>>> e. good tutorials for non-multicast environments (EC2).
>>> f. good tutorials for security, including living with firewalls and
>>> constraining ports.
>>> g. maybe a json interface to reggie so applications can easily see what's
>>> in the various registrars?
>>>
>>> Thanks,
>>> Bryan
>>>
>>>
>>>
>>> ----
>>> Bryan Thompson
>>> Chief Scientist & Founder
>>> SYSTAP, LLC
>>> 4501 Tower Road
>>> Greensboro, NC 27410
>>> bryan@systap.com
>>> http://blazegraph.com
>>> http://blog.bigdata.com <http://bigdata.com>
>>> http://mapgraph.io
>>>
>>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
>>> technology to use GPUs to accelerate data-parallel graph analytics.
>>>
>>> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
>>> for the sole use of the intended recipient(s) and are confidential or
>>> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>>> dissemination or copying of this email or its contents or attachments is
>>> prohibited. If you have received this communication in error, please
>>> notify
>>> the sender by reply email and permanently delete all copies of the email
>>> and its contents and attachments.
>>>
>>> On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com>
>>> wrote:
>>>
>>>
>>>> Well, here’s what I’ve used Jini for:
>>>>
>>>> Desktop Java applications communicating with a back-end implemented as a
>>>> set of Jini services in “LAN” scope. In this scenario, Jini has the
>>>> following valuable features:
>>>> - Zero-configuration on the client side. The client is able to use
>>>> multicast discovery to find the lookup service(s) and then lookup the
>>>> service it wants to use. There is no need for the client app to have a
>>>> configured url to reach the back-end.
>>>> - If the backend needs to move to a different host, the clients don’t
>>>> need
>>>> to be reconfigured.
>>>> - The client can export a service endpoint, even without publishing it to
>>>> the service registrar. It can use that endpoint to receive event
>>>> notifications. So the user interface can be completely asynchronous.
>>>> This
>>>> is not easy to do using a request/response protocol like http (i.e. SOAP
>>>> or
>>>> RESTful services).
>>>>
>>>> Shared Data Store on a LAN. For example, one service scanned a
>>>> programmable logic controller for process data. Other clients would
>>>> display selected portions of that data. For instance, a daemon would
>>>> display the aforementioned process data on overhead displays. Another
>>>> daemon took the process data and logged it to a database every eight
>>>> hours
>>>> or so. Apart from the zero-configuration aspects mentioned above, Jini
>>>> lets us avoid setting up a central data store. You want to find a tag
>>>> called “/cell-006/smt1/placement-defects”? Simply query the lookup
>>>> service
>>>> for services implementing the “SharedDataStore” interface, then select
>>>> the
>>>> one that lists a service attribute that matches the prefix of the tag you
>>>> want, then subscribe to change events for it. The entire idea of a
>>>> distributed key-value store, along with the Paxos-based consistency
>>>> algorithm and leader-elections that go along with it ——is simply not
>>>> necessary ! —. You just get the data from the source.
>>>>
>>>> Distributed computing framework, roughly based on data flow architecture,
>>>> using JavaSpaces in a leader/follower pattern. Like any tuple-space
>>>> system, the architecture has flexible scaling of resources (just add more
>>>> followers as required). As well, you get automatically optimal dynamic
>>>> load balancing (since faster processors finish a packet of work faster,
>>>> they simply pull work out of the JavaSpace in proportion to their speed.
>>>> Being JavaSpaces, the strong typing and dynamic class loading turns out
>>>> to
>>>> be useful in a number of ways, mainly that you can distribute code to the
>>>> followers without any extra effort.
>>>>
>>>> All of the above had aspects of what has recently come to be known as
>>>> “micro-services architecture” (I’ve been doing this kind of architecture
>>>> for 30 years, but that’s a different story, and the kids today won’t
>>>> believe it anyway). In all cases, the ability of Jini services to easily
>>>> discover and coordinate with other services, with very little static
>>>> configuration makes the system very flexible.
>>>>
>>>> Which leads to my thoughts for the future of River - service integration
>>>> in the cloud/data centre. I look at the convolutions that people are
>>>> going
>>>> through to get service discovery working in a Docker environment (e.g.
>>>> https://www.digitalocean.com/community/tutorials/the-
>>>> docker-ecosystem-service-discovery-and-distributed-configuration-stores
>>>> ),
>>>> and I think that Jini has solved this problem already. The dynamic
>>>> discovery and zero-configuration nature of Jini, not to mention the
>>>> inherent fault-tolerance that goes along with leasing, etc, makes Jini
>>>> perfectly suited to a dynamically-scalable environment. We just haven’t
>>>> made it easy to get started. Also, in the past, people were often left
>>>> with the impression that Jini was too complex. I think that people have
>>>> come around to the idea that the problem-space for distributed computing
>>>> is
>>>> complex, so the solution-space is necessarily complex as well.
>>>>
>>>> Cheers,
>>>>
>>>> Greg Trasuk.
>>>>
>>>
>>>
>
Re: River future direction
Posted by Bryan Thompson <br...@systap.com>.
Sure. Maybe we could draw together 10-20 success stories and publish them
out one a week for a few months? We could certainly point our user base at
the success story posts for river. It would be nice to use this as an
opportunity to convince users that jini/river was a good idea as a
distributed systems platform. This is purely a marketing issue, so having a
flurry of good press would be nice.
Thanks,
Bryan
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
bryan@systap.com
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io
Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.
On Mon, Apr 6, 2015 at 8:48 AM, Patricia Shanahan <pa...@acm.org> wrote:
> On 4/6/2015 4:55 AM, Bryan Thompson wrote:
>
>> We've built a scalable distributed graph database using river (
>> www.blazegraph.com, formally www.bigdata.com). There are actually two
>> versions that use river.
>>
>> 1. High availability and self-healing based on low write replication. This
>> uses zookeeper for the leader election. It would be nice if river
>> supported
>> these semantics in the base distribution.
>>
>> 2. Horizontally scaled using dynamic sharding. This implementation uses
>> river to expose the shards on the data services, to distribute the
>> evaluation of high level queries over the shards and services, etc.
>>
>> River has made it relatively easy to have services export RMI interfaces.
>>
>
> Perhaps a "success" story we could post on the web site?
>
>
>> Some possible issues to consider:
>>
>> a. better communications about the project (i.e., more marketing). I think
>> that most of the technical documentation for the river ecosystem (other
>> than javadoc) is pretty old. It might still be valid, but there is a
>> perception of stagnation. Maybe a weekly blog post, project of the month,
>> etc.
>>
>
> I don't know whether I am right about this, and it is one of the topics on
> which I am really looking for feedback, but I feel we have to fix our
> "Getting started" experience first. Once that is done, we have a better
> chance of converting marketing leads into active users.
>
> b. bindings for other languages (yes, this is not that easy).
>> c. leader election semantics and similar interesting patterns in the base
>> platform.
>> d. design patterns for building scalable applications on river (it is
>> really not that easy to get started if I remember back to when I first
>> used
>> the platform in the mid 2000s.)
>> e. good tutorials for non-multicast environments (EC2).
>> f. good tutorials for security, including living with firewalls and
>> constraining ports.
>> g. maybe a json interface to reggie so applications can easily see what's
>> in the various registrars?
>>
>> Thanks,
>> Bryan
>>
>>
>>
>> ----
>> Bryan Thompson
>> Chief Scientist & Founder
>> SYSTAP, LLC
>> 4501 Tower Road
>> Greensboro, NC 27410
>> bryan@systap.com
>> http://blazegraph.com
>> http://blog.bigdata.com <http://bigdata.com>
>> http://mapgraph.io
>>
>> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
>> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
>> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
>> technology to use GPUs to accelerate data-parallel graph analytics.
>>
>> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
>> for the sole use of the intended recipient(s) and are confidential or
>> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
>> dissemination or copying of this email or its contents or attachments is
>> prohibited. If you have received this communication in error, please
>> notify
>> the sender by reply email and permanently delete all copies of the email
>> and its contents and attachments.
>>
>> On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com>
>> wrote:
>>
>>
>>> Well, here’s what I’ve used Jini for:
>>>
>>> Desktop Java applications communicating with a back-end implemented as a
>>> set of Jini services in “LAN” scope. In this scenario, Jini has the
>>> following valuable features:
>>> - Zero-configuration on the client side. The client is able to use
>>> multicast discovery to find the lookup service(s) and then lookup the
>>> service it wants to use. There is no need for the client app to have a
>>> configured url to reach the back-end.
>>> - If the backend needs to move to a different host, the clients don’t
>>> need
>>> to be reconfigured.
>>> - The client can export a service endpoint, even without publishing it to
>>> the service registrar. It can use that endpoint to receive event
>>> notifications. So the user interface can be completely asynchronous.
>>> This
>>> is not easy to do using a request/response protocol like http (i.e. SOAP
>>> or
>>> RESTful services).
>>>
>>> Shared Data Store on a LAN. For example, one service scanned a
>>> programmable logic controller for process data. Other clients would
>>> display selected portions of that data. For instance, a daemon would
>>> display the aforementioned process data on overhead displays. Another
>>> daemon took the process data and logged it to a database every eight
>>> hours
>>> or so. Apart from the zero-configuration aspects mentioned above, Jini
>>> lets us avoid setting up a central data store. You want to find a tag
>>> called “/cell-006/smt1/placement-defects”? Simply query the lookup
>>> service
>>> for services implementing the “SharedDataStore” interface, then select
>>> the
>>> one that lists a service attribute that matches the prefix of the tag you
>>> want, then subscribe to change events for it. The entire idea of a
>>> distributed key-value store, along with the Paxos-based consistency
>>> algorithm and leader-elections that go along with it ——is simply not
>>> necessary ! —. You just get the data from the source.
>>>
>>> Distributed computing framework, roughly based on data flow architecture,
>>> using JavaSpaces in a leader/follower pattern. Like any tuple-space
>>> system, the architecture has flexible scaling of resources (just add more
>>> followers as required). As well, you get automatically optimal dynamic
>>> load balancing (since faster processors finish a packet of work faster,
>>> they simply pull work out of the JavaSpace in proportion to their speed.
>>> Being JavaSpaces, the strong typing and dynamic class loading turns out
>>> to
>>> be useful in a number of ways, mainly that you can distribute code to the
>>> followers without any extra effort.
>>>
>>> All of the above had aspects of what has recently come to be known as
>>> “micro-services architecture” (I’ve been doing this kind of architecture
>>> for 30 years, but that’s a different story, and the kids today won’t
>>> believe it anyway). In all cases, the ability of Jini services to easily
>>> discover and coordinate with other services, with very little static
>>> configuration makes the system very flexible.
>>>
>>> Which leads to my thoughts for the future of River - service integration
>>> in the cloud/data centre. I look at the convolutions that people are
>>> going
>>> through to get service discovery working in a Docker environment (e.g.
>>> https://www.digitalocean.com/community/tutorials/the-
>>> docker-ecosystem-service-discovery-and-distributed-configuration-stores
>>> ),
>>> and I think that Jini has solved this problem already. The dynamic
>>> discovery and zero-configuration nature of Jini, not to mention the
>>> inherent fault-tolerance that goes along with leasing, etc, makes Jini
>>> perfectly suited to a dynamically-scalable environment. We just haven’t
>>> made it easy to get started. Also, in the past, people were often left
>>> with the impression that Jini was too complex. I think that people have
>>> come around to the idea that the problem-space for distributed computing
>>> is
>>> complex, so the solution-space is necessarily complex as well.
>>>
>>> Cheers,
>>>
>>> Greg Trasuk.
>>>
>>
>>
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
On 4/6/2015 4:55 AM, Bryan Thompson wrote:
> We've built a scalable distributed graph database using river (
> www.blazegraph.com, formally www.bigdata.com). There are actually two
> versions that use river.
>
> 1. High availability and self-healing based on low write replication. This
> uses zookeeper for the leader election. It would be nice if river supported
> these semantics in the base distribution.
>
> 2. Horizontally scaled using dynamic sharding. This implementation uses
> river to expose the shards on the data services, to distribute the
> evaluation of high level queries over the shards and services, etc.
>
> River has made it relatively easy to have services export RMI interfaces.
Perhaps a "success" story we could post on the web site?
>
> Some possible issues to consider:
>
> a. better communications about the project (i.e., more marketing). I think
> that most of the technical documentation for the river ecosystem (other
> than javadoc) is pretty old. It might still be valid, but there is a
> perception of stagnation. Maybe a weekly blog post, project of the month,
> etc.
I don't know whether I am right about this, and it is one of the topics
on which I am really looking for feedback, but I feel we have to fix our
"Getting started" experience first. Once that is done, we have a better
chance of converting marketing leads into active users.
> b. bindings for other languages (yes, this is not that easy).
> c. leader election semantics and similar interesting patterns in the base
> platform.
> d. design patterns for building scalable applications on river (it is
> really not that easy to get started if I remember back to when I first used
> the platform in the mid 2000s.)
> e. good tutorials for non-multicast environments (EC2).
> f. good tutorials for security, including living with firewalls and
> constraining ports.
> g. maybe a json interface to reggie so applications can easily see what's
> in the various registrars?
>
> Thanks,
> Bryan
>
>
>
> ----
> Bryan Thompson
> Chief Scientist & Founder
> SYSTAP, LLC
> 4501 Tower Road
> Greensboro, NC 27410
> bryan@systap.com
> http://blazegraph.com
> http://blog.bigdata.com <http://bigdata.com>
> http://mapgraph.io
>
> Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
> graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
> APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
> technology to use GPUs to accelerate data-parallel graph analytics.
>
> CONFIDENTIALITY NOTICE: This email and its contents and attachments are
> for the sole use of the intended recipient(s) and are confidential or
> proprietary to SYSTAP. Any unauthorized review, use, disclosure,
> dissemination or copying of this email or its contents or attachments is
> prohibited. If you have received this communication in error, please notify
> the sender by reply email and permanently delete all copies of the email
> and its contents and attachments.
>
> On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com> wrote:
>
>>
>> Well, here’s what I’ve used Jini for:
>>
>> Desktop Java applications communicating with a back-end implemented as a
>> set of Jini services in “LAN” scope. In this scenario, Jini has the
>> following valuable features:
>> - Zero-configuration on the client side. The client is able to use
>> multicast discovery to find the lookup service(s) and then lookup the
>> service it wants to use. There is no need for the client app to have a
>> configured url to reach the back-end.
>> - If the backend needs to move to a different host, the clients don’t need
>> to be reconfigured.
>> - The client can export a service endpoint, even without publishing it to
>> the service registrar. It can use that endpoint to receive event
>> notifications. So the user interface can be completely asynchronous. This
>> is not easy to do using a request/response protocol like http (i.e. SOAP or
>> RESTful services).
>>
>> Shared Data Store on a LAN. For example, one service scanned a
>> programmable logic controller for process data. Other clients would
>> display selected portions of that data. For instance, a daemon would
>> display the aforementioned process data on overhead displays. Another
>> daemon took the process data and logged it to a database every eight hours
>> or so. Apart from the zero-configuration aspects mentioned above, Jini
>> lets us avoid setting up a central data store. You want to find a tag
>> called “/cell-006/smt1/placement-defects”? Simply query the lookup service
>> for services implementing the “SharedDataStore” interface, then select the
>> one that lists a service attribute that matches the prefix of the tag you
>> want, then subscribe to change events for it. The entire idea of a
>> distributed key-value store, along with the Paxos-based consistency
>> algorithm and leader-elections that go along with it ——is simply not
>> necessary ! —. You just get the data from the source.
>>
>> Distributed computing framework, roughly based on data flow architecture,
>> using JavaSpaces in a leader/follower pattern. Like any tuple-space
>> system, the architecture has flexible scaling of resources (just add more
>> followers as required). As well, you get automatically optimal dynamic
>> load balancing (since faster processors finish a packet of work faster,
>> they simply pull work out of the JavaSpace in proportion to their speed.
>> Being JavaSpaces, the strong typing and dynamic class loading turns out to
>> be useful in a number of ways, mainly that you can distribute code to the
>> followers without any extra effort.
>>
>> All of the above had aspects of what has recently come to be known as
>> “micro-services architecture” (I’ve been doing this kind of architecture
>> for 30 years, but that’s a different story, and the kids today won’t
>> believe it anyway). In all cases, the ability of Jini services to easily
>> discover and coordinate with other services, with very little static
>> configuration makes the system very flexible.
>>
>> Which leads to my thoughts for the future of River - service integration
>> in the cloud/data centre. I look at the convolutions that people are going
>> through to get service discovery working in a Docker environment (e.g.
>> https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores),
>> and I think that Jini has solved this problem already. The dynamic
>> discovery and zero-configuration nature of Jini, not to mention the
>> inherent fault-tolerance that goes along with leasing, etc, makes Jini
>> perfectly suited to a dynamically-scalable environment. We just haven’t
>> made it easy to get started. Also, in the past, people were often left
>> with the impression that Jini was too complex. I think that people have
>> come around to the idea that the problem-space for distributed computing is
>> complex, so the solution-space is necessarily complex as well.
>>
>> Cheers,
>>
>> Greg Trasuk.
>
Re: River future direction
Posted by Bryan Thompson <br...@systap.com>.
We've built a scalable distributed graph database using river (
www.blazegraph.com, formally www.bigdata.com). There are actually two
versions that use river.
1. High availability and self-healing based on low write replication. This
uses zookeeper for the leader election. It would be nice if river supported
these semantics in the base distribution.
2. Horizontally scaled using dynamic sharding. This implementation uses
river to expose the shards on the data services, to distribute the
evaluation of high level queries over the shards and services, etc.
River has made it relatively easy to have services export RMI interfaces.
Some possible issues to consider:
a. better communications about the project (i.e., more marketing). I think
that most of the technical documentation for the river ecosystem (other
than javadoc) is pretty old. It might still be valid, but there is a
perception of stagnation. Maybe a weekly blog post, project of the month,
etc.
b. bindings for other languages (yes, this is not that easy).
c. leader election semantics and similar interesting patterns in the base
platform.
d. design patterns for building scalable applications on river (it is
really not that easy to get started if I remember back to when I first used
the platform in the mid 2000s.)
e. good tutorials for non-multicast environments (EC2).
f. good tutorials for security, including living with firewalls and
constraining ports.
g. maybe a json interface to reggie so applications can easily see what's
in the various registrars?
Thanks,
Bryan
----
Bryan Thompson
Chief Scientist & Founder
SYSTAP, LLC
4501 Tower Road
Greensboro, NC 27410
bryan@systap.com
http://blazegraph.com
http://blog.bigdata.com <http://bigdata.com>
http://mapgraph.io
Blazegraph™ <http://www.blazegraph.com/> is our ultra high-performance
graph database that supports both RDF/SPARQL and Tinkerpop/Blueprints
APIs. MapGraph™ <http://www.systap.com/mapgraph> is our disruptive new
technology to use GPUs to accelerate data-parallel graph analytics.
CONFIDENTIALITY NOTICE: This email and its contents and attachments are
for the sole use of the intended recipient(s) and are confidential or
proprietary to SYSTAP. Any unauthorized review, use, disclosure,
dissemination or copying of this email or its contents or attachments is
prohibited. If you have received this communication in error, please notify
the sender by reply email and permanently delete all copies of the email
and its contents and attachments.
On Mon, Apr 6, 2015 at 4:40 AM, Greg Trasuk <tr...@stratuscom.com> wrote:
>
> Well, here’s what I’ve used Jini for:
>
> Desktop Java applications communicating with a back-end implemented as a
> set of Jini services in “LAN” scope. In this scenario, Jini has the
> following valuable features:
> - Zero-configuration on the client side. The client is able to use
> multicast discovery to find the lookup service(s) and then lookup the
> service it wants to use. There is no need for the client app to have a
> configured url to reach the back-end.
> - If the backend needs to move to a different host, the clients don’t need
> to be reconfigured.
> - The client can export a service endpoint, even without publishing it to
> the service registrar. It can use that endpoint to receive event
> notifications. So the user interface can be completely asynchronous. This
> is not easy to do using a request/response protocol like http (i.e. SOAP or
> RESTful services).
>
> Shared Data Store on a LAN. For example, one service scanned a
> programmable logic controller for process data. Other clients would
> display selected portions of that data. For instance, a daemon would
> display the aforementioned process data on overhead displays. Another
> daemon took the process data and logged it to a database every eight hours
> or so. Apart from the zero-configuration aspects mentioned above, Jini
> lets us avoid setting up a central data store. You want to find a tag
> called “/cell-006/smt1/placement-defects”? Simply query the lookup service
> for services implementing the “SharedDataStore” interface, then select the
> one that lists a service attribute that matches the prefix of the tag you
> want, then subscribe to change events for it. The entire idea of a
> distributed key-value store, along with the Paxos-based consistency
> algorithm and leader-elections that go along with it ——is simply not
> necessary ! —. You just get the data from the source.
>
> Distributed computing framework, roughly based on data flow architecture,
> using JavaSpaces in a leader/follower pattern. Like any tuple-space
> system, the architecture has flexible scaling of resources (just add more
> followers as required). As well, you get automatically optimal dynamic
> load balancing (since faster processors finish a packet of work faster,
> they simply pull work out of the JavaSpace in proportion to their speed.
> Being JavaSpaces, the strong typing and dynamic class loading turns out to
> be useful in a number of ways, mainly that you can distribute code to the
> followers without any extra effort.
>
> All of the above had aspects of what has recently come to be known as
> “micro-services architecture” (I’ve been doing this kind of architecture
> for 30 years, but that’s a different story, and the kids today won’t
> believe it anyway). In all cases, the ability of Jini services to easily
> discover and coordinate with other services, with very little static
> configuration makes the system very flexible.
>
> Which leads to my thoughts for the future of River - service integration
> in the cloud/data centre. I look at the convolutions that people are going
> through to get service discovery working in a Docker environment (e.g.
> https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores),
> and I think that Jini has solved this problem already. The dynamic
> discovery and zero-configuration nature of Jini, not to mention the
> inherent fault-tolerance that goes along with leasing, etc, makes Jini
> perfectly suited to a dynamically-scalable environment. We just haven’t
> made it easy to get started. Also, in the past, people were often left
> with the impression that Jini was too complex. I think that people have
> come around to the idea that the problem-space for distributed computing is
> complex, so the solution-space is necessarily complex as well.
>
> Cheers,
>
> Greg Trasuk.
Re: River future direction
Posted by Greg Trasuk <tr...@stratuscom.com>.
Well, here’s what I’ve used Jini for:
Desktop Java applications communicating with a back-end implemented as a set of Jini services in “LAN” scope. In this scenario, Jini has the following valuable features:
- Zero-configuration on the client side. The client is able to use multicast discovery to find the lookup service(s) and then lookup the service it wants to use. There is no need for the client app to have a configured url to reach the back-end.
- If the backend needs to move to a different host, the clients don’t need to be reconfigured.
- The client can export a service endpoint, even without publishing it to the service registrar. It can use that endpoint to receive event notifications. So the user interface can be completely asynchronous. This is not easy to do using a request/response protocol like http (i.e. SOAP or RESTful services).
Shared Data Store on a LAN. For example, one service scanned a programmable logic controller for process data. Other clients would display selected portions of that data. For instance, a daemon would display the aforementioned process data on overhead displays. Another daemon took the process data and logged it to a database every eight hours or so. Apart from the zero-configuration aspects mentioned above, Jini lets us avoid setting up a central data store. You want to find a tag called “/cell-006/smt1/placement-defects”? Simply query the lookup service for services implementing the “SharedDataStore” interface, then select the one that lists a service attribute that matches the prefix of the tag you want, then subscribe to change events for it. The entire idea of a distributed key-value store, along with the Paxos-based consistency algorithm and leader-elections that go along with it ——is simply not necessary ! —. You just get the data from the source.
Distributed computing framework, roughly based on data flow architecture, using JavaSpaces in a leader/follower pattern. Like any tuple-space system, the architecture has flexible scaling of resources (just add more followers as required). As well, you get automatically optimal dynamic load balancing (since faster processors finish a packet of work faster, they simply pull work out of the JavaSpace in proportion to their speed. Being JavaSpaces, the strong typing and dynamic class loading turns out to be useful in a number of ways, mainly that you can distribute code to the followers without any extra effort.
All of the above had aspects of what has recently come to be known as “micro-services architecture” (I’ve been doing this kind of architecture for 30 years, but that’s a different story, and the kids today won’t believe it anyway). In all cases, the ability of Jini services to easily discover and coordinate with other services, with very little static configuration makes the system very flexible.
Which leads to my thoughts for the future of River - service integration in the cloud/data centre. I look at the convolutions that people are going through to get service discovery working in a Docker environment (e.g. https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-service-discovery-and-distributed-configuration-stores), and I think that Jini has solved this problem already. The dynamic discovery and zero-configuration nature of Jini, not to mention the inherent fault-tolerance that goes along with leasing, etc, makes Jini perfectly suited to a dynamically-scalable environment. We just haven’t made it easy to get started. Also, in the past, people were often left with the impression that Jini was too complex. I think that people have come around to the idea that the problem-space for distributed computing is complex, so the solution-space is necessarily complex as well.
Cheers,
Greg Trasuk.
Re: River future direction
Posted by Mike <mi...@gmail.com>.
On 04/04/15 03:04, Patricia Shanahan wrote:
> I am getting the feeling that the real strength of River is going to be
> in what I think of as the "household area network" which includes a
Fully agree! That's the space I have been nibbling at when time permits,
and it seems to me that the Surrogate Architecture plays a critical role
in enabling less capable devices to play, too.
--
Mike Morris
http://www.mikro2nd.net/
EarthStuff: http://blog.mikro2nd.net/
TechStuff : http://onemikro2nd.blogspot.com/
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
I think a tutorial on using River with Raspberry Pi would be a valuable
asset for the web site.
I am getting the feeling that the real strength of River is going to be
in what I think of as the "household area network" which includes a
house's computers, printers, and IoT devices, as well as the cars and
personal devices that travel with household members even when they are
away from home.
Patricia
On 4/2/2015 6:01 PM, Bishnu Gautam wrote:
> Hello AllJust a quick note !! I can contribute for "Getting Started"
> and tutorials of River. We are working on how to implement services
> developed on River by using Raspberry Pi. So, let me know if you
> guys are interested about it. I can prepare it and send you the
> tutorial of using River on Raspberry Pi within a month. I think once
> we are able to provide some tutorial on it, we will be able to bring
> River for "Fog Computing" technology. Personally, I propose to bring
> River on Internet so that we will have more users and developers and
> there will be better future for the people who are involved in
> River/Jini technologies. RegardsBishnu
>
> Bishnu Prasad Gautam
>
>
>> Date: Wed, 1 Apr 2015 08:57:34 -0700 From: pats@acm.org To:
>> dev@river.apache.org; user@river.apache.org Subject: Re: River
>> future direction
>>
>> I realized I made a mistake initially posting this only to the
>> "dev" mailing list. I definitely want the opinions of any River
>> user.
>>
>> Additionally, a River user who is not interested in River internals
>> may be the perfect person to contribute to "Getting Started"
>> examples and tutorials.
>>
>> Patricia (Chair, River PMC)
>>
>> On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
>>> I would like to start a discussion of future direction for the
>>> River project, and whether we have the right resources to get
>>> there.
>>>
>>> Recently, I have been trying to push improving the "Getting
>>> Started" experience in the hope of attracting and keeping new
>>> users who may turn into committers. I am not sure what to do next
>>> to make that happen. I feel we may have a problem of not having
>>> enough committer time to get more committers.
>>>
>>> I do have some free time, but not quite the right skills. When I
>>> got involved in River I expected to be working on Java
>>> programming for refactoring, performance enhancement, and
>>> multithread/multiprocessor issues. Writing good "Getting Started"
>>> materials calls for actual River expertise and experience, which
>>> I lack.
>>>
>>> I have about five weeks before I need to file my next board
>>> report. I would like to use that time to build a consensus on
>>> where we are going, and whether we need help from outside the
>>> project.
>>>
>>> Patricia
>
>
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
I think a tutorial on using River with Raspberry Pi would be a valuable
asset for the web site.
I am getting the feeling that the real strength of River is going to be
in what I think of as the "household area network" which includes a
house's computers, printers, and IoT devices, as well as the cars and
personal devices that travel with household members even when they are
away from home.
Patricia
On 4/2/2015 6:01 PM, Bishnu Gautam wrote:
> Hello AllJust a quick note !! I can contribute for "Getting Started"
> and tutorials of River. We are working on how to implement services
> developed on River by using Raspberry Pi. So, let me know if you
> guys are interested about it. I can prepare it and send you the
> tutorial of using River on Raspberry Pi within a month. I think once
> we are able to provide some tutorial on it, we will be able to bring
> River for "Fog Computing" technology. Personally, I propose to bring
> River on Internet so that we will have more users and developers and
> there will be better future for the people who are involved in
> River/Jini technologies. RegardsBishnu
>
> Bishnu Prasad Gautam
>
>
>> Date: Wed, 1 Apr 2015 08:57:34 -0700 From: pats@acm.org To:
>> dev@river.apache.org; user@river.apache.org Subject: Re: River
>> future direction
>>
>> I realized I made a mistake initially posting this only to the
>> "dev" mailing list. I definitely want the opinions of any River
>> user.
>>
>> Additionally, a River user who is not interested in River internals
>> may be the perfect person to contribute to "Getting Started"
>> examples and tutorials.
>>
>> Patricia (Chair, River PMC)
>>
>> On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
>>> I would like to start a discussion of future direction for the
>>> River project, and whether we have the right resources to get
>>> there.
>>>
>>> Recently, I have been trying to push improving the "Getting
>>> Started" experience in the hope of attracting and keeping new
>>> users who may turn into committers. I am not sure what to do next
>>> to make that happen. I feel we may have a problem of not having
>>> enough committer time to get more committers.
>>>
>>> I do have some free time, but not quite the right skills. When I
>>> got involved in River I expected to be working on Java
>>> programming for refactoring, performance enhancement, and
>>> multithread/multiprocessor issues. Writing good "Getting Started"
>>> materials calls for actual River expertise and experience, which
>>> I lack.
>>>
>>> I have about five weeks before I need to file my next board
>>> report. I would like to use that time to build a consensus on
>>> where we are going, and whether we need help from outside the
>>> project.
>>>
>>> Patricia
>
>
RE: River future direction
Posted by Bishnu Gautam <bi...@hotmail.com>.
Hello AllJust a quick note !!
I can contribute for "Getting Started" and tutorials of River. We are working on how to implement services developed on River by using Raspberry Pi. So, let me know if you guys are interested about it. I can prepare it and send you the tutorial of using River on Raspberry Pi within a month. I think once we are able to provide some tutorial on it, we will be able to bring River for "Fog Computing" technology. Personally, I propose to bring River on Internet so that we will have more users and developers and there will be better future for the people who are involved in River/Jini technologies.
RegardsBishnu
Bishnu Prasad Gautam
> Date: Wed, 1 Apr 2015 08:57:34 -0700
> From: pats@acm.org
> To: dev@river.apache.org; user@river.apache.org
> Subject: Re: River future direction
>
> I realized I made a mistake initially posting this only to the "dev"
> mailing list. I definitely want the opinions of any River user.
>
> Additionally, a River user who is not interested in River internals may
> be the perfect person to contribute to "Getting Started" examples and
> tutorials.
>
> Patricia
> (Chair, River PMC)
>
> On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
> > I would like to start a discussion of future direction for the River
> > project, and whether we have the right resources to get there.
> >
> > Recently, I have been trying to push improving the "Getting Started"
> > experience in the hope of attracting and keeping new users who may turn
> > into committers. I am not sure what to do next to make that happen. I
> > feel we may have a problem of not having enough committer time to get
> > more committers.
> >
> > I do have some free time, but not quite the right skills. When I got
> > involved in River I expected to be working on Java programming for
> > refactoring, performance enhancement, and multithread/multiprocessor
> > issues. Writing good "Getting Started" materials calls for actual River
> > expertise and experience, which I lack.
> >
> > I have about five weeks before I need to file my next board report. I
> > would like to use that time to build a consensus on where we are going,
> > and whether we need help from outside the project.
> >
> > Patricia
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
I realized I made a mistake initially posting this only to the "dev"
mailing list. I definitely want the opinions of any River user.
Additionally, a River user who is not interested in River internals may
be the perfect person to contribute to "Getting Started" examples and
tutorials.
Patricia
(Chair, River PMC)
On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
> I would like to start a discussion of future direction for the River
> project, and whether we have the right resources to get there.
>
> Recently, I have been trying to push improving the "Getting Started"
> experience in the hope of attracting and keeping new users who may turn
> into committers. I am not sure what to do next to make that happen. I
> feel we may have a problem of not having enough committer time to get
> more committers.
>
> I do have some free time, but not quite the right skills. When I got
> involved in River I expected to be working on Java programming for
> refactoring, performance enhancement, and multithread/multiprocessor
> issues. Writing good "Getting Started" materials calls for actual River
> expertise and experience, which I lack.
>
> I have about five weeks before I need to file my next board report. I
> would like to use that time to build a consensus on where we are going,
> and whether we need help from outside the project.
>
> Patricia
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
I realized I made a mistake initially posting this only to the "dev"
mailing list. I definitely want the opinions of any River user.
Additionally, a River user who is not interested in River internals may
be the perfect person to contribute to "Getting Started" examples and
tutorials.
Patricia
(Chair, River PMC)
On 4/1/2015 7:40 AM, Patricia Shanahan wrote:
> I would like to start a discussion of future direction for the River
> project, and whether we have the right resources to get there.
>
> Recently, I have been trying to push improving the "Getting Started"
> experience in the hope of attracting and keeping new users who may turn
> into committers. I am not sure what to do next to make that happen. I
> feel we may have a problem of not having enough committer time to get
> more committers.
>
> I do have some free time, but not quite the right skills. When I got
> involved in River I expected to be working on Java programming for
> refactoring, performance enhancement, and multithread/multiprocessor
> issues. Writing good "Getting Started" materials calls for actual River
> expertise and experience, which I lack.
>
> I have about five weeks before I need to file my next board report. I
> would like to use that time to build a consensus on where we are going,
> and whether we need help from outside the project.
>
> Patricia
Re: River future direction
Posted by Patricia Shanahan <pa...@acm.org>.
It is indeed interesting. I'm afraid it will be a while before we can
advance to Java 9 for the River trunk.
Meanwhile, what are your views on future direction?
On 4/5/2015 8:25 PM, Peter wrote:
> I don't propose the following as River's future direction, it's an
> item of interest.
>
> Space based computing overcomes NAT limitations. That is provided
> javaspace servervices are internet visible to clients behind NAT
> routers, even though clients that can't directly contact each other,
> they can work together in a completely uncoupled fashion with spaces.
> This ignores security concerns
>
> Java 9 is introducing a new module url format, this will make life
> much easier in a number of ways. Firstly protection domains won't be
> dependant on file paths, secondly we could automate policy file
> generation for administrators. We've also previously discussed build
> tools for generating preferred lists. These sorts of tools should
> reduce setup time.
>
> One consequence of Java 9's modular design is the removal of support
> for policy providers using the java extensions directory and
> classloader. I've made some local modifications to test using the
> policy provider in the application classloader, this resulted in 4
> test failures, at least one of which failed only because it was
> expecting the extensions classloader. Note this was using the new
> policy providers in qa-refactor, I haven't tested the older policy
> providers in the current release branch.
>
> Regards,
>
> Peter.
>
>
> ----- Original message -----
>> I would like to start a discussion of future direction for the
>> River project, and whether we have the right resources to get
>> there.
>>
>> Recently, I have been trying to push improving the "Getting
>> Started" experience in the hope of attracting and keeping new users
>> who may turn into committers. I am not sure what to do next to make
>> that happen. I feel we may have a problem of not having enough
>> committer time to get more committers.
>>
>> I do have some free time, but not quite the right skills. When I
>> got involved in River I expected to be working on Java programming
>> for refactoring, performance enhancement, and
>> multithread/multiprocessor issues. Writing good "Getting Started"
>> materials calls for actual River expertise and experience, which I
>> lack.
>>
>> I have about five weeks before I need to file my next board report.
>> I would like to use that time to build a consensus on where we are
>> going, and whether we need help from outside the project.
>>
>> Patricia
>
>
Re: River future direction
Posted by Peter <ji...@zeus.net.au>.
I don't propose the following as River's future direction, it's an item of interest.
Space based computing overcomes NAT limitations. That is provided javaspace servervices are internet visible to clients behind NAT routers, even though clients that can't directly contact each other, they can work together in a completely uncoupled fashion with spaces. This ignores security concerns
Java 9 is introducing a new module url format, this will make life much easier in a number of ways. Firstly protection domains won't be dependant on file paths, secondly we could automate policy file generation for administrators. We've also previously discussed build tools for generating preferred lists. These sorts of tools should reduce setup time.
One consequence of Java 9's modular design is the removal of support for policy providers using the java extensions directory and classloader. I've made some local modifications to test using the policy provider in the application classloader, this resulted in 4 test failures, at least one of which failed only because it was expecting the extensions classloader. Note this was using the new policy providers in qa-refactor, I haven't tested the older policy providers in the current release branch.
Regards,
Peter.
----- Original message -----
> I would like to start a discussion of future direction for the River
> project, and whether we have the right resources to get there.
>
> Recently, I have been trying to push improving the "Getting Started"
> experience in the hope of attracting and keeping new users who may turn
> into committers. I am not sure what to do next to make that happen. I
> feel we may have a problem of not having enough committer time to get
> more committers.
>
> I do have some free time, but not quite the right skills. When I got
> involved in River I expected to be working on Java programming for
> refactoring, performance enhancement, and multithread/multiprocessor
> issues. Writing good "Getting Started" materials calls for actual River
> expertise and experience, which I lack.
>
> I have about five weeks before I need to file my next board report. I
> would like to use that time to build a consensus on where we are going,
> and whether we need help from outside the project.
>
> Patricia