You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@couchdb.apache.org by madhuka udantha <ma...@gmail.com> on 2015/03/19 07:15:15 UTC
[GSOC 2015] COUCHDB-2605 Visualize the CouchDB Cluster
Hi. All
I'm Udantha. I'm a MSc. Student at University of Moratuwa. This particular
project looks interesting to me. Since I have previous experiences on
visualization(web portals / dashboards [1]), javascript [2], html5, REST
etc.
Under guidance of Alexander Shorin, Robert Kowalski and Sebastian
Rothbucher, I have started the initial phase. Building
(/developer-preview/2.0/) from source and pointing out few small bugs in
build process. Going through the REST api[3] and following few
tutorials[4]. If there are any material or ideas that can help for
$subject, please let me know.
Here is the list of features that I gain over the period of time (this can
be change). Your ideas and view will be grateful in here.
- Visualize the Couch DB Clusters with nodes (icons, grouping)
- Memory usage on each node in the cluster.
- Charting and graph representation of memory usage over the nodes
- Notification center
- Where it will give notification for user when he/she reaches memory
limits
- Notification over documents/ document count
- Summary of each node (name of the Databases, recent DB updates on each)
- List databases cluster/node
- Drag and Drop supporting for DB over the nodes *
- Basic operation for node/DB can be done with interactive user
interface*
- Retrieving stats/actives*
Your ideas are welcome in here.
[1] https://github.com/wso2/product-ues/
[2] https://github.com/wso2/jaggery
[3] http://couchdb.readthedocs.org/en/1.6.1/api/server/common.html
[4] https://github.com/ryanflorence/react-training/tree/gh-pages/lessons
Thanks,
--
Madhuka Udantha
http://madhukaudantha.blogspot.com
Re: [GSOC 2015] COUCHDB-2605 Visualize the CouchDB Cluster
Posted by madhuka udantha <ma...@gmail.com>.
Hi,
I covered the CouchDB 2.0 API[1]. And also worked out clustering features
in CouchDB. Also I covered Flux and React[2] regarding Fauxton UI[3]
Components. I also built a react github repo following the pointed
resources. It contains 4 samples[4] that cover lot of basics and tips of
react and each sample contains their own readme file explaining the react
features.
Thanks
[1]
http://madhukaudantha.blogspot.com/2015/04/couchdb-20-developer-preview-with-http.html
[2] https://github.com/Madhuka/reactjs-tutorial
[3] http://madhukaudantha.blogspot.com/search/label/Fauxton
[4] https://github.com/Madhuka/reactjs-tutorial/tree/master/code/
On Wed, Apr 1, 2015 at 10:59 AM, madhuka udantha <ma...@gmail.com>
wrote:
> Hi Sebastian,
>
> Thanks for the quick reply for my query. I will go through the resources'
> links and update you all on it.
>
> On Wed, Apr 1, 2015 at 2:39 AM, Sebastian Rothbucher <
> sebastianrothbucher@googlemail.com> wrote:
>
>> Hi Madhuka,
>>
>> concerning Revisions specifically, there is some great writing at
>> http://guide.couchdb.org/draft/conflicts.html and
>> http://writing.jan.io/2013/12/19/understanding-couchdb-conflicts.html
>> The essence is that CouchDB (more or less) arbitrarily but
>> deterministically (!) picks a winner in case of a conflict. Be aware that
>> old revisions are not necessarily kept. As soon as you "compact" the DB
>> (in
>> Futon / Fauxon or via curl), they are gone. So you can't use them as SVN/
>> git replacement ;-)
>>
>> In terms of internal code structure, I'm sorry I can't really help out, I
>> guess building does the trick to quite some extent. Basically, before
>> Couch
>> 2.0 there was one HTTP endpoint written in Erlang (port 5984) which did
>> all
>> the magic (including the possibility to add custom endpoints to it). With
>> Couch 2.0, there is a 2nd one for the cluster (which is why there is two
>> ports: 5984 - pointing to the NEW cluster http handling, as far as I know
>> in "chttpd") and the existing one, now at port 5986 (or 15984 / 15986,
>> 25984/25986, etc. when running several nodes).
>>
>> Probably s/o else can help you with the Erlang stuff
>>
>> Good luck
>> Sebastian
>>
>> On Tue, Mar 31, 2015 at 7:25 PM, madhuka udantha <
>> madhukaudantha@gmail.com>
>> wrote:
>>
>> > Hi All,
>> >
>> > Can you point out some resources / tutorials to understand the code
>> base
>> > of couchDB 2.0?
>> > Since I am trying to familiarize myself with the couchDB2.0 code
>> structure
>> > and architecture (mainly for cluster level)
>> >
>> > Thanks
>> >
>> > On Thu, Mar 19, 2015 at 11:45 AM, madhuka udantha <
>> > madhukaudantha@gmail.com>
>> > wrote:
>> >
>> > > Hi. All
>> > >
>> > >
>> > >
>> > > I'm Udantha. I'm a MSc. Student at University of Moratuwa. This
>> > particular
>> > > project looks interesting to me. Since I have previous experiences on
>> > > visualization(web portals / dashboards [1]), javascript [2], html5,
>> REST
>> > > etc.
>> > >
>> > >
>> > >
>> > > Under guidance of Alexander Shorin, Robert Kowalski and Sebastian
>> > > Rothbucher, I have started the initial phase. Building
>> > > (/developer-preview/2.0/) from source and pointing out few small bugs
>> in
>> > > build process. Going through the REST api[3] and following few
>> > > tutorials[4]. If there are any material or ideas that can help for
>> > > $subject, please let me know.
>> > >
>> > >
>> > > Here is the list of features that I gain over the period of time (this
>> > can
>> > > be change). Your ideas and view will be grateful in here.
>> > >
>> > >
>> > >
>> > > - Visualize the Couch DB Clusters with nodes (icons, grouping)
>> > > - Memory usage on each node in the cluster.
>> > > - Charting and graph representation of memory usage over the nodes
>> > > - Notification center
>> > > - Where it will give notification for user when he/she reaches
>> memory
>> > > limits
>> > > - Notification over documents/ document count
>> > > - Summary of each node (name of the Databases, recent DB updates on
>> > > each)
>> > > - List databases cluster/node
>> > > - Drag and Drop supporting for DB over the nodes *
>> > > - Basic operation for node/DB can be done with interactive user
>> > > interface*
>> > > - Retrieving stats/actives*
>> > >
>> > >
>> > > Your ideas are welcome in here.
>> > >
>> > >
>> > >
>> > > [1] https://github.com/wso2/product-ues/
>> > >
>> > > [2] https://github.com/wso2/jaggery
>> > >
>> > > [3] http://couchdb.readthedocs.org/en/1.6.1/api/server/common.html
>> > >
>> > > [4]
>> https://github.com/ryanflorence/react-training/tree/gh-pages/lessons
>> > >
>> > >
>> > > Thanks,
>> > > --
>> > > Madhuka Udantha
>> > > http://madhukaudantha.blogspot.com
>> > >
>> >
>> >
>> >
>> > --
>> > Cheers,
>> > Madhuka Udantha
>> > http://madhukaudantha.blogspot.com
>> >
>>
>
>
>
> --
> Cheers,
> Madhuka Udantha
> http://madhukaudantha.blogspot.com
>
--
Cheers,
Madhuka Udantha
http://madhukaudantha.blogspot.com
Re: [GSOC 2015] COUCHDB-2605 Visualize the CouchDB Cluster
Posted by madhuka udantha <ma...@gmail.com>.
Hi Sebastian,
Thanks for the quick reply for my query. I will go through the resources'
links and update you all on it.
On Wed, Apr 1, 2015 at 2:39 AM, Sebastian Rothbucher <
sebastianrothbucher@googlemail.com> wrote:
> Hi Madhuka,
>
> concerning Revisions specifically, there is some great writing at
> http://guide.couchdb.org/draft/conflicts.html and
> http://writing.jan.io/2013/12/19/understanding-couchdb-conflicts.html
> The essence is that CouchDB (more or less) arbitrarily but
> deterministically (!) picks a winner in case of a conflict. Be aware that
> old revisions are not necessarily kept. As soon as you "compact" the DB (in
> Futon / Fauxon or via curl), they are gone. So you can't use them as SVN/
> git replacement ;-)
>
> In terms of internal code structure, I'm sorry I can't really help out, I
> guess building does the trick to quite some extent. Basically, before Couch
> 2.0 there was one HTTP endpoint written in Erlang (port 5984) which did all
> the magic (including the possibility to add custom endpoints to it). With
> Couch 2.0, there is a 2nd one for the cluster (which is why there is two
> ports: 5984 - pointing to the NEW cluster http handling, as far as I know
> in "chttpd") and the existing one, now at port 5986 (or 15984 / 15986,
> 25984/25986, etc. when running several nodes).
>
> Probably s/o else can help you with the Erlang stuff
>
> Good luck
> Sebastian
>
> On Tue, Mar 31, 2015 at 7:25 PM, madhuka udantha <madhukaudantha@gmail.com
> >
> wrote:
>
> > Hi All,
> >
> > Can you point out some resources / tutorials to understand the code
> base
> > of couchDB 2.0?
> > Since I am trying to familiarize myself with the couchDB2.0 code
> structure
> > and architecture (mainly for cluster level)
> >
> > Thanks
> >
> > On Thu, Mar 19, 2015 at 11:45 AM, madhuka udantha <
> > madhukaudantha@gmail.com>
> > wrote:
> >
> > > Hi. All
> > >
> > >
> > >
> > > I'm Udantha. I'm a MSc. Student at University of Moratuwa. This
> > particular
> > > project looks interesting to me. Since I have previous experiences on
> > > visualization(web portals / dashboards [1]), javascript [2], html5,
> REST
> > > etc.
> > >
> > >
> > >
> > > Under guidance of Alexander Shorin, Robert Kowalski and Sebastian
> > > Rothbucher, I have started the initial phase. Building
> > > (/developer-preview/2.0/) from source and pointing out few small bugs
> in
> > > build process. Going through the REST api[3] and following few
> > > tutorials[4]. If there are any material or ideas that can help for
> > > $subject, please let me know.
> > >
> > >
> > > Here is the list of features that I gain over the period of time (this
> > can
> > > be change). Your ideas and view will be grateful in here.
> > >
> > >
> > >
> > > - Visualize the Couch DB Clusters with nodes (icons, grouping)
> > > - Memory usage on each node in the cluster.
> > > - Charting and graph representation of memory usage over the nodes
> > > - Notification center
> > > - Where it will give notification for user when he/she reaches
> memory
> > > limits
> > > - Notification over documents/ document count
> > > - Summary of each node (name of the Databases, recent DB updates on
> > > each)
> > > - List databases cluster/node
> > > - Drag and Drop supporting for DB over the nodes *
> > > - Basic operation for node/DB can be done with interactive user
> > > interface*
> > > - Retrieving stats/actives*
> > >
> > >
> > > Your ideas are welcome in here.
> > >
> > >
> > >
> > > [1] https://github.com/wso2/product-ues/
> > >
> > > [2] https://github.com/wso2/jaggery
> > >
> > > [3] http://couchdb.readthedocs.org/en/1.6.1/api/server/common.html
> > >
> > > [4]
> https://github.com/ryanflorence/react-training/tree/gh-pages/lessons
> > >
> > >
> > > Thanks,
> > > --
> > > Madhuka Udantha
> > > http://madhukaudantha.blogspot.com
> > >
> >
> >
> >
> > --
> > Cheers,
> > Madhuka Udantha
> > http://madhukaudantha.blogspot.com
> >
>
--
Cheers,
Madhuka Udantha
http://madhukaudantha.blogspot.com
Re: [GSOC 2015] COUCHDB-2605 Visualize the CouchDB Cluster
Posted by Sebastian Rothbucher <se...@googlemail.com>.
Hi Madhuka,
concerning Revisions specifically, there is some great writing at
http://guide.couchdb.org/draft/conflicts.html and
http://writing.jan.io/2013/12/19/understanding-couchdb-conflicts.html
The essence is that CouchDB (more or less) arbitrarily but
deterministically (!) picks a winner in case of a conflict. Be aware that
old revisions are not necessarily kept. As soon as you "compact" the DB (in
Futon / Fauxon or via curl), they are gone. So you can't use them as SVN/
git replacement ;-)
In terms of internal code structure, I'm sorry I can't really help out, I
guess building does the trick to quite some extent. Basically, before Couch
2.0 there was one HTTP endpoint written in Erlang (port 5984) which did all
the magic (including the possibility to add custom endpoints to it). With
Couch 2.0, there is a 2nd one for the cluster (which is why there is two
ports: 5984 - pointing to the NEW cluster http handling, as far as I know
in "chttpd") and the existing one, now at port 5986 (or 15984 / 15986,
25984/25986, etc. when running several nodes).
Probably s/o else can help you with the Erlang stuff
Good luck
Sebastian
On Tue, Mar 31, 2015 at 7:25 PM, madhuka udantha <ma...@gmail.com>
wrote:
> Hi All,
>
> Can you point out some resources / tutorials to understand the code base
> of couchDB 2.0?
> Since I am trying to familiarize myself with the couchDB2.0 code structure
> and architecture (mainly for cluster level)
>
> Thanks
>
> On Thu, Mar 19, 2015 at 11:45 AM, madhuka udantha <
> madhukaudantha@gmail.com>
> wrote:
>
> > Hi. All
> >
> >
> >
> > I'm Udantha. I'm a MSc. Student at University of Moratuwa. This
> particular
> > project looks interesting to me. Since I have previous experiences on
> > visualization(web portals / dashboards [1]), javascript [2], html5, REST
> > etc.
> >
> >
> >
> > Under guidance of Alexander Shorin, Robert Kowalski and Sebastian
> > Rothbucher, I have started the initial phase. Building
> > (/developer-preview/2.0/) from source and pointing out few small bugs in
> > build process. Going through the REST api[3] and following few
> > tutorials[4]. If there are any material or ideas that can help for
> > $subject, please let me know.
> >
> >
> > Here is the list of features that I gain over the period of time (this
> can
> > be change). Your ideas and view will be grateful in here.
> >
> >
> >
> > - Visualize the Couch DB Clusters with nodes (icons, grouping)
> > - Memory usage on each node in the cluster.
> > - Charting and graph representation of memory usage over the nodes
> > - Notification center
> > - Where it will give notification for user when he/she reaches memory
> > limits
> > - Notification over documents/ document count
> > - Summary of each node (name of the Databases, recent DB updates on
> > each)
> > - List databases cluster/node
> > - Drag and Drop supporting for DB over the nodes *
> > - Basic operation for node/DB can be done with interactive user
> > interface*
> > - Retrieving stats/actives*
> >
> >
> > Your ideas are welcome in here.
> >
> >
> >
> > [1] https://github.com/wso2/product-ues/
> >
> > [2] https://github.com/wso2/jaggery
> >
> > [3] http://couchdb.readthedocs.org/en/1.6.1/api/server/common.html
> >
> > [4] https://github.com/ryanflorence/react-training/tree/gh-pages/lessons
> >
> >
> > Thanks,
> > --
> > Madhuka Udantha
> > http://madhukaudantha.blogspot.com
> >
>
>
>
> --
> Cheers,
> Madhuka Udantha
> http://madhukaudantha.blogspot.com
>
Re: [GSOC 2015] COUCHDB-2605 Visualize the CouchDB Cluster
Posted by madhuka udantha <ma...@gmail.com>.
Hi All,
Can you point out some resources / tutorials to understand the code base
of couchDB 2.0?
Since I am trying to familiarize myself with the couchDB2.0 code structure
and architecture (mainly for cluster level)
Thanks
On Thu, Mar 19, 2015 at 11:45 AM, madhuka udantha <ma...@gmail.com>
wrote:
> Hi. All
>
>
>
> I'm Udantha. I'm a MSc. Student at University of Moratuwa. This particular
> project looks interesting to me. Since I have previous experiences on
> visualization(web portals / dashboards [1]), javascript [2], html5, REST
> etc.
>
>
>
> Under guidance of Alexander Shorin, Robert Kowalski and Sebastian
> Rothbucher, I have started the initial phase. Building
> (/developer-preview/2.0/) from source and pointing out few small bugs in
> build process. Going through the REST api[3] and following few
> tutorials[4]. If there are any material or ideas that can help for
> $subject, please let me know.
>
>
> Here is the list of features that I gain over the period of time (this can
> be change). Your ideas and view will be grateful in here.
>
>
>
> - Visualize the Couch DB Clusters with nodes (icons, grouping)
> - Memory usage on each node in the cluster.
> - Charting and graph representation of memory usage over the nodes
> - Notification center
> - Where it will give notification for user when he/she reaches memory
> limits
> - Notification over documents/ document count
> - Summary of each node (name of the Databases, recent DB updates on
> each)
> - List databases cluster/node
> - Drag and Drop supporting for DB over the nodes *
> - Basic operation for node/DB can be done with interactive user
> interface*
> - Retrieving stats/actives*
>
>
> Your ideas are welcome in here.
>
>
>
> [1] https://github.com/wso2/product-ues/
>
> [2] https://github.com/wso2/jaggery
>
> [3] http://couchdb.readthedocs.org/en/1.6.1/api/server/common.html
>
> [4] https://github.com/ryanflorence/react-training/tree/gh-pages/lessons
>
>
> Thanks,
> --
> Madhuka Udantha
> http://madhukaudantha.blogspot.com
>
--
Cheers,
Madhuka Udantha
http://madhukaudantha.blogspot.com
Re: [GSOC 2015] COUCHDB-2605 Visualize the CouchDB Cluster
Posted by Alexander Shorin <kx...@gmail.com>.
On Fri, Mar 20, 2015 at 7:50 PM, madhuka udantha
<ma...@gmail.com> wrote:
>
> Q1) I also went through the REST API and I like to know weather we can add
> nodes into clusters from the REST API (else only in config.)?
Documentation is in progress, but here is python[1] example of how dev
nodes becomes a cluster. You can also look on _nodes database which is
available on backdoor (5986) port.
[1]: https://github.com/apache/couchdb/blob/master/dev/run#L298-L307
> Q2) When I am running the fauxton (http://localhost:8000/#/_all_dbs), I see
> the list of DBs but when I click one of them and move to '
> http://localhost:8000/#/database/test8000/_all_docs' it gives an error in
> web UI with RED text saying 'bad request'. Some pages it shows 'Databases
> is not exit' Here i am sharing my screen shoots.
> Is it the way it work?
> I think the reason for above Q2. is round-robin nature of Loadbalancer. Am
> I correct?
Could you open browser network console and say what the actual request
Fauxton made to receive such response?
--
,,,^..^,,,
Re: [GSOC 2015] COUCHDB-2605 Visualize the CouchDB Cluster
Posted by madhuka udantha <ma...@gmail.com>.
Hi,
I built and ran developer-perview 2.0 as explained previously and here are
the steps of it [1].
I have 2 Questions to ask.
Q1) I also went through the REST API and I like to know weather we can add
nodes into clusters from the REST API (else only in config.)?
Q2) When I am running the fauxton (http://localhost:8000/#/_all_dbs), I see
the list of DBs but when I click one of them and move to '
http://localhost:8000/#/database/test8000/_all_docs' it gives an error in
web UI with RED text saying 'bad request'. Some pages it shows 'Databases
is not exit' Here i am sharing my screen shoots.
Is it the way it work?
I think the reason for above Q2. is round-robin nature of Loadbalancer. Am
I correct?
I went one step on this issue (Q2) to find the root then I noticed (I am
using *haproxy.cfg*) it is acting in round robbing manner, So 1st request
hit for node 1, then second one hit for node2, as My test_db was in node1,
not in node2 that's why it says "database is not exist'. It is correct as
request was treated by node2[3] and not from the node1[4]. After 2 request
sent, it shows fine as it hit again by round-robin
Is it normal? or am I missing anything here?
[1]
http://madhukaudantha.blogspot.com/2015/03/couchdb-fauxton-introduction.html
[2] https://app.box.com/s/vunvxxfepzdle33b70710b6omj2rjo2y
[3] https://app.box.com/s/h3izlnadghnj6zfsgww93kernqn9x2gy
[4] https://app.box.com/s/pdsnvle5x1oihubfryojkixj82uz2qe2
On Thu, Mar 19, 2015 at 11:45 AM, madhuka udantha <ma...@gmail.com>
wrote:
> Hi. All
>
>
>
> I'm Udantha. I'm a MSc. Student at University of Moratuwa. This particular
> project looks interesting to me. Since I have previous experiences on
> visualization(web portals / dashboards [1]), javascript [2], html5, REST
> etc.
>
>
>
> Under guidance of Alexander Shorin, Robert Kowalski and Sebastian
> Rothbucher, I have started the initial phase. Building
> (/developer-preview/2.0/) from source and pointing out few small bugs in
> build process. Going through the REST api[3] and following few
> tutorials[4]. If there are any material or ideas that can help for
> $subject, please let me know.
>
>
> Here is the list of features that I gain over the period of time (this can
> be change). Your ideas and view will be grateful in here.
>
>
>
> - Visualize the Couch DB Clusters with nodes (icons, grouping)
> - Memory usage on each node in the cluster.
> - Charting and graph representation of memory usage over the nodes
> - Notification center
> - Where it will give notification for user when he/she reaches memory
> limits
> - Notification over documents/ document count
> - Summary of each node (name of the Databases, recent DB updates on
> each)
> - List databases cluster/node
> - Drag and Drop supporting for DB over the nodes *
> - Basic operation for node/DB can be done with interactive user
> interface*
> - Retrieving stats/actives*
>
>
> Your ideas are welcome in here.
>
>
>
> [1] https://github.com/wso2/product-ues/
>
> [2] https://github.com/wso2/jaggery
>
> [3] http://couchdb.readthedocs.org/en/1.6.1/api/server/common.html
>
> [4] https://github.com/ryanflorence/react-training/tree/gh-pages/lessons
>
>
> Thanks,
> --
> Madhuka Udantha
> http://madhukaudantha.blogspot.com
>
--
Cheers,
Madhuka Udantha
http://madhukaudantha.blogspot.com