Re: Node.js npm December usage

[Noah Slater <ns...@apache.org>]

Man, this is totally great.

Perhaps we could write it up as a case-study and promote it on our homepage?

Does that sound like a good idea? Something you could help with?


On 1 January 2013 05:32, Jason Smith <jh...@iriscouch.com> wrote:

> Hi, all. Sorry to be distant from the community recently. No excuse.
>
> I thought I might share December stats from one of Apache CouchDB's most
> well-known deployments and killer apps: the Node.js npm registry.
>
> ## Facts
>
> * Zero downtime
> * Three data centers: SoftLayer, EC2, Joyent
> * 99,327,470 HTTP queries served = 37/sec
>
> * Slowest minute: Dec 08 09:35, 578 queries = 9.6/sec
> * Busiest minute: Dec 20 18:43, 19,776 queries = 329/sec
>
> * Slowest second: (many), 0 queries
> * Busiest second: Dec 20 18:43:03, 932 queries/sec
>
> ## Reflections
>
> This is only the public registry. Our customers and also independent third
> parties run their own replicas. We do not or cannot (respectively) publish
> their usage stats.
>
> Think about that. Isaac owns the registry. We run the registry. Yet neither
> of us can even **know** its entire function, much less do anything about
> it. That is empowerment. That is why I joined CouchDB. CouchDB is free
> software for free data. It carries the ideals of the Free Software movement
> into the 21st century.
>
> Plenty of sites can produce more impressive numbers than these. There are
> even larger CouchDB sites out there. But I am still proud. This is not a
> multi-million dollar venture-capitalized eyeball something something. We
> run standard, orthodox Apache CouchDB. That is encouraging. I did not
> deliver these numbers. Apache CouchDB did. These are not benchmarks. These
> are production logs. That is nine-hundred thirty-two satisfied customers in
> one second! (Well, a true sysadmin would say "not-yet disappointed
> customers" which is all one can ask for.) It shows that anybody can wield
> CouchDB to similar effect.
>
> There are general-purpose programming languages, and there are
> domain-specific programming languages. Nobody gets upset because you can't
> write a web server in YAML. Nobody uses .java configuration files.
>
> Apache CouchDB is a domain-specific database. The npm registry shows: for
> the domain CouchDB addresses, it has no peer.
>
> --
> Iris Couch
>



-- 
NS

Re: Node.js npm December usage

[Noah Slater <ns...@apache.org>]

Jason,

The wiki is a good place for it. Feel free to create a case studies
section, and add your own as the first.

Case studies help to establish legitimacy of our product. I intend to
solicit a few more this year.

I am looking forward to reading yours and being able to share it with
others!


On 18 February 2013 04:01, Jason Smith <jh...@iriscouch.com> wrote:

> Sure!
>
> Not mentioned in that email (and pardon me for banging on about it) is that
> usage grows 15% monthly, i.e. doubling every 5 months. February is a short
> month but we will probably hit 130M queries, a 1/3 growth since I wrote
> that email. Pretty exciting!
>
> We are working on publishing reports and stats about individual packages
> and things, so this is a good time to work on this.
>
> Next steps? Maybe I'll start scribbling down ideas on the wiki?
>
> On Mon, Feb 18, 2013 at 5:25 AM, Noah Slater <ns...@apache.org> wrote:
>
> > Man, this is totally great.
> >
> > Perhaps we could write it up as a case-study and promote it on our
> > homepage?
> >
> > Does that sound like a good idea? Something you could help with?
> >
> >
> > On 1 January 2013 05:32, Jason Smith <jh...@iriscouch.com> wrote:
> >
> > > Hi, all. Sorry to be distant from the community recently. No excuse.
> > >
> > > I thought I might share December stats from one of Apache CouchDB's
> most
> > > well-known deployments and killer apps: the Node.js npm registry.
> > >
> > > ## Facts
> > >
> > > * Zero downtime
> > > * Three data centers: SoftLayer, EC2, Joyent
> > > * 99,327,470 HTTP queries served = 37/sec
> > >
> > > * Slowest minute: Dec 08 09:35, 578 queries = 9.6/sec
> > > * Busiest minute: Dec 20 18:43, 19,776 queries = 329/sec
> > >
> > > * Slowest second: (many), 0 queries
> > > * Busiest second: Dec 20 18:43:03, 932 queries/sec
> > >
> > > ## Reflections
> > >
> > > This is only the public registry. Our customers and also independent
> > third
> > > parties run their own replicas. We do not or cannot (respectively)
> > publish
> > > their usage stats.
> > >
> > > Think about that. Isaac owns the registry. We run the registry. Yet
> > neither
> > > of us can even **know** its entire function, much less do anything
> about
> > > it. That is empowerment. That is why I joined CouchDB. CouchDB is free
> > > software for free data. It carries the ideals of the Free Software
> > movement
> > > into the 21st century.
> > >
> > > Plenty of sites can produce more impressive numbers than these. There
> are
> > > even larger CouchDB sites out there. But I am still proud. This is not
> a
> > > multi-million dollar venture-capitalized eyeball something something.
> We
> > > run standard, orthodox Apache CouchDB. That is encouraging. I did not
> > > deliver these numbers. Apache CouchDB did. These are not benchmarks.
> > These
> > > are production logs. That is nine-hundred thirty-two satisfied
> customers
> > in
> > > one second! (Well, a true sysadmin would say "not-yet disappointed
> > > customers" which is all one can ask for.) It shows that anybody can
> wield
> > > CouchDB to similar effect.
> > >
> > > There are general-purpose programming languages, and there are
> > > domain-specific programming languages. Nobody gets upset because you
> > can't
> > > write a web server in YAML. Nobody uses .java configuration files.
> > >
> > > Apache CouchDB is a domain-specific database. The npm registry shows:
> for
> > > the domain CouchDB addresses, it has no peer.
> > >
> > > --
> > > Iris Couch
> > >
> >
> >
> >
> > --
> > NS
> >
>
>
>
> --
> Iris Couch
>



-- 
NS

Re: Node.js npm December usage

[Hans J Schroeder <hs...@cloudno.de>]

Thanks for the explanation. Again I am impressed with what can be done with
CouchDB. Iriscouch's service is awesome, performance-wise and
availability-wise.

- Hans

On Tuesday, February 19, 2013, Jason Smith wrote:

> Hi, Hans! I just emailed asking if you were on this list. I guess I should
> pay better attention myself!
>
> Yes we use, if you will, wide-area load balancing and also local-area load
> balancing.
>
> Locally, we have CouchDB replicas and reverse-proxies.
>
> Globally, we run a content distribution network, primarily using a custom
> Node.js DNS server: https://github.com/iriscouch/dnsd. We use geolocation
> to route to the best data center.
>
> Both systems also provide high-availability features. If a couch is down,
> we do not (well, ideally!) route to it. If the reverse-proxies are down, we
> do not include its IP address in the DNS response.
>
>
>
> On Tue, Feb 19, 2013 at 2:39 AM, Hans J Schroeder <hs...@cloudno.de> wrote:
>
> > Hi Jason,
> >
> > These stats are totally impressive. Especially because it is real world
> > data and no result of a synthetic benchmark.
> >
> > I am interested how the three data centres are used with standard
> couchdb.
> > A combination of load balancing and master-master replication?
> >
> > - Hans
> >
> > On Feb 18, 2013, at 5:01 AM, Jason Smith <jh...@iriscouch.com> wrote:
> >
> > > Sure!
> > >
> > > Not mentioned in that email (and pardon me for banging on about it) is
> > that
> > > usage grows 15% monthly, i.e. doubling every 5 months. February is a
> > short
> > > month but we will probably hit 130M queries, a 1/3 growth since I wrote
> > > that email. Pretty exciting!
> > >
> > > We are working on publishing reports and stats about individual
> packages
> > > and things, so this is a good time to work on this.
> > >
> > > Next steps? Maybe I'll start scribbling down ideas on the wiki?
> > >
> > > On Mon, Feb 18, 2013 at 5:25 AM, Noah Slater <ns...@apache.org>
> wrote:
> > >
> > >> Man, this is totally great.
> > >>
> > >> Perhaps we could write it up as a case-study and promote it on our
> > >> homepage?
> > >>
> > >> Does that sound like a good idea? Something you could help with?
> > >>
> > >>
> > >> On 1 January 2013 05:32, Jason Smith <jh...@iriscouch.com> wrote:
> > >>
> > >>> Hi, all. Sorry to be distant from the community recently. No excuse.
> > >>>
> > >>> I thought I might share December stats from one of Apache CouchDB's
> > most
> > >>> well-known deployments and killer apps: the Node.js npm registry.
> > >>>
> > >>> ## Facts
> > >>>
> > >>> * Zero downtime
> > >>> * Three data centers: SoftLayer, EC2, Joyent
> > >>> * 99,327,470 HTTP queries served = 37/sec
> > >>>
> > >>> * Slowest minute: Dec 08 09:35, 578 queries = 9.6/sec
> > >>> * Busiest minute: Dec 20 18:43, 19,776 queries = 329/sec
> > >>>
> > >>> * Slowest second: (many), 0 queries
> > >>> * Busiest second: Dec 20 18:43:03, 932 queries/sec
> > >>>
> > >>> ## Reflections
> > >>>
> > >>> This is only the public registry. Our customers and also independent
> > >> third
> > >>> parties run their own replicas. We do not or cannot (respectively)
> > >> publish
> > >>> their usage stats.
> > >>>
> > >>> Think about that. Isaac owns the registry. We run the registry. Yet
> > >> neither
> > >>> of us can even **know** its entire function, much less do anything
> > about
> > >>> it. That is empowerment. That is why I joined CouchDB. CouchDB is
> free
> > >>> software for free data. It carries the ideals of the Free Software
> > >> movement
> > >>> into the 21st century.
> > >>>
> > >>> Plenty of sites can produce more impressive numbers than these. There
> > are
> > >>> even larger CouchDB sites out there. But I am still proud. This is
> not
> > a
> > >>> multi-million dollar venture-capitalized eyeball something something.
> > We
> > >>> run standard, orthodox Apache CouchDB. That is encouraging. I did not
> > >>> deliver these numbers. Apache CouchDB did. These are not benchmarks.
> > >> These
> > >>> are production logs. That is nine-hundred thirty-two satisfied
> > customers
> > >> in
> > >>> one second! (Well, a true sysadmin would say "not-yet disappointed
> > >>> customers" which is all one can ask for.) It shows that anybody can
> > wield
> > >>> CouchDB to similar effect.
> > >>>
> > >>> There are general-purpose programming languages, and there are
> > >>> domain-specific programming languages. Nobody gets upset because you
> > >> can't
> > >>> write a web server --
> Iris Couch
>

Re: Node.js npm December usage

[Jason Smith <jh...@iriscouch.com>]

Hi, Hans! I just emailed asking if you were on this list. I guess I should
pay better attention myself!

Yes we use, if you will, wide-area load balancing and also local-area load
balancing.

Locally, we have CouchDB replicas and reverse-proxies.

Globally, we run a content distribution network, primarily using a custom
Node.js DNS server: https://github.com/iriscouch/dnsd. We use geolocation
to route to the best data center.

Both systems also provide high-availability features. If a couch is down,
we do not (well, ideally!) route to it. If the reverse-proxies are down, we
do not include its IP address in the DNS response.



On Tue, Feb 19, 2013 at 2:39 AM, Hans J Schroeder <hs...@cloudno.de> wrote:

> Hi Jason,
>
> These stats are totally impressive. Especially because it is real world
> data and no result of a synthetic benchmark.
>
> I am interested how the three data centres are used with standard couchdb.
> A combination of load balancing and master-master replication?
>
> - Hans
>
> On Feb 18, 2013, at 5:01 AM, Jason Smith <jh...@iriscouch.com> wrote:
>
> > Sure!
> >
> > Not mentioned in that email (and pardon me for banging on about it) is
> that
> > usage grows 15% monthly, i.e. doubling every 5 months. February is a
> short
> > month but we will probably hit 130M queries, a 1/3 growth since I wrote
> > that email. Pretty exciting!
> >
> > We are working on publishing reports and stats about individual packages
> > and things, so this is a good time to work on this.
> >
> > Next steps? Maybe I'll start scribbling down ideas on the wiki?
> >
> > On Mon, Feb 18, 2013 at 5:25 AM, Noah Slater <ns...@apache.org> wrote:
> >
> >> Man, this is totally great.
> >>
> >> Perhaps we could write it up as a case-study and promote it on our
> >> homepage?
> >>
> >> Does that sound like a good idea? Something you could help with?
> >>
> >>
> >> On 1 January 2013 05:32, Jason Smith <jh...@iriscouch.com> wrote:
> >>
> >>> Hi, all. Sorry to be distant from the community recently. No excuse.
> >>>
> >>> I thought I might share December stats from one of Apache CouchDB's
> most
> >>> well-known deployments and killer apps: the Node.js npm registry.
> >>>
> >>> ## Facts
> >>>
> >>> * Zero downtime
> >>> * Three data centers: SoftLayer, EC2, Joyent
> >>> * 99,327,470 HTTP queries served = 37/sec
> >>>
> >>> * Slowest minute: Dec 08 09:35, 578 queries = 9.6/sec
> >>> * Busiest minute: Dec 20 18:43, 19,776 queries = 329/sec
> >>>
> >>> * Slowest second: (many), 0 queries
> >>> * Busiest second: Dec 20 18:43:03, 932 queries/sec
> >>>
> >>> ## Reflections
> >>>
> >>> This is only the public registry. Our customers and also independent
> >> third
> >>> parties run their own replicas. We do not or cannot (respectively)
> >> publish
> >>> their usage stats.
> >>>
> >>> Think about that. Isaac owns the registry. We run the registry. Yet
> >> neither
> >>> of us can even **know** its entire function, much less do anything
> about
> >>> it. That is empowerment. That is why I joined CouchDB. CouchDB is free
> >>> software for free data. It carries the ideals of the Free Software
> >> movement
> >>> into the 21st century.
> >>>
> >>> Plenty of sites can produce more impressive numbers than these. There
> are
> >>> even larger CouchDB sites out there. But I am still proud. This is not
> a
> >>> multi-million dollar venture-capitalized eyeball something something.
> We
> >>> run standard, orthodox Apache CouchDB. That is encouraging. I did not
> >>> deliver these numbers. Apache CouchDB did. These are not benchmarks.
> >> These
> >>> are production logs. That is nine-hundred thirty-two satisfied
> customers
> >> in
> >>> one second! (Well, a true sysadmin would say "not-yet disappointed
> >>> customers" which is all one can ask for.) It shows that anybody can
> wield
> >>> CouchDB to similar effect.
> >>>
> >>> There are general-purpose programming languages, and there are
> >>> domain-specific programming languages. Nobody gets upset because you
> >> can't
> >>> write a web server in YAML. Nobody uses .java configuration files.
> >>>
> >>> Apache CouchDB is a domain-specific database. The npm registry shows:
> for
> >>> the domain CouchDB addresses, it has no peer.
> >>>
> >>> --
> >>> Iris Couch
> >>>
> >>
> >>
> >>
> >> --
> >> NS
> >>
> >
> >
> >
> > --
> > Iris Couch
>
>


-- 
Iris Couch

Re: Node.js npm December usage

[Hans J Schroeder <hs...@cloudno.de>]

Hi Jason,

These stats are totally impressive. Especially because it is real world data and no result of a synthetic benchmark.

I am interested how the three data centres are used with standard couchdb. A combination of load balancing and master-master replication?

- Hans

On Feb 18, 2013, at 5:01 AM, Jason Smith <jh...@iriscouch.com> wrote:

> Sure!
> 
> Not mentioned in that email (and pardon me for banging on about it) is that
> usage grows 15% monthly, i.e. doubling every 5 months. February is a short
> month but we will probably hit 130M queries, a 1/3 growth since I wrote
> that email. Pretty exciting!
> 
> We are working on publishing reports and stats about individual packages
> and things, so this is a good time to work on this.
> 
> Next steps? Maybe I'll start scribbling down ideas on the wiki?
> 
> On Mon, Feb 18, 2013 at 5:25 AM, Noah Slater <ns...@apache.org> wrote:
> 
>> Man, this is totally great.
>> 
>> Perhaps we could write it up as a case-study and promote it on our
>> homepage?
>> 
>> Does that sound like a good idea? Something you could help with?
>> 
>> 
>> On 1 January 2013 05:32, Jason Smith <jh...@iriscouch.com> wrote:
>> 
>>> Hi, all. Sorry to be distant from the community recently. No excuse.
>>> 
>>> I thought I might share December stats from one of Apache CouchDB's most
>>> well-known deployments and killer apps: the Node.js npm registry.
>>> 
>>> ## Facts
>>> 
>>> * Zero downtime
>>> * Three data centers: SoftLayer, EC2, Joyent
>>> * 99,327,470 HTTP queries served = 37/sec
>>> 
>>> * Slowest minute: Dec 08 09:35, 578 queries = 9.6/sec
>>> * Busiest minute: Dec 20 18:43, 19,776 queries = 329/sec
>>> 
>>> * Slowest second: (many), 0 queries
>>> * Busiest second: Dec 20 18:43:03, 932 queries/sec
>>> 
>>> ## Reflections
>>> 
>>> This is only the public registry. Our customers and also independent
>> third
>>> parties run their own replicas. We do not or cannot (respectively)
>> publish
>>> their usage stats.
>>> 
>>> Think about that. Isaac owns the registry. We run the registry. Yet
>> neither
>>> of us can even **know** its entire function, much less do anything about
>>> it. That is empowerment. That is why I joined CouchDB. CouchDB is free
>>> software for free data. It carries the ideals of the Free Software
>> movement
>>> into the 21st century.
>>> 
>>> Plenty of sites can produce more impressive numbers than these. There are
>>> even larger CouchDB sites out there. But I am still proud. This is not a
>>> multi-million dollar venture-capitalized eyeball something something. We
>>> run standard, orthodox Apache CouchDB. That is encouraging. I did not
>>> deliver these numbers. Apache CouchDB did. These are not benchmarks.
>> These
>>> are production logs. That is nine-hundred thirty-two satisfied customers
>> in
>>> one second! (Well, a true sysadmin would say "not-yet disappointed
>>> customers" which is all one can ask for.) It shows that anybody can wield
>>> CouchDB to similar effect.
>>> 
>>> There are general-purpose programming languages, and there are
>>> domain-specific programming languages. Nobody gets upset because you
>> can't
>>> write a web server in YAML. Nobody uses .java configuration files.
>>> 
>>> Apache CouchDB is a domain-specific database. The npm registry shows: for
>>> the domain CouchDB addresses, it has no peer.
>>> 
>>> --
>>> Iris Couch
>>> 
>> 
>> 
>> 
>> --
>> NS
>> 
> 
> 
> 
> -- 
> Iris Couch

Re: Node.js npm December usage

[Jason Smith <jh...@iriscouch.com>]

Sure!

Not mentioned in that email (and pardon me for banging on about it) is that
usage grows 15% monthly, i.e. doubling every 5 months. February is a short
month but we will probably hit 130M queries, a 1/3 growth since I wrote
that email. Pretty exciting!

We are working on publishing reports and stats about individual packages
and things, so this is a good time to work on this.

Next steps? Maybe I'll start scribbling down ideas on the wiki?

On Mon, Feb 18, 2013 at 5:25 AM, Noah Slater <ns...@apache.org> wrote:

> Man, this is totally great.
>
> Perhaps we could write it up as a case-study and promote it on our
> homepage?
>
> Does that sound like a good idea? Something you could help with?
>
>
> On 1 January 2013 05:32, Jason Smith <jh...@iriscouch.com> wrote:
>
> > Hi, all. Sorry to be distant from the community recently. No excuse.
> >
> > I thought I might share December stats from one of Apache CouchDB's most
> > well-known deployments and killer apps: the Node.js npm registry.
> >
> > ## Facts
> >
> > * Zero downtime
> > * Three data centers: SoftLayer, EC2, Joyent
> > * 99,327,470 HTTP queries served = 37/sec
> >
> > * Slowest minute: Dec 08 09:35, 578 queries = 9.6/sec
> > * Busiest minute: Dec 20 18:43, 19,776 queries = 329/sec
> >
> > * Slowest second: (many), 0 queries
> > * Busiest second: Dec 20 18:43:03, 932 queries/sec
> >
> > ## Reflections
> >
> > This is only the public registry. Our customers and also independent
> third
> > parties run their own replicas. We do not or cannot (respectively)
> publish
> > their usage stats.
> >
> > Think about that. Isaac owns the registry. We run the registry. Yet
> neither
> > of us can even **know** its entire function, much less do anything about
> > it. That is empowerment. That is why I joined CouchDB. CouchDB is free
> > software for free data. It carries the ideals of the Free Software
> movement
> > into the 21st century.
> >
> > Plenty of sites can produce more impressive numbers than these. There are
> > even larger CouchDB sites out there. But I am still proud. This is not a
> > multi-million dollar venture-capitalized eyeball something something. We
> > run standard, orthodox Apache CouchDB. That is encouraging. I did not
> > deliver these numbers. Apache CouchDB did. These are not benchmarks.
> These
> > are production logs. That is nine-hundred thirty-two satisfied customers
> in
> > one second! (Well, a true sysadmin would say "not-yet disappointed
> > customers" which is all one can ask for.) It shows that anybody can wield
> > CouchDB to similar effect.
> >
> > There are general-purpose programming languages, and there are
> > domain-specific programming languages. Nobody gets upset because you
> can't
> > write a web server in YAML. Nobody uses .java configuration files.
> >
> > Apache CouchDB is a domain-specific database. The npm registry shows: for
> > the domain CouchDB addresses, it has no peer.
> >
> > --
> > Iris Couch
> >
>
>
>
> --
> NS
>



-- 
Iris Couch