Documentation on internal LevelDB design?

[Kevin Burton <bu...@spinn3r.com>]

Is there any documentation on the internal LevelDB design?

I can just dive into the source of course but I was curious if there was a
high level document explaining how it works.

I’m curious how individual queues are backed in LevelDB as I can’t
understand why GCing them takes too long.

I started working on a proposal for a new in-memory , journaled persistence
layer, at least to think out loud, but then realized that in theory LevelDB
*may* already be laid out like this.

During destination removal the LevelDB code must be doing something rather
silly since each operation takes about 100ms..

Kevin

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>

Re: Documentation on internal LevelDB design?

[Kevin Burton <bu...@spinn3r.com>]

I think what I want is in-memory but journaled persistence with crash-only
semantics.

Basically, log each message to disk, but keep them in memory.  This would
mean that even on a HDD you would be seeing 100MB throughput.

In theory , leveldb should be able to sustain that, I’m just not sure why
deleting the destinations is so expensive.  Perhaps each one is a separate
database ...

On Sat, Feb 14, 2015 at 6:09 PM, artnaseef <ar...@artnaseef.com> wrote:

> If you want "in-memory persistence" only, just disable persistence and the
> broker automatically uses an in-memory persistence adapter.  That's the
> fastest configuration for ActiveMQ, but obviously loses reliability in the
> case of the broker shutting down since all messages still pending in memory
> are lost.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Documentation-on-internal-LevelDB-design-tp4691567p4691577.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>



-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>
<http://spinn3r.com>

Re: Documentation on internal LevelDB design?

[artnaseef <ar...@artnaseef.com>]

If you want "in-memory persistence" only, just disable persistence and the
broker automatically uses an in-memory persistence adapter.  That's the
fastest configuration for ActiveMQ, but obviously loses reliability in the
case of the broker shutting down since all messages still pending in memory
are lost.



--
View this message in context: http://activemq.2283324.n4.nabble.com/Documentation-on-internal-LevelDB-design-tp4691567p4691577.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.