Improvements to local stream builder

[Ryan Ebanks <ry...@gmail.com>]

I created some pull requests for review.  The main purpose is to improve
the local stream builder runtime.  The idea was to remove busy waits,
unnecessary waiting, and allow more transparency into what is happening
when a stream is running.  This only scratching the service on all of the
needs improvements but it is a start.

In the pull request I have included code that allows monitoring of the
queues that linked together the processors/writers/providers through JMX.
This should allow you to view where data is backing up and help you
configure your streams in the future.

I plan on continuing to add more JMX enabled monitoring in the future.  The
next step would be to refactor the DatumCountable objects to use JMX
instead of printing results to the console.  This will reduce thread usage,
and make the information available remotely and easier to read.

Please let me know if there is any feed back on adding JMX functionality.
I will create JIRA tickets and would love help in creating these
improvements.

-Ryan Ebanks

Re: Improvements to local stream builder

[Renato Marroquín Mogrovejo <re...@gmail.com>]

Cool thing Ryan! If you put those into JIRAs I would be happy to help out.


Renato M.

2014-10-08 21:11 GMT+02:00 Ryan Ebanks <ry...@gmail.com>:

> I created some pull requests for review.  The main purpose is to improve
> the local stream builder runtime.  The idea was to remove busy waits,
> unnecessary waiting, and allow more transparency into what is happening
> when a stream is running.  This only scratching the service on all of the
> needs improvements but it is a start.
>
> In the pull request I have included code that allows monitoring of the
> queues that linked together the processors/writers/providers through JMX.
> This should allow you to view where data is backing up and help you
> configure your streams in the future.
>
> I plan on continuing to add more JMX enabled monitoring in the future.  The
> next step would be to refactor the DatumCountable objects to use JMX
> instead of printing results to the console.  This will reduce thread usage,
> and make the information available remotely and easier to read.
>
> Please let me know if there is any feed back on adding JMX functionality.
> I will create JIRA tickets and would love help in creating these
> improvements.
>
> -Ryan Ebanks
>