Exposing Bugs

[Ryan Ebanks <ry...@gmail.com>]

I have issued a pull request that allows local streams to  shutdown if a
provider, processor, or writer tasks stops execution due to an unhandled
throwable.  This should be a step forward in making streams more
transparent and reliable on the local level.  The Throwable will appear in
the logs and it will note which Runnable class threw the error.

This is intended to only be a temporary fix.  This will illuminate many
streams operations that have bugs, and need better testing.  Please create
tickets for all exceptions that are caused by bugs and issue pull requests
(with appropriate tests and documentation) to fix them.

In the future we can relaunch threads that exit due to error, and reprocess
the failed datum.  Until we implemented better datum tacking this is not
possible, but it is on the road map.

Please create tickets or notify the dev list of bugs you encounter.  This
push should expose bugs that previously were causing streams to fail
silently or stall/hang.  We need to have big push for better tested code in
the project.

Note: Threads that exit due to error that are not started/handled by the
local stream builder will still fail silently.  The fact that so many
providers, processors, and writers have to launch their own threads means
we should work on the local frame work to remove those needs push almost
all thread management responsibility to the framework so we can ensure
consistent monitoring/error handling of threads.

Thanks,
Ryan Ebanks

Re: Exposing Bugs

[Renato Marroquín Mogrovejo <re...@gmail.com>]

Sorry Ryan I sent that too fast I saw you created STREAMS-179.
Thanks! I will try debugging RSS provider with that :)

Renato M.
On Sep 20, 2014 3:16 AM, "Renato Marroquín Mogrovejo" <
renatoj.marroquin@gmail.com> wrote:

> These are great news Ryan! So will you update us when the pull request is
> up? Are you going to create a JIRA for keeping track of this?
> I am really eager to get the RSS provider working :-)
>
> Renato M.
> On Sep 19, 2014 10:44 PM, "Ryan Ebanks" <ry...@gmail.com> wrote:
>
>> I have issued a pull request that allows local streams to  shutdown if a
>> provider, processor, or writer tasks stops execution due to an unhandled
>> throwable.  This should be a step forward in making streams more
>> transparent and reliable on the local level.  The Throwable will appear in
>> the logs and it will note which Runnable class threw the error.
>>
>> This is intended to only be a temporary fix.  This will illuminate many
>> streams operations that have bugs, and need better testing.  Please create
>> tickets for all exceptions that are caused by bugs and issue pull requests
>> (with appropriate tests and documentation) to fix them.
>>
>> In the future we can relaunch threads that exit due to error, and
>> reprocess
>> the failed datum.  Until we implemented better datum tacking this is not
>> possible, but it is on the road map.
>>
>> Please create tickets or notify the dev list of bugs you encounter.  This
>> push should expose bugs that previously were causing streams to fail
>> silently or stall/hang.  We need to have big push for better tested code
>> in
>> the project.
>>
>> Note: Threads that exit due to error that are not started/handled by the
>> local stream builder will still fail silently.  The fact that so many
>> providers, processors, and writers have to launch their own threads means
>> we should work on the local frame work to remove those needs push almost
>> all thread management responsibility to the framework so we can ensure
>> consistent monitoring/error handling of threads.
>>
>> Thanks,
>> Ryan Ebanks
>>
>

Re: Exposing Bugs

[Renato Marroquín Mogrovejo <re...@gmail.com>]

These are great news Ryan! So will you update us when the pull request is
up? Are you going to create a JIRA for keeping track of this?
I am really eager to get the RSS provider working :-)

Renato M.
On Sep 19, 2014 10:44 PM, "Ryan Ebanks" <ry...@gmail.com> wrote:

> I have issued a pull request that allows local streams to  shutdown if a
> provider, processor, or writer tasks stops execution due to an unhandled
> throwable.  This should be a step forward in making streams more
> transparent and reliable on the local level.  The Throwable will appear in
> the logs and it will note which Runnable class threw the error.
>
> This is intended to only be a temporary fix.  This will illuminate many
> streams operations that have bugs, and need better testing.  Please create
> tickets for all exceptions that are caused by bugs and issue pull requests
> (with appropriate tests and documentation) to fix them.
>
> In the future we can relaunch threads that exit due to error, and reprocess
> the failed datum.  Until we implemented better datum tacking this is not
> possible, but it is on the road map.
>
> Please create tickets or notify the dev list of bugs you encounter.  This
> push should expose bugs that previously were causing streams to fail
> silently or stall/hang.  We need to have big push for better tested code in
> the project.
>
> Note: Threads that exit due to error that are not started/handled by the
> local stream builder will still fail silently.  The fact that so many
> providers, processors, and writers have to launch their own threads means
> we should work on the local frame work to remove those needs push almost
> all thread management responsibility to the framework so we can ensure
> consistent monitoring/error handling of threads.
>
> Thanks,
> Ryan Ebanks
>