You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Yan Fang <ya...@gmail.com> on 2014/07/02 18:58:23 UTC
Review Request 23235: SAMZA-59
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23235/
-----------------------------------------------------------
Review request for samza.
Repository: samza
Description
-------
Support dropping messages on serde failures.
1. added a task property: task.drop.deserialization.errors . If set to true, the system will skip error messages. If set to false, will throw exceptions.
2. use try/catch to wrap butter.update in SystemConsumers.
3. added method in TaskConfig.
4. added Test method in TestSystemConsumers
5. updated the configuration page to include the new property.
Diffs
-----
docs/learn/documentation/0.7.0/jobs/configuration-table.html 07cb83f
samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 18a9510
samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 356adbb
samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala b537046
samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala e1a4c15
Diff: https://reviews.apache.org/r/23235/diff/
Testing
-------
Thanks,
Yan Fang
Re: Review Request 23235: SAMZA-59
Posted by Chris Riccomini <cr...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23235/#review47232
-----------------------------------------------------------
samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala
<https://reviews.apache.org/r/23235/#comment82868>
Isolate the buffer.update call outside of this try/catch so that the only thing that can throw an exception is the serdeManager.fromBytes call. Right now buffering message chooser or underlying chooser exceptions could be caught rather than deserialization errors.
samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala
<https://reviews.apache.org/r/23235/#comment82867>
Should add a debug statement and metric here.
- Chris Riccomini
On July 2, 2014, 4:58 p.m., Yan Fang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23235/
> -----------------------------------------------------------
>
> (Updated July 2, 2014, 4:58 p.m.)
>
>
> Review request for samza.
>
>
> Repository: samza
>
>
> Description
> -------
>
> Support dropping messages on serde failures.
>
> 1. added a task property: task.drop.deserialization.errors . If set to true, the system will skip error messages. If set to false, will throw exceptions.
> 2. use try/catch to wrap butter.update in SystemConsumers.
> 3. added method in TaskConfig.
> 4. added Test method in TestSystemConsumers
> 5. updated the configuration page to include the new property.
>
>
> Diffs
> -----
>
> docs/learn/documentation/0.7.0/jobs/configuration-table.html 07cb83f
> samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 18a9510
> samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 356adbb
> samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala b537046
> samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala e1a4c15
>
> Diff: https://reviews.apache.org/r/23235/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Yan Fang
>
>
Re: Review Request 23235: SAMZA-59
Posted by Yan Fang <ya...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23235/
-----------------------------------------------------------
(Updated July 13, 2014, 5:44 a.m.)
Review request for samza.
Changes
-------
1. added debug and metrics
2. isolated the serdeManager.fromBytes and serdeManager.toBytes
3. added the same feature to SystemProducers: a new property "task.drop.serialization.errors" to define whether dropping the messages or not when serialization error occurs.
Repository: samza
Description
-------
Support dropping messages on serde failures.
1. added a task property: task.drop.deserialization.errors . If set to true, the system will skip error messages. If set to false, will throw exceptions.
2. use try/catch to wrap butter.update in SystemConsumers.
3. added method in TaskConfig.
4. added Test method in TestSystemConsumers
5. updated the configuration page to include the new property.
Diffs (updated)
-----
docs/learn/documentation/0.7.0/jobs/configuration-table.html 07cb83f
samza-core/src/main/scala/org/apache/samza/config/TaskConfig.scala 18a9510
samza-core/src/main/scala/org/apache/samza/container/SamzaContainer.scala 356adbb
samza-core/src/main/scala/org/apache/samza/system/SystemConsumers.scala b537046
samza-core/src/main/scala/org/apache/samza/system/SystemConsumersMetrics.scala d632314
samza-core/src/main/scala/org/apache/samza/system/SystemProducers.scala 8fb36b3
samza-core/src/main/scala/org/apache/samza/system/SystemProducersMetrics.scala 594dd51
samza-core/src/test/scala/org/apache/samza/system/TestSystemConsumers.scala e1a4c15
samza-core/src/test/scala/org/apache/samza/system/TestSystemProducers.scala PRE-CREATION
Diff: https://reviews.apache.org/r/23235/diff/
Testing
-------
Thanks,
Yan Fang