Fwd: AutoMetric not working with pojo?

[Siyuan Hua <si...@datatorrent.com>]

> Annotations is not a good idea. We can add Interface Aggregator the way we
> have for partitions as Partitioner
>
> Thanks
> - Gaurav
>
> On Dec 17, 2015, at 10:36 PM, Timothy Farkas <ti...@datatorrent.com> wrote:
>
> -1 For specifying an aggregator in an annotation. This will become
> confusing / ill defined when there are multiple AutoMetrics in an operator
> +1 For adding a new interface. The operator developer is already familiar
> with that method of specification for partitioners.
> -1 For defining single metric aggregators via a map. Too many mechanisms
> for defining the same thing leads to a complex api and complex validations
> to check for conflicting configurations. If it is necessary to define an
> aggregator on a per metric basis this could be done in the current api by
> implementing an aggregator which allows single field aggregators to be
> defined on it as properties.
>
> On Thu, Dec 17, 2015 at 10:28 PM, Chinmay Kolhatkar <
> chinmay@datatorrent.com> wrote:
>
>> Please ignore. The MetricsAggregator object can take of multiple fields
>> aggregations.
>>
>> But that still is quite some work for Operator writer.
>>
>> How about method returning Map of field->SinfleMetricAggregator ?
>>
>> ~ Chinmay.
>>
>> On Fri, Dec 18, 2015 at 11:54 AM, Chinmay Kolhatkar <
>> chinmay@datatorrent.com> wrote:
>>
>>> Maybe it can return a Map of field->MetricsAggregator. As a operator
>>> developer I might have fields which needs different types of
>>> MetricsAggregator.
>>>
>>> ~ Chinmay.
>>>
>>> On Fri, Dec 18, 2015 at 11:52 AM, Chandni Singh <chandni@datatorrent.com
>>> > wrote:
>>>
>>>> Specifying MetricsAggregator as annotation is not useful.
>>>>
>>>> A better solution is to introduce a new interface say
>>>> DefaultMetricsAggregatorProvider
>>>> {
>>>>  MetricsAggregator getDefaultMetricsAggregator()
>>>> }
>>>> which when implemented by the Operator can provide the instance to
>>>> aggregate.
>>>>
>>>> Chandni
>>>>
>>>> On Thu, Dec 17, 2015 at 9:52 PM, Thomas Weise <th...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Curious what others think about this. Would it be useful for the
>>>>> operator developer to be able to specify a default aggregator?
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 17, 2015 at 9:43 PM, Chandni Singh <
>>>>> chandni@datatorrent.com> wrote:
>>>>>
>>>>>> That is not accurate. We did think about this. We had design
>>>>>> discussions and multiple iterations for AutoMetric. It was reviewed by
>>>>>> several folks.
>>>>>> In fact specifying aggregator as an annotation param is not a new
>>>>>> idea. The idea was dropped because then you cannot provide an object and
>>>>>> therefore cannot configure an aggregator.
>>>>>>
>>>>>> The current way is flexible that multiple physical metrics can be
>>>>>> aggregated to provide a single value for example bytes/sec.
>>>>>>
>>>>>> Chandni
>>>>>>
>>>>>> On Thu, Dec 17, 2015 at 9:34 PM, Thomas Weise <thomas@datatorrent.com
>>>>>> > wrote:
>>>>>>
>>>>>>> The operator developer defines the metrics and would very likely
>>>>>>> know how they are best aggregated. Just because we did not think about it
>>>>>>> in the first cut of the feature and it wasn't needed then, does not mean we
>>>>>>> cannot add the capability should there be consensus that it is useful.
>>>>>>>
>>>>>>> To me, this is similar to partitioners and stats listeners. Some
>>>>>>> operators have a default implementation for reasonable out of box behavior,
>>>>>>> others don't. The application assembler should have the ability to override.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Dec 17, 2015 at 7:39 PM, Chandni Singh <
>>>>>>> chandni@datatorrent.com> wrote:
>>>>>>>
>>>>>>>> Operator developer provides the auto metric at physical operator
>>>>>>>> level. The application developer chooses how to aggregate them.
>>>>>>>> The pojo should be visible in the physical level.
>>>>>>>>
>>>>>>>> For logical level, the platform inserts a default metrics
>>>>>>>> aggregator for primitive numeric types. For  a complex type, application
>>>>>>>> developer needs to provide a custom aggregator.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Dec 17, 2015 at 7:23 PM, Thomas Weise <
>>>>>>>> thomas@datatorrent.com> wrote:
>>>>>>>>
>>>>>>>>> Operator developer needs to be able provide stats if that is the
>>>>>>>>> intention. Strange that this would apply to simple values and not to
>>>>>>>>> complex types.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> sent from mobile
>>>>>>>>> On Dec 17, 2015 7:08 PM, "Chandni Singh" <ch...@datatorrent.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Why? Application developer wires the application and therefore
>>>>>>>>>> chooses how to aggregate metrics of an operator.
>>>>>>>>>>
>>>>>>>>>> It is similar to partitioning where application developer chooses
>>>>>>>>>> how to partition.
>>>>>>>>>>
>>>>>>>>>> Chandni
>>>>>>>>>>
>>>>>>>>>> On Thu, Dec 17, 2015 at 7:01 PM, Thomas Weise <
>>>>>>>>>> thomas@datatorrent.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Application developer shouldn't have to know about this.
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> sent from mobile
>>>>>>>>>>> On Dec 17, 2015 6:52 PM, "Siyuan Hua" <si...@datatorrent.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I just followed the document, the document doesn't say
>>>>>>>>>>>> aggregator is necessary for complex object. And I feel the aggregator
>>>>>>>>>>>> should be set as an parameter of the annotation, otherwise developer always
>>>>>>>>>>>> need to remember to set the attribute to make it work?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks!
>>>>>>>>>>>>
>>>>>>>>>>>> Sent from my iPhone
>>>>>>>>>>>>
>>>>>>>>>>>> On Dec 17, 2015, at 17:19, Chandni Singh <
>>>>>>>>>>>> chandni@datatorrent.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Siyuan,
>>>>>>>>>>>> Did you add a Metrics aggregator for the POJO autometric?
>>>>>>>>>>>> If an auto-metric is a number, by default a metrics aggregator
>>>>>>>>>>>> is added. When not you need to set an aggregator.
>>>>>>>>>>>>
>>>>>>>>>>>> Please refer this document:
>>>>>>>>>>>> http://docs.datatorrent.com/autometrics/
>>>>>>>>>>>>
>>>>>>>>>>>> Chandni
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Dec 17, 2015 at 5:15 PM, Chinmay Kolhatkar <
>>>>>>>>>>>> chinmay@datatorrent.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Last I checked it worked with simple variables only not POJO.
>>>>>>>>>>>>> I thin AutoMetrics are identified for simple variable types
>>>>>>>>>>>>> only.
>>>>>>>>>>>>>
>>>>>>>>>>>>> ~ Chinmay.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Dec 18, 2015 at 6:25 AM, Siyuan Hua <
>>>>>>>>>>>>> siyuan@datatorrent.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> When I put AutoMetric on a pojo property, it is always null
>>>>>>>>>>>>>> in the REST response.
>>>>>>>>>>>>>> It works if I flatten the pojo. Can any one verify this?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Siyuan
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
>