You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Lukasz Gajowy (JIRA)" <ji...@apache.org> on 2019/05/27 12:35:00 UTC
[jira] [Commented] (BEAM-7431) ObjectSizeCalculator causing
regressions in load tests
[ https://issues.apache.org/jira/browse/BEAM-7431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848895#comment-16848895 ]
Lukasz Gajowy commented on BEAM-7431:
-------------------------------------
Doubts that we have after an offline discussion with [~mwalenia]
- measuring bytes this way has a huge overhead
- how to measure bytes (correctly) without such overhead from objects of generic T type?
-should we rely on ByteMonitor or maybe use a pipeline option such as "avgRecordSize" and multiply it times records count (we already have a "countMonitor")? Will this approach work for IO tests?
> ObjectSizeCalculator causing regressions in load tests
> ------------------------------------------------------
>
> Key: BEAM-7431
> URL: https://issues.apache.org/jira/browse/BEAM-7431
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Lukasz Gajowy
> Assignee: Lukasz Gajowy
> Priority: Critical
>
> It looks (based on my investigation) that `ObjectSizeCalculator` is causing a regression in load tests of core operations. the runtime increased from ~400s to ~2000s. Calculating object size this way seems to be an expensive operation - we probably could change it to something simpler (or not?).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)