You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Alex Amato (JIRA)" <ji...@apache.org> on 2019/06/01 00:48:00 UTC

[jira] [Commented] (BEAM-7462) Add Sampled Byte Count Metric to the Java SDK

    [ https://issues.apache.org/jira/browse/BEAM-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16853513#comment-16853513 ] 

Alex Amato commented on BEAM-7462:
----------------------------------

DF Job  with more logging

[2019-05-31_10_08_10-16779835117472819827|https://pantheon.corp.google.com/logs/viewer?resource=dataflow_step%2Fjob_id%2F2019-05-31_10_08_10-16779835117472819827&logName=projects%2Fapache-beam-testing%2Flogs%2Fdataflow.googleapis.com%252Fworker&interval=NO_LIMIT&project=apache-beam-testing&e=13803970,13803205,13804191,13804201,13804391,13807341&organizationId=433637338589&minLogLevel=0&expandAll=false&timestamp=2019-06-01T00:40:51.628000000Z&customFacets=&limitCustomFacetWidth=true&scrollTimestamp=2019-05-31T17:10:08.924000000Z]

 

We suspected that the pcollectionId might belong to a side input. But I was unable to confirm, as I wasn't able to print the deserialized side input data where I added debug logging in DoFnPtransformRunner. There was still more serialized data.

ajamato sample with pCollectionId: generatedPcollection-191 pColl.getName(): generatedPcollection-191 Coder: org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder

 

 [https://pantheon.corp.google.com/logs/viewer?resource=dataflow_step%2Fjob_id%2F2019-05-31_10_08_10-16779835117472819827&logName=projects%2Fapache-beam-testing%2Flogs%2Fdataflow.googleapis.com%252Fworker&interval=NO_LIMIT&project=apache-beam-testing&e=13803970,13803205,13804191,13804201,13804391,13807341&organizationId=433637338589&minLogLevel=0&expandAll=false&timestamp=2019-06-01T00:40:51.628000000Z&customFacets=&limitCustomFacetWidth=true&scrollTimestamp=2019-05-31T17:10:08.924000000Z]

> Add Sampled Byte Count Metric to the Java SDK
> ---------------------------------------------
>
>                 Key: BEAM-7462
>                 URL: https://issues.apache.org/jira/browse/BEAM-7462
>             Project: Beam
>          Issue Type: New Feature
>          Components: java-fn-execution
>            Reporter: Alex Amato
>            Priority: Major
>         Attachments: bundle_descriptor_dump.txt
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)