[GitHub] [beam] kennknowles opened a new issue, #19128: Logging on Dataflow for Python's Beam 2.7.0 and 2.8.0 not working

[GitBox <gi...@apache.org>]

kennknowles opened a new issue, #19128:
URL: https://github.com/apache/beam/issues/19128

   Hello, my name is Eugene. This is my first bug I am submitting to the Apache Beam project!
   
   I'm using Python's SDK on Google Cloud Platform to prototype with Beam locally inside Datalab and then submit a job to Dataflow. In GCP, I'm using Dataflow and use the logging as basically a print statement to check the state of objects. For example, after a GroupByKey, if I run Beam locally, the grouped elements are in a list. In Dataflow, the group elements sometimes will be a generator. Hence, logging is very important to me.
   
    
   
   When submitting a job to Dataflow using Beam 2.6.0, the logging works correctly. When I click on a box on the Dataflow job's user interface, I can see a bunch of logs. However, when submitting a job to Dataflow using Beam 2.7.0 or 2.8.0, the logs do not appear at all.
   
   I believe this logging problem is specific to Dataflow/GCP and might not apply to the other runners. The reason is that I look in the StackDriver for the job submitted for Dataflow with Beam versions 2.6.0, 2.7.0, and 2.8.0–all the logs exist in StackDriver. Hence, the logging was correctly accounted for. However, in the Dataflow user interface, the logs do not appear for Beam 2.7.0 or 2.8.0 when you click on the box for a transformation.
   
   It appears that this problem is GCP specific. I'm creating this bug here since I don't know where else to raise this issue. Sorry if this is not the appropriate place to post the bug. If there is a more appropriate place, please tell me and I'll raise the bug there.
   
   If there are any questions, I can provide screenshots or anything you like.
   
   Thank you,
    Eugene
   
   Imported from Jira [BEAM-6107](https://issues.apache.org/jira/browse/BEAM-6107). Original Jira may contain additional context.
   Reported by: eugeneh101.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org