You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Matthew Willson (JIRA)" <ji...@apache.org> on 2019/01/09 21:12:00 UTC
[jira] [Updated] (BEAM-6389) Sources using Metrics.counter fail
with 'Can't pickle ... DelegatingCounter'
[ https://issues.apache.org/jira/browse/BEAM-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matthew Willson updated BEAM-6389:
----------------------------------
Priority: Major (was: Minor)
Description:
This fails under Python 2.7 but works fine under Python 3 – looks like an easy fix.
I am reading from a beam.io.utils.CountingSource:
{{beam.io.Read(beam_io_utils.CountingSource(NUM_ROWS))}}
This fails under python 2.7 with:
Python exception: Can't pickle <class 'apache_beam.metrics.metric.DelegatingCounter'>: attribute lookup apache_beam.metrics.metric.DelegatingCounter failed
Looks like this is because DelegatingCounter is actually an inner class defined inside apache_beam.metrics.metric.Metric. Under Python 3 pickling can resolve it under its qualified name apache_beam.metrics.metric.Metric.DelegatingCounter, but not in Python 2. A fix would be to move it to a top-level class in that module.
This also applies to other sources which use Metrics.counter.
was:
This fails under Python 2.7 but works fine under Python 3 – looks like an easy fix.
I am reading from a beam.io.utils.CountingSource:
{{beam.io.Read(beam_io_utils.CountingSource(NUM_ROWS))}}
This fails under python 2.7 with:
Python exception: Can't pickle <class 'apache_beam.metrics.metric.DelegatingCounter'>: attribute lookup apache_beam.metrics.metric.DelegatingCounter failed
Looks like this is because DelegatingCounter is actually an inner class defined inside apache_beam.metrics.metric.Metric. Under Python 3 pickling can resolve it under its qualified name apache_beam.metrics.metric.Metric.DelegatingCounter, but not in Python 2. A fix would be to move it to a top-level class in that module.
Summary: Sources using Metrics.counter fail with 'Can't pickle ... DelegatingCounter' (was: Read from python CountingSource fails with 'Can't pickle ... DelegatingCounter')
> Sources using Metrics.counter fail with 'Can't pickle ... DelegatingCounter'
> ----------------------------------------------------------------------------
>
> Key: BEAM-6389
> URL: https://issues.apache.org/jira/browse/BEAM-6389
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.9.0
> Reporter: Matthew Willson
> Assignee: Ahmet Altay
> Priority: Major
>
> This fails under Python 2.7 but works fine under Python 3 – looks like an easy fix.
> I am reading from a beam.io.utils.CountingSource:
> {{beam.io.Read(beam_io_utils.CountingSource(NUM_ROWS))}}
> This fails under python 2.7 with:
> Python exception: Can't pickle <class 'apache_beam.metrics.metric.DelegatingCounter'>: attribute lookup apache_beam.metrics.metric.DelegatingCounter failed
> Looks like this is because DelegatingCounter is actually an inner class defined inside apache_beam.metrics.metric.Metric. Under Python 3 pickling can resolve it under its qualified name apache_beam.metrics.metric.Metric.DelegatingCounter, but not in Python 2. A fix would be to move it to a top-level class in that module.
> This also applies to other sources which use Metrics.counter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)