You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@beam.apache.org by Yu Watanabe <yu...@gmail.com> on 2019/07/25 01:54:15 UTC

Which memory profiler is used for python 3.7.3 ?

Hello.

The built in memory profiler used guppy for the profiler but  unfortunately
guppy is not supported on python3.

https://beam.apache.org/releases/pydoc/2.13.0/_modules/apache_beam/utils/profiler.html


    try:
      from guppy import hpy  # pylint: disable=import-error
      self._hpy = hpy
      self._interval_second = interval_second
      self._timer = None
    except ImportError:
      warnings.warn('guppy is not installed; MemoryReporter not available.')
      self._hpy = None
    self._enabled = False

I am thinking of using memory profiler  but  I was wondering which memory
profiler do users of apache beam use for python 3.7.3 ?

https://pypi.org/project/memory-profiler/

Thanks,
Yu Watanabe

-- 
Yu Watanabe
Weekend Freelancer who loves to challenge building data platform
yu.w.tennis@gmail.com
[image: LinkedIn icon] <https://www.linkedin.com/in/yuwatanabe1>  [image:
Twitter icon] <https://twitter.com/yuwtennis>

Re: Which memory profiler is used for python 3.7.3 ?

Posted by Valentyn Tymofieiev <va...@google.com>.
Hello,

I have not experimented with profilers on Python 3.7 but filed
https://issues.apache.org/jira/browse/BEAM-7815 to update Beam's profiler
to use a Python3-compatible library.

Feel free to give memory profiler a try and report back how it goes, it may
well be suitable a replacement.

Thanks,
Valentyn

On Wed, Jul 24, 2019 at 6:54 PM Yu Watanabe <yu...@gmail.com> wrote:

> Hello.
>
> The built in memory profiler used guppy for the profiler but
> unfortunately guppy is not supported on python3.
>
>
> https://beam.apache.org/releases/pydoc/2.13.0/_modules/apache_beam/utils/profiler.html
>
>
>     try:
>       from guppy import hpy  # pylint: disable=import-error
>       self._hpy = hpy
>       self._interval_second = interval_second
>       self._timer = None
>     except ImportError:
>       warnings.warn('guppy is not installed; MemoryReporter not
> available.')
>       self._hpy = None
>     self._enabled = False
>
> I am thinking of using memory profiler  but  I was wondering which memory
> profiler do users of apache beam use for python 3.7.3 ?
>
> https://pypi.org/project/memory-profiler/
>
> Thanks,
> Yu Watanabe
>
> --
> Yu Watanabe
> Weekend Freelancer who loves to challenge building data platform
> yu.w.tennis@gmail.com
> [image: LinkedIn icon] <https://www.linkedin.com/in/yuwatanabe1>  [image:
> Twitter icon] <https://twitter.com/yuwtennis>
>