You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Shoaib Zafar <sh...@venturedive.com> on 2020/02/10 19:05:30 UTC
Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Hello Beamers.
Just curious does anyone having trouble running
':sdks:python:test-suites:tox:pycommon:docs' command locally?
After rebasing with master recently, I am facing sphinx thread fork error
with on my macos mojave, using python 3.7.0.
I Tried to add system variable "export
OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but no
luck!
Any suggestions/help?
Thanks!
Console Log:
------------------------------------------------------
....
Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
Creating file target/docs/source/apache_beam.utils.retry.rst.
Creating file target/docs/source/apache_beam.utils.subprocess_server.rst.
Creating file target/docs/source/apache_beam.utils.thread_pool_executor.rst.
Creating file target/docs/source/apache_beam.utils.timestamp.rst.
Creating file target/docs/source/apache_beam.utils.urns.rst.
Creating file target/docs/source/apache_beam.utils.rst.
objc[8384]: +[__NSCFConstantString initialize] may have been in progress in
another thread when fork() was called.
objc[8384]: +[__NSCFConstantString initialize] may have been in progress in
another thread when fork() was called. We cannot safely call it or ignore
it in the fork() child process. Crashing instead. Set a breakpoint on
objc_initializeAfterForkError to debug.
Traceback (most recent call last):
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
line 304, in build_main
app.build(args.force_all, filenames)
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
line 335, in build
self.builder.build_all()
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
line 305, in build_all
self.build(None, summary=__('all source files'), method='all')
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
line 360, in build
updated_docnames = set(self.read())
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
line 466, in read
self._read_parallel(docnames, nproc=self.app.parallel)
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
line 521, in _read_parallel
tasks.join()
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
line 114, in join
self._join_one()
File
"/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
line 120, in _join_one
exc, logs, result = pipe.recv()
File
"/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
line 250, in recv
buf = self._recv_bytes()
File
"/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
line 407, in _recv_bytes
buf = self._recv(4)
File
"/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
line 383, in _recv
raise EOFError
EOFError
Exception occurred:
File
"/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
line 383, in _recv
raise EOFError
EOFError
The full traceback has been saved in
/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error
message can be provided next time.
A bug report can be filed in the tracker at <
https://github.com/sphinx-doc/sphinx/issues>. Thanks!
objc[8385]: +[__NSCFConstantString initialize] may have been in progress in
another thread when fork() was called.
objc[8385]: +[__NSCFConstantString initialize] may have been in progress in
another thread when fork() was called. We cannot safely call it or ignore
it in the fork() child process. Crashing instead. Set a breakpoint on
objc_initializeAfterForkError to debug.
objc[8386]: +[__NSCFConstantString initialize] may have been in progress in
another thread when fork() was called.
objc[8386]: +[__NSCFConstantString initialize] may have been in progress in
another thread when fork() was called. We cannot safely call it or ignore
it in the fork() child process. Crashing instead. Set a breakpoint on
objc_initializeAfterForkError to debug.
....
....
....
9.54 real 4.59 user 1.69 sys
ERROR: InvocationError for command /usr/bin/time scripts/generate_pydoc.sh
(exited with code 2)
py37-docs run-test-post: commands[0] |
/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
___________________________________ summary
____________________________________
ERROR: py37-docs: commands failed
> Task :sdks:python:test-suites:tox:pycommon:docs FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:python:test-suites:tox:pycommon:docs'.
> Process 'command 'sh'' finished with non-zero exit value 1
------------------------------------------------------
*Shoaib Zafar*
Software Engineering Lead
Mobile: +92 333 274 6242
Skype: live:shoaibzafar_1
<http://venturedive.com/>
Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Posted by Ahmet Altay <al...@google.com>.
Sent https://github.com/apache/beam/pull/10874 to remove this flag.
On Tue, Feb 11, 2020 at 5:38 PM jincheng sun <su...@gmail.com>
wrote:
> I think it's a good advice to remove the "-j 8" option if it doesn't
> affect the performance much.
>
>
> Udi Meiri <eh...@google.com> 于2020年2月12日周三 上午2:20写道：
>
>> For me the difference was about 20s longer (40s -> 60s approx). Not
>> significant IMO
>>
>> On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay <al...@google.com> wrote:
>>
>>> Should we remove the "-j 8" option by default? Sphinx docs says this is
>>> an experimental option [1]. I do not recall docs generation taking a long
>>> time, does this increase significantly without this option?
>>>
>>> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
>>>
>>> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar <
>>> shoaib.zafar@venturedive.com> wrote:
>>>
>>>> Thanks, Udi and Jincheng for the response.
>>>> The suggested solution worked for me as well.
>>>>
>>>> Regards,
>>>>
>>>> *Shoaib Zafar*
>>>> Software Engineering Lead
>>>> Mobile: +92 333 274 6242
>>>> Skype: live:shoaibzafar_1
>>>>
>>>> <http://venturedive.com/>
>>>>
>>>>
>>>> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <su...@gmail.com>
>>>> wrote:
>>>>
>>>>> I have verified that this issue could be reproduced in my local
>>>>> environment (MacOS) and the solution suggested by Udi could work!
>>>>>
>>>>> Best,
>>>>> Jincheng
>>>>>
>>>>> Udi Meiri <eh...@google.com> 于2020年2月11日周二 上午8:51写道：
>>>>>
>>>>>> I don't have those issues (running on Linux), but a possible
>>>>>> workaround could be to remove the "-j 8" flags (2 locations) in
>>>>>> generate_pydoc.sh.
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>>>>>> shoaib.zafar@venturedive.com> wrote:
>>>>>>
>>>>>>> Hello Beamers.
>>>>>>>
>>>>>>> Just curious does anyone having trouble running
>>>>>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>>>>>>
>>>>>>> After rebasing with master recently, I am facing sphinx thread fork
>>>>>>> error with on my macos mojave, using python 3.7.0.
>>>>>>> I Tried to add system variable "export
>>>>>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google)
>>>>>>> but no luck!
>>>>>>>
>>>>>>> Any suggestions/help?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> Console Log:
>>>>>>> ------------------------------------------------------
>>>>>>> ....
>>>>>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>>>>>>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>>>>>>> Creating file
>>>>>>> target/docs/source/apache_beam.utils.subprocess_server.rst.
>>>>>>> Creating file
>>>>>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>>>>>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>>>>>>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>>>>>>> Creating file target/docs/source/apache_beam.utils.rst.
>>>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>>>>> progress in another thread when fork() was called.
>>>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>>>>
>>>>>>> Traceback (most recent call last):
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>>>>>>> line 304, in build_main
>>>>>>> app.build(args.force_all, filenames)
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>>>>>>> line 335, in build
>>>>>>> self.builder.build_all()
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>>> line 305, in build_all
>>>>>>> self.build(None, summary=__('all source files'), method='all')
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>>> line 360, in build
>>>>>>> updated_docnames = set(self.read())
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>>> line 466, in read
>>>>>>> self._read_parallel(docnames, nproc=self.app.parallel)
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>>> line 521, in _read_parallel
>>>>>>> tasks.join()
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>>>>> line 114, in join
>>>>>>> self._join_one()
>>>>>>> File
>>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>>>>> line 120, in _join_one
>>>>>>> exc, logs, result = pipe.recv()
>>>>>>> File
>>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>>> line 250, in recv
>>>>>>> buf = self._recv_bytes()
>>>>>>> File
>>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>>> line 407, in _recv_bytes
>>>>>>> buf = self._recv(4)
>>>>>>> File
>>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>>> line 383, in _recv
>>>>>>> raise EOFError
>>>>>>> EOFError
>>>>>>>
>>>>>>> Exception occurred:
>>>>>>> File
>>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>>> line 383, in _recv
>>>>>>> raise EOFError
>>>>>>> EOFError
>>>>>>> The full traceback has been saved in
>>>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
>>>>>>> if you want to report the issue to the developers.
>>>>>>> Please also report this if it was a user error, so that a better
>>>>>>> error message can be provided next time.
>>>>>>> A bug report can be filed in the tracker at <
>>>>>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>>>>> progress in another thread when fork() was called.
>>>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>>>>> progress in another thread when fork() was called.
>>>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>>>> ....
>>>>>>> ....
>>>>>>> ....
>>>>>>> 9.54 real 4.59 user 1.69 sys
>>>>>>> ERROR: InvocationError for command /usr/bin/time
>>>>>>> scripts/generate_pydoc.sh (exited with code 2)
>>>>>>> py37-docs run-test-post: commands[0] |
>>>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
>>>>>>> ___________________________________ summary
>>>>>>> ____________________________________
>>>>>>> ERROR: py37-docs: commands failed
>>>>>>>
>>>>>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED
>>>>>>>
>>>>>>> FAILURE: Build failed with an exception.
>>>>>>>
>>>>>>> * What went wrong:
>>>>>>> Execution failed for task
>>>>>>> ':sdks:python:test-suites:tox:pycommon:docs'.
>>>>>>> > Process 'command 'sh'' finished with non-zero exit value 1
>>>>>>> ------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *Shoaib Zafar*
>>>>>>> Software Engineering Lead
>>>>>>> Mobile: +92 333 274 6242
>>>>>>> Skype: live:shoaibzafar_1
>>>>>>>
>>>>>>> <http://venturedive.com/>
>>>>>>>
>>>>>>
Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Posted by jincheng sun <su...@gmail.com>.
I think it's a good advice to remove the "-j 8" option if it doesn't affect
the performance much.
Udi Meiri <eh...@google.com> 于2020年2月12日周三 上午2:20写道：
> For me the difference was about 20s longer (40s -> 60s approx). Not
> significant IMO
>
> On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay <al...@google.com> wrote:
>
>> Should we remove the "-j 8" option by default? Sphinx docs says this is
>> an experimental option [1]. I do not recall docs generation taking a long
>> time, does this increase significantly without this option?
>>
>> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
>>
>> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar <
>> shoaib.zafar@venturedive.com> wrote:
>>
>>> Thanks, Udi and Jincheng for the response.
>>> The suggested solution worked for me as well.
>>>
>>> Regards,
>>>
>>> *Shoaib Zafar*
>>> Software Engineering Lead
>>> Mobile: +92 333 274 6242
>>> Skype: live:shoaibzafar_1
>>>
>>> <http://venturedive.com/>
>>>
>>>
>>> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <su...@gmail.com>
>>> wrote:
>>>
>>>> I have verified that this issue could be reproduced in my local
>>>> environment (MacOS) and the solution suggested by Udi could work!
>>>>
>>>> Best,
>>>> Jincheng
>>>>
>>>> Udi Meiri <eh...@google.com> 于2020年2月11日周二 上午8:51写道：
>>>>
>>>>> I don't have those issues (running on Linux), but a possible
>>>>> workaround could be to remove the "-j 8" flags (2 locations) in
>>>>> generate_pydoc.sh.
>>>>>
>>>>>
>>>>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>>>>> shoaib.zafar@venturedive.com> wrote:
>>>>>
>>>>>> Hello Beamers.
>>>>>>
>>>>>> Just curious does anyone having trouble running
>>>>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>>>>>
>>>>>> After rebasing with master recently, I am facing sphinx thread fork
>>>>>> error with on my macos mojave, using python 3.7.0.
>>>>>> I Tried to add system variable "export
>>>>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google)
>>>>>> but no luck!
>>>>>>
>>>>>> Any suggestions/help?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> Console Log:
>>>>>> ------------------------------------------------------
>>>>>> ....
>>>>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>>>>>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>>>>>> Creating file
>>>>>> target/docs/source/apache_beam.utils.subprocess_server.rst.
>>>>>> Creating file
>>>>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>>>>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>>>>>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>>>>>> Creating file target/docs/source/apache_beam.utils.rst.
>>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>>>> progress in another thread when fork() was called.
>>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>>>
>>>>>> Traceback (most recent call last):
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>>>>>> line 304, in build_main
>>>>>> app.build(args.force_all, filenames)
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>>>>>> line 335, in build
>>>>>> self.builder.build_all()
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>> line 305, in build_all
>>>>>> self.build(None, summary=__('all source files'), method='all')
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>> line 360, in build
>>>>>> updated_docnames = set(self.read())
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>> line 466, in read
>>>>>> self._read_parallel(docnames, nproc=self.app.parallel)
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>>> line 521, in _read_parallel
>>>>>> tasks.join()
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>>>> line 114, in join
>>>>>> self._join_one()
>>>>>> File
>>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>>>> line 120, in _join_one
>>>>>> exc, logs, result = pipe.recv()
>>>>>> File
>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>> line 250, in recv
>>>>>> buf = self._recv_bytes()
>>>>>> File
>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>> line 407, in _recv_bytes
>>>>>> buf = self._recv(4)
>>>>>> File
>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>> line 383, in _recv
>>>>>> raise EOFError
>>>>>> EOFError
>>>>>>
>>>>>> Exception occurred:
>>>>>> File
>>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>>> line 383, in _recv
>>>>>> raise EOFError
>>>>>> EOFError
>>>>>> The full traceback has been saved in
>>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
>>>>>> if you want to report the issue to the developers.
>>>>>> Please also report this if it was a user error, so that a better
>>>>>> error message can be provided next time.
>>>>>> A bug report can be filed in the tracker at <
>>>>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>>>> progress in another thread when fork() was called.
>>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>>>> progress in another thread when fork() was called.
>>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>>> ....
>>>>>> ....
>>>>>> ....
>>>>>> 9.54 real 4.59 user 1.69 sys
>>>>>> ERROR: InvocationError for command /usr/bin/time
>>>>>> scripts/generate_pydoc.sh (exited with code 2)
>>>>>> py37-docs run-test-post: commands[0] |
>>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
>>>>>> ___________________________________ summary
>>>>>> ____________________________________
>>>>>> ERROR: py37-docs: commands failed
>>>>>>
>>>>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED
>>>>>>
>>>>>> FAILURE: Build failed with an exception.
>>>>>>
>>>>>> * What went wrong:
>>>>>> Execution failed for task
>>>>>> ':sdks:python:test-suites:tox:pycommon:docs'.
>>>>>> > Process 'command 'sh'' finished with non-zero exit value 1
>>>>>> ------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>> *Shoaib Zafar*
>>>>>> Software Engineering Lead
>>>>>> Mobile: +92 333 274 6242
>>>>>> Skype: live:shoaibzafar_1
>>>>>>
>>>>>> <http://venturedive.com/>
>>>>>>
>>>>>
Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Posted by Udi Meiri <eh...@google.com>.
For me the difference was about 20s longer (40s -> 60s approx). Not
significant IMO
On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay <al...@google.com> wrote:
> Should we remove the "-j 8" option by default? Sphinx docs says this is an
> experimental option [1]. I do not recall docs generation taking a long
> time, does this increase significantly without this option?
>
> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
>
> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar <sh...@venturedive.com>
> wrote:
>
>> Thanks, Udi and Jincheng for the response.
>> The suggested solution worked for me as well.
>>
>> Regards,
>>
>> *Shoaib Zafar*
>> Software Engineering Lead
>> Mobile: +92 333 274 6242
>> Skype: live:shoaibzafar_1
>>
>> <http://venturedive.com/>
>>
>>
>> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <su...@gmail.com>
>> wrote:
>>
>>> I have verified that this issue could be reproduced in my local
>>> environment (MacOS) and the solution suggested by Udi could work!
>>>
>>> Best,
>>> Jincheng
>>>
>>> Udi Meiri <eh...@google.com> 于2020年2月11日周二 上午8:51写道：
>>>
>>>> I don't have those issues (running on Linux), but a possible workaround
>>>> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
>>>>
>>>>
>>>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>>>> shoaib.zafar@venturedive.com> wrote:
>>>>
>>>>> Hello Beamers.
>>>>>
>>>>> Just curious does anyone having trouble running
>>>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>>>>
>>>>> After rebasing with master recently, I am facing sphinx thread fork
>>>>> error with on my macos mojave, using python 3.7.0.
>>>>> I Tried to add system variable "export
>>>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google)
>>>>> but no luck!
>>>>>
>>>>> Any suggestions/help?
>>>>>
>>>>> Thanks!
>>>>>
>>>>> Console Log:
>>>>> ------------------------------------------------------
>>>>> ....
>>>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>>>>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>>>>> Creating file
>>>>> target/docs/source/apache_beam.utils.subprocess_server.rst.
>>>>> Creating file
>>>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>>>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>>>>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>>>>> Creating file target/docs/source/apache_beam.utils.rst.
>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>>> progress in another thread when fork() was called.
>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>>
>>>>> Traceback (most recent call last):
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>>>>> line 304, in build_main
>>>>> app.build(args.force_all, filenames)
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>>>>> line 335, in build
>>>>> self.builder.build_all()
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>> line 305, in build_all
>>>>> self.build(None, summary=__('all source files'), method='all')
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>> line 360, in build
>>>>> updated_docnames = set(self.read())
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>> line 466, in read
>>>>> self._read_parallel(docnames, nproc=self.app.parallel)
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>>> line 521, in _read_parallel
>>>>> tasks.join()
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>>> line 114, in join
>>>>> self._join_one()
>>>>> File
>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>>> line 120, in _join_one
>>>>> exc, logs, result = pipe.recv()
>>>>> File
>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>> line 250, in recv
>>>>> buf = self._recv_bytes()
>>>>> File
>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>> line 407, in _recv_bytes
>>>>> buf = self._recv(4)
>>>>> File
>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>> line 383, in _recv
>>>>> raise EOFError
>>>>> EOFError
>>>>>
>>>>> Exception occurred:
>>>>> File
>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>>> line 383, in _recv
>>>>> raise EOFError
>>>>> EOFError
>>>>> The full traceback has been saved in
>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
>>>>> if you want to report the issue to the developers.
>>>>> Please also report this if it was a user error, so that a better error
>>>>> message can be provided next time.
>>>>> A bug report can be filed in the tracker at <
>>>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>>> progress in another thread when fork() was called.
>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>>> progress in another thread when fork() was called.
>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>>> progress in another thread when fork() was called. We cannot safely call it
>>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>> ....
>>>>> ....
>>>>> ....
>>>>> 9.54 real 4.59 user 1.69 sys
>>>>> ERROR: InvocationError for command /usr/bin/time
>>>>> scripts/generate_pydoc.sh (exited with code 2)
>>>>> py37-docs run-test-post: commands[0] |
>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
>>>>> ___________________________________ summary
>>>>> ____________________________________
>>>>> ERROR: py37-docs: commands failed
>>>>>
>>>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED
>>>>>
>>>>> FAILURE: Build failed with an exception.
>>>>>
>>>>> * What went wrong:
>>>>> Execution failed for task ':sdks:python:test-suites:tox:pycommon:docs'.
>>>>> > Process 'command 'sh'' finished with non-zero exit value 1
>>>>> ------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> *Shoaib Zafar*
>>>>> Software Engineering Lead
>>>>> Mobile: +92 333 274 6242
>>>>> Skype: live:shoaibzafar_1
>>>>>
>>>>> <http://venturedive.com/>
>>>>>
>>>>
Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Posted by Ahmet Altay <al...@google.com>.
Should we remove the "-j 8" option by default? Sphinx docs says this is an
experimental option [1]. I do not recall docs generation taking a long
time, does this increase significantly without this option?
[1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar <sh...@venturedive.com>
wrote:
> Thanks, Udi and Jincheng for the response.
> The suggested solution worked for me as well.
>
> Regards,
>
> *Shoaib Zafar*
> Software Engineering Lead
> Mobile: +92 333 274 6242
> Skype: live:shoaibzafar_1
>
> <http://venturedive.com/>
>
>
> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <su...@gmail.com>
> wrote:
>
>> I have verified that this issue could be reproduced in my local
>> environment (MacOS) and the solution suggested by Udi could work!
>>
>> Best,
>> Jincheng
>>
>> Udi Meiri <eh...@google.com> 于2020年2月11日周二 上午8:51写道：
>>
>>> I don't have those issues (running on Linux), but a possible workaround
>>> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
>>>
>>>
>>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>>> shoaib.zafar@venturedive.com> wrote:
>>>
>>>> Hello Beamers.
>>>>
>>>> Just curious does anyone having trouble running
>>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>>>
>>>> After rebasing with master recently, I am facing sphinx thread fork
>>>> error with on my macos mojave, using python 3.7.0.
>>>> I Tried to add system variable "export
>>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but
>>>> no luck!
>>>>
>>>> Any suggestions/help?
>>>>
>>>> Thanks!
>>>>
>>>> Console Log:
>>>> ------------------------------------------------------
>>>> ....
>>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>>>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>>>> Creating file
>>>> target/docs/source/apache_beam.utils.subprocess_server.rst.
>>>> Creating file
>>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>>>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>>>> Creating file target/docs/source/apache_beam.utils.rst.
>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>> progress in another thread when fork() was called.
>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>>> progress in another thread when fork() was called. We cannot safely call it
>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>>
>>>> Traceback (most recent call last):
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>>>> line 304, in build_main
>>>> app.build(args.force_all, filenames)
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>>>> line 335, in build
>>>> self.builder.build_all()
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>> line 305, in build_all
>>>> self.build(None, summary=__('all source files'), method='all')
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>> line 360, in build
>>>> updated_docnames = set(self.read())
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>> line 466, in read
>>>> self._read_parallel(docnames, nproc=self.app.parallel)
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>>> line 521, in _read_parallel
>>>> tasks.join()
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>> line 114, in join
>>>> self._join_one()
>>>> File
>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>>> line 120, in _join_one
>>>> exc, logs, result = pipe.recv()
>>>> File
>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>> line 250, in recv
>>>> buf = self._recv_bytes()
>>>> File
>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>> line 407, in _recv_bytes
>>>> buf = self._recv(4)
>>>> File
>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>> line 383, in _recv
>>>> raise EOFError
>>>> EOFError
>>>>
>>>> Exception occurred:
>>>> File
>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>>> line 383, in _recv
>>>> raise EOFError
>>>> EOFError
>>>> The full traceback has been saved in
>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
>>>> if you want to report the issue to the developers.
>>>> Please also report this if it was a user error, so that a better error
>>>> message can be provided next time.
>>>> A bug report can be filed in the tracker at <
>>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>> progress in another thread when fork() was called.
>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in
>>>> progress in another thread when fork() was called. We cannot safely call it
>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>> progress in another thread when fork() was called.
>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in
>>>> progress in another thread when fork() was called. We cannot safely call it
>>>> or ignore it in the fork() child process. Crashing instead. Set a
>>>> breakpoint on objc_initializeAfterForkError to debug.
>>>> ....
>>>> ....
>>>> ....
>>>> 9.54 real 4.59 user 1.69 sys
>>>> ERROR: InvocationError for command /usr/bin/time
>>>> scripts/generate_pydoc.sh (exited with code 2)
>>>> py37-docs run-test-post: commands[0] |
>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
>>>> ___________________________________ summary
>>>> ____________________________________
>>>> ERROR: py37-docs: commands failed
>>>>
>>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED
>>>>
>>>> FAILURE: Build failed with an exception.
>>>>
>>>> * What went wrong:
>>>> Execution failed for task ':sdks:python:test-suites:tox:pycommon:docs'.
>>>> > Process 'command 'sh'' finished with non-zero exit value 1
>>>> ------------------------------------------------------
>>>>
>>>>
>>>>
>>>> *Shoaib Zafar*
>>>> Software Engineering Lead
>>>> Mobile: +92 333 274 6242
>>>> Skype: live:shoaibzafar_1
>>>>
>>>> <http://venturedive.com/>
>>>>
>>>
Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Posted by Shoaib Zafar <sh...@venturedive.com>.
Thanks, Udi and Jincheng for the response.
The suggested solution worked for me as well.
Regards,
*Shoaib Zafar*
Software Engineering Lead
Mobile: +92 333 274 6242
Skype: live:shoaibzafar_1
<http://venturedive.com/>
On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <su...@gmail.com>
wrote:
> I have verified that this issue could be reproduced in my local
> environment (MacOS) and the solution suggested by Udi could work!
>
> Best,
> Jincheng
>
> Udi Meiri <eh...@google.com> 于2020年2月11日周二 上午8:51写道：
>
>> I don't have those issues (running on Linux), but a possible workaround
>> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
>>
>>
>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>> shoaib.zafar@venturedive.com> wrote:
>>
>>> Hello Beamers.
>>>
>>> Just curious does anyone having trouble running
>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>>
>>> After rebasing with master recently, I am facing sphinx thread fork
>>> error with on my macos mojave, using python 3.7.0.
>>> I Tried to add system variable "export
>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but
>>> no luck!
>>>
>>> Any suggestions/help?
>>>
>>> Thanks!
>>>
>>> Console Log:
>>> ------------------------------------------------------
>>> ....
>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>>> Creating file target/docs/source/apache_beam.utils.subprocess_server.rst.
>>> Creating file
>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>>> Creating file target/docs/source/apache_beam.utils.rst.
>>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called.
>>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called. We cannot safely call it or
>>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>>> on objc_initializeAfterForkError to debug.
>>>
>>> Traceback (most recent call last):
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>>> line 304, in build_main
>>> app.build(args.force_all, filenames)
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>>> line 335, in build
>>> self.builder.build_all()
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 305, in build_all
>>> self.build(None, summary=__('all source files'), method='all')
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 360, in build
>>> updated_docnames = set(self.read())
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 466, in read
>>> self._read_parallel(docnames, nproc=self.app.parallel)
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 521, in _read_parallel
>>> tasks.join()
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>> line 114, in join
>>> self._join_one()
>>> File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>> line 120, in _join_one
>>> exc, logs, result = pipe.recv()
>>> File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 250, in recv
>>> buf = self._recv_bytes()
>>> File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 407, in _recv_bytes
>>> buf = self._recv(4)
>>> File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 383, in _recv
>>> raise EOFError
>>> EOFError
>>>
>>> Exception occurred:
>>> File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 383, in _recv
>>> raise EOFError
>>> EOFError
>>> The full traceback has been saved in
>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
>>> if you want to report the issue to the developers.
>>> Please also report this if it was a user error, so that a better error
>>> message can be provided next time.
>>> A bug report can be filed in the tracker at <
>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>>> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called.
>>> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called. We cannot safely call it or
>>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>>> on objc_initializeAfterForkError to debug.
>>> objc[8386]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called.
>>> objc[8386]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called. We cannot safely call it or
>>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>>> on objc_initializeAfterForkError to debug.
>>> ....
>>> ....
>>> ....
>>> 9.54 real 4.59 user 1.69 sys
>>> ERROR: InvocationError for command /usr/bin/time
>>> scripts/generate_pydoc.sh (exited with code 2)
>>> py37-docs run-test-post: commands[0] |
>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
>>> ___________________________________ summary
>>> ____________________________________
>>> ERROR: py37-docs: commands failed
>>>
>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED
>>>
>>> FAILURE: Build failed with an exception.
>>>
>>> * What went wrong:
>>> Execution failed for task ':sdks:python:test-suites:tox:pycommon:docs'.
>>> > Process 'command 'sh'' finished with non-zero exit value 1
>>> ------------------------------------------------------
>>>
>>>
>>>
>>> *Shoaib Zafar*
>>> Software Engineering Lead
>>> Mobile: +92 333 274 6242
>>> Skype: live:shoaibzafar_1
>>>
>>> <http://venturedive.com/>
>>>
>>
Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Posted by jincheng sun <su...@gmail.com>.
I have verified that this issue could be reproduced in my local environment
(MacOS) and the solution suggested by Udi could work!
Best,
Jincheng
Udi Meiri <eh...@google.com> 于2020年2月11日周二 上午8:51写道：
> I don't have those issues (running on Linux), but a possible workaround
> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
>
>
> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
> shoaib.zafar@venturedive.com> wrote:
>
>> Hello Beamers.
>>
>> Just curious does anyone having trouble running
>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>
>> After rebasing with master recently, I am facing sphinx thread fork error
>> with on my macos mojave, using python 3.7.0.
>> I Tried to add system variable "export
>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but
>> no luck!
>>
>> Any suggestions/help?
>>
>> Thanks!
>>
>> Console Log:
>> ------------------------------------------------------
>> ....
>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>> Creating file target/docs/source/apache_beam.utils.subprocess_server.rst.
>> Creating file
>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>> Creating file target/docs/source/apache_beam.utils.rst.
>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called.
>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called. We cannot safely call it or
>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>> on objc_initializeAfterForkError to debug.
>>
>> Traceback (most recent call last):
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>> line 304, in build_main
>> app.build(args.force_all, filenames)
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>> line 335, in build
>> self.builder.build_all()
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 305, in build_all
>> self.build(None, summary=__('all source files'), method='all')
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 360, in build
>> updated_docnames = set(self.read())
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 466, in read
>> self._read_parallel(docnames, nproc=self.app.parallel)
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 521, in _read_parallel
>> tasks.join()
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>> line 114, in join
>> self._join_one()
>> File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>> line 120, in _join_one
>> exc, logs, result = pipe.recv()
>> File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 250, in recv
>> buf = self._recv_bytes()
>> File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 407, in _recv_bytes
>> buf = self._recv(4)
>> File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 383, in _recv
>> raise EOFError
>> EOFError
>>
>> Exception occurred:
>> File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 383, in _recv
>> raise EOFError
>> EOFError
>> The full traceback has been saved in
>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
>> if you want to report the issue to the developers.
>> Please also report this if it was a user error, so that a better error
>> message can be provided next time.
>> A bug report can be filed in the tracker at <
>> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
>> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called.
>> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called. We cannot safely call it or
>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>> on objc_initializeAfterForkError to debug.
>> objc[8386]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called.
>> objc[8386]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called. We cannot safely call it or
>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>> on objc_initializeAfterForkError to debug.
>> ....
>> ....
>> ....
>> 9.54 real 4.59 user 1.69 sys
>> ERROR: InvocationError for command /usr/bin/time
>> scripts/generate_pydoc.sh (exited with code 2)
>> py37-docs run-test-post: commands[0] |
>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
>> ___________________________________ summary
>> ____________________________________
>> ERROR: py37-docs: commands failed
>>
>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED
>>
>> FAILURE: Build failed with an exception.
>>
>> * What went wrong:
>> Execution failed for task ':sdks:python:test-suites:tox:pycommon:docs'.
>> > Process 'command 'sh'' finished with non-zero exit value 1
>> ------------------------------------------------------
>>
>>
>>
>> *Shoaib Zafar*
>> Software Engineering Lead
>> Mobile: +92 333 274 6242
>> Skype: live:shoaibzafar_1
>>
>> <http://venturedive.com/>
>>
>
Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)
Posted by Udi Meiri <eh...@google.com>.
I don't have those issues (running on Linux), but a possible workaround
could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <sh...@venturedive.com>
wrote:
> Hello Beamers.
>
> Just curious does anyone having trouble running
> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>
> After rebasing with master recently, I am facing sphinx thread fork error
> with on my macos mojave, using python 3.7.0.
> I Tried to add system variable "export
> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but no
> luck!
>
> Any suggestions/help?
>
> Thanks!
>
> Console Log:
> ------------------------------------------------------
> ....
> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
> Creating file target/docs/source/apache_beam.utils.retry.rst.
> Creating file target/docs/source/apache_beam.utils.subprocess_server.rst.
> Creating file
> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
> Creating file target/docs/source/apache_beam.utils.urns.rst.
> Creating file target/docs/source/apache_beam.utils.rst.
> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called.
> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called. We cannot safely call it or
> ignore it in the fork() child process. Crashing instead. Set a breakpoint
> on objc_initializeAfterForkError to debug.
>
> Traceback (most recent call last):
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
> line 304, in build_main
> app.build(args.force_all, filenames)
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
> line 335, in build
> self.builder.build_all()
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 305, in build_all
> self.build(None, summary=__('all source files'), method='all')
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 360, in build
> updated_docnames = set(self.read())
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 466, in read
> self._read_parallel(docnames, nproc=self.app.parallel)
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 521, in _read_parallel
> tasks.join()
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
> line 114, in join
> self._join_one()
> File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
> line 120, in _join_one
> exc, logs, result = pipe.recv()
> File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 250, in recv
> buf = self._recv_bytes()
> File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 407, in _recv_bytes
> buf = self._recv(4)
> File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 383, in _recv
> raise EOFError
> EOFError
>
> Exception occurred:
> File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 383, in _recv
> raise EOFError
> EOFError
> The full traceback has been saved in
> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
> if you want to report the issue to the developers.
> Please also report this if it was a user error, so that a better error
> message can be provided next time.
> A bug report can be filed in the tracker at <
> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called.
> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called. We cannot safely call it or
> ignore it in the fork() child process. Crashing instead. Set a breakpoint
> on objc_initializeAfterForkError to debug.
> objc[8386]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called.
> objc[8386]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called. We cannot safely call it or
> ignore it in the fork() child process. Crashing instead. Set a breakpoint
> on objc_initializeAfterForkError to debug.
> ....
> ....
> ....
> 9.54 real 4.59 user 1.69 sys
> ERROR: InvocationError for command /usr/bin/time scripts/generate_pydoc.sh
> (exited with code 2)
> py37-docs run-test-post: commands[0] |
> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh
> ___________________________________ summary
> ____________________________________
> ERROR: py37-docs: commands failed
>
> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED
>
> FAILURE: Build failed with an exception.
>
> * What went wrong:
> Execution failed for task ':sdks:python:test-suites:tox:pycommon:docs'.
> > Process 'command 'sh'' finished with non-zero exit value 1
> ------------------------------------------------------
>
>
>
> *Shoaib Zafar*
> Software Engineering Lead
> Mobile: +92 333 274 6242
> Skype: live:shoaibzafar_1
>
> <http://venturedive.com/>
>