You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Beam JIRA Bot (Jira)" <ji...@apache.org> on 2020/09/11 17:08:04 UTC
[jira] [Commented] (BEAM-10627) tests fails on windows -
interactive tests fails due to FileNotFoundError
[ https://issues.apache.org/jira/browse/BEAM-10627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17194413#comment-17194413 ]
Beam JIRA Bot commented on BEAM-10627:
--------------------------------------
This issue is assigned but has not received an update in 30 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned.
> tests fails on windows - interactive tests fails due to FileNotFoundError
> -------------------------------------------------------------------------
>
> Key: BEAM-10627
> URL: https://issues.apache.org/jira/browse/BEAM-10627
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core, testing
> Reporter: Tobiasz Kedzierski
> Assignee: Ning Kang
> Priority: P2
> Labels: stale-assigned
> Attachments: BEAM-10627.txt
>
>
> Failing tests:
> apache_beam.runners.interactive.interactive_runner_test.InteractiveRunnerTest.test_basic
> apache_beam.runners.interactive.interactive_runner_test.InteractiveRunnerTest.test_wordcount
> apache_beam.runners.interactive.interactive_beam_test.InteractiveBeamTest.test_show_always_watch_given_pcolls
> apache_beam.runners.interactive.interactive_beam_test.InteractiveBeamTest.test_show_mark_pcolls_computed_when_done
> Link to the github workflow run with mentioned error:
> [https://github.com/TobKed/beam/runs/937336438?check_suite_focus=true]
> partial log:
> 2020-08-02T11:05:43.5852779Z ___________ InteractiveBeamTest.test_show_always_watch_given_pcolls ___________
> 2020-08-02T11:05:43.5853476Z [gw3] win32 -- Python 3.5.4 d:\a\beam\beam\sdks\python\target\.tox\py35-win\scripts\python.exe
> 2020-08-02T11:05:43.5853847Z
> 2020-08-02T11:05:43.5855313Z self = <apache_beam.runners.interactive.interactive_beam_test.InteractiveBeamTest testMethod=test_show_always_watch_given_pcolls>
> 2020-08-02T11:05:43.5855658Z
> 2020-08-02T11:05:43.5855975Z def test_show_always_watch_given_pcolls(self):
> 2020-08-02T11:05:43.5856278Z p = beam.Pipeline(ir.InteractiveRunner())
> 2020-08-02T11:05:43.5856566Z # pylint: disable=range-builtin-not-iterating
> 2020-08-02T11:05:43.5856845Z pcoll = p | 'Create' >> beam.Create(range(10))
> 2020-08-02T11:05:43.5857355Z # The pcoll is not watched since watch(locals()) is not explicitly called.
> 2020-08-02T11:05:43.5858106Z self.assertFalse(pcoll in _get_watched_pcollections_with_variable_names())
> 2020-08-02T11:05:43.5858620Z # The call of show watches pcoll.
> 2020-08-02T11:05:43.5859235Z > ib.show(pcoll)
> 2020-08-02T11:05:43.5859475Z
> 2020-08-02T11:05:43.5860015Z apache_beam\runners\interactive\interactive_beam_test.py:96:
> 2020-08-02T11:05:43.5861024Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2020-08-02T11:05:43.5861944Z apache_beam\runners\interactive\utils.py:205: in run_within_progress_indicator
> 2020-08-02T11:05:43.5862682Z return func(*args, **kwargs)
> 2020-08-02T11:05:43.5863214Z apache_beam\runners\interactive\interactive_beam.py:411: in show
> 2020-08-02T11:05:43.5863760Z result = pf.PipelineFragment(list(pcolls), user_pipeline.options).run()
> 2020-08-02T11:05:43.5864291Z apache_beam\runners\interactive\pipeline_fragment.py:113: in run
> 2020-08-02T11:05:43.5864746Z return self.deduce_fragment().run()
> 2020-08-02T11:05:43.5865292Z apache_beam\pipeline.py:521: in run
> 2020-08-02T11:05:43.5865633Z allow_proto_holders=True).run(False)
> 2020-08-02T11:05:43.5866159Z apache_beam\pipeline.py:534: in run
> 2020-08-02T11:05:43.5866638Z return self.runner.run_pipeline(self, self._options)
> 2020-08-02T11:05:43.5867299Z apache_beam\runners\interactive\interactive_runner.py:194: in run_pipeline
> 2020-08-02T11:05:43.5867667Z pipeline_to_execute.run(), pipeline_instrument)
> 2020-08-02T11:05:43.5868119Z apache_beam\pipeline.py:534: in run
> 2020-08-02T11:05:43.5868627Z return self.runner.run_pipeline(self, self._options)
> 2020-08-02T11:05:43.5869401Z apache_beam\runners\direct\direct_runner.py:119: in run_pipeline
> 2020-08-02T11:05:43.5869735Z return runner.run_pipeline(pipeline, options)
> 2020-08-02T11:05:43.5870201Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:176: in run_pipeline
> 2020-08-02T11:05:43.5870665Z pipeline.to_runner_api(default_environment=self._default_environment))
> 2020-08-02T11:05:43.5871520Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:186: in run_via_runner_api
> 2020-08-02T11:05:43.5871987Z return self.run_stages(stage_context, stages)
> 2020-08-02T11:05:43.5872612Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:344: in run_stages
> 2020-08-02T11:05:43.5872918Z bundle_context_manager,
> 2020-08-02T11:05:43.5873512Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:523: in _run_stage
> 2020-08-02T11:05:43.5873851Z bundle_manager)
> 2020-08-02T11:05:43.5874423Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:561: in _run_bundle
> 2020-08-02T11:05:43.5875072Z data_input, data_output, input_timers, expected_timer_output)
> 2020-08-02T11:05:43.5875740Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:945: in process_bundle
> 2020-08-02T11:05:43.5876099Z timer_inputs)):
> 2020-08-02T11:05:43.5877043Z c:\hostedtoolcache\windows\python\3.5.4\x64\lib\concurrent\futures\_base.py:556: in result_iterator
> 2020-08-02T11:05:43.5877296Z yield future.result()
> 2020-08-02T11:05:43.5878056Z c:\hostedtoolcache\windows\python\3.5.4\x64\lib\concurrent\futures\_base.py:405: in result
> 2020-08-02T11:05:43.5878412Z return self.__get_result()
> 2020-08-02T11:05:43.5879053Z c:\hostedtoolcache\windows\python\3.5.4\x64\lib\concurrent\futures\_base.py:357: in __get_result
> 2020-08-02T11:05:43.5879419Z raise self._exception
> 2020-08-02T11:05:43.5879879Z apache_beam\utils\thread_pool_executor.py:44: in run
> 2020-08-02T11:05:43.5880617Z self._future.set_result(self._fn(*self._fn_args, **self._fn_kwargs))
> 2020-08-02T11:05:43.5881245Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:941: in execute
> 2020-08-02T11:05:43.5881486Z dry_run)
> 2020-08-02T11:05:43.5882151Z apache_beam\runners\portability\fn_api_runner\fn_runner.py:841: in process_bundle
> 2020-08-02T11:05:43.5882648Z result_future = self._worker_handler.control_conn.push(process_bundle_req)
> 2020-08-02T11:05:43.5883269Z apache_beam\runners\portability\fn_api_runner\worker_handlers.py:352: in push
> 2020-08-02T11:05:43.5883663Z response = self.worker.do_instruction(request)
> 2020-08-02T11:05:43.5884260Z apache_beam\runners\worker\sdk_worker.py:483: in do_instruction
> 2020-08-02T11:05:43.5884688Z getattr(request, request_type), request.instruction_id)
> 2020-08-02T11:05:43.5885411Z apache_beam\runners\worker\sdk_worker.py:518: in process_bundle
> 2020-08-02T11:05:43.5885697Z bundle_processor.process_bundle(instruction_id))
> 2020-08-02T11:05:43.5886325Z apache_beam\runners\worker\bundle_processor.py:978: in process_bundle
> 2020-08-02T11:05:43.5886632Z element.data)
> 2020-08-02T11:05:43.5887166Z apache_beam\runners\worker\bundle_processor.py:218: in process_encoded
> 2020-08-02T11:05:43.5887466Z self.output(decoded_value)
> 2020-08-02T11:05:43.5888015Z apache_beam\runners\worker\operations.py:332: in output
> 2020-08-02T11:05:43.5888562Z cython.cast(Receiver, self.receivers[output_index]).receive(windowed_value)
> 2020-08-02T11:05:43.5889136Z apache_beam\runners\worker\operations.py:195: in receive
> 2020-08-02T11:05:43.5889440Z self.consumer.process(windowed_value)
> 2020-08-02T11:05:43.5890038Z apache_beam\runners\worker\operations.py:671: in process
> 2020-08-02T11:05:43.5890501Z delayed_application = self.dofn_runner.process(o)
> 2020-08-02T11:05:43.5890938Z apache_beam\runners\common.py:1215: in process
> 2020-08-02T11:05:43.5891219Z self._reraise_augmented(exn)
> 2020-08-02T11:05:43.5891754Z apache_beam\runners\common.py:1213: in process
> 2020-08-02T11:05:43.5892205Z return self.do_fn_invoker.invoke_process(windowed_value)
> 2020-08-02T11:05:43.5892742Z apache_beam\runners\common.py:570: in invoke_process
> 2020-08-02T11:05:43.5893286Z windowed_value, self.process_method(windowed_value.value))
> 2020-08-02T11:05:43.5893773Z apache_beam\runners\common.py:1371: in process_outputs
> 2020-08-02T11:05:43.5894086Z self.main_receivers.receive(windowed_value)
> 2020-08-02T11:05:43.5894677Z apache_beam\runners\worker\operations.py:195: in receive
> 2020-08-02T11:05:43.5895258Z self.consumer.process(windowed_value)
> 2020-08-02T11:05:43.5895850Z apache_beam\runners\worker\operations.py:671: in process
> 2020-08-02T11:05:43.5896279Z delayed_application = self.dofn_runner.process(o)
> 2020-08-02T11:05:43.5896853Z apache_beam\runners\common.py:1215: in process
> 2020-08-02T11:05:43.5897261Z self._reraise_augmented(exn)
> 2020-08-02T11:05:43.5897794Z apache_beam\runners\common.py:1213: in process
> 2020-08-02T11:05:43.5898242Z return self.do_fn_invoker.invoke_process(windowed_value)
> 2020-08-02T11:05:43.5898818Z apache_beam\runners\common.py:570: in invoke_process
> 2020-08-02T11:05:43.5899253Z windowed_value, self.process_method(windowed_value.value))
> 2020-08-02T11:05:43.5899693Z apache_beam\runners\common.py:1371: in process_outputs
> 2020-08-02T11:05:43.5900009Z self.main_receivers.receive(windowed_value)
> 2020-08-02T11:05:43.5900593Z apache_beam\runners\worker\operations.py:195: in receive
> 2020-08-02T11:05:43.5900907Z self.consumer.process(windowed_value)
> 2020-08-02T11:05:43.5901487Z apache_beam\runners\worker\operations.py:671: in process
> 2020-08-02T11:05:43.5901895Z delayed_application = self.dofn_runner.process(o)
> 2020-08-02T11:05:43.5902425Z apache_beam\runners\common.py:1215: in process
> 2020-08-02T11:05:43.5902747Z self._reraise_augmented(exn)
> 2020-08-02T11:05:43.5903273Z apache_beam\runners\common.py:1213: in process
> 2020-08-02T11:05:43.5903722Z return self.do_fn_invoker.invoke_process(windowed_value)
> 2020-08-02T11:05:43.5904244Z apache_beam\runners\common.py:570: in invoke_process
> 2020-08-02T11:05:43.5906019Z windowed_value, self.process_method(windowed_value.value))
> 2020-08-02T11:05:43.5907258Z apache_beam\runners\common.py:1371: in process_outputs
> 2020-08-02T11:05:43.5907654Z self.main_receivers.receive(windowed_value)
> 2020-08-02T11:05:43.5908042Z apache_beam\runners\worker\operations.py:195: in receive
> 2020-08-02T11:05:43.5908684Z self.consumer.process(windowed_value)
> 2020-08-02T11:05:43.5909344Z apache_beam\runners\worker\operations.py:671: in process
> 2020-08-02T11:05:43.5910421Z delayed_application = self.dofn_runner.process(o)
> 2020-08-02T11:05:43.5910866Z apache_beam\runners\common.py:1215: in process
> 2020-08-02T11:05:43.5911116Z self._reraise_augmented(exn)
> 2020-08-02T11:05:43.5911450Z apache_beam\runners\common.py:1213: in process
> 2020-08-02T11:05:43.5911759Z return self.do_fn_invoker.invoke_process(windowed_value)
> 2020-08-02T11:05:43.5912095Z apache_beam\runners\common.py:743: in invoke_process
> 2020-08-02T11:05:43.5912432Z windowed_value, additional_args, additional_kwargs)
> 2020-08-02T11:05:43.5913114Z apache_beam\runners\common.py:868: in _invoke_process_per_window
> 2020-08-02T11:05:43.5913534Z self.threadsafe_watermark_estimator)
> 2020-08-02T11:05:43.5914166Z apache_beam\runners\common.py:1371: in process_outputs
> 2020-08-02T11:05:43.5914896Z self.main_receivers.receive(windowed_value)
> 2020-08-02T11:05:43.5915860Z apache_beam\runners\worker\operations.py:195: in receive
> 2020-08-02T11:05:43.5916467Z self.consumer.process(windowed_value)
> 2020-08-02T11:05:43.5917277Z apache_beam\runners\worker\operations.py:671: in process
> 2020-08-02T11:05:43.5918044Z delayed_application = self.dofn_runner.process(o)
> 2020-08-02T11:05:43.5918715Z apache_beam\runners\common.py:1215: in process
> 2020-08-02T11:05:43.5919012Z self._reraise_augmented(exn)
> 2020-08-02T11:05:43.5919716Z apache_beam\runners\common.py:1213: in process
> 2020-08-02T11:05:43.5920223Z return self.do_fn_invoker.invoke_process(windowed_value)
> 2020-08-02T11:05:43.5920932Z apache_beam\runners\common.py:743: in invoke_process
> 2020-08-02T11:05:43.5922244Z windowed_value, additional_args, additional_kwargs)
> 2020-08-02T11:05:43.5922797Z apache_beam\runners\common.py:868: in _invoke_process_per_window
> 2020-08-02T11:05:43.5923112Z self.threadsafe_watermark_estimator)
> 2020-08-02T11:05:43.5923454Z apache_beam\runners\common.py:1371: in process_outputs
> 2020-08-02T11:05:43.5923791Z self.main_receivers.receive(windowed_value)
> 2020-08-02T11:05:43.5924916Z apache_beam\runners\worker\operations.py:195: in receive
> 2020-08-02T11:05:43.5925359Z self.consumer.process(windowed_value)
> 2020-08-02T11:05:43.5925741Z apache_beam\runners\worker\operations.py:671: in process
> 2020-08-02T11:05:43.5926284Z delayed_application = self.dofn_runner.process(o)
> 2020-08-02T11:05:43.5926952Z apache_beam\runners\common.py:1215: in process
> 2020-08-02T11:05:43.5927818Z self._reraise_augmented(exn)
> 2020-08-02T11:05:43.5928947Z apache_beam\runners\common.py:1294: in _reraise_augmented
> 2020-08-02T11:05:43.5929923Z raise_with_traceback(new_exn)
> 2020-08-02T11:05:43.5930261Z target\.tox\py35-win\lib\site-packages\future\utils\__init__.py:446: in raise_with_traceback
> 2020-08-02T11:05:43.5930538Z raise exc.with_traceback(traceback)
> 2020-08-02T11:05:43.5930828Z apache_beam\runners\common.py:1213: in process
> 2020-08-02T11:05:43.5931267Z return self.do_fn_invoker.invoke_process(windowed_value)
> 2020-08-02T11:05:43.5931563Z apache_beam\runners\common.py:743: in invoke_process
> 2020-08-02T11:05:43.5932030Z windowed_value, additional_args, additional_kwargs)
> 2020-08-02T11:05:43.5932856Z apache_beam\runners\common.py:867: in _invoke_process_per_window
> 2020-08-02T11:05:43.5933758Z self.process_method(*args_for_process),
> 2020-08-02T11:05:43.5934275Z apache_beam\io\iobase.py:1061: in process
> 2020-08-02T11:05:43.5935058Z self.writer = self.sink.open_writer(init_result, str(uuid.uuid4()))
> 2020-08-02T11:05:43.5935807Z apache_beam\options\value_provider.py:135: in _f
> 2020-08-02T11:05:43.5936345Z return fnc(self, *args, **kwargs)
> 2020-08-02T11:05:43.5937517Z apache_beam\io\filebasedsink.py:196: in open_writer
> 2020-08-02T11:05:43.5938203Z return FileBasedSinkWriter(self, writer_path)
> 2020-08-02T11:05:43.5938575Z apache_beam\io\filebasedsink.py:417: in __init__
> 2020-08-02T11:05:43.5939191Z self.temp_handle = self.sink.open(temp_shard_path)
> 2020-08-02T11:05:43.5939486Z apache_beam\io\textio.py:401: in open
> 2020-08-02T11:05:43.5939961Z file_handle = super(_TextSink, self).open(temp_path)
> 2020-08-02T11:05:43.5940778Z apache_beam\options\value_provider.py:135: in _f
> 2020-08-02T11:05:43.5941063Z return fnc(self, *args, **kwargs)
> 2020-08-02T11:05:43.5941626Z apache_beam\io\filebasedsink.py:138: in open
> 2020-08-02T11:05:43.5942248Z return FileSystems.create(temp_path, self.mime_type, self.compression_type)
> 2020-08-02T11:05:43.5942734Z apache_beam\io\filesystems.py:224: in create
> 2020-08-02T11:05:43.5943258Z return filesystem.create(path, mime_type, compression_type)
> 2020-08-02T11:05:43.5943943Z apache_beam\io\localfilesystem.py:168: in create
> 2020-08-02T11:05:43.5944260Z return self._path_open(path, 'wb', mime_type, compression_type)
> 2020-08-02T11:05:43.5946106Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> 2020-08-02T11:05:43.5946356Z
> 2020-08-02T11:05:43.5946637Z self = <apache_beam.io.localfilesystem.LocalFileSystem object at 0x0000017D1F14FEF0>
> 2020-08-02T11:05:43.5947908Z path = 'D:\\a\\beam\\beam\\sdks\\python\\target\\.tox\\py35-win\\tmp\\it-9as6naq51636906604248\\full\\beam-temp-47e439f812-16...8ecd4af11eabb14000d3a7b9d20\\130e2734-6324-4fb8-9383-4d46b2ece316.47e439f812-1636908653816-1636908654152-1636906604248'
> 2020-08-02T11:05:43.5948751Z mode = 'wb', mime_type = 'text/plain', compression_type = 'uncompressed'
> 2020-08-02T11:05:43.5949030Z
> 2020-08-02T11:05:43.5949268Z def _path_open(
> 2020-08-02T11:05:43.5949544Z self,
> 2020-08-02T11:05:43.5949775Z path,
> 2020-08-02T11:05:43.5950052Z mode,
> 2020-08-02T11:05:43.5950403Z mime_type='application/octet-stream',
> 2020-08-02T11:05:43.5950753Z compression_type=CompressionTypes.AUTO):
> 2020-08-02T11:05:43.5951011Z """Helper functions to open a file in the provided mode.
> 2020-08-02T11:05:43.5951286Z """
> 2020-08-02T11:05:43.5951520Z compression_type = FileSystem._get_compression_type(path, compression_type)
> 2020-08-02T11:05:43.5951798Z > raw_file = io.open(path, mode)
> 2020-08-02T11:05:43.5953920Z E RuntimeError: FileNotFoundError: [Errno 2] No such file or directory: 'D:\\a\\beam\\beam\\sdks\\python\\target\\.tox\\py35-win\\tmp\\it-9as6naq51636906604248\\full\\beam-temp-47e439f812-1636908653816-1636908654152-1636906604248-45f788ecd4af11eabb14000d3a7b9d20\\130e2734-6324-4fb8-9383-4d46b2ece316.47e439f812-1636908653816-1636908654152-1636906604248' [while running 'ref_AppliedPTransform_AppliedPTransform_1/_WriteCache_47e439f812-1636908653816-1636908654152-1636906604248/Write/Write/WriteImpl/WriteBundles']
--
This message was sent by Atlassian Jira
(v8.3.4#803005)