You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kamil Wasilewski (Jira)" <ji...@apache.org> on 2020/08/11 09:23:00 UTC
[jira] [Commented] (BEAM-9154) Move Chicago Taxi Example to Python
3
[ https://issues.apache.org/jira/browse/BEAM-9154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17175397#comment-17175397 ]
Kamil Wasilewski commented on BEAM-9154:
----------------------------------------
As fas as I remember, yes, it still affects us.
[~tvalentyn] Do you have contact with the TFX team? If so, could you ask them what's their opinion on how to approach the problem? I saw something called "Beam orchestrator", which is, I think, a new way of running chicago taxi (there's also a ticket: https://issues.apache.org/jira/browse/BEAM-8892). Should we take this into account?
> Move Chicago Taxi Example to Python 3
> -------------------------------------
>
> Key: BEAM-9154
> URL: https://issues.apache.org/jira/browse/BEAM-9154
> Project: Beam
> Issue Type: Improvement
> Components: testing
> Reporter: Kamil Wasilewski
> Assignee: Kamil Wasilewski
> Priority: P1
>
> The Chicago Taxi Example[1] should be moved to the latest version of Python supported by Beam (currently it's Python 3.7).
> At the moment, the following error occurs when running the benchmark on Python 3.7 (requires futher investigation):
> {code:java}
> Traceback (most recent call last):
> File "preprocess.py", line 259, in <module>
> main()
> File "preprocess.py", line 254, in main
> project=known_args.metric_reporting_project
> File "preprocess.py", line 155, in transform_data
> ('Analyze' >> tft_beam.AnalyzeDataset(preprocessing_fn)))
> File "/Users/kamilwasilewski/proj/beam/sdks/python/apache_beam/transforms/ptransform.py", line 987, in __ror__
> return self.transform.__ror__(pvalueish, self.label)
> File "/Users/kamilwasilewski/proj/beam/sdks/python/apache_beam/transforms/ptransform.py", line 547, in __ror__
> result = p.apply(self, pvalueish, label)
> File "/Users/kamilwasilewski/proj/beam/sdks/python/apache_beam/pipeline.py", line 532, in apply
> return self.apply(transform, pvalueish)
> File "/Users/kamilwasilewski/proj/beam/sdks/python/apache_beam/pipeline.py", line 573, in apply
> pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
> File "/Users/kamilwasilewski/proj/beam/sdks/python/apache_beam/runners/runner.py", line 193, in apply
> return m(transform, input, options)
> File "/Users/kamilwasilewski/proj/beam/sdks/python/apache_beam/runners/runner.py", line 223, in apply_PTransform
> return transform.expand(input)
> File "/Users/kamilwasilewski/proj/beam/build/gradleenv/2022703441/lib/python3.7/site-packages/tensorflow_transform/beam/impl.py", line 825, in expand
> input_metadata))
> File "/Users/kamilwasilewski/proj/beam/build/gradleenv/2022703441/lib/python3.7/site-packages/tensorflow_transform/beam/impl.py", line 716, in expand
> output_signature = self._preprocessing_fn(copied_inputs)
> File "preprocess.py", line 102, in preprocessing_fn
> _fill_in_missing(inputs[key]),
> KeyError: 'company'
> {code}
> [1] sdks/python/apache_beam/testing/benchmarks/chicago_taxi
--
This message was sent by Atlassian Jira
(v8.3.4#803005)