You are viewing a plain text version of this content. The canonical link for it is here.
Posted to gsoc@community.apache.org by "Pablo Estrada (Jira)" <ji...@apache.org> on 2026/02/01 20:12:00 UTC
[jira] [Updated] (GSOC-311) A learning path to using accelerators with Beam
[ https://issues.apache.org/jira/browse/GSOC-311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pablo Estrada updated GSOC-311:
-------------------------------
Description:
The Beam project has a few examples where hardware accelerators can be used to run models. See [https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/dataflow_tpu_examples.ipynb]
This project is to improve on the available set of examples by building starter examples that allow a user to write code that slowly builds up to using these hardware accelerators. The idea would be:
was:
This project consists in a series of tasks that build a sort of 'infra platform' for Beam. Some tasks include:
* Automated cleaning of infrastructure: [[Task]: Build a cleaner for assets in the GCP test environment #33644|https://github.com/apache/beam/issues/33644]
* Implement Infra-as-code for Beam infrastructure
** Implement access permissions using IaC: [[Task]: Build a cleaner for assets in the GCP test environment #33644|https://github.com/apache/beam/issues/33644]
** Implement drift detection for IaC resources for Beam
** Implement 'best-practice' key management for Beam (i.e. force key rotation for service account keys, and store in secret manager secrets)
A quality proposal will include a series of features beyond the ones listed above. Some ideas:
* Detection of policy breakages, and nagging to fix
* Security detections based on cloud logging
* others?
> A learning path to using accelerators with Beam
> -----------------------------------------------
>
> Key: GSOC-311
> URL: https://issues.apache.org/jira/browse/GSOC-311
> Project: Comdev GSOC
> Issue Type: Task
> Reporter: Pablo Estrada
> Priority: Major
> Labels: Beam, gsoc, gsoc2026, mentor
>
> The Beam project has a few examples where hardware accelerators can be used to run models. See [https://github.com/apache/beam/blob/master/examples/notebooks/beam-ml/dataflow_tpu_examples.ipynb]
>
> This project is to improve on the available set of examples by building starter examples that allow a user to write code that slowly builds up to using these hardware accelerators. The idea would be:
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: gsoc-unsubscribe@community.apache.org
For additional commands, e-mail: gsoc-help@community.apache.org