You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Brian Hulette <bh...@google.com> on 2021/06/21 17:30:41 UTC

[PROPOSAL] Add a "dsls-dataframe-api" component in jira

Hi all,
I'd like to propose that we add a new component in jira for the DataFarme
API [1]. I've just been using the label "dataframe-api" [2] to keep these
jiras organized, but I think it's time to graduate to a component, which
should be more discoverable for anyone else filing a bug or feature
request. I think we should use the name "dsls-dataframe-api" for the
component, to be consistent with SQL's "dsl-sql" component.

If there aren't any objections could a PMC member help with this?

Thanks!
Brian

PS If anyone is confused about the "DSL" designation I opted to document it
that way since it is analogous to SQL - it's possible to embed DataFrames
within a larger Python pipeline via DataframeTransform (similar to
SqlTransform), but it's also possible to describe a complete pipeline using
the DataFrame API because it includes its own IOs [3].

[1] https://beam.apache.org/documentation/dsls/dataframes/overview/
[2]
https://issues.apache.org/jira/issues/?jql=project=BEAM%20AND%20labels=dataframe-api
[3]
https://beam.apache.org/releases/pydoc/2.30.0/apache_beam.dataframe.io.html

Re: [PROPOSAL] Add a "dsls-dataframe-api" component in jira

Posted by Kenneth Knowles <ke...@apache.org>.
Done

On Mon, Jun 21, 2021 at 10:30 AM Brian Hulette <bh...@google.com> wrote:

> Hi all,
> I'd like to propose that we add a new component in jira for the DataFarme
> API [1]. I've just been using the label "dataframe-api" [2] to keep these
> jiras organized, but I think it's time to graduate to a component, which
> should be more discoverable for anyone else filing a bug or feature
> request. I think we should use the name "dsls-dataframe-api" for the
> component, to be consistent with SQL's "dsl-sql" component.
>
> If there aren't any objections could a PMC member help with this?
>
> Thanks!
> Brian
>
> PS If anyone is confused about the "DSL" designation I opted to document
> it that way since it is analogous to SQL - it's possible to embed
> DataFrames within a larger Python pipeline via DataframeTransform (similar
> to SqlTransform), but it's also possible to describe a complete pipeline
> using the DataFrame API because it includes its own IOs [3].
>
> [1] https://beam.apache.org/documentation/dsls/dataframes/overview/
> [2]
> https://issues.apache.org/jira/issues/?jql=project=BEAM%20AND%20labels=dataframe-api
> [3]
> https://beam.apache.org/releases/pydoc/2.30.0/apache_beam.dataframe.io.html
>