You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Aizhamal Nurmamat kyzy <ai...@apache.org> on 2019/06/03 21:23:28 UTC
[DISCUSS] Apache Airflow website requirements
Hello everyone,
As some of you may remember, earlier this year I have suggested adding a
landing page for Apache Airflow[1], also introduced a prototype we could
use [2], but we may want to have something professionally done for Airflow
:)
I have been working with the Google Cloud Composer team to allocate some
budget for revamping the Airflow website and it got approved, yay!
I would like to discuss the requirements for the website with the community
as I am not an expert in this matter. I started this document[3] where you
can add your thoughts and comments.
After we finalize the requirements, the vendors we will choose to work with
will:
- Design the website: the design approval will be done in the dev@ list by
the community
- Develop the website: testing and improvement suggestions will be done in
the dev@ list by the community
While we think of the requirements, I especially want to emphasize the work
we need to do around restructuring the Documentation page for Airflow
(navigations, meaningful divisions into related topics/subtopics, etc).
Other pages like Blog, Community, etc are more straightforward in my
opinion.
Let me know if you have any questions.
Thanks,
Aizhamal
[1]
https://lists.apache.org/thread.html/94e9f68ef8a3c42df0c1f78d3eff711c4761c1cf27fe2d3603ca961b@%3Cdev.airflow.apache.org%3E
[2]
https://lists.apache.org/thread.html/d12fa36f75e9a35cb70ba5d3a91003dac79fd92977c465c091e12b77@%3Cdev.airflow.apache.org%3E
[3]
https://docs.google.com/document/d/15nFhFlC__nm2_kRqCCT4a6fYRLnnIf6ATjDqlQACdKo/edit?usp=sharing
Re: [DISCUSS] Apache Airflow website requirements
Posted by Jarek Potiuk <Ja...@polidea.com>.
Current process uses sphinx to generate documentation out of the .rst files
and documentation, and uses some examples from the code.
Sphinx has some nice i18n tools that can extract translatable "messages"
already (for example here:
https://jareddillard.com/blog/documentation-internationalization-using-sphinx-and-zanata.html).
This includes skipping "code" and "links" automatically - and likely we can
tweak it to get pretty good set of texts ready to translate.
On Mon, Jul 29, 2019 at 5:48 PM Austin Bennett <wh...@gmail.com>
wrote:
> Had same in mind :-)
>
> Should have some time late this week to dig into specifics of Airflow
> process. And, need to explore how docs (source) organized, esp. whether
> the parts to be translated are separate in some specific way.
>
> On Sun, Jul 28, 2019 at 12:02 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
>> Such translation could be performed on every master merge/ and for each
>> official release. I do not expect a lot of overhead/time/cost. We have to
>> run documentation building/release anyway and then translation of such
>> generated documentation should happen right after.
>>
>> On Fri, Jul 26, 2019 at 6:29 PM Austin Bennett <
>> whatwouldaustindo@gmail.com>
>> wrote:
>>
>> > Seems a suitable and reasonable approach, esp given current
>> circumstances.
>> >
>> > I would imagine the desire would be to then update anything in other
>> > languages each time English updated? Or on a daily basis? Ongoing
>> would
>> > absolutely be important - in a predictable manner - rather than just
>> > running this once.
>> >
>> > It sounds interesting to get involved helping build a system that would
>> do
>> > this; and I'd be happy to explore how I might be able to help with this
>> > effort, should that be deemed a desirable direction.
>> >
>> >
>> > On Tue, Jul 16, 2019 at 11:24 PM Jarek Potiuk <Jarek.Potiuk@polidea.com
>> >
>> > wrote:
>> >
>> > > I understand the inclusivity need, and that's perfectly fine. However
>> I
>> > am
>> > > afraid community will not have a process (and people) to keep the
>> > > documentation up-to-date - it will almost immediately become obsolete
>> and
>> > > useless - we have like 10 commits coming in every day, most of them
>> > > including some documentation changes, and it's simply not feasible to
>> > keep
>> > > technical documentation updated and we have no-one to do it.
>> > >
>> > > However maybe another approach is be better. We could actually
>> automate a
>> > > process of generating translation using Google AI translation
>> > capabilities
>> > > and GCP's Machine-Learning based Translation API
>> > > <https://cloud.google.com/translate/docs/> and use the generated
>> > > documentation in English as source for even more inclusive
>> > > <https://cloud.google.com/translate/docs/languages> set of
>> > multi-language
>> > > translations of Airflow documentation. Of course it will not be
>> perfect
>> > > from linguistic point of view, but at least it will be accurate from
>> the
>> > > content point of view and we can automatically translate it to
>> multiple
>> > > languages, add a disclaimer: "This has been automatically translated
>> by
>> > > Google Translation API. For official documentation please refer to the
>> > > (original) English version" or similar.
>> > >
>> > > We just need some (very little - rather in 100s than 1000s of dollars
>> > > credits from google to run translation regularly over the API with
>> this
>> > > pricing: https://cloud.google.com/translate/pricing
>> > >
>> > > We could still keep the main "slow changing" documentation translated
>> by
>> > > Human Translators into multiple languages of course and it would be
>> great
>> > > to have it linguistically correct.
>> > >
>> > > I think that might be much more reasonable and totally feasible
>> approach.
>> > >
>> > > J.
>> > >
>> > >
>> > > On Wed, Jul 17, 2019 at 7:44 AM Aizhamal Nurmamat kyzy
>> > > <ai...@google.com.invalid> wrote:
>> > >
>> > > > Hi Jiajie, Kamil,
>> > > >
>> > > > My intention with a multilingual website is to make the project
>> > > accessible
>> > > > to users around the world, grow inclusive community, and drive the
>> > > adoption
>> > > > of technology faster among groups that speak and understand English
>> > less
>> > > > than well. As you said, Chinese community benefitted from Chinese
>> > > > documentation and were able to set things up faster because they
>> could
>> > > > understand it better. That is the outcome we want for other groups
>> as
>> > > well.
>> > > >
>> > > >
>> > > > However I understand that keeping docs up to date can be
>> challenging.
>> > > What
>> > > > I propose is to add disclaimers in the translations, eg. “this
>> > > > documentation was translated by community members. For official
>> > > > documentation please refer to the (original) English version” or
>> > > something
>> > > > similar in those terms.
>> > > >
>> > > > Translating documentation can also open up opportunities for many to
>> > > > contribute back to the project and be part of the community.
>> > > >
>> > > > Let me know what you all think.
>> > > >
>> > > > Thanks,
>> > > > Aizhamal
>> > > >
>> > > > On Tue, Jul 16, 2019 at 22:17 Jiajie Zhong <
>> zhongjiajie955@hotmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Thanks Kamil remind me in
>> > > > > https://github.com/apachecn/airflow-doc-zh/issues/65
>> > > > > I agree with you multi languages point, I think Airflow leading
>> page
>> > > > > should have multi languages
>> > > > > But for the documentation, I think it hard to up to date
>> > > > >
>> > > > > I’m one of contributor of
>> https://github.com/apachecn/airflow-doc-zh
>> > > and
>> > > > > this repo translate still in Airflow 1.10.2
>> > > > > Airflow documentation change too fast, and translation it’s hard
>> to
>> > > catch
>> > > > > up
>> > > > >
>> > > > > But, personally, Airflow documentation support multi language will
>> > > > > increasing Airflow users
>> > > > > especially not native English users. I join a Airflow Chinese
>> users
>> > > > > Tencent QQ group,
>> > > > > when them hear Airflow have Chinese translation their so happy
>> > because
>> > > > > their could make a quick
>> > > > > start of Airflow more easier.
>> > > > >
>> > > > > What I want to said it's If we could tolerate other language
>> > > translation
>> > > > > not up to date, add translation
>> > > > > in Airflow website is good for not native English users.
>> > > > >
>> > > > >
>> > > > > Best Wish
>> > > > > — Jiajie
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > > --
>> > >
>> > > Jarek Potiuk
>> > > Polidea <https://www.polidea.com/> | Principal Software Engineer
>> > >
>> > > M: +48 660 796 129 <+48660796129>
>> > > [image: Polidea] <https://www.polidea.com/>
>> > >
>> >
>>
>>
>> --
>>
>> Jarek Potiuk
>> Polidea <https://www.polidea.com/> | Principal Software Engineer
>>
>> M: +48 660 796 129 <+48660796129>
>> [image: Polidea] <https://www.polidea.com/>
>>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: [DISCUSS] Apache Airflow website requirements
Posted by Austin Bennett <wh...@gmail.com>.
Had same in mind :-)
Should have some time late this week to dig into specifics of Airflow
process. And, need to explore how docs (source) organized, esp. whether
the parts to be translated are separate in some specific way.
On Sun, Jul 28, 2019 at 12:02 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> Such translation could be performed on every master merge/ and for each
> official release. I do not expect a lot of overhead/time/cost. We have to
> run documentation building/release anyway and then translation of such
> generated documentation should happen right after.
>
> On Fri, Jul 26, 2019 at 6:29 PM Austin Bennett <
> whatwouldaustindo@gmail.com>
> wrote:
>
> > Seems a suitable and reasonable approach, esp given current
> circumstances.
> >
> > I would imagine the desire would be to then update anything in other
> > languages each time English updated? Or on a daily basis? Ongoing would
> > absolutely be important - in a predictable manner - rather than just
> > running this once.
> >
> > It sounds interesting to get involved helping build a system that would
> do
> > this; and I'd be happy to explore how I might be able to help with this
> > effort, should that be deemed a desirable direction.
> >
> >
> > On Tue, Jul 16, 2019 at 11:24 PM Jarek Potiuk <Ja...@polidea.com>
> > wrote:
> >
> > > I understand the inclusivity need, and that's perfectly fine. However I
> > am
> > > afraid community will not have a process (and people) to keep the
> > > documentation up-to-date - it will almost immediately become obsolete
> and
> > > useless - we have like 10 commits coming in every day, most of them
> > > including some documentation changes, and it's simply not feasible to
> > keep
> > > technical documentation updated and we have no-one to do it.
> > >
> > > However maybe another approach is be better. We could actually
> automate a
> > > process of generating translation using Google AI translation
> > capabilities
> > > and GCP's Machine-Learning based Translation API
> > > <https://cloud.google.com/translate/docs/> and use the generated
> > > documentation in English as source for even more inclusive
> > > <https://cloud.google.com/translate/docs/languages> set of
> > multi-language
> > > translations of Airflow documentation. Of course it will not be perfect
> > > from linguistic point of view, but at least it will be accurate from
> the
> > > content point of view and we can automatically translate it to multiple
> > > languages, add a disclaimer: "This has been automatically translated by
> > > Google Translation API. For official documentation please refer to the
> > > (original) English version" or similar.
> > >
> > > We just need some (very little - rather in 100s than 1000s of dollars
> > > credits from google to run translation regularly over the API with this
> > > pricing: https://cloud.google.com/translate/pricing
> > >
> > > We could still keep the main "slow changing" documentation translated
> by
> > > Human Translators into multiple languages of course and it would be
> great
> > > to have it linguistically correct.
> > >
> > > I think that might be much more reasonable and totally feasible
> approach.
> > >
> > > J.
> > >
> > >
> > > On Wed, Jul 17, 2019 at 7:44 AM Aizhamal Nurmamat kyzy
> > > <ai...@google.com.invalid> wrote:
> > >
> > > > Hi Jiajie, Kamil,
> > > >
> > > > My intention with a multilingual website is to make the project
> > > accessible
> > > > to users around the world, grow inclusive community, and drive the
> > > adoption
> > > > of technology faster among groups that speak and understand English
> > less
> > > > than well. As you said, Chinese community benefitted from Chinese
> > > > documentation and were able to set things up faster because they
> could
> > > > understand it better. That is the outcome we want for other groups as
> > > well.
> > > >
> > > >
> > > > However I understand that keeping docs up to date can be challenging.
> > > What
> > > > I propose is to add disclaimers in the translations, eg. “this
> > > > documentation was translated by community members. For official
> > > > documentation please refer to the (original) English version” or
> > > something
> > > > similar in those terms.
> > > >
> > > > Translating documentation can also open up opportunities for many to
> > > > contribute back to the project and be part of the community.
> > > >
> > > > Let me know what you all think.
> > > >
> > > > Thanks,
> > > > Aizhamal
> > > >
> > > > On Tue, Jul 16, 2019 at 22:17 Jiajie Zhong <
> zhongjiajie955@hotmail.com
> > >
> > > > wrote:
> > > >
> > > > > Thanks Kamil remind me in
> > > > > https://github.com/apachecn/airflow-doc-zh/issues/65
> > > > > I agree with you multi languages point, I think Airflow leading
> page
> > > > > should have multi languages
> > > > > But for the documentation, I think it hard to up to date
> > > > >
> > > > > I’m one of contributor of
> https://github.com/apachecn/airflow-doc-zh
> > > and
> > > > > this repo translate still in Airflow 1.10.2
> > > > > Airflow documentation change too fast, and translation it’s hard to
> > > catch
> > > > > up
> > > > >
> > > > > But, personally, Airflow documentation support multi language will
> > > > > increasing Airflow users
> > > > > especially not native English users. I join a Airflow Chinese users
> > > > > Tencent QQ group,
> > > > > when them hear Airflow have Chinese translation their so happy
> > because
> > > > > their could make a quick
> > > > > start of Airflow more easier.
> > > > >
> > > > > What I want to said it's If we could tolerate other language
> > > translation
> > > > > not up to date, add translation
> > > > > in Airflow website is good for not native English users.
> > > > >
> > > > >
> > > > > Best Wish
> > > > > — Jiajie
> > > > >
> > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Jarek Potiuk
> > > Polidea <https://www.polidea.com/> | Principal Software Engineer
> > >
> > > M: +48 660 796 129 <+48660796129>
> > > [image: Polidea] <https://www.polidea.com/>
> > >
> >
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
Re: [DISCUSS] Apache Airflow website requirements
Posted by Jarek Potiuk <Ja...@polidea.com>.
Such translation could be performed on every master merge/ and for each
official release. I do not expect a lot of overhead/time/cost. We have to
run documentation building/release anyway and then translation of such
generated documentation should happen right after.
On Fri, Jul 26, 2019 at 6:29 PM Austin Bennett <wh...@gmail.com>
wrote:
> Seems a suitable and reasonable approach, esp given current circumstances.
>
> I would imagine the desire would be to then update anything in other
> languages each time English updated? Or on a daily basis? Ongoing would
> absolutely be important - in a predictable manner - rather than just
> running this once.
>
> It sounds interesting to get involved helping build a system that would do
> this; and I'd be happy to explore how I might be able to help with this
> effort, should that be deemed a desirable direction.
>
>
> On Tue, Jul 16, 2019 at 11:24 PM Jarek Potiuk <Ja...@polidea.com>
> wrote:
>
> > I understand the inclusivity need, and that's perfectly fine. However I
> am
> > afraid community will not have a process (and people) to keep the
> > documentation up-to-date - it will almost immediately become obsolete and
> > useless - we have like 10 commits coming in every day, most of them
> > including some documentation changes, and it's simply not feasible to
> keep
> > technical documentation updated and we have no-one to do it.
> >
> > However maybe another approach is be better. We could actually automate a
> > process of generating translation using Google AI translation
> capabilities
> > and GCP's Machine-Learning based Translation API
> > <https://cloud.google.com/translate/docs/> and use the generated
> > documentation in English as source for even more inclusive
> > <https://cloud.google.com/translate/docs/languages> set of
> multi-language
> > translations of Airflow documentation. Of course it will not be perfect
> > from linguistic point of view, but at least it will be accurate from the
> > content point of view and we can automatically translate it to multiple
> > languages, add a disclaimer: "This has been automatically translated by
> > Google Translation API. For official documentation please refer to the
> > (original) English version" or similar.
> >
> > We just need some (very little - rather in 100s than 1000s of dollars
> > credits from google to run translation regularly over the API with this
> > pricing: https://cloud.google.com/translate/pricing
> >
> > We could still keep the main "slow changing" documentation translated by
> > Human Translators into multiple languages of course and it would be great
> > to have it linguistically correct.
> >
> > I think that might be much more reasonable and totally feasible approach.
> >
> > J.
> >
> >
> > On Wed, Jul 17, 2019 at 7:44 AM Aizhamal Nurmamat kyzy
> > <ai...@google.com.invalid> wrote:
> >
> > > Hi Jiajie, Kamil,
> > >
> > > My intention with a multilingual website is to make the project
> > accessible
> > > to users around the world, grow inclusive community, and drive the
> > adoption
> > > of technology faster among groups that speak and understand English
> less
> > > than well. As you said, Chinese community benefitted from Chinese
> > > documentation and were able to set things up faster because they could
> > > understand it better. That is the outcome we want for other groups as
> > well.
> > >
> > >
> > > However I understand that keeping docs up to date can be challenging.
> > What
> > > I propose is to add disclaimers in the translations, eg. “this
> > > documentation was translated by community members. For official
> > > documentation please refer to the (original) English version” or
> > something
> > > similar in those terms.
> > >
> > > Translating documentation can also open up opportunities for many to
> > > contribute back to the project and be part of the community.
> > >
> > > Let me know what you all think.
> > >
> > > Thanks,
> > > Aizhamal
> > >
> > > On Tue, Jul 16, 2019 at 22:17 Jiajie Zhong <zhongjiajie955@hotmail.com
> >
> > > wrote:
> > >
> > > > Thanks Kamil remind me in
> > > > https://github.com/apachecn/airflow-doc-zh/issues/65
> > > > I agree with you multi languages point, I think Airflow leading page
> > > > should have multi languages
> > > > But for the documentation, I think it hard to up to date
> > > >
> > > > I’m one of contributor of https://github.com/apachecn/airflow-doc-zh
> > and
> > > > this repo translate still in Airflow 1.10.2
> > > > Airflow documentation change too fast, and translation it’s hard to
> > catch
> > > > up
> > > >
> > > > But, personally, Airflow documentation support multi language will
> > > > increasing Airflow users
> > > > especially not native English users. I join a Airflow Chinese users
> > > > Tencent QQ group,
> > > > when them hear Airflow have Chinese translation their so happy
> because
> > > > their could make a quick
> > > > start of Airflow more easier.
> > > >
> > > > What I want to said it's If we could tolerate other language
> > translation
> > > > not up to date, add translation
> > > > in Airflow website is good for not native English users.
> > > >
> > > >
> > > > Best Wish
> > > > — Jiajie
> > > >
> > > >
> > > >
> > >
> >
> >
> > --
> >
> > Jarek Potiuk
> > Polidea <https://www.polidea.com/> | Principal Software Engineer
> >
> > M: +48 660 796 129 <+48660796129>
> > [image: Polidea] <https://www.polidea.com/>
> >
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: [DISCUSS] Apache Airflow website requirements
Posted by Austin Bennett <wh...@gmail.com>.
Seems a suitable and reasonable approach, esp given current circumstances.
I would imagine the desire would be to then update anything in other
languages each time English updated? Or on a daily basis? Ongoing would
absolutely be important - in a predictable manner - rather than just
running this once.
It sounds interesting to get involved helping build a system that would do
this; and I'd be happy to explore how I might be able to help with this
effort, should that be deemed a desirable direction.
On Tue, Jul 16, 2019 at 11:24 PM Jarek Potiuk <Ja...@polidea.com>
wrote:
> I understand the inclusivity need, and that's perfectly fine. However I am
> afraid community will not have a process (and people) to keep the
> documentation up-to-date - it will almost immediately become obsolete and
> useless - we have like 10 commits coming in every day, most of them
> including some documentation changes, and it's simply not feasible to keep
> technical documentation updated and we have no-one to do it.
>
> However maybe another approach is be better. We could actually automate a
> process of generating translation using Google AI translation capabilities
> and GCP's Machine-Learning based Translation API
> <https://cloud.google.com/translate/docs/> and use the generated
> documentation in English as source for even more inclusive
> <https://cloud.google.com/translate/docs/languages> set of multi-language
> translations of Airflow documentation. Of course it will not be perfect
> from linguistic point of view, but at least it will be accurate from the
> content point of view and we can automatically translate it to multiple
> languages, add a disclaimer: "This has been automatically translated by
> Google Translation API. For official documentation please refer to the
> (original) English version" or similar.
>
> We just need some (very little - rather in 100s than 1000s of dollars
> credits from google to run translation regularly over the API with this
> pricing: https://cloud.google.com/translate/pricing
>
> We could still keep the main "slow changing" documentation translated by
> Human Translators into multiple languages of course and it would be great
> to have it linguistically correct.
>
> I think that might be much more reasonable and totally feasible approach.
>
> J.
>
>
> On Wed, Jul 17, 2019 at 7:44 AM Aizhamal Nurmamat kyzy
> <ai...@google.com.invalid> wrote:
>
> > Hi Jiajie, Kamil,
> >
> > My intention with a multilingual website is to make the project
> accessible
> > to users around the world, grow inclusive community, and drive the
> adoption
> > of technology faster among groups that speak and understand English less
> > than well. As you said, Chinese community benefitted from Chinese
> > documentation and were able to set things up faster because they could
> > understand it better. That is the outcome we want for other groups as
> well.
> >
> >
> > However I understand that keeping docs up to date can be challenging.
> What
> > I propose is to add disclaimers in the translations, eg. “this
> > documentation was translated by community members. For official
> > documentation please refer to the (original) English version” or
> something
> > similar in those terms.
> >
> > Translating documentation can also open up opportunities for many to
> > contribute back to the project and be part of the community.
> >
> > Let me know what you all think.
> >
> > Thanks,
> > Aizhamal
> >
> > On Tue, Jul 16, 2019 at 22:17 Jiajie Zhong <zh...@hotmail.com>
> > wrote:
> >
> > > Thanks Kamil remind me in
> > > https://github.com/apachecn/airflow-doc-zh/issues/65
> > > I agree with you multi languages point, I think Airflow leading page
> > > should have multi languages
> > > But for the documentation, I think it hard to up to date
> > >
> > > I’m one of contributor of https://github.com/apachecn/airflow-doc-zh
> and
> > > this repo translate still in Airflow 1.10.2
> > > Airflow documentation change too fast, and translation it’s hard to
> catch
> > > up
> > >
> > > But, personally, Airflow documentation support multi language will
> > > increasing Airflow users
> > > especially not native English users. I join a Airflow Chinese users
> > > Tencent QQ group,
> > > when them hear Airflow have Chinese translation their so happy because
> > > their could make a quick
> > > start of Airflow more easier.
> > >
> > > What I want to said it's If we could tolerate other language
> translation
> > > not up to date, add translation
> > > in Airflow website is good for not native English users.
> > >
> > >
> > > Best Wish
> > > — Jiajie
> > >
> > >
> > >
> >
>
>
> --
>
> Jarek Potiuk
> Polidea <https://www.polidea.com/> | Principal Software Engineer
>
> M: +48 660 796 129 <+48660796129>
> [image: Polidea] <https://www.polidea.com/>
>
Re: [DISCUSS] Apache Airflow website requirements
Posted by Jarek Potiuk <Ja...@polidea.com>.
I understand the inclusivity need, and that's perfectly fine. However I am
afraid community will not have a process (and people) to keep the
documentation up-to-date - it will almost immediately become obsolete and
useless - we have like 10 commits coming in every day, most of them
including some documentation changes, and it's simply not feasible to keep
technical documentation updated and we have no-one to do it.
However maybe another approach is be better. We could actually automate a
process of generating translation using Google AI translation capabilities
and GCP's Machine-Learning based Translation API
<https://cloud.google.com/translate/docs/> and use the generated
documentation in English as source for even more inclusive
<https://cloud.google.com/translate/docs/languages> set of multi-language
translations of Airflow documentation. Of course it will not be perfect
from linguistic point of view, but at least it will be accurate from the
content point of view and we can automatically translate it to multiple
languages, add a disclaimer: "This has been automatically translated by
Google Translation API. For official documentation please refer to the
(original) English version" or similar.
We just need some (very little - rather in 100s than 1000s of dollars
credits from google to run translation regularly over the API with this
pricing: https://cloud.google.com/translate/pricing
We could still keep the main "slow changing" documentation translated by
Human Translators into multiple languages of course and it would be great
to have it linguistically correct.
I think that might be much more reasonable and totally feasible approach.
J.
On Wed, Jul 17, 2019 at 7:44 AM Aizhamal Nurmamat kyzy
<ai...@google.com.invalid> wrote:
> Hi Jiajie, Kamil,
>
> My intention with a multilingual website is to make the project accessible
> to users around the world, grow inclusive community, and drive the adoption
> of technology faster among groups that speak and understand English less
> than well. As you said, Chinese community benefitted from Chinese
> documentation and were able to set things up faster because they could
> understand it better. That is the outcome we want for other groups as well.
>
>
> However I understand that keeping docs up to date can be challenging. What
> I propose is to add disclaimers in the translations, eg. “this
> documentation was translated by community members. For official
> documentation please refer to the (original) English version” or something
> similar in those terms.
>
> Translating documentation can also open up opportunities for many to
> contribute back to the project and be part of the community.
>
> Let me know what you all think.
>
> Thanks,
> Aizhamal
>
> On Tue, Jul 16, 2019 at 22:17 Jiajie Zhong <zh...@hotmail.com>
> wrote:
>
> > Thanks Kamil remind me in
> > https://github.com/apachecn/airflow-doc-zh/issues/65
> > I agree with you multi languages point, I think Airflow leading page
> > should have multi languages
> > But for the documentation, I think it hard to up to date
> >
> > I’m one of contributor of https://github.com/apachecn/airflow-doc-zh and
> > this repo translate still in Airflow 1.10.2
> > Airflow documentation change too fast, and translation it’s hard to catch
> > up
> >
> > But, personally, Airflow documentation support multi language will
> > increasing Airflow users
> > especially not native English users. I join a Airflow Chinese users
> > Tencent QQ group,
> > when them hear Airflow have Chinese translation their so happy because
> > their could make a quick
> > start of Airflow more easier.
> >
> > What I want to said it's If we could tolerate other language translation
> > not up to date, add translation
> > in Airflow website is good for not native English users.
> >
> >
> > Best Wish
> > — Jiajie
> >
> >
> >
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: [DISCUSS] Apache Airflow website requirements
Posted by Aizhamal Nurmamat kyzy <ai...@google.com.INVALID>.
Hi Jiajie, Kamil,
My intention with a multilingual website is to make the project accessible
to users around the world, grow inclusive community, and drive the adoption
of technology faster among groups that speak and understand English less
than well. As you said, Chinese community benefitted from Chinese
documentation and were able to set things up faster because they could
understand it better. That is the outcome we want for other groups as well.
However I understand that keeping docs up to date can be challenging. What
I propose is to add disclaimers in the translations, eg. “this
documentation was translated by community members. For official
documentation please refer to the (original) English version” or something
similar in those terms.
Translating documentation can also open up opportunities for many to
contribute back to the project and be part of the community.
Let me know what you all think.
Thanks,
Aizhamal
On Tue, Jul 16, 2019 at 22:17 Jiajie Zhong <zh...@hotmail.com>
wrote:
> Thanks Kamil remind me in
> https://github.com/apachecn/airflow-doc-zh/issues/65
> I agree with you multi languages point, I think Airflow leading page
> should have multi languages
> But for the documentation, I think it hard to up to date
>
> I’m one of contributor of https://github.com/apachecn/airflow-doc-zh and
> this repo translate still in Airflow 1.10.2
> Airflow documentation change too fast, and translation it’s hard to catch
> up
>
> But, personally, Airflow documentation support multi language will
> increasing Airflow users
> especially not native English users. I join a Airflow Chinese users
> Tencent QQ group,
> when them hear Airflow have Chinese translation their so happy because
> their could make a quick
> start of Airflow more easier.
>
> What I want to said it's If we could tolerate other language translation
> not up to date, add translation
> in Airflow website is good for not native English users.
>
>
> Best Wish
> — Jiajie
>
>
>
Re: [DISCUSS] Apache Airflow website requirements
Posted by Jarek Potiuk <Ja...@polidea.com>.
I agree with everything Kamil said - we need to split the docs into:
- website/landing page (simple, static, slow-changing, multi-language,
managed by designated people)
- Tutorial/Documentation (fast changing, every contributor can change
and update it, automatically generated from the code of Airflow)
The problem with multi-language documentation/tutorial of fast-evolving
product is that you cannot ask every contribute to make translation. For
the tutorial/documentation, correctness. freshness are the most important -
so basically it has to change together with the code. Our code is in
English and everyone contributing must understand it, so English is the
only choice for such documentation.
J.
On Wed, Jul 17, 2019 at 7:17 AM Jiajie Zhong <zh...@hotmail.com>
wrote:
> Thanks Kamil remind me in
> https://github.com/apachecn/airflow-doc-zh/issues/65
> I agree with you multi languages point, I think Airflow leading page
> should have multi languages
> But for the documentation, I think it hard to up to date
>
> I’m one of contributor of https://github.com/apachecn/airflow-doc-zh and
> this repo translate still in Airflow 1.10.2
> Airflow documentation change too fast, and translation it’s hard to catch
> up
>
> But, personally, Airflow documentation support multi language will
> increasing Airflow users
> especially not native English users. I join a Airflow Chinese users
> Tencent QQ group,
> when them hear Airflow have Chinese translation their so happy because
> their could make a quick
> start of Airflow more easier.
>
> What I want to said it's If we could tolerate other language translation
> not up to date, add translation
> in Airflow website is good for not native English users.
>
>
> Best Wish
> — Jiajie
>
>
>
--
Jarek Potiuk
Polidea <https://www.polidea.com/> | Principal Software Engineer
M: +48 660 796 129 <+48660796129>
[image: Polidea] <https://www.polidea.com/>
Re: [DISCUSS] Apache Airflow website requirements
Posted by Jiajie Zhong <zh...@hotmail.com>.
Thanks Kamil remind me in https://github.com/apachecn/airflow-doc-zh/issues/65
I agree with you multi languages point, I think Airflow leading page should have multi languages
But for the documentation, I think it hard to up to date
I’m one of contributor of https://github.com/apachecn/airflow-doc-zh and this repo translate still in Airflow 1.10.2
Airflow documentation change too fast, and translation it’s hard to catch up
But, personally, Airflow documentation support multi language will increasing Airflow users
especially not native English users. I join a Airflow Chinese users Tencent QQ group,
when them hear Airflow have Chinese translation their so happy because their could make a quick
start of Airflow more easier.
What I want to said it's If we could tolerate other language translation not up to date, add translation
in Airflow website is good for not native English users.
Best Wish
— Jiajie
Re: [DISCUSS] Apache Airflow website requirements
Posted by Kamil Breguła <ka...@polidea.com>.
I read the document and left a few simple comments. I would like us to
discuss about three things.
1. Architecture from the technical side.
I'm not sure if it's about creating a Airflow's site or creating
documentation for Airflow. For me, these are two independent things.
Pytorch consist of multiple components:
* Site:
This is built using the Jekyll static page generator
https://pytorch.org/get-started/locally/
https://pytorch.org/features
https://pytorch.org/hub
https://pytorch.org/blog/
Repo: https://github.com/pytorch/pytorch.github.io
* Tutorial:
This is sphinx-based docs
https://pytorch.org/tutorials/
Repo: https://github.com/pytorch/tutorials
* Documentation:
This is sphinx-based docs
https://pytorch.org/docs/1.0.0/
https://pytorch.org/docs/0.2.0/
Repo: https://github.com/pytorch/pytorch/tree/master/docs
* Sphinx theme. It is used by documentation and tutorial. The site
does not use this
Repo: https://github.com/pytorch/pytorch_sphinx_theme
There are still a few other components, but they do not matter.
I would not want all things mixed together in the case of Airflow. If
we agree that we want to have a similar architecture to Pytorch, then
we need a static website ex. Jekyll and sphinx documentation. The
current documentation should be updated to be based on a new theme.
We also need a way to easily publish changes and documentation that
will describe how to make popular changes on the site. For example:
publication of documentation for a new version of the product.
In such architecture, the contractor should provide the following elements:
a) design files including icons. It should be accepted by the
community on dev list.
b) sphinx theme. It should be accepted by the community on Github as a PR.
c) static website. It should be accepted by the community on Github as a PR.
2. Integration with CI/CD.
The document "ReadTheDocs" is often used in the document. It is a
service that allows publishing documentation based on a master branch.
Unfortunately, it causes a lot of problems on our scale of source
code.
Currently ongoing and unresolved problem:
https://apache-airflow.slack.com/archives/CJ1LVREHX/p1561724137011200
A big plus is easy configuration and automatic content publishing.
Unfortunately, this is also a big restriction because we do not have
much control over the building process. I think it's worth thinking
about a different mechanism.
I spoke about it on the Slack channel - #documentation, so I will copy
my statement:
> Can we add an additional step in CI that will send documentation to GCS or other service? In the near future, we should have a some information about the credits in GCP. GCS is very cheap, so cutting out of these credits should not be a problem If we get credits then the configuration of sending documentation for each branch/PR should not be problematic. This is just a vision of the future of course. Building on RTD is becoming problematic and I think it is worth rethinking it.
> This is especially important if we realize that the documentation/the website will be intensively developed in the future.
> https://docs.google.com/document/d/15nFhFlC__nm2_kRqCCT4a6fYRLnnIf6ATjDqlQACdKo/edit#
> I do not know who will take care of this task ( ;-) ), but I think that he will have to take care of this problem. I did a similar task for another project last week. The documentation is available for viewing for each person and works more stable than ReadTheDocs.
3. Multilingual website
Is the whole documentation to be in several languages? I think it
will be very difficult to maintain the high quality of each version. I
think that the website can be in all languages, but the documentation
should be in English only.
https://airflow.apachecn.org/#/
The Chinese-speaking community has created documentation in their
language, but I'm afraid it is up to date.
Greetings,
Kamil Breguła
On Mon, Jun 3, 2019 at 11:24 PM Aizhamal Nurmamat kyzy
<ai...@apache.org> wrote:
>
> Hello everyone,
>
> As some of you may remember, earlier this year I have suggested adding a
> landing page for Apache Airflow[1], also introduced a prototype we could
> use [2], but we may want to have something professionally done for Airflow
> :)
>
> I have been working with the Google Cloud Composer team to allocate some
> budget for revamping the Airflow website and it got approved, yay!
>
> I would like to discuss the requirements for the website with the community
> as I am not an expert in this matter. I started this document[3] where you
> can add your thoughts and comments.
>
> After we finalize the requirements, the vendors we will choose to work with
> will:
> - Design the website: the design approval will be done in the dev@ list by
> the community
> - Develop the website: testing and improvement suggestions will be done in
> the dev@ list by the community
>
> While we think of the requirements, I especially want to emphasize the work
> we need to do around restructuring the Documentation page for Airflow
> (navigations, meaningful divisions into related topics/subtopics, etc).
> Other pages like Blog, Community, etc are more straightforward in my
> opinion.
>
> Let me know if you have any questions.
>
> Thanks,
> Aizhamal
>
>
>
> [1]
> https://lists.apache.org/thread.html/94e9f68ef8a3c42df0c1f78d3eff711c4761c1cf27fe2d3603ca961b@%3Cdev.airflow.apache.org%3E
> [2]
> https://lists.apache.org/thread.html/d12fa36f75e9a35cb70ba5d3a91003dac79fd92977c465c091e12b77@%3Cdev.airflow.apache.org%3E
> [3]
> https://docs.google.com/document/d/15nFhFlC__nm2_kRqCCT4a6fYRLnnIf6ATjDqlQACdKo/edit?usp=sharing