You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@airflow.apache.org by da...@gmail.com, da...@gmail.com on 2018/08/06 21:37:41 UTC

Re: Airflow's JS code (and dependencies) manageable via npm and webpack

Hi Verdan

Which version of Ace js was used? I couldn't track that

Cheers
David 

On 2018/07/09 10:45:00, Verdan Mahmood <ve...@gmail.com> wrote: 
> Hey Guys, 
> 
> In an effort to simplify the JS dependencies of Airflow
> 
> ,
> I've
>  introduce
> d
>  npm and webpack for the package management. For now, it only implements
> this in the www_rbac version of the web server.
> 
> 
> Pull Request: https://github.com/apache/incubator-airflow/pull/3572
> 
> The problem with the
> existing 
> frontend (
> JS
> ) code of Airflow is that most of the custom JS is written
> with
> in the html files, using the Flask's (Jinja) variables in that JS. The next
> step of this effort would be to extract that custom
> JS
>  code in separate JS files
> ,
> use the dependencies in those files using require or import
>  and introduce the JS automated test suite eventually. 
> (At the moment, I'm simply using the CopyWebPackPlugin to copy the required
> dependencies for use)
> .
> 
> There are also some dependencies which are directly modified in the codebase
>  or are outdated
> . I couldn't found the
>  correct
> npm versions of those libraries. (dagre-d3.js and gantt-chart-d3v2.js).
> Apparently dagre-d3.js that we are using is one of the gist or is very old
> version
>  not supported with webpack 4
> , while the gantt-chart-d3v2 has been modified according to Airflow's
> requirements
>  I believe
> .
>  Used the existing libraries for now. 
> 
> I am currently working in a separate branch to upgrade the DagreD3
> library, and updating the custom JS related to DagreD3 accordingly. 
> 
> This PR also introduces the pypi_push.sh
> <https://github.com/apache/incubator-airflow/pull/3572/files#diff-8fae684cdcc8cc8df2232c8df16f64cb>
> script that will generate all the JS statics before creating and uploading
> the package.
> 
> Please let me know if you guys have any questions or suggestions and I'd
> be happy to answer that. 
> 
> Best,
> *Verdan Mahmood*
> (+31) 655 576 560
>

Re: Airflow's JS code (and dependencies) manageable via npm and webpack

Posted by Bolke de Bruin <bd...@gmail.com>.

After the release of 1.10 we will start removing the old “www”.

B.

> On 7 Aug 2018, at 11:03, Verdan Mahmood <ve...@gmail.com> wrote:
> 
> Hi Dave,
> 
> AceJS is being used only in "www" version of webserver.
> "www_rbac" does not use AceJS, and we implemented npm and webpack only for
> www_rbac.
> 
> Best,
> *Verdan Mahmood*
> 
> On Mon, Aug 6, 2018 at 11:49 PM dave.allan.us@gmail.com <
> dave.allan.us@gmail.com> wrote:
> 
>> Hi Verdan
>> 
>> Which version of Ace js was used? I couldn't track that
>> 
>> Cheers
>> David
>> 
>> On 2018/07/09 10:45:00, Verdan Mahmood <ve...@gmail.com> wrote:
>>> Hey Guys, 
>>> 
>>> In an effort to simplify the JS dependencies of Airflow
>>> 
>>> ,
>>> I've
>>> introduce
>>> d
>>> npm and webpack for the package management. For now, it only implements
>>> this in the www_rbac version of the web server.
>>> 
>>> 
>>> Pull Request: https://github.com/apache/incubator-airflow/pull/3572
>>> 
>>> The problem with the
>>> existing 
>>> frontend (
>>> JS
>>> ) code of Airflow is that most of the custom JS is written
>>> with
>>> in the html files, using the Flask's (Jinja) variables in that JS. The
>> next
>>> step of this effort would be to extract that custom
>>> JS
>>> code in separate JS files
>>> ,
>>> use the dependencies in those files using require or import
>>>  and introduce the JS automated test suite eventually. 
>>> (At the moment, I'm simply using the CopyWebPackPlugin to copy the
>> required
>>> dependencies for use)
>>> .
>>> 
>>> There are also some dependencies which are directly modified in the
>> codebase
>>>  or are outdated
>>> . I couldn't found the
>>>  correct
>>> npm versions of those libraries. (dagre-d3.js and gantt-chart-d3v2.js).
>>> Apparently dagre-d3.js that we are using is one of the gist or is very
>> old
>>> version
>>>  not supported with webpack 4
>>> , while the gantt-chart-d3v2 has been modified according to Airflow's
>>> requirements
>>>  I believe
>>> .
>>>  Used the existing libraries for now. 
>>> 
>>> I am currently working in a separate branch to upgrade the DagreD3
>>> library, and updating the custom JS related to DagreD3 accordingly. 
>>> 
>>> This PR also introduces the pypi_push.sh
>>> <
>> https://github.com/apache/incubator-airflow/pull/3572/files#diff-8fae684cdcc8cc8df2232c8df16f64cb
>>> 
>>> script that will generate all the JS statics before creating and
>> uploading
>>> the package.
>>> 
>>> Please let me know if you guys have any questions or suggestions and I'd
>>> be happy to answer that. 
>>> 
>>> Best,
>>> *Verdan Mahmood*
>>> (+31) 655 576 560
>>> 
>>

Re: Airflow's JS code (and dependencies) manageable via npm and webpack

Posted by Verdan Mahmood <ve...@gmail.com>.

Hi Dave,

AceJS is being used only in "www" version of webserver.
"www_rbac" does not use AceJS, and we implemented npm and webpack only for
www_rbac.

Best,
*Verdan Mahmood*

On Mon, Aug 6, 2018 at 11:49 PM dave.allan.us@gmail.com <
dave.allan.us@gmail.com> wrote:

> Hi Verdan
>
> Which version of Ace js was used? I couldn't track that
>
> Cheers
> David
>
> On 2018/07/09 10:45:00, Verdan Mahmood <ve...@gmail.com> wrote:
> > Hey Guys, 
> >
> > In an effort to simplify the JS dependencies of Airflow
> > 
> > ,
> > I've
> >  introduce
> > d
> >  npm and webpack for the package management. For now, it only implements
> > this in the www_rbac version of the web server.
> > 
> >
> > Pull Request: https://github.com/apache/incubator-airflow/pull/3572
> >
> > The problem with the
> > existing 
> > frontend (
> > JS
> > ) code of Airflow is that most of the custom JS is written
> > with
> > in the html files, using the Flask's (Jinja) variables in that JS. The
> next
> > step of this effort would be to extract that custom
> > JS
> >  code in separate JS files
> > ,
> > use the dependencies in those files using require or import
> >  and introduce the JS automated test suite eventually. 
> > (At the moment, I'm simply using the CopyWebPackPlugin to copy the
> required
> > dependencies for use)
> > .
> >
> > There are also some dependencies which are directly modified in the
> codebase
> >  or are outdated
> > . I couldn't found the
> >  correct
> > npm versions of those libraries. (dagre-d3.js and gantt-chart-d3v2.js).
> > Apparently dagre-d3.js that we are using is one of the gist or is very
> old
> > version
> >  not supported with webpack 4
> > , while the gantt-chart-d3v2 has been modified according to Airflow's
> > requirements
> >  I believe
> > .
> >  Used the existing libraries for now. 
> >
> > I am currently working in a separate branch to upgrade the DagreD3
> > library, and updating the custom JS related to DagreD3 accordingly. 
> >
> > This PR also introduces the pypi_push.sh
> > <
> https://github.com/apache/incubator-airflow/pull/3572/files#diff-8fae684cdcc8cc8df2232c8df16f64cb
> >
> > script that will generate all the JS statics before creating and
> uploading
> > the package.
> > 
> > Please let me know if you guys have any questions or suggestions and I'd
> > be happy to answer that. 
> >
> > Best,
> > *Verdan Mahmood*
> > (+31) 655 576 560
> >
>