You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by Maxime Beauchemin <ma...@gmail.com> on 2018/08/29 19:15:17 UTC
Missing operators in the docs
Hey committers,
I noticed that some of the operators are missing from the API reference
part of the docs (HiveOperator for instance). I'm guessing a committer
generated / pushed the docs with some libs missing and that the operators
depending on those missing libs got skipped.
We may have to improve the doc-generation wiki page or make a bulletproof
shell script that ensures all libs are installed prior to generating the
docs.
Max
Re: Missing operators in the docs
Posted by Maxime Beauchemin <ma...@gmail.com>.
Nice, thanks for fixing this!
On Wed, Aug 29, 2018 at 1:53 PM Kaxil Naik <ka...@gmail.com> wrote:
> I have fixed the issue on https://airflow.apache.org/ , added a comment on
> confluence as well. Will try to fix ReadTheDocs environment (Have opened a
> Jira for it) as it can't install all the dependencies that depend on C
> Modules (
>
> https://read-the-docs.readthedocs.io/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
> )
>
> Regards,
> Kaxil
>
> On Wed, Aug 29, 2018 at 9:15 PM Kaxil Naik <ka...@gmail.com> wrote:
>
> > Will have a look and resolve it.
> >
> > On Wed, Aug 29, 2018 at 8:25 PM Maxime Beauchemin <
> > maximebeauchemin@gmail.com> wrote:
> >
> >> Looks like both.
> >>
> >> On Wed, Aug 29, 2018 at 12:18 PM Kaxil Naik <ka...@gmail.com>
> wrote:
> >>
> >> > Hi Max,
> >> >
> >> > Did you see that on readthedocs or airflow.apache one?
> >> >
> >> > On Wed, 29 Aug 2018, 20:15 Maxime Beauchemin, <
> >> maximebeauchemin@gmail.com>
> >> > wrote:
> >> >
> >> > > Hey committers,
> >> > >
> >> > > I noticed that some of the operators are missing from the API
> >> reference
> >> > > part of the docs (HiveOperator for instance). I'm guessing a
> committer
> >> > > generated / pushed the docs with some libs missing and that the
> >> operators
> >> > > depending on those missing libs got skipped.
> >> > >
> >> > > We may have to improve the doc-generation wiki page or make a
> >> bulletproof
> >> > > shell script that ensures all libs are installed prior to generating
> >> the
> >> > > docs.
> >> > >
> >> > > Max
> >> > >
> >> >
> >>
> >
> >
> > --
> > *Kaxil Naik*
> > *Big Data Consultant *@ *Data Reply UK*
> > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> > Developer
> > *Phone: *+44 (0) 74820 88992
> > *LinkedIn*: https://www.linkedin.com/in/kaxil
> >
>
>
> --
> *Kaxil Naik*
> *Big Data Consultant *@ *Data Reply UK*
> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> Developer
> *Phone: *+44 (0) 74820 88992
> *LinkedIn*: https://www.linkedin.com/in/kaxil
>
Re: Missing operators in the docs
Posted by Kaxil Naik <ka...@gmail.com>.
@ashb Yes, we are mocking out some of the Google Cloud modules in
docs/conf.py .
On Thu, 30 Aug 2018, 11:20 Ash Berlin-Taylor, <as...@apache.org> wrote:
> There's a setting available for sphinx projects where imports/modules can
> be mocked - it might be worth exploring
> http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html?highlight=autodoc_mock_imports#confval-autodoc_mock_imports
> <
> http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html?highlight=autodoc_mock_imports#confval-autodoc_mock_imports>
> for some of the harder-to-install modules?
>
> -ash
>
> > On 29 Aug 2018, at 21:53, Kaxil Naik <ka...@gmail.com> wrote:
> >
> > I have fixed the issue on https://airflow.apache.org/ , added a comment
> on
> > confluence as well. Will try to fix ReadTheDocs environment (Have opened
> a
> > Jira for it) as it can't install all the dependencies that depend on C
> > Modules (
> >
> https://read-the-docs.readthedocs.io/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
> > )
> >
> > Regards,
> > Kaxil
> >
> > On Wed, Aug 29, 2018 at 9:15 PM Kaxil Naik <ka...@gmail.com> wrote:
> >
> >> Will have a look and resolve it.
> >>
> >> On Wed, Aug 29, 2018 at 8:25 PM Maxime Beauchemin <
> >> maximebeauchemin@gmail.com> wrote:
> >>
> >>> Looks like both.
> >>>
> >>> On Wed, Aug 29, 2018 at 12:18 PM Kaxil Naik <ka...@gmail.com>
> wrote:
> >>>
> >>>> Hi Max,
> >>>>
> >>>> Did you see that on readthedocs or airflow.apache one?
> >>>>
> >>>> On Wed, 29 Aug 2018, 20:15 Maxime Beauchemin, <
> >>> maximebeauchemin@gmail.com>
> >>>> wrote:
> >>>>
> >>>>> Hey committers,
> >>>>>
> >>>>> I noticed that some of the operators are missing from the API
> >>> reference
> >>>>> part of the docs (HiveOperator for instance). I'm guessing a
> committer
> >>>>> generated / pushed the docs with some libs missing and that the
> >>> operators
> >>>>> depending on those missing libs got skipped.
> >>>>>
> >>>>> We may have to improve the doc-generation wiki page or make a
> >>> bulletproof
> >>>>> shell script that ensures all libs are installed prior to generating
> >>> the
> >>>>> docs.
> >>>>>
> >>>>> Max
> >>>>>
> >>>>
> >>>
> >>
> >>
> >> --
> >> *Kaxil Naik*
> >> *Big Data Consultant *@ *Data Reply UK*
> >> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark &
> Neo4j
> >> Developer
> >> *Phone: *+44 (0) 74820 88992
> >> *LinkedIn*: https://www.linkedin.com/in/kaxil
> >>
> >
> >
> > --
> > *Kaxil Naik*
> > *Big Data Consultant *@ *Data Reply UK*
> > *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> > Developer
> > *Phone: *+44 (0) 74820 88992
> > *LinkedIn*: https://www.linkedin.com/in/kaxil
>
>
Re: Missing operators in the docs
Posted by Ash Berlin-Taylor <as...@apache.org>.
There's a setting available for sphinx projects where imports/modules can be mocked - it might be worth exploring http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html?highlight=autodoc_mock_imports#confval-autodoc_mock_imports <http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html?highlight=autodoc_mock_imports#confval-autodoc_mock_imports> for some of the harder-to-install modules?
-ash
> On 29 Aug 2018, at 21:53, Kaxil Naik <ka...@gmail.com> wrote:
>
> I have fixed the issue on https://airflow.apache.org/ , added a comment on
> confluence as well. Will try to fix ReadTheDocs environment (Have opened a
> Jira for it) as it can't install all the dependencies that depend on C
> Modules (
> https://read-the-docs.readthedocs.io/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
> )
>
> Regards,
> Kaxil
>
> On Wed, Aug 29, 2018 at 9:15 PM Kaxil Naik <ka...@gmail.com> wrote:
>
>> Will have a look and resolve it.
>>
>> On Wed, Aug 29, 2018 at 8:25 PM Maxime Beauchemin <
>> maximebeauchemin@gmail.com> wrote:
>>
>>> Looks like both.
>>>
>>> On Wed, Aug 29, 2018 at 12:18 PM Kaxil Naik <ka...@gmail.com> wrote:
>>>
>>>> Hi Max,
>>>>
>>>> Did you see that on readthedocs or airflow.apache one?
>>>>
>>>> On Wed, 29 Aug 2018, 20:15 Maxime Beauchemin, <
>>> maximebeauchemin@gmail.com>
>>>> wrote:
>>>>
>>>>> Hey committers,
>>>>>
>>>>> I noticed that some of the operators are missing from the API
>>> reference
>>>>> part of the docs (HiveOperator for instance). I'm guessing a committer
>>>>> generated / pushed the docs with some libs missing and that the
>>> operators
>>>>> depending on those missing libs got skipped.
>>>>>
>>>>> We may have to improve the doc-generation wiki page or make a
>>> bulletproof
>>>>> shell script that ensures all libs are installed prior to generating
>>> the
>>>>> docs.
>>>>>
>>>>> Max
>>>>>
>>>>
>>>
>>
>>
>> --
>> *Kaxil Naik*
>> *Big Data Consultant *@ *Data Reply UK*
>> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
>> Developer
>> *Phone: *+44 (0) 74820 88992
>> *LinkedIn*: https://www.linkedin.com/in/kaxil
>>
>
>
> --
> *Kaxil Naik*
> *Big Data Consultant *@ *Data Reply UK*
> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> Developer
> *Phone: *+44 (0) 74820 88992
> *LinkedIn*: https://www.linkedin.com/in/kaxil
Re: Missing operators in the docs
Posted by Kaxil Naik <ka...@gmail.com>.
I have fixed the issue on https://airflow.apache.org/ , added a comment on
confluence as well. Will try to fix ReadTheDocs environment (Have opened a
Jira for it) as it can't install all the dependencies that depend on C
Modules (
https://read-the-docs.readthedocs.io/en/latest/faq.html#i-get-import-errors-on-libraries-that-depend-on-c-modules
)
Regards,
Kaxil
On Wed, Aug 29, 2018 at 9:15 PM Kaxil Naik <ka...@gmail.com> wrote:
> Will have a look and resolve it.
>
> On Wed, Aug 29, 2018 at 8:25 PM Maxime Beauchemin <
> maximebeauchemin@gmail.com> wrote:
>
>> Looks like both.
>>
>> On Wed, Aug 29, 2018 at 12:18 PM Kaxil Naik <ka...@gmail.com> wrote:
>>
>> > Hi Max,
>> >
>> > Did you see that on readthedocs or airflow.apache one?
>> >
>> > On Wed, 29 Aug 2018, 20:15 Maxime Beauchemin, <
>> maximebeauchemin@gmail.com>
>> > wrote:
>> >
>> > > Hey committers,
>> > >
>> > > I noticed that some of the operators are missing from the API
>> reference
>> > > part of the docs (HiveOperator for instance). I'm guessing a committer
>> > > generated / pushed the docs with some libs missing and that the
>> operators
>> > > depending on those missing libs got skipped.
>> > >
>> > > We may have to improve the doc-generation wiki page or make a
>> bulletproof
>> > > shell script that ensures all libs are installed prior to generating
>> the
>> > > docs.
>> > >
>> > > Max
>> > >
>> >
>>
>
>
> --
> *Kaxil Naik*
> *Big Data Consultant *@ *Data Reply UK*
> *Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
> Developer
> *Phone: *+44 (0) 74820 88992
> *LinkedIn*: https://www.linkedin.com/in/kaxil
>
--
*Kaxil Naik*
*Big Data Consultant *@ *Data Reply UK*
*Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
Developer
*Phone: *+44 (0) 74820 88992
*LinkedIn*: https://www.linkedin.com/in/kaxil
Re: Missing operators in the docs
Posted by Kaxil Naik <ka...@gmail.com>.
Will have a look and resolve it.
On Wed, Aug 29, 2018 at 8:25 PM Maxime Beauchemin <
maximebeauchemin@gmail.com> wrote:
> Looks like both.
>
> On Wed, Aug 29, 2018 at 12:18 PM Kaxil Naik <ka...@gmail.com> wrote:
>
> > Hi Max,
> >
> > Did you see that on readthedocs or airflow.apache one?
> >
> > On Wed, 29 Aug 2018, 20:15 Maxime Beauchemin, <
> maximebeauchemin@gmail.com>
> > wrote:
> >
> > > Hey committers,
> > >
> > > I noticed that some of the operators are missing from the API reference
> > > part of the docs (HiveOperator for instance). I'm guessing a committer
> > > generated / pushed the docs with some libs missing and that the
> operators
> > > depending on those missing libs got skipped.
> > >
> > > We may have to improve the doc-generation wiki page or make a
> bulletproof
> > > shell script that ensures all libs are installed prior to generating
> the
> > > docs.
> > >
> > > Max
> > >
> >
>
--
*Kaxil Naik*
*Big Data Consultant *@ *Data Reply UK*
*Certified *Google Cloud Data Engineer | *Certified* Apache Spark & Neo4j
Developer
*Phone: *+44 (0) 74820 88992
*LinkedIn*: https://www.linkedin.com/in/kaxil
Re: Missing operators in the docs
Posted by Maxime Beauchemin <ma...@gmail.com>.
Looks like both.
On Wed, Aug 29, 2018 at 12:18 PM Kaxil Naik <ka...@gmail.com> wrote:
> Hi Max,
>
> Did you see that on readthedocs or airflow.apache one?
>
> On Wed, 29 Aug 2018, 20:15 Maxime Beauchemin, <ma...@gmail.com>
> wrote:
>
> > Hey committers,
> >
> > I noticed that some of the operators are missing from the API reference
> > part of the docs (HiveOperator for instance). I'm guessing a committer
> > generated / pushed the docs with some libs missing and that the operators
> > depending on those missing libs got skipped.
> >
> > We may have to improve the doc-generation wiki page or make a bulletproof
> > shell script that ensures all libs are installed prior to generating the
> > docs.
> >
> > Max
> >
>
Re: Missing operators in the docs
Posted by Kaxil Naik <ka...@gmail.com>.
Hi Max,
Did you see that on readthedocs or airflow.apache one?
On Wed, 29 Aug 2018, 20:15 Maxime Beauchemin, <ma...@gmail.com>
wrote:
> Hey committers,
>
> I noticed that some of the operators are missing from the API reference
> part of the docs (HiveOperator for instance). I'm guessing a committer
> generated / pushed the docs with some libs missing and that the operators
> depending on those missing libs got skipped.
>
> We may have to improve the doc-generation wiki page or make a bulletproof
> shell script that ensures all libs are installed prior to generating the
> docs.
>
> Max
>