You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@airflow.apache.org by הילה ויזן <hi...@gmail.com> on 2016/06/13 12:20:06 UTC
airflow API using java code
Hi,
We write our project in java, and i am looking for a way to run tasks in
airflow from the code,
without using Runtime.exec(args) command.
Is it possible?
Thanks
Re: airflow API using java code
Posted by hi...@gmail.com.
As you suggested, I execute:
"airflow run <date> <task> <dag>" command from Java code.
(Using Runtime.exec).
I see the task logs in the logs directory, so there is no doubt that the job was executed.
But- I can't see the task execution on the webserver, and don't see the logs from the GUI.
Do you have any clue?!
ב-14 ביוני 2016, בשעה 7:05, Chris Riccomini <cr...@apache.org> כתב/ה:
> For this kind of thing, we're running a Java web service, and we have
> airflow call out to it using the SimpleHttpOperator.
>
> On Mon, Jun 13, 2016 at 1:15 PM, Lance Norskog <la...@gmail.com>
> wrote:
>
>> There is no HTTP API for these things- it's on the list. You will have to
>> use the command-line tools.
>>
>>
>>> On Mon, Jun 13, 2016 at 11:44 AM, <hi...@gmail.com> wrote:
>>>
>>> Hi Arthur,
>>> I meant that I want to call airflow from Java code (the second option).
>>> I want to run tasks or query tasks status (simple airflow commands).
>>> Thanks a lot!
>>>
>>> נשלח מה-iPhone שלי
>>>
>>> ב-13 ביוני 2016, בשעה 20:14, Arthur Wiedmer <
>> arthur.wiedmer@gmail.com>
>>> כתב/ה:
>>>
>>>> Hi,
>>>>
>>>> I am unsure what you mean:
>>>> - Do you mean that the code you have to execute is written in Java? (If
>>> so
>>>> you could either use a BashOperator or write a custom operator deriving
>>> it)
>>>> - Do you want to call Airflow from Java? If so I am not sure what your
>>> use
>>>> case is here, but Runtime.exec sounds like it would be the way to do
>>> this...
>>>>
>>>> If you provide more details I can try to answer further.
>>>>
>>>> Best,
>>>> Arthur
>>>>
>>>>> On Mon, Jun 13, 2016 at 5:20 AM, הילה ויזן <hi...@gmail.com> wrote:
>>>>>
>>>>> Hi,
>>>>> We write our project in java, and i am looking for a way to run tasks
>> in
>>>>> airflow from the code,
>>>>> without using Runtime.exec(args) command.
>>>>> Is it possible?
>>>>>
>>>>> Thanks
>>
>>
>>
>> --
>> Lance Norskog
>> lance.norskog@gmail.com
>> Redwood City, CA
>>
Re: airflow API using java code
Posted by Chris Riccomini <cr...@apache.org>.
For this kind of thing, we're running a Java web service, and we have
airflow call out to it using the SimpleHttpOperator.
On Mon, Jun 13, 2016 at 1:15 PM, Lance Norskog <la...@gmail.com>
wrote:
> There is no HTTP API for these things- it's on the list. You will have to
> use the command-line tools.
>
>
> On Mon, Jun 13, 2016 at 11:44 AM, <hi...@gmail.com> wrote:
>
> > Hi Arthur,
> > I meant that I want to call airflow from Java code (the second option).
> > I want to run tasks or query tasks status (simple airflow commands).
> > Thanks a lot!
> >
> > נשלח מה-iPhone שלי
> >
> > ב-13 ביוני 2016, בשעה 20:14, Arthur Wiedmer <
> arthur.wiedmer@gmail.com>
> > כתב/ה:
> >
> > > Hi,
> > >
> > > I am unsure what you mean:
> > > - Do you mean that the code you have to execute is written in Java? (If
> > so
> > > you could either use a BashOperator or write a custom operator deriving
> > it)
> > > - Do you want to call Airflow from Java? If so I am not sure what your
> > use
> > > case is here, but Runtime.exec sounds like it would be the way to do
> > this...
> > >
> > > If you provide more details I can try to answer further.
> > >
> > > Best,
> > > Arthur
> > >
> > >> On Mon, Jun 13, 2016 at 5:20 AM, הילה ויזן <hi...@gmail.com> wrote:
> > >>
> > >> Hi,
> > >> We write our project in java, and i am looking for a way to run tasks
> in
> > >> airflow from the code,
> > >> without using Runtime.exec(args) command.
> > >> Is it possible?
> > >>
> > >> Thanks
> > >>
> >
>
>
>
> --
> Lance Norskog
> lance.norskog@gmail.com
> Redwood City, CA
>
Re: airflow API using java code
Posted by Lance Norskog <la...@gmail.com>.
There is no HTTP API for these things- it's on the list. You will have to
use the command-line tools.
On Mon, Jun 13, 2016 at 11:44 AM, <hi...@gmail.com> wrote:
> Hi Arthur,
> I meant that I want to call airflow from Java code (the second option).
> I want to run tasks or query tasks status (simple airflow commands).
> Thanks a lot!
>
> נשלח מה-iPhone שלי
>
> ב-13 ביוני 2016, בשעה 20:14, Arthur Wiedmer <ar...@gmail.com>
> כתב/ה:
>
> > Hi,
> >
> > I am unsure what you mean:
> > - Do you mean that the code you have to execute is written in Java? (If
> so
> > you could either use a BashOperator or write a custom operator deriving
> it)
> > - Do you want to call Airflow from Java? If so I am not sure what your
> use
> > case is here, but Runtime.exec sounds like it would be the way to do
> this...
> >
> > If you provide more details I can try to answer further.
> >
> > Best,
> > Arthur
> >
> >> On Mon, Jun 13, 2016 at 5:20 AM, הילה ויזן <hi...@gmail.com> wrote:
> >>
> >> Hi,
> >> We write our project in java, and i am looking for a way to run tasks in
> >> airflow from the code,
> >> without using Runtime.exec(args) command.
> >> Is it possible?
> >>
> >> Thanks
> >>
>
--
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA
Re: airflow API using java code
Posted by hi...@gmail.com.
Hi Arthur,
I meant that I want to call airflow from Java code (the second option).
I want to run tasks or query tasks status (simple airflow commands).
Thanks a lot!
נשלח מה-iPhone שלי
ב-13 ביוני 2016, בשעה 20:14, Arthur Wiedmer <ar...@gmail.com> כתב/ה:
> Hi,
>
> I am unsure what you mean:
> - Do you mean that the code you have to execute is written in Java? (If so
> you could either use a BashOperator or write a custom operator deriving it)
> - Do you want to call Airflow from Java? If so I am not sure what your use
> case is here, but Runtime.exec sounds like it would be the way to do this...
>
> If you provide more details I can try to answer further.
>
> Best,
> Arthur
>
>> On Mon, Jun 13, 2016 at 5:20 AM, הילה ויזן <hi...@gmail.com> wrote:
>>
>> Hi,
>> We write our project in java, and i am looking for a way to run tasks in
>> airflow from the code,
>> without using Runtime.exec(args) command.
>> Is it possible?
>>
>> Thanks
>>
Re: airflow API using java code
Posted by Arthur Wiedmer <ar...@gmail.com>.
Hi,
I am unsure what you mean:
- Do you mean that the code you have to execute is written in Java? (If so
you could either use a BashOperator or write a custom operator deriving it)
- Do you want to call Airflow from Java? If so I am not sure what your use
case is here, but Runtime.exec sounds like it would be the way to do this...
If you provide more details I can try to answer further.
Best,
Arthur
On Mon, Jun 13, 2016 at 5:20 AM, הילה ויזן <hi...@gmail.com> wrote:
> Hi,
> We write our project in java, and i am looking for a way to run tasks in
> airflow from the code,
> without using Runtime.exec(args) command.
> Is it possible?
>
> Thanks
>