Is airflow a fit?

[ah...@ensofia.com]

Hi,

I am Ahlem Triki, software Engineer. Recently we started working with
Airflow and I would ask if you see Airflow as a fit to solve what we working
on, or is it an over kill. We are really in need for an airflow expert point
of view.

Here is a brief description of what  we working on:

*	Running health programs (10's to low 100's program) e.g., diabetic,
cardio, new born..  For different 100's clinics (each one customize their
own) including low 1000's of doctors
*	1000's of patients in each clinic, with a patient enrolled in one or
more of these programs
*	Each program has 

*	a time table (Time triggered) actions of sending SMS, notifications,
updating application (e.g., reminders of appointments or lab, setting calls,
sending and receiving surveys/ results AND
*	Event Triggered action, like receiving confirmation a mobile, or
email, SMS (lab results high or low, med inventory status.)
*	Each program runs for month or years or for ever (or just one single
day)

My confusion it seems Airflow is meant more for data pipelines i.e, in my
mind computational level, vs what I'd doing is more like an application
level, hence my question is AirFlow a fit? If so any guidance, if not any
recommendations

 

Best regards, 

Ahem



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Re: Is airflow a fit?

[James Meickle <jm...@quantopian.com.INVALID>]

We use Airflow for a mixture of traditional data pipeline work, and
orchestration tasks that need to express complex date and dependency logic.
The former is a bit better supported, but it's still a great tool for the
latter.

But it sounds like your format is "this task should happen whenever this
other event happens". Airflow is oriented around "this task should happen
once per day, but not until this other condition happens". Therefore, I
would suggest that you probably want a live/streaming architecture where
tasks can subscribe to events.

On Thu, Oct 18, 2018 at 3:14 PM <ah...@ensofia.com> wrote:

> Hi,
>
> I am Ahlem Triki, software Engineer. Recently we started working with
> Airflow and I would ask if you see Airflow as a fit to solve what we
> working
> on, or is it an over kill. We are really in need for an airflow expert
> point
> of view.
>
> Here is a brief description of what  we working on:
>
> *       Running health programs (10's to low 100's program) e.g., diabetic,
> cardio, new born..  For different 100's clinics (each one customize their
> own) including low 1000's of doctors
> *       1000's of patients in each clinic, with a patient enrolled in one
> or
> more of these programs
> *       Each program has
>
> *       a time table (Time triggered) actions of sending SMS,
> notifications,
> updating application (e.g., reminders of appointments or lab, setting
> calls,
> sending and receiving surveys/ results AND
> *       Event Triggered action, like receiving confirmation a mobile, or
> email, SMS (lab results high or low, med inventory status.)
> *       Each program runs for month or years or for ever (or just one
> single
> day)
>
> My confusion it seems Airflow is meant more for data pipelines i.e, in my
> mind computational level, vs what I'd doing is more like an application
> level, hence my question is AirFlow a fit? If so any guidance, if not any
> recommendations
>
>
>
> Best regards,
>
> Ahem
>
>
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://www.avast.com/antivirus
>