You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/12/13 08:22:07 UTC
[GitHub] [airflow] rikirolly opened a new issue, #28329: DAG start before data_interval_end
rikirolly opened a new issue, #28329:
URL: https://github.com/apache/airflow/issues/28329
### Apache Airflow version
Other Airflow 2 version (please specify below)
### What happened
The DAG start 1 hour before the data_interval_end.
The schedule is set to be triggered every 2 hours:
`schedule='0 */2 * * *'`
This is the screen shot showing the DAG started at 2022-12-12, 12:00:01 UTC when the Data interval end is 2022-12-12, 13:00:00 UTC.
### What you think should happen instead
The DAG should wait till data_interval_end datetime.
### How to reproduce
_No response_
### Operating System
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==6.0.0
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.4.0
apache-airflow-providers-common-sql==1.3.0
apache-airflow-providers-docker==3.2.0
apache-airflow-providers-elasticsearch==4.2.1
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-google==8.4.0
apache-airflow-providers-grpc==3.0.0
apache-airflow-providers-hashicorp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-microsoft-azure==4.3.0
apache-airflow-providers-microsoft-mssql==3.3.0
apache-airflow-providers-mysql==3.2.1
apache-airflow-providers-odbc==3.1.2
apache-airflow-providers-postgres==5.2.2
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-samba==4.1.0
apache-airflow-providers-sendgrid==3.0.0
apache-airflow-providers-sftp==4.1.0
apache-airflow-providers-slack==6.0.0
apache-airflow-providers-sqlite==3.2.1
apache-airflow-providers-ssh==3.2.0
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Deployed with Official Helm chart on Kubernetes v1.25.0.
### Anything else
Running on VirtualBox nodes deployed via Vagrant.
The timezone on the nodes is UTC even if the server is Europe/Rome timezone.
Anyway on the nodes the Time is correcly set to the corresponding UTC time.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1354474188
Could you provide an example DAG and check what timezone Airflow is using? (`[core] default_timezone` in `airflow.cfg`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAG start before data_interval_end [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-2002060705
MSSQL support is dropped finally in Airflow 2.9.0, I'd not expect a fix for this issue. Therefore closing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1347922106
> Is this a one-time incident, or does this happen for all runs to this DAG?
This happens regularly for all runs of this DAG and other DAGs with the same schedule.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1354710193
> Could you provide an example DAG and check what timezone Airflow is using? (`[core] default_timezone` in `airflow.cfg`)
```
@dag(
schedule='0 */2 * * *',
start_date=pendulum.datetime(2022, 12, 7, 10, 00, tz="Europe/Rome"),
catchup=True,
max_active_runs=1,
tags=['riki'],
)
def neorouter_v08():
@task()
def load_neorouter_log():
context = get_current_context()
ti = context["ti"]
print(f"data_interval_start: {ti.get_dagrun().data_interval_start}")
print(f"data_interval_end: {ti.get_dagrun().data_interval_end}")
load_neorouter_log()
dag_neorouter_v07 = neorouter_v08()
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by "rikirolly (via GitHub)" <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1486448017
I started a new airflow cluster based on a Postgres database and the problem is not present.
The original cluster is using a Microsoft SQL database, it seems the issue is related to this.
The Microsoft SQL server is in CET timezone, I checked timestamps in different airflow tables and seems everything is in UTC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #28329: DAG start before data_interval_end
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1488112822
Took some effort to find but I think this is the issue #21171
I’m not sure how best to implement this without littering `if mssql:` all over the place, any help from someone who better knows how SQLAlchemy works would be appreciated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #28329: DAG start before data_interval_end
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1486550857
> Microsoft SQL database
Ahhhh I think I seem to recall there’s an issue with SQL Server treats `NOW()` differently or something. That could be the issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by "rikirolly (via GitHub)" <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1429715957
@uranusjr
I have updated to airflow 2.5.1 and the problem persist.
Have you found something?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by "rikirolly (via GitHub)" <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1486890651
> > Microsoft SQL database
>
> Ahhhh I think I seem to recall there’s an issue with SQL Server treats `NOW()` differently or something. That could be the issue.
It seems Microsoft SQL require the application to use the specific function GETUTCDATE() to get the datetime in UTC.
The alternative is to set the system time of the server in UTC.
Do you foresee a possible fix in the airflow code or you think the only viable solution is to change the system clock on the database server?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1347917334
Is this a one-time incident, or does this happen for all runs to this DAG?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1354358338
>
I can confirm it's not an UI issue because I have seen the DAG running 1 hour before.
This is the LOG related to the same RUN of the previous screenshot.
`
*** Reading remote log from s3://roj-airflow-log/airflow-log/dag_id=neorouter_v08/run_id=scheduled__2022-12-12T11:00:00+00:00/task_id=load_neorouter_log/attempt=1.log.
[2022-12-12, 12:00:05 UTC] {taskinstance.py:1165} INFO - Dependencies all met for <TaskInstance: neorouter_v08.load_neorouter_log scheduled__2022-12-12T11:00:00+00:00 [queued]>
[2022-12-12, 12:00:05 UTC] {taskinstance.py:1165} INFO - Dependencies all met for <TaskInstance: neorouter_v08.load_neorouter_log scheduled__2022-12-12T11:00:00+00:00 [queued]>
[2022-12-12, 12:00:05 UTC] {taskinstance.py:1362} INFO -
--------------------------------------------------------------------------------
[2022-12-12, 12:00:05 UTC] {taskinstance.py:1363} INFO - Starting attempt 1 of 1
[2022-12-12, 12:00:05 UTC] {taskinstance.py:1364} INFO -
--------------------------------------------------------------------------------
[2022-12-12, 12:00:05 UTC] {taskinstance.py:1383} INFO - Executing <Task(_PythonDecoratedOperator): load_neorouter_log> on 2022-12-12 11:00:00+00:00
[2022-12-12, 12:00:05 UTC] {standard_task_runner.py:82} INFO - Running: ['airflow', 'tasks', 'run', 'neorouter_v08', 'load_neorouter_log', 'scheduled__2022-12-12T11:00:00+00:00', '--job-id', '32119', '--raw', '--subdir', 'DAGS_FOLDER/neorouter.py', '--cfg-path', '/tmp/tmpo94brkol']
[2022-12-12, 12:00:05 UTC] {standard_task_runner.py:83} INFO - Job 32119: Subtask load_neorouter_log
[2022-12-12, 12:00:05 UTC] {standard_task_runner.py:55} INFO - Started process 8644 to run task
[2022-12-12, 12:00:06 UTC] {task_command.py:376} INFO - Running <TaskInstance: neorouter_v08.load_neorouter_log scheduled__2022-12-12T11:00:00+00:00 [running]> on host airflow-release-worker-0.airflow-release-worker.airflow-namespace.svc.cluster.local
[2022-12-12, 12:00:06 UTC] {taskinstance.py:1592} INFO - Exporting the following env vars:
AIRFLOW_CTX_DAG_OWNER=airflow
AIRFLOW_CTX_DAG_ID=neorouter_v08
AIRFLOW_CTX_TASK_ID=load_neorouter_log
AIRFLOW_CTX_EXECUTION_DATE=2022-12-12T11:00:00+00:00
AIRFLOW_CTX_TRY_NUMBER=1
AIRFLOW_CTX_DAG_RUN_ID=scheduled__2022-12-12T11:00:00+00:00
[2022-12-12, 12:00:06 UTC] {logging_mixin.py:137} INFO - data_interval_start: 2022-12-12 11:00:00+00:00
[2022-12-12, 12:00:06 UTC] {logging_mixin.py:137} INFO - data_interval_end: 2022-12-12 13:00:00+00:00
[2022-12-12, 12:00:06 UTC] {python.py:177} INFO - Done. Returned value was: None
[2022-12-12, 12:00:06 UTC] {taskinstance.py:1406} INFO - Marking task as SUCCESS. dag_id=neorouter_v08, task_id=load_neorouter_log, execution_date=20221212T110000, start_date=20221212T120005, end_date=20221212T120006
[2022-12-12, 12:00:07 UTC] {local_task_job.py:159} INFO - Task exited with return code 0
[2022-12-12, 12:00:07 UTC] {taskinstance.py:2623} INFO - 0 downstream tasks scheduled from follow-on schedule check
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1384012119
@uranusjr
Have you discovered something?
Thanks
Riki
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] DAG start before data_interval_end [airflow]
Posted by "jscheffl (via GitHub)" <gi...@apache.org>.
jscheffl closed issue #28329: DAG start before data_interval_end
URL: https://github.com/apache/airflow/issues/28329
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1353541964
Hmm, could you check the logs (and perhaps the `dag_run` database table) to see if the run actually happens one hour early, or is this only a UI issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1348031814
Thanks for the clarification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #28329: DAG start before data_interval_end
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1347914488
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by "rikirolly (via GitHub)" <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1487088742
I have seen in sqlachemy documentation they suggest this:
```"UTC timestamp" function
-------------------------
A function that works like "CURRENT_TIMESTAMP" except applies the
appropriate conversions so that the time is in UTC time. Timestamps are best
stored in relational databases as UTC, without time zones. UTC so that your
database doesn't think time has gone backwards in the hour when daylight
savings ends, without timezones because timezones are like character
encodings - they're best applied only at the endpoints of an application
(i.e. convert to UTC upon user input, re-apply desired timezone upon display).
For PostgreSQL and Microsoft SQL Server::
from sqlalchemy.sql import expression
from sqlalchemy.ext.compiler import compiles
from sqlalchemy.types import DateTime
class utcnow(expression.FunctionElement):
type = DateTime()
inherit_cache = True
@compiles(utcnow, 'postgresql')
def pg_utcnow(element, compiler, **kw):
return "TIMEZONE('utc', CURRENT_TIMESTAMP)"
@compiles(utcnow, 'mssql')
def ms_utcnow(element, compiler, **kw):
return "GETUTCDATE()"
Example usage::
from sqlalchemy import (
Table, Column, Integer, String, DateTime, MetaData
)
metadata = MetaData()
event = Table("event", metadata,
Column("id", Integer, primary_key=True),
Column("description", String(50), nullable=False),
Column("timestamp", DateTime, server_default=utcnow())
)
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] rikirolly commented on issue #28329: DAG start before data_interval_end
Posted by "rikirolly (via GitHub)" <gi...@apache.org>.
rikirolly commented on issue #28329:
URL: https://github.com/apache/airflow/issues/28329#issuecomment-1487056770
Could be a fix changing func.now() into func.utcnow() in dag.py and dagrun.py modules?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org