You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/03/29 12:05:49 UTC
[GitHub] [airflow] bintocher opened a new issue #15066: try to open admin->xcom and get error message
bintocher opened a new issue #15066:
URL: https://github.com/apache/airflow/issues/15066
Hi,
i'll try open Admin-XComs (http://url/xcom/list/) and get error page:
Something bad has happened.
Please consider letting us know by creating a bug report using GitHub.
Python version: 3.7.3
Airflow version: 2.0.1
Node: hostname.local
-------------------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python3.7/dist-packages/flask/_compat.py", line 39, in reraise
raise value
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python3.7/dist-packages/flask_appbuilder/security/decorators.py", line 109, in wraps
return f(self, *args, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/flask_appbuilder/views.py", line 551, in list
widgets = self._list()
File "/usr/local/lib/python3.7/dist-packages/flask_appbuilder/baseviews.py", line 1127, in _list
page_size=page_size,
File "/usr/local/lib/python3.7/dist-packages/flask_appbuilder/baseviews.py", line 1026, in _get_list_widget
page_size=page_size,
File "/usr/local/lib/python3.7/dist-packages/flask_appbuilder/models/sqla/interface.py", line 435, in query
query_results = query.all()
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/query.py", line 3373, in all
return list(self)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py", line 100, in instances
cursor.close()
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
with_traceback=exc_tb,
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py", line 80, in instances
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py", line 80, in <listcomp>
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/loading.py", line 601, in _instance
state.manager.dispatch.load(state, context)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/event/attr.py", line 322, in __call__
fn(*args, **kw)
File "/usr/local/lib/python3.7/dist-packages/sqlalchemy/orm/mapper.py", line 3397, in _event_on_load
instrumenting_mapper._reconstructor(state.obj())
File "/usr/local/lib/python3.7/dist-packages/airflow/models/xcom.py", line 66, in init_on_load
self.value = self.orm_deserialize_value()
File "/usr/local/lib/python3.7/dist-packages/airflow/models/xcom.py", line 274, in orm_deserialize_value
return BaseXCom.deserialize_value(self)
File "/usr/local/lib/python3.7/dist-packages/airflow/models/xcom.py", line 251, in deserialize_value
return pickle.loads(result.value)
ModuleNotFoundError: No module named 'unusual_prefix_06d086af3242da40550adef1deae85607495751f_np_run_test_task'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-984170157
This issue has been closed because it has not received response from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bintocher edited a comment on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
bintocher edited a comment on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-809845930
np_run_test_task file:
``` python
# sudo pip3 install requests_ntlm
"""Example DAG demonstrating the usage of the PythonOperator."""
import time
import requests
import json
# from pprint import pprint
from requests_ntlm import HttpNtlmAuth
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
default_args = {
'owner': 'nprinting',
'depends_on_past': False,
#'retries': 1,
}
# раписание шедулера можно конфигурировать тут: https://crontab.guru/
# время UTC !!!
dag = DAG(
dag_id = 'np_run_test_task_1_3-20_A_A_A',
default_args = default_args ,
start_date = days_ago(1) ,
schedule_interval = '1 3-20 * * *',
description = 'NPrinting Мониторинг обновления',
tags = ['nprinting','apps','мониторинг обновления'],
catchup = False
)
# schedule_interval=None,
requests.packages.urllib3.disable_warnings()
tokenstring = ''
headers = {"Accept": "application/json",
"Content-Type": "application/json"}
session = requests.session()
class ConnectnPrinting:
"""
Instantiates the nPrinting connection class
"""
def __init__(self, server, root = False
, userid = False, credential = False, password = False):
"""
Establishes connectivity with Qlik Sense Repository Service
:param server: servername.domain:4242
:param userid: user to use for queries
:param credential: domain\\username for Windows Authentication
:param password: password of windows credential
"""
self.server = server
self.root = root
self.credential = credential
self.password = password
def get(self,endpoint):
if self.credential is not False:
global tokenstring
session.auth = HttpNtlmAuth(self.credential, self.password, session)
headers['User-Agent'] = 'Windows'
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
response = session.get('http://{0}/{1}'.format (self.server, endpoint),
headers=headers, verify=self.root)
tokenstring = (session.cookies['NPWEBCONSOLE_XSRF-TOKEN'])
return response.content
def post(self, endpoint, data=None):
global tokenstring
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
response = session.post('http://{0}/{1}'.format (self.server, endpoint),
headers=headers, verify=self.root, data=data)
return response.status_code
def auth(self):
path = 'login/ntlm'
return json.loads(self.get(path))
def run_task(self, task_id = None):
path = 'tasks'
global tokenstring
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
urltorun = 'http://{0}/{1}/{2}/executions'.format (self.server, path,task_id)
response = session.post(urltorun, headers=headers, verify=self.root)
print (response)
# [START howto_operator_python]
def run_test_task(ds, **kwargs):
"""Print the Airflow context and ds variable from the context."""
print(ds)
np = ConnectnPrinting(server='server-qv:4993/api/v1',
credential='user',
password='pass')
# Авторизуемся, и запоминаем TOKEN, с помощью которого потом выполняем запросы
auth = np.auth()
# print (tokenstring)
# Запускаем таск по его ID
np.run_task('bc58a89f-4401-4769-abbd-f515e13d386e')
return np
run_this = PythonOperator(
task_id='NPrinting_run_test_task',
python_callable=run_test_task,
dag=dag,
)
# [END howto_operator_python]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bintocher commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
bintocher commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-809845930
np_run_test_task file:
`# sudo pip3 install requests_ntlm
"""Example DAG demonstrating the usage of the PythonOperator."""
import time
import requests
import json
# from pprint import pprint
from requests_ntlm import HttpNtlmAuth
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
default_args = {
'owner': 'nprinting',
'depends_on_past': False,
#'retries': 1,
}
# раписание шедулера можно конфигурировать тут: https://crontab.guru/
# время UTC !!!
dag = DAG(
dag_id = 'np_run_test_task_1_3-20_A_A_A',
default_args = default_args ,
start_date = days_ago(1) ,
schedule_interval = '1 3-20 * * *',
description = 'NPrinting Мониторинг обновления',
tags = ['nprinting','apps','мониторинг обновления'],
catchup = False
)
# schedule_interval=None,
requests.packages.urllib3.disable_warnings()
tokenstring = ''
headers = {"Accept": "application/json",
"Content-Type": "application/json"}
session = requests.session()
class ConnectnPrinting:
"""
Instantiates the nPrinting connection class
"""
def __init__(self, server, root = False
, userid = False, credential = False, password = False):
"""
Establishes connectivity with Qlik Sense Repository Service
:param server: servername.domain:4242
:param userid: user to use for queries
:param credential: domain\\username for Windows Authentication
:param password: password of windows credential
"""
self.server = server
self.root = root
self.credential = credential
self.password = password
def get(self,endpoint):
if self.credential is not False:
global tokenstring
session.auth = HttpNtlmAuth(self.credential, self.password, session)
headers['User-Agent'] = 'Windows'
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
response = session.get('http://{0}/{1}'.format (self.server, endpoint),
headers=headers, verify=self.root)
tokenstring = (session.cookies['NPWEBCONSOLE_XSRF-TOKEN'])
return response.content
def post(self, endpoint, data=None):
global tokenstring
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
response = session.post('http://{0}/{1}'.format (self.server, endpoint),
headers=headers, verify=self.root, data=data)
return response.status_code
def auth(self):
path = 'login/ntlm'
return json.loads(self.get(path))
def run_task(self, task_id = None):
path = 'tasks'
global tokenstring
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
urltorun = 'http://{0}/{1}/{2}/executions'.format (self.server, path,task_id)
response = session.post(urltorun, headers=headers, verify=self.root)
print (response)
# [START howto_operator_python]
def run_test_task(ds, **kwargs):
"""Print the Airflow context and ds variable from the context."""
print(ds)
np = ConnectnPrinting(server='server-qv3.luding.ru:4993/api/v1',
credential='luding\\Qvtaskmgr',
password='Gthver75')
# Авторизуемся, и запоминаем TOKEN, с помощью которого потом выполняем запросы
auth = np.auth()
# print (tokenstring)
# Запускаем таск по его ID
np.run_task('bc58a89f-4401-4769-abbd-f515e13d386e')
return np
run_this = PythonOperator(
task_id='NPrinting_run_test_task',
python_callable=run_test_task,
dag=dag,
)
# [END howto_operator_python]
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bintocher commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
bintocher commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-813303623
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-833125332
This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-978514697
This issue has been automatically marked as stale because it has been open for 30 days with no response from the author. It will be closed in next 7 days if no further activity occurs from the issue author.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
github-actions[bot] closed issue #15066:
URL: https://github.com/apache/airflow/issues/15066
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-947661998
@bintocher does this still happen in latest airflow version?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] vikramkoka commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
vikramkoka commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-812224323
@bintocher What are the values in your airflow.cfg with respect to Xcom?
Specifically:
1. Do you have pickling set to True of False, and
2. Are you using a custom_xcom_backend?
Out of the box, Xcom only supports values that are automatically serializable into JSON.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-809642725
Hmmm, this is something odd about the value that is pickled.
Do you happen to know what sort of value you returned from the tasks in np_run_test_tasks.py dag file?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] bintocher edited a comment on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
bintocher edited a comment on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-809845930
np_run_test_task file:
` python
# sudo pip3 install requests_ntlm
"""Example DAG demonstrating the usage of the PythonOperator."""
import time
import requests
import json
# from pprint import pprint
from requests_ntlm import HttpNtlmAuth
from airflow import DAG
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
default_args = {
'owner': 'nprinting',
'depends_on_past': False,
#'retries': 1,
}
# раписание шедулера можно конфигурировать тут: https://crontab.guru/
# время UTC !!!
dag = DAG(
dag_id = 'np_run_test_task_1_3-20_A_A_A',
default_args = default_args ,
start_date = days_ago(1) ,
schedule_interval = '1 3-20 * * *',
description = 'NPrinting Мониторинг обновления',
tags = ['nprinting','apps','мониторинг обновления'],
catchup = False
)
# schedule_interval=None,
requests.packages.urllib3.disable_warnings()
tokenstring = ''
headers = {"Accept": "application/json",
"Content-Type": "application/json"}
session = requests.session()
class ConnectnPrinting:
"""
Instantiates the nPrinting connection class
"""
def __init__(self, server, root = False
, userid = False, credential = False, password = False):
"""
Establishes connectivity with Qlik Sense Repository Service
:param server: servername.domain:4242
:param userid: user to use for queries
:param credential: domain\\username for Windows Authentication
:param password: password of windows credential
"""
self.server = server
self.root = root
self.credential = credential
self.password = password
def get(self,endpoint):
if self.credential is not False:
global tokenstring
session.auth = HttpNtlmAuth(self.credential, self.password, session)
headers['User-Agent'] = 'Windows'
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
response = session.get('http://{0}/{1}'.format (self.server, endpoint),
headers=headers, verify=self.root)
tokenstring = (session.cookies['NPWEBCONSOLE_XSRF-TOKEN'])
return response.content
def post(self, endpoint, data=None):
global tokenstring
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
response = session.post('http://{0}/{1}'.format (self.server, endpoint),
headers=headers, verify=self.root, data=data)
return response.status_code
def auth(self):
path = 'login/ntlm'
return json.loads(self.get(path))
def run_task(self, task_id = None):
path = 'tasks'
global tokenstring
if tokenstring != '':
headers['X-XSRF-TOKEN'] = tokenstring
urltorun = 'http://{0}/{1}/{2}/executions'.format (self.server, path,task_id)
response = session.post(urltorun, headers=headers, verify=self.root)
print (response)
# [START howto_operator_python]
def run_test_task(ds, **kwargs):
"""Print the Airflow context and ds variable from the context."""
print(ds)
np = ConnectnPrinting(server='server-qv:4993/api/v1',
credential='user',
password='pass')
# Авторизуемся, и запоминаем TOKEN, с помощью которого потом выполняем запросы
auth = np.auth()
# print (tokenstring)
# Запускаем таск по его ID
np.run_task('bc58a89f-4401-4769-abbd-f515e13d386e')
return np
run_this = PythonOperator(
task_id='NPrinting_run_test_task',
python_callable=run_test_task,
dag=dag,
)
# [END howto_operator_python]
`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #15066: try to open admin->xcom and get error message
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #15066:
URL: https://github.com/apache/airflow/issues/15066#issuecomment-809323893
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org