You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/12/27 03:17:04 UTC
[GitHub] [airflow] subkanthi opened a new pull request #13324: Subkanthi/neo4j operator 12873
subkanthi opened a new pull request #13324:
URL: https://github.com/apache/airflow/pull/13324
<!--
Thank you for contributing! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
In case of existing issue, reference it using one of the following:
closes: #ISSUE
related: #ISSUE
How to write a good git commit message:
http://chris.beams.io/posts/git-commit/
-->
---
**^ Add meaningful description above**
Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more information.
In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-751812735
[The Workflow run](https://github.com/apache/airflow/actions/runs/449280739) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r553069719
##########
File path: airflow/providers/neo4j/hooks/neo4j.py
##########
@@ -0,0 +1,116 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module allows to connect to a Neo4j database."""
+
+from neo4j import GraphDatabase, Neo4jDriver, Result
+
+from airflow.hooks.base import BaseHook
+from airflow.models import Connection
+
+
+class Neo4jHook(BaseHook):
+ """
+ Interact with Neo4j.
+
+ Performs a connection to Neo4j
+ and runs the query.
Review comment:
```suggestion
Performs a connection to Neo4j and runs the query.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-751705467
[The Workflow run](https://github.com/apache/airflow/actions/runs/448838804) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-751842928
[The Workflow run](https://github.com/apache/airflow/actions/runs/449421332) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-757905572
[The Workflow run](https://github.com/apache/airflow/actions/runs/477291570) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r549408935
##########
File path: airflow/example_dags/example_neo4j.py
##########
@@ -0,0 +1 @@
+../providers/neo4j/example_dags/example_neo4j.py
Review comment:
Removed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r549188571
##########
File path: airflow/example_dags/example_neo4j.py
##########
@@ -0,0 +1 @@
+../providers/neo4j/example_dags/example_neo4j.py
Review comment:
Thanks, will remove it.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-752315568
[The Workflow run](https://github.com/apache/airflow/actions/runs/451950370) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-757929249
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest master at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-755902352
[The Workflow run](https://github.com/apache/airflow/actions/runs/468012488) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
subkanthi commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-760292828
> Nice work. Let wait for the test results
Hi @kaxil , can this PR be merged, the failed check seems to be a timeout(not related to the change)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-756325805
[The Workflow run](https://github.com/apache/airflow/actions/runs/469769431) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-757905999
[The Workflow run](https://github.com/apache/airflow/actions/runs/477298707) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-751790886
[The Workflow run](https://github.com/apache/airflow/actions/runs/449198495) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] smowden commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
smowden commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-757325632
I am eagerly awaiting this, commenting so I am subscribed to this PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-751753096
[The Workflow run](https://github.com/apache/airflow/actions/runs/449057430) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] smowden commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
smowden commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-757325632
I am eagerly awaiting this, commenting so I am subscribed to this PR
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-759423693
[The Workflow run](https://github.com/apache/airflow/actions/runs/482359034) is cancelling this PR. Building images for the PR has failed. Follow the the workflow link to check the reason.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r553415848
##########
File path: airflow/providers/neo4j/hooks/neo4j.py
##########
@@ -0,0 +1,116 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module allows to connect to a Neo4j database."""
+
+from neo4j import GraphDatabase, Neo4jDriver, Result
+
+from airflow.hooks.base import BaseHook
+from airflow.models import Connection
+
+
+class Neo4jHook(BaseHook):
+ """
+ Interact with Neo4j.
+
+ Performs a connection to Neo4j and runs the query.
+ """
+
+ conn_name_attr = 'neo4j_conn_id'
+ default_conn_name = 'neo4j_default'
+ conn_type = 'neo4j'
+ hook_name = 'Neo4j'
+
+ def __init__(self, conn_id: str = default_conn_name, *args, **kwargs) -> None:
+ super().__init__(*args, **kwargs)
+ self.neo4j_conn_id = conn_id
+ self.connection = kwargs.pop("connection", None)
+ self.client = None
+ self.extras = None
+ self.uri = None
+
+ def get_conn(self) -> Neo4jDriver:
+ """
+ Function that initiates a new Neo4j connection
+ with username, password and database schema.
+ """
+ self.connection = self.get_connection(self.neo4j_conn_id)
+ self.extras = self.connection.extra_dejson.copy()
+
+ self.uri = self.get_uri(self.connection)
+ self.log.info('URI: %s', self.uri)
+
+ if self.client is not None:
+ return self.client
+
+ is_encrypted = self.connection.extra_dejson.get('encrypted', False)
+
+ self.client = GraphDatabase.driver(
+ self.uri, auth=(self.connection.login, self.connection.password), encrypted=is_encrypted
+ )
+
+ return self.client
+
+ def get_uri(self, conn: Connection) -> str:
+ """
+ Build the uri based on extras
+ - Default - uses bolt scheme(bolt://)
+ - neo4j_scheme - neo4j://
+ - certs_self_signed - neo4j+ssc://
+ - certs_trusted_ca - neo4j+s://
+ :param conn: connection object.
+ :return: uri
+ """
+ use_neo4j_scheme = conn.extra_dejson.get('neo4j_scheme', False)
+ scheme = 'neo4j' if use_neo4j_scheme else 'bolt'
+
+ # Self signed certificates
+ ssc = conn.extra_dejson.get('certs_self_signed', False)
+
+ # Only certificates signed by CA.
+ trusted_ca = conn.extra_dejson.get('certs_trusted_ca', False)
+ encryption_scheme = ''
+
+ if ssc:
+ encryption_scheme = '+ssc'
+ elif trusted_ca:
+ encryption_scheme = '+s'
+
+ return '{scheme}{encryption_scheme}://{host}:{port}'.format(
+ scheme=scheme,
+ encryption_scheme=encryption_scheme,
+ host=conn.host,
+ port='7687' if conn.port is None else f'{conn.port}',
+ )
+
+ def run(self, query) -> Result:
+ """
+ Function to create a neo4j session
+ and execute the query in the session.
+
Review comment:
```suggestion
```
This change should fix the failing static check: https://github.com/apache/airflow/pull/13324/checks?check_run_id=1660621219#step:7:181
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r553070787
##########
File path: tests/providers/neo4j/operators/test_neo4j.py
##########
@@ -0,0 +1,16 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
Review comment:
Can you add atleast a single test for the operator -- otherwise empty file does not make sense
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
subkanthi commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r553086119
##########
File path: tests/providers/neo4j/operators/test_neo4j.py
##########
@@ -0,0 +1,16 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
Review comment:
sure will do, thanks for reviewing.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-753216224
[The Workflow run](https://github.com/apache/airflow/actions/runs/455203770) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r553069920
##########
File path: airflow/providers/neo4j/hooks/neo4j.py
##########
@@ -0,0 +1,116 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""This module allows to connect to a Neo4j database."""
+
+from neo4j import GraphDatabase, Neo4jDriver, Result
+
+from airflow.hooks.base import BaseHook
+from airflow.models import Connection
+
+
+class Neo4jHook(BaseHook):
+ """
+ Interact with Neo4j.
+
+ Performs a connection to Neo4j
+ and runs the query.
+ """
+
+ conn_name_attr = 'neo4j_conn_id'
+ default_conn_name = 'neo4j_default'
+ conn_type = 'neo4j'
+ hook_name = 'Neo4j'
+
+ def __init__(self, conn_id: str = default_conn_name, *args, **kwargs) -> None:
+ super().__init__(*args, **kwargs)
+ self.neo4j_conn_id = conn_id
+ self.connection = kwargs.pop("connection", None)
+ self.client = None
+ self.extras = None
+ self.uri = None
+
+ def get_conn(self) -> Neo4jDriver:
+ """
+ Function that initiates a new Neo4j connection
+ with username, password and database schema.
+ """
+ self.connection = self.get_connection(self.neo4j_conn_id)
+ self.extras = self.connection.extra_dejson.copy()
+
+ self.uri = self.get_uri(self.connection)
+ self.log.info('URI: %s', self.uri)
+
+ if self.client is not None:
+ return self.client
+
+ is_encrypted = self.connection.extra_dejson.get('encrypted', False)
+
+ self.client = GraphDatabase.driver(
+ self.uri, auth=(self.connection.login, self.connection.password), encrypted=is_encrypted
+ )
+
+ return self.client
+
+ def get_uri(self, conn: Connection) -> str:
+ """
+ Build the uri based on extras
+ - Default - uses bolt scheme(bolt://)
+ - neo4j_scheme - neo4j://
+ - certs_self_signed - neo4j+ssc://
+ - certs_trusted_ca - neo4j+s://
+ :param conn: connection object.
+ :return: uri
+ """
+ use_neo4j_scheme = conn.extra_dejson.get('neo4j_scheme', False)
+ scheme = 'neo4j' if use_neo4j_scheme else 'bolt'
+
+ # Self signed certificates
+ ssc = conn.extra_dejson.get('certs_self_signed', False)
+
+ # Only certificates signed by CA.
+ trusted_ca = conn.extra_dejson.get('certs_trusted_ca', False)
+ encryption_scheme = ''
+
+ if ssc:
+ encryption_scheme = '+ssc'
+ elif trusted_ca:
+ encryption_scheme = '+s'
+
+ return '{scheme}{encryption_scheme}://{host}:{port}'.format(
+ scheme=scheme,
+ encryption_scheme=encryption_scheme,
+ host=conn.host,
+ port='7687' if conn.port is None else f'{conn.port}',
+ )
+
+ def run(self, query) -> Result:
+ """
+ Function to create a neo4j session
+ and execute the query in the session.
+ :param query - Neo4j query
+ :return: Result
Review comment:
```suggestion
Function to create a neo4j session
and execute the query in the session.
:param query: Neo4j query
:return: Result
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RosterIn commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
RosterIn commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r551914956
##########
File path: tests/providers/mysql/hooks/test_mysql.py
##########
@@ -1,4 +1,3 @@
-#
Review comment:
this seems unrelated change to this pr
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-758285868
[The Workflow run](https://github.com/apache/airflow/actions/runs/478597573) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] subkanthi commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
subkanthi commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-754377239
#12873
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-752698159
[The Workflow run](https://github.com/apache/airflow/actions/runs/453080671) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-751501376
[The Workflow run](https://github.com/apache/airflow/actions/runs/447494191) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
kaxil commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r553070407
##########
File path: docs/apache-airflow-providers-neo4j/connections/neo4j.rst
##########
@@ -0,0 +1,63 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+
+Neo4j Connection
+================
+The Neo4j connection type provides connection to a Neo4j database.
+
+Configuring the Connection
+--------------------------
+Host (required)
+ The host to connect to.
+
+Schema (optional)
+ Specify the schema name to be used in the database.
+
+Login (required)
+ Specify the user name to connect.
+
+Password (required)
+ Specify the password to connect.
+
+Extra (optional)
+ Specify the extra parameters (as json dictionary) that can be used in Neo4j
+ connection.
+
+ The following extras are supported:
+
+ - Default - uses bolt scheme(bolt://)
+ - neo4j_scheme - neo4j://
+ - certs_self_signed - neo4j+ssc://
+ - certs_trusted_ca - neo4j+s://
+
+ * ``encrypted``: Sets encrypted=True/False for GraphDatabase.driver, Set to True for Neo4j Aura.
+ * ``neo4j_scheme``: Specifies the scheme to neo4j://, default is bolt://
+ * ``certs_self_signed``: Sets the URI scheme to support self-signed certificates(neo4j+ssc://)
+ * ``certs_trusted_ca``: Sets the URI scheme to support only trusted CA(neo4j+s://)
Review comment:
```suggestion
* ``encrypted``: Sets encrypted=True/False for GraphDatabase.driver, Set to ``True`` for Neo4j Aura.
* ``neo4j_scheme``: Specifies the scheme to ``neo4j://``, default is ``bolt://``
* ``certs_self_signed``: Sets the URI scheme to support self-signed certificates(``neo4j+ssc://``)
* ``certs_trusted_ca``: Sets the URI scheme to support only trusted CA(``neo4j+s://``)
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-758286348
[The Workflow run](https://github.com/apache/airflow/actions/runs/478613522) is cancelling this PR. It has some failed jobs matching ^Pylint$,^Static checks,^Build docs$,^Spell check docs$,^Backport packages$,^Provider packages,^Checks: Helm tests$,^Test OpenAPI*.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#issuecomment-751843152
[The Workflow run](https://github.com/apache/airflow/actions/runs/449383986) is cancelling this PR. Building image for the PR has been cancelled
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on a change in pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
mik-laj commented on a change in pull request #13324:
URL: https://github.com/apache/airflow/pull/13324#discussion_r549173270
##########
File path: airflow/example_dags/example_neo4j.py
##########
@@ -0,0 +1 @@
+../providers/neo4j/example_dags/example_neo4j.py
Review comment:
In the airflow / example_dags directory, we only store examples that use operators that are part of the core.
http://apache-airflow-docs.s3-website.eu-central-1.amazonaws.com/docs/apache-airflow/latest/operators-and-hooks-ref.html
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil merged pull request #13324: Add Neo4j hook and operator
Posted by GitBox <gi...@apache.org>.
kaxil merged pull request #13324:
URL: https://github.com/apache/airflow/pull/13324
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org