You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sedona.apache.org by ji...@apache.org on 2022/08/28 05:11:10 UTC
[incubator-sedona] branch master updated: [DOCS] Fix the copyright issues (#676)
This is an automated email from the ASF dual-hosted git repository.
jiayu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-sedona.git
The following commit(s) were added to refs/heads/master by this push:
new 26d65d4e [DOCS] Fix the copyright issues (#676)
26d65d4e is described below
commit 26d65d4e7f5add374c8089dfdf974d7cb830ee01
Author: Jia Yu <ji...@apache.org>
AuthorDate: Sat Aug 27 22:11:05 2022 -0700
[DOCS] Fix the copyright issues (#676)
---
DISCLAIMER | 1 +
DISCLAIMER-WIP | 5 -
NOTICE | 2 +-
binder/ApacheSedonaCore.ipynb | 26 +-
binder/ApacheSedonaRaster.ipynb | 28 +-
binder/ApacheSedonaSQL.ipynb | 421 +++++++++-------
...eSedonaSQL_SpatialJoin_AirportsPerCountry.ipynb | 31 +-
binder/contrib/ApacheSedonaImageFilter.ipynb | 95 ++--
binder/contrib/DownloadImageFromGEE.ipynb | 41 +-
binder/contrib/NdviSentinelApacheSedona.ipynb | 81 ++--
.../contrib/PostgresqlConnectionApacheSedona.ipynb | 531 +++++++++++----------
binder/contrib/VectorAnalisisApacheSedona.ipynb | 53 +-
12 files changed, 764 insertions(+), 551 deletions(-)
diff --git a/DISCLAIMER b/DISCLAIMER
new file mode 100644
index 00000000..f5f96288
--- /dev/null
+++ b/DISCLAIMER
@@ -0,0 +1 @@
+Apache Sedona is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project [...]
\ No newline at end of file
diff --git a/DISCLAIMER-WIP b/DISCLAIMER-WIP
deleted file mode 100644
index 93343ef3..00000000
--- a/DISCLAIMER-WIP
+++ /dev/null
@@ -1,5 +0,0 @@
-Apache Sedona is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the name of Apache TLP sponsor. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that th [...]
-
-Some of the incubating project’s releases may not be fully compliant with ASF policy. For example, releases may have incomplete or un-reviewed licensing conditions.
-
-If you are planning to incorporate this work into your product/project, please be aware that you will need to conduct a thorough licensing review to determine the overall implications of including this work. For the current status of this project through the Apache Incubator visit: https://incubator.apache.org/projects/sedona.html
\ No newline at end of file
diff --git a/NOTICE b/NOTICE
index a1ed253d..f1f65c13 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,5 +1,5 @@
Apache Sedona (incubating)
-Copyright 2020 and onwards The Apache Software Foundation
+Copyright 2020-2022 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
diff --git a/binder/ApacheSedonaCore.ipynb b/binder/ApacheSedonaCore.ipynb
index 800230ee..2470bf48 100644
--- a/binder/ApacheSedonaCore.ipynb
+++ b/binder/ApacheSedonaCore.ipynb
@@ -1,5 +1,27 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 6,
@@ -2124,7 +2146,7 @@
],
"metadata": {
"kernelspec": {
- "display_name": "Python 3",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
@@ -2138,7 +2160,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.7"
+ "version": "3.7.12"
}
},
"nbformat": 4,
diff --git a/binder/ApacheSedonaRaster.ipynb b/binder/ApacheSedonaRaster.ipynb
index ba77429c..624eabb9 100644
--- a/binder/ApacheSedonaRaster.ipynb
+++ b/binder/ApacheSedonaRaster.ipynb
@@ -1,5 +1,27 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
@@ -1054,9 +1076,9 @@
],
"metadata": {
"kernelspec": {
- "display_name": "apache-sedona",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
- "name": "apache-sedona"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
@@ -1068,7 +1090,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.7.3"
+ "version": "3.7.12"
}
},
"nbformat": 4,
diff --git a/binder/ApacheSedonaSQL.ipynb b/binder/ApacheSedonaSQL.ipynb
index c1697c42..290e2a5f 100644
--- a/binder/ApacheSedonaSQL.ipynb
+++ b/binder/ApacheSedonaSQL.ipynb
@@ -1,8 +1,30 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
{
"cell_type": "code",
- "execution_count": 21,
+ "execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
@@ -17,9 +39,82 @@
},
{
"cell_type": "code",
- "execution_count": 22,
+ "execution_count": 2,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ ":: loading settings :: url = jar:file:/home/jovyan/spark-3.1.2-bin-hadoop3.2/jars/ivy-2.4.0.jar!/org/apache/ivy/core/settings/ivysettings.xml\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "Ivy Default Cache set to: /home/jovyan/.ivy2/cache\n",
+ "The jars for the packages stored in: /home/jovyan/.ivy2/jars\n",
+ "org.apache.sedona#sedona-python-adapter-3.0_2.12 added as a dependency\n",
+ "org.datasyslab#geotools-wrapper added as a dependency\n",
+ ":: resolving dependencies :: org.apache.spark#spark-submit-parent-3625fe59-a0fc-45df-ae07-108b2b675db5;1.0\n",
+ "\tconfs: [default]\n",
+ "\tfound org.apache.sedona#sedona-python-adapter-3.0_2.12;1.1.0-incubating in central\n",
+ "\tfound org.locationtech.jts#jts-core;1.18.0 in central\n",
+ "\tfound org.wololo#jts2geojson;0.16.1 in central\n",
+ "\tfound com.fasterxml.jackson.core#jackson-databind;2.12.2 in central\n",
+ "\tfound com.fasterxml.jackson.core#jackson-annotations;2.12.2 in central\n",
+ "\tfound com.fasterxml.jackson.core#jackson-core;2.12.2 in central\n",
+ "\tfound org.apache.sedona#sedona-core-3.0_2.12;1.1.0-incubating in central\n",
+ "\tfound org.apache.sedona#sedona-sql-3.0_2.12;1.1.0-incubating in central\n",
+ "\tfound org.datasyslab#geotools-wrapper;1.1.0-25.2 in central\n",
+ "downloading https://repo1.maven.org/maven2/org/apache/sedona/sedona-python-adapter-3.0_2.12/1.1.0-incubating/sedona-python-adapter-3.0_2.12-1.1.0-incubating.jar ...\n",
+ "\t[SUCCESSFUL ] org.apache.sedona#sedona-python-adapter-3.0_2.12;1.1.0-incubating!sedona-python-adapter-3.0_2.12.jar (85ms)\n",
+ "downloading https://repo1.maven.org/maven2/org/datasyslab/geotools-wrapper/1.1.0-25.2/geotools-wrapper-1.1.0-25.2.jar ...\n",
+ "\t[SUCCESSFUL ] org.datasyslab#geotools-wrapper;1.1.0-25.2!geotools-wrapper.jar (687ms)\n",
+ "downloading https://repo1.maven.org/maven2/org/locationtech/jts/jts-core/1.18.0/jts-core-1.18.0.jar ...\n",
+ "\t[SUCCESSFUL ] org.locationtech.jts#jts-core;1.18.0!jts-core.jar(bundle) (31ms)\n",
+ "downloading https://repo1.maven.org/maven2/org/wololo/jts2geojson/0.16.1/jts2geojson-0.16.1.jar ...\n",
+ "\t[SUCCESSFUL ] org.wololo#jts2geojson;0.16.1!jts2geojson.jar (10ms)\n",
+ "downloading https://repo1.maven.org/maven2/org/apache/sedona/sedona-core-3.0_2.12/1.1.0-incubating/sedona-core-3.0_2.12-1.1.0-incubating.jar ...\n",
+ "\t[SUCCESSFUL ] org.apache.sedona#sedona-core-3.0_2.12;1.1.0-incubating!sedona-core-3.0_2.12.jar (15ms)\n",
+ "downloading https://repo1.maven.org/maven2/org/apache/sedona/sedona-sql-3.0_2.12/1.1.0-incubating/sedona-sql-3.0_2.12-1.1.0-incubating.jar ...\n",
+ "\t[SUCCESSFUL ] org.apache.sedona#sedona-sql-3.0_2.12;1.1.0-incubating!sedona-sql-3.0_2.12.jar (24ms)\n",
+ "downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.12.2/jackson-databind-2.12.2.jar ...\n",
+ "\t[SUCCESSFUL ] com.fasterxml.jackson.core#jackson-databind;2.12.2!jackson-databind.jar(bundle) (43ms)\n",
+ "downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.12.2/jackson-annotations-2.12.2.jar ...\n",
+ "\t[SUCCESSFUL ] com.fasterxml.jackson.core#jackson-annotations;2.12.2!jackson-annotations.jar(bundle) (11ms)\n",
+ "downloading https://repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.12.2/jackson-core-2.12.2.jar ...\n",
+ "\t[SUCCESSFUL ] com.fasterxml.jackson.core#jackson-core;2.12.2!jackson-core.jar(bundle) (17ms)\n",
+ ":: resolution report :: resolve 9110ms :: artifacts dl 931ms\n",
+ "\t:: modules in use:\n",
+ "\tcom.fasterxml.jackson.core#jackson-annotations;2.12.2 from central in [default]\n",
+ "\tcom.fasterxml.jackson.core#jackson-core;2.12.2 from central in [default]\n",
+ "\tcom.fasterxml.jackson.core#jackson-databind;2.12.2 from central in [default]\n",
+ "\torg.apache.sedona#sedona-core-3.0_2.12;1.1.0-incubating from central in [default]\n",
+ "\torg.apache.sedona#sedona-python-adapter-3.0_2.12;1.1.0-incubating from central in [default]\n",
+ "\torg.apache.sedona#sedona-sql-3.0_2.12;1.1.0-incubating from central in [default]\n",
+ "\torg.datasyslab#geotools-wrapper;1.1.0-25.2 from central in [default]\n",
+ "\torg.locationtech.jts#jts-core;1.18.0 from central in [default]\n",
+ "\torg.wololo#jts2geojson;0.16.1 from central in [default]\n",
+ "\t:: evicted modules:\n",
+ "\torg.locationtech.jts#jts-core;1.18.1 by [org.locationtech.jts#jts-core;1.18.0] in [default]\n",
+ "\t---------------------------------------------------------------------\n",
+ "\t| | modules || artifacts |\n",
+ "\t| conf | number| search|dwnlded|evicted|| number|dwnlded|\n",
+ "\t---------------------------------------------------------------------\n",
+ "\t| default | 10 | 9 | 9 | 1 || 9 | 9 |\n",
+ "\t---------------------------------------------------------------------\n",
+ ":: retrieving :: org.apache.spark#spark-submit-parent-3625fe59-a0fc-45df-ae07-108b2b675db5\n",
+ "\tconfs: [default]\n",
+ "\t9 artifacts copied, 0 already retrieved (35019kB/52ms)\n",
+ "22/08/28 04:59:46 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\n",
+ "Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties\n",
+ "Setting default log level to \"WARN\".\n",
+ "To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).\n"
+ ]
+ }
+ ],
"source": [
"spark = SparkSession. \\\n",
" builder. \\\n",
@@ -33,66 +128,14 @@
},
{
"cell_type": "code",
- "execution_count": 23,
+ "execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "21/10/08 19:57:20 WARN UDTRegistration: Cannot register UDT for org.locationtech.jts.geom.Geometry, which is already registered.\n",
- "21/10/08 19:57:20 WARN UDTRegistration: Cannot register UDT for org.locationtech.jts.index.SpatialIndex, which is already registered.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_pointfromtext replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_polygonfromtext replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_linestringfromtext replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_geomfromtext replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_geomfromwkt replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_geomfromwkb replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_geomfromgeojson replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_point replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_polygonfromenvelope replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_contains replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_intersects replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_within replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_distance replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_convexhull replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_npoints replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_buffer replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_envelope replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_length replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_area replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_centroid replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_transform replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_intersection replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_isvalid replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_precisionreduce replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_equals replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_touches replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_overlaps replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_crosses replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_issimple replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_makevalid replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_simplifypreservetopology replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_astext replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_asgeojson replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_geometrytype replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_numgeometries replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_linemerge replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_azimuth replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_x replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_y replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_startpoint replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_boundary replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_minimumboundingradius replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_minimumboundingcircle replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_endpoint replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_exteriorring replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_geometryn replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_interiorringn replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_dump replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_dumppoints replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_isclosed replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function"
+ " \r"
]
},
{
@@ -101,52 +144,9 @@
"True"
]
},
- "execution_count": 23,
+ "execution_count": 3,
"metadata": {},
"output_type": "execute_result"
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- " st_numinteriorrings replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_addpoint replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_removepoint replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_isring replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_flipcoordinates replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_linesubstring replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_lineinterpolatepoint replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_subdivideexplode replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_subdivide replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_normalizeddifference replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_mean replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_mode replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_fetchregion replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_greaterthan replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_greaterthanequal replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_lessthan replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_lessthanequal replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_addbands replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_subtractbands replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_dividebands replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_multiplyfactor replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_multiplybands replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_bitwiseand replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_bitwiseor replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_count replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_modulo replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_getband replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_squareroot replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_logicaldifference replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_logicalover replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_base64 replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_html replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_array replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function rs_normalize replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_union_aggr replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_envelope_aggr replaced a previously registered function.\n",
- "21/10/08 19:57:20 WARN SimpleFunctionRegistry: The function st_intersection_aggr replaced a previously registered function.\n"
- ]
}
],
"source": [
@@ -169,7 +169,7 @@
},
{
"cell_type": "code",
- "execution_count": 24,
+ "execution_count": 4,
"metadata": {},
"outputs": [
{
@@ -211,7 +211,7 @@
},
{
"cell_type": "code",
- "execution_count": 25,
+ "execution_count": 5,
"metadata": {},
"outputs": [
{
@@ -252,7 +252,7 @@
},
{
"cell_type": "code",
- "execution_count": 26,
+ "execution_count": 6,
"metadata": {},
"outputs": [
{
@@ -293,7 +293,7 @@
},
{
"cell_type": "code",
- "execution_count": 27,
+ "execution_count": 7,
"metadata": {},
"outputs": [
{
@@ -341,7 +341,7 @@
},
{
"cell_type": "code",
- "execution_count": 28,
+ "execution_count": 8,
"metadata": {},
"outputs": [
{
@@ -349,13 +349,26 @@
"output_type": "stream",
"text": [
"== Physical Plan ==\n",
- "DistanceJoin pointshape1#651: geometry, pointshape2#675: geometry, 2.0, false\n",
- ":- Project [st_point(cast(_c0#647 as decimal(24,20)), cast(_c1#648 as decimal(24,20))) AS pointshape1#651, abc AS name1#652]\n",
- ": +- FileScan csv [_c0#647,_c1#648] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[file:/Users/jiayu/GitHub/GeoSpark-datasys-repo/binder/data/testpoint.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string,_c1:string>\n",
- "+- Project [st_point(cast(_c0#671 as decimal(24,20)), cast(_c1#672 as decimal(24,20))) AS pointshape2#675, def AS name2#676]\n",
- " +- FileScan csv [_c0#671,_c1#672] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[file:/Users/jiayu/GitHub/GeoSpark-datasys-repo/binder/data/testpoint.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string,_c1:string>\n",
- "\n",
+ "DistanceJoin pointshape1#259: geometry, pointshape2#283: geometry, 2.0, false\n",
+ ":- Project [st_point(cast(_c0#255 as decimal(24,20)), cast(_c1#256 as decimal(24,20))) AS pointshape1#259, abc AS name1#260]\n",
+ ": +- FileScan csv [_c0#255,_c1#256] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[file:/home/jovyan/binder/data/testpoint.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string,_c1:string>\n",
+ "+- Project [st_point(cast(_c0#279 as decimal(24,20)), cast(_c1#280 as decimal(24,20))) AS pointshape2#283, def AS name2#284]\n",
+ " +- FileScan csv [_c0#279,_c1#280] Batched: false, DataFilters: [], Format: CSV, Location: InMemoryFileIndex[file:/home/jovyan/binder/data/testpoint.csv], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<_c0:string,_c1:string>\n",
"\n",
+ "\n"
+ ]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "22/08/28 05:00:01 WARN JoinQuery: UseIndex is true, but no index exists. Will build index on the fly.\n"
+ ]
+ },
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
"+-----------------+-----+-----------------+-----+\n",
"| pointshape1|name1| pointshape2|name2|\n",
"+-----------------+-----+-----------------+-----+\n",
@@ -368,13 +381,6 @@
"only showing top 5 rows\n",
"\n"
]
- },
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "21/10/08 19:57:21 WARN JoinQuery: UseIndex is true, but no index exists. Will build index on the fly.\n"
- ]
}
],
"source": [
@@ -418,9 +424,18 @@
},
{
"cell_type": "code",
- "execution_count": 29,
+ "execution_count": 9,
"metadata": {},
- "outputs": [],
+ "outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "/srv/conda/envs/notebook/lib/python3.7/site-packages/geopandas/geodataframe.py:35: ShapelyDeprecationWarning: The array interface is deprecated and will no longer work in Shapely 2.0. Convert the '.coords' to a numpy array instead.\n",
+ " out = from_shapely(data)\n"
+ ]
+ }
+ ],
"source": [
"gdf = gpd.read_file(\"data/gis_osm_pois_free_1.shp\")\n",
"\n",
@@ -431,7 +446,7 @@
},
{
"cell_type": "code",
- "execution_count": 30,
+ "execution_count": 10,
"metadata": {},
"outputs": [
{
@@ -454,9 +469,16 @@
},
{
"cell_type": "code",
- "execution_count": 31,
+ "execution_count": 11,
"metadata": {},
"outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "[Stage 17:> (0 + 1) / 1]\r"
+ ]
+ },
{
"name": "stdout",
"output_type": "stream",
@@ -473,6 +495,13 @@
"only showing top 5 rows\n",
"\n"
]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ " \r"
+ ]
}
],
"source": [
@@ -481,7 +510,7 @@
},
{
"cell_type": "code",
- "execution_count": 32,
+ "execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
@@ -490,7 +519,7 @@
},
{
"cell_type": "code",
- "execution_count": 33,
+ "execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
@@ -507,9 +536,16 @@
},
{
"cell_type": "code",
- "execution_count": 34,
+ "execution_count": 14,
"metadata": {},
"outputs": [
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "[Stage 18:> (0 + 1) / 1]\r"
+ ]
+ },
{
"name": "stdout",
"output_type": "stream",
@@ -526,6 +562,13 @@
"only showing top 5 rows\n",
"\n"
]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ " \r"
+ ]
}
],
"source": [
@@ -534,7 +577,7 @@
},
{
"cell_type": "code",
- "execution_count": 35,
+ "execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
@@ -543,7 +586,7 @@
},
{
"cell_type": "code",
- "execution_count": 36,
+ "execution_count": 16,
"metadata": {},
"outputs": [],
"source": [
@@ -558,14 +601,15 @@
},
{
"cell_type": "code",
- "execution_count": 37,
+ "execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "21/10/08 19:57:23 WARN JoinQuery: UseIndex is true, but no index exists. Will build index on the fly.\n"
+ "22/08/28 05:00:09 WARN JoinQuery: UseIndex is true, but no index exists. Will build index on the fly.\n",
+ "[Stage 22:=============================> (1 + 1) / 2]\r"
]
},
{
@@ -575,6 +619,16 @@
"+----------+---------+--------------------+\n",
"| id_1| id_2| geom|\n",
"+----------+---------+--------------------+\n",
+ "| 26860294| 26860294|POINT (-3341183.9...|\n",
+ "| 29947493| 29947493|POINT (-3320466.5...|\n",
+ "|4165181885| 29947498|POINT (-3323204.4...|\n",
+ "|5818905324| 29947498|POINT (-3323210.6...|\n",
+ "| 29947498| 29947498|POINT (-3323205.7...|\n",
+ "| 29947499| 29947499|POINT (-3323655.1...|\n",
+ "| 30077461| 29947499|POINT (-3323697.1...|\n",
+ "| 29947505| 29947505|POINT (-3330369.2...|\n",
+ "| 29947499| 30077461|POINT (-3323655.1...|\n",
+ "| 30077461| 30077461|POINT (-3323697.1...|\n",
"| 197624402|197624402|POINT (-3383818.5...|\n",
"| 197663196|197663196|POINT (-3383367.1...|\n",
"| 197953474|197953474|POINT (-3383763.3...|\n",
@@ -584,21 +638,18 @@
"|1074232906|270281140|POINT (-3385408.6...|\n",
"| 270306609|270306609|POINT (-3383982.8...|\n",
"| 270306746|270306746|POINT (-3383898.4...|\n",
- "| 293896571|293896571|POINT (-3385029.0...|\n",
- "|3256728465|293896571|POINT (-3385002.4...|\n",
- "| 360178884|360178884|POINT (-3377483.1...|\n",
- "| 360178897|360178897|POINT (-3374350.0...|\n",
- "| 360178897|360178897|POINT (-3374350.0...|\n",
- "|5546280698|360178897|POINT (-3374344.9...|\n",
- "|5546280699|360178897|POINT (-3374339.7...|\n",
- "| 360178897|360178897|POINT (-3374350.0...|\n",
- "| 360178897|360178897|POINT (-3374350.0...|\n",
- "|5546280698|360178897|POINT (-3374344.9...|\n",
- "|5546280699|360178897|POINT (-3374339.7...|\n",
+ "| 273101780|273101780|POINT (-3389705.7...|\n",
"+----------+---------+--------------------+\n",
"only showing top 20 rows\n",
"\n"
]
+ },
+ {
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ " \r"
+ ]
}
],
"source": [
@@ -614,14 +665,14 @@
},
{
"cell_type": "code",
- "execution_count": 38,
+ "execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
- "21/10/08 19:57:23 WARN JoinQuery: UseIndex is true, but no index exists. Will build index on the fly.\n",
+ "22/08/28 05:00:15 WARN JoinQuery: UseIndex is true, but no index exists. Will build index on the fly.\n",
" \r"
]
}
@@ -632,7 +683,7 @@
},
{
"cell_type": "code",
- "execution_count": 39,
+ "execution_count": 19,
"metadata": {},
"outputs": [],
"source": [
@@ -641,7 +692,7 @@
},
{
"cell_type": "code",
- "execution_count": 40,
+ "execution_count": 20,
"metadata": {},
"outputs": [
{
@@ -673,33 +724,33 @@
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
- " <td>197624402</td>\n",
- " <td>197624402</td>\n",
- " <td>POINT (-3383818.580 4179182.169)</td>\n",
+ " <td>26860294</td>\n",
+ " <td>26860294</td>\n",
+ " <td>POINT (-3341183.976 4318356.064)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
- " <td>197663196</td>\n",
- " <td>197663196</td>\n",
- " <td>POINT (-3383367.151 4179427.096)</td>\n",
+ " <td>29947493</td>\n",
+ " <td>29947493</td>\n",
+ " <td>POINT (-3320466.547 4265941.760)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
- " <td>197953474</td>\n",
- " <td>197953474</td>\n",
- " <td>POINT (-3383763.332 4179408.785)</td>\n",
+ " <td>4165181885</td>\n",
+ " <td>29947498</td>\n",
+ " <td>POINT (-3323204.491 4266510.379)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
- " <td>262310516</td>\n",
- " <td>262310516</td>\n",
- " <td>POINT (-3384257.682 4178033.053)</td>\n",
+ " <td>5818905324</td>\n",
+ " <td>29947498</td>\n",
+ " <td>POINT (-3323210.654 4266502.772)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
- " <td>1074233123</td>\n",
- " <td>262310516</td>\n",
- " <td>POINT (-3384262.187 4178036.442)</td>\n",
+ " <td>29947498</td>\n",
+ " <td>29947498</td>\n",
+ " <td>POINT (-3323205.784 4266548.416)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>...</th>\n",
@@ -709,27 +760,27 @@
" </tr>\n",
" <tr>\n",
" <th>45314</th>\n",
- " <td>6617406900</td>\n",
- " <td>6617406900</td>\n",
- " <td>POINT (-3202030.997 4313940.216)</td>\n",
+ " <td>6815618439</td>\n",
+ " <td>6815618435</td>\n",
+ " <td>POINT (-3285827.820 4250345.966)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>45315</th>\n",
- " <td>6617371185</td>\n",
- " <td>6619204985</td>\n",
- " <td>POINT (-3224898.369 4313308.131)</td>\n",
+ " <td>6815618435</td>\n",
+ " <td>6815618439</td>\n",
+ " <td>POINT (-3285831.862 4250347.684)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>45316</th>\n",
- " <td>6619204985</td>\n",
- " <td>6619204985</td>\n",
- " <td>POINT (-3224887.255 4313298.404)</td>\n",
+ " <td>6815618439</td>\n",
+ " <td>6815618439</td>\n",
+ " <td>POINT (-3285827.820 4250345.966)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>45317</th>\n",
- " <td>6736772185</td>\n",
- " <td>6736772185</td>\n",
- " <td>POINT (-3204857.139 4313763.361)</td>\n",
+ " <td>6815883980</td>\n",
+ " <td>6815883980</td>\n",
+ " <td>POINT (-3286165.443 4249818.008)</td>\n",
" </tr>\n",
" <tr>\n",
" <th>45318</th>\n",
@@ -744,22 +795,22 @@
],
"text/plain": [
" id_1 id_2 geom\n",
- "0 197624402 197624402 POINT (-3383818.580 4179182.169)\n",
- "1 197663196 197663196 POINT (-3383367.151 4179427.096)\n",
- "2 197953474 197953474 POINT (-3383763.332 4179408.785)\n",
- "3 262310516 262310516 POINT (-3384257.682 4178033.053)\n",
- "4 1074233123 262310516 POINT (-3384262.187 4178036.442)\n",
+ "0 26860294 26860294 POINT (-3341183.976 4318356.064)\n",
+ "1 29947493 29947493 POINT (-3320466.547 4265941.760)\n",
+ "2 4165181885 29947498 POINT (-3323204.491 4266510.379)\n",
+ "3 5818905324 29947498 POINT (-3323210.654 4266502.772)\n",
+ "4 29947498 29947498 POINT (-3323205.784 4266548.416)\n",
"... ... ... ...\n",
- "45314 6617406900 6617406900 POINT (-3202030.997 4313940.216)\n",
- "45315 6617371185 6619204985 POINT (-3224898.369 4313308.131)\n",
- "45316 6619204985 6619204985 POINT (-3224887.255 4313298.404)\n",
- "45317 6736772185 6736772185 POINT (-3204857.139 4313763.361)\n",
+ "45314 6815618439 6815618435 POINT (-3285827.820 4250345.966)\n",
+ "45315 6815618435 6815618439 POINT (-3285831.862 4250347.684)\n",
+ "45316 6815618439 6815618439 POINT (-3285827.820 4250345.966)\n",
+ "45317 6815883980 6815883980 POINT (-3286165.443 4249818.008)\n",
"45318 6817416704 6817416704 POINT (-3214549.268 4314872.904)\n",
"\n",
"[45319 rows x 3 columns]"
]
},
- "execution_count": 40,
+ "execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
@@ -771,9 +822,9 @@
],
"metadata": {
"kernelspec": {
- "display_name": "apache-sedona",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
- "name": "apache-sedona"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
@@ -785,7 +836,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.7"
+ "version": "3.7.12"
}
},
"nbformat": 4,
diff --git a/binder/ApacheSedonaSQL_SpatialJoin_AirportsPerCountry.ipynb b/binder/ApacheSedonaSQL_SpatialJoin_AirportsPerCountry.ipynb
index 2bde1bf8..0b6f73b7 100644
--- a/binder/ApacheSedonaSQL_SpatialJoin_AirportsPerCountry.ipynb
+++ b/binder/ApacheSedonaSQL_SpatialJoin_AirportsPerCountry.ipynb
@@ -1,5 +1,27 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
@@ -441,9 +463,6 @@
"name": "stderr",
"output_type": "stream",
"text": [
- "\r",
- "[Stage 19:=================================================> (6 + 1) / 7]\r",
- "\r",
" \r"
]
},
@@ -544,9 +563,9 @@
],
"metadata": {
"kernelspec": {
- "display_name": "apache-sedona",
+ "display_name": "Python 3 (ipykernel)",
"language": "python",
- "name": "apache-sedona"
+ "name": "python3"
},
"language_info": {
"codemirror_mode": {
@@ -558,7 +577,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.7"
+ "version": "3.7.12"
}
},
"nbformat": 4,
diff --git a/binder/contrib/ApacheSedonaImageFilter.ipynb b/binder/contrib/ApacheSedonaImageFilter.ipynb
index ed248092..8dea56b1 100644
--- a/binder/contrib/ApacheSedonaImageFilter.ipynb
+++ b/binder/contrib/ApacheSedonaImageFilter.ipynb
@@ -1,17 +1,32 @@
{
"cells": [
{
- "cell_type": "code",
- "execution_count": null,
- "id": "f00c8864",
+ "cell_type": "markdown",
+ "id": "7c57cbe7",
"metadata": {},
- "outputs": [],
- "source": []
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
},
{
"cell_type": "code",
"execution_count": 1,
- "id": "a90d47f8",
+ "id": "12b92e9b",
"metadata": {},
"outputs": [],
"source": [
@@ -29,7 +44,7 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "1184ba86",
+ "id": "238c5977",
"metadata": {},
"outputs": [],
"source": [
@@ -51,7 +66,7 @@
{
"cell_type": "code",
"execution_count": 3,
- "id": "661a3fc3",
+ "id": "b22f111b",
"metadata": {},
"outputs": [
{
@@ -128,7 +143,7 @@
{
"cell_type": "code",
"execution_count": 4,
- "id": "f1eb9469",
+ "id": "104f1bc7",
"metadata": {},
"outputs": [],
"source": [
@@ -140,7 +155,7 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "f88896a0",
+ "id": "ee7e4b6b",
"metadata": {},
"outputs": [
{
@@ -168,7 +183,7 @@
{
"cell_type": "code",
"execution_count": 6,
- "id": "8024ef1e",
+ "id": "6ffb52de",
"metadata": {},
"outputs": [
{
@@ -189,7 +204,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "c567af25",
+ "id": "7cc6f6a1",
"metadata": {},
"outputs": [
{
@@ -211,7 +226,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "6d16bdcc",
+ "id": "565f5033",
"metadata": {},
"outputs": [],
"source": [
@@ -224,7 +239,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "9ad7d109",
+ "id": "8a3e9e3c",
"metadata": {},
"outputs": [],
"source": [
@@ -235,7 +250,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "19ef669a",
+ "id": "8e3c42f5",
"metadata": {},
"outputs": [],
"source": [
@@ -246,7 +261,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "f26ef1a4",
+ "id": "6ca544de",
"metadata": {},
"outputs": [],
"source": [
@@ -257,7 +272,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "98e8350a",
+ "id": "32476a7b",
"metadata": {},
"outputs": [],
"source": [
@@ -268,7 +283,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "5cf6c21b",
+ "id": "fe4cfcc0",
"metadata": {},
"outputs": [],
"source": [
@@ -279,7 +294,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "c2360bad",
+ "id": "593ef04b",
"metadata": {},
"outputs": [],
"source": [
@@ -290,7 +305,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "bde3a7a5",
+ "id": "86714634",
"metadata": {},
"outputs": [],
"source": [
@@ -301,7 +316,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "7315e340",
+ "id": "375c16ba",
"metadata": {},
"outputs": [],
"source": [
@@ -312,7 +327,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "373920b7",
+ "id": "dd4676bd",
"metadata": {},
"outputs": [],
"source": [
@@ -323,7 +338,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "92be3eba",
+ "id": "37fc95a2",
"metadata": {},
"outputs": [],
"source": [
@@ -334,7 +349,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "c52af8bb",
+ "id": "ae8005ac",
"metadata": {},
"outputs": [],
"source": [
@@ -345,7 +360,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "b7e186ca",
+ "id": "21c311e5",
"metadata": {},
"outputs": [],
"source": [
@@ -356,7 +371,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "68b56ea9",
+ "id": "094b971f",
"metadata": {},
"outputs": [],
"source": [
@@ -367,7 +382,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "8ca30b96",
+ "id": "1f55226e",
"metadata": {},
"outputs": [],
"source": [
@@ -378,7 +393,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "67b08806",
+ "id": "dc7c622c",
"metadata": {},
"outputs": [],
"source": [
@@ -389,7 +404,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "ead2d2a0",
+ "id": "41c20289",
"metadata": {},
"outputs": [],
"source": [
@@ -400,7 +415,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "8174502f",
+ "id": "ae3b6c2c",
"metadata": {},
"outputs": [],
"source": [
@@ -411,7 +426,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "4f7cb317",
+ "id": "b69e4460",
"metadata": {},
"outputs": [],
"source": [
@@ -422,7 +437,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "23835487",
+ "id": "43c52163",
"metadata": {},
"outputs": [],
"source": [
@@ -433,7 +448,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "0bd25e2e",
+ "id": "e3b67f3a",
"metadata": {},
"outputs": [],
"source": [
@@ -444,7 +459,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "b0418cb5",
+ "id": "89f857b3",
"metadata": {},
"outputs": [],
"source": [
@@ -460,7 +475,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "63c86f79",
+ "id": "6458aeaa",
"metadata": {},
"outputs": [],
"source": [
@@ -470,7 +485,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "aca271df",
+ "id": "1dc49744",
"metadata": {},
"outputs": [],
"source": [
@@ -492,7 +507,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "211e89c9",
+ "id": "f29f5a94",
"metadata": {},
"outputs": [],
"source": [
@@ -516,7 +531,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "209005e7",
+ "id": "01295f56",
"metadata": {},
"outputs": [],
"source": [
@@ -526,7 +541,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "da5fb78b",
+ "id": "73ec8d8a",
"metadata": {},
"outputs": [],
"source": []
@@ -548,7 +563,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.2"
+ "version": "3.7.12"
}
},
"nbformat": 4,
diff --git a/binder/contrib/DownloadImageFromGEE.ipynb b/binder/contrib/DownloadImageFromGEE.ipynb
index 1acd9c38..9d5a873c 100644
--- a/binder/contrib/DownloadImageFromGEE.ipynb
+++ b/binder/contrib/DownloadImageFromGEE.ipynb
@@ -1,9 +1,32 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "id": "38d5c352",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
- "id": "5ab40e60-e58c-4b50-a9ee-72bd875209ca",
+ "id": "c690240e",
"metadata": {},
"outputs": [],
"source": [
@@ -16,7 +39,7 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "98df74a0-1730-4433-9ec8-8d3d77c54b50",
+ "id": "e0cd62f0",
"metadata": {},
"outputs": [],
"source": [
@@ -26,7 +49,7 @@
{
"cell_type": "code",
"execution_count": 8,
- "id": "ed1bb739-a013-417d-9b2a-566ae89b0292",
+ "id": "eba7a27e",
"metadata": {},
"outputs": [],
"source": [
@@ -40,7 +63,7 @@
{
"cell_type": "code",
"execution_count": 4,
- "id": "ffaa907b-3c69-461c-84a9-7e533ee8589c",
+ "id": "50898a76",
"metadata": {},
"outputs": [
{
@@ -87,7 +110,7 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "2d866b25-8895-4d71-989a-8bde2c2becaa",
+ "id": "89dbb23a",
"metadata": {},
"outputs": [],
"source": [
@@ -109,7 +132,7 @@
{
"cell_type": "code",
"execution_count": 9,
- "id": "68d64f0b-040a-4ccf-980e-0e8f201ce719",
+ "id": "f89fa5a9",
"metadata": {},
"outputs": [
{
@@ -231,7 +254,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "25c9d1be-9a86-4ec5-bd3d-4f23ee2ca024",
+ "id": "9c142a84",
"metadata": {},
"outputs": [],
"source": [
@@ -243,7 +266,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "e0961125-15bf-4ff2-b851-e7b90a177c16",
+ "id": "62199777",
"metadata": {},
"outputs": [],
"source": [
@@ -274,7 +297,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.2"
+ "version": "3.7.12"
}
},
"nbformat": 4,
diff --git a/binder/contrib/NdviSentinelApacheSedona.ipynb b/binder/contrib/NdviSentinelApacheSedona.ipynb
index 97eb8c18..3eb7aa31 100644
--- a/binder/contrib/NdviSentinelApacheSedona.ipynb
+++ b/binder/contrib/NdviSentinelApacheSedona.ipynb
@@ -1,9 +1,32 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "id": "bdcd9d53",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
- "id": "778f8d92-8bf3-4e39-8b2e-42c644af4734",
+ "id": "00514c5b",
"metadata": {},
"outputs": [],
"source": [
@@ -15,7 +38,7 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "7a0ae142-47e9-4e04-8187-3a93baaa2cb5",
+ "id": "92e81634",
"metadata": {},
"outputs": [],
"source": [
@@ -39,7 +62,7 @@
{
"cell_type": "code",
"execution_count": 3,
- "id": "4edd3ce3-7e1f-409b-8a7b-acdeb0659835",
+ "id": "6f504f33",
"metadata": {},
"outputs": [
{
@@ -62,7 +85,7 @@
{
"cell_type": "code",
"execution_count": 4,
- "id": "6b3d9ae8",
+ "id": "d4d8efa5",
"metadata": {},
"outputs": [
{
@@ -148,7 +171,7 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "68018b95",
+ "id": "03300d21",
"metadata": {},
"outputs": [
{
@@ -168,7 +191,7 @@
{
"cell_type": "code",
"execution_count": 6,
- "id": "6c538082-3500-4a2c-ba93-16eb6f141dad",
+ "id": "53550d54",
"metadata": {},
"outputs": [
{
@@ -188,7 +211,7 @@
{
"cell_type": "code",
"execution_count": 7,
- "id": "e2308653-b592-41e4-b495-6eb1efbdd0c1",
+ "id": "0907df80",
"metadata": {},
"outputs": [
{
@@ -223,7 +246,7 @@
{
"cell_type": "code",
"execution_count": 8,
- "id": "de05ddb3-83e8-40ed-b3b4-cbbf90ae08a8",
+ "id": "80ee3d06",
"metadata": {},
"outputs": [
{
@@ -282,7 +305,7 @@
{
"cell_type": "code",
"execution_count": 9,
- "id": "d0017088-c448-48af-97fd-4607db94c0de",
+ "id": "39e61e2a",
"metadata": {},
"outputs": [
{
@@ -314,7 +337,7 @@
{
"cell_type": "code",
"execution_count": 10,
- "id": "25ad0246-fc8f-42ca-866e-209a471c180f",
+ "id": "089277c3",
"metadata": {},
"outputs": [],
"source": [
@@ -329,7 +352,7 @@
{
"cell_type": "code",
"execution_count": 11,
- "id": "0026d557-5926-4759-8c98-5424a0e54b81",
+ "id": "abba167b",
"metadata": {},
"outputs": [],
"source": [
@@ -340,7 +363,7 @@
{
"cell_type": "code",
"execution_count": 12,
- "id": "a0516441-a907-4b68-bf23-a2ad95490eb7",
+ "id": "0c3ae716",
"metadata": {},
"outputs": [],
"source": [
@@ -355,7 +378,7 @@
{
"cell_type": "code",
"execution_count": 13,
- "id": "5b870bc9",
+ "id": "5f9a5f60",
"metadata": {},
"outputs": [
{
@@ -405,7 +428,7 @@
{
"cell_type": "code",
"execution_count": 14,
- "id": "bc0ff12f-f9bf-48bd-b0c4-874446cd4dd5",
+ "id": "9044edaa",
"metadata": {},
"outputs": [
{
@@ -458,7 +481,7 @@
{
"cell_type": "code",
"execution_count": 15,
- "id": "ebfbf8b5-dfac-4a45-a8d8-a3529c6426b7",
+ "id": "767d580c",
"metadata": {},
"outputs": [
{
@@ -508,7 +531,7 @@
{
"cell_type": "code",
"execution_count": 16,
- "id": "c5617a09-a3ba-4769-bbd9-9420ad7d9fb8",
+ "id": "627e4dbe",
"metadata": {},
"outputs": [
{
@@ -559,7 +582,7 @@
{
"cell_type": "code",
"execution_count": 17,
- "id": "1efbe09b-e174-4e95-adca-e788618b4ef0",
+ "id": "33d5ed66",
"metadata": {},
"outputs": [
{
@@ -638,7 +661,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "82026c96-13ec-4555-b6a8-cf5b3072fb50",
+ "id": "156303b2",
"metadata": {},
"outputs": [],
"source": []
@@ -646,7 +669,7 @@
{
"cell_type": "code",
"execution_count": 18,
- "id": "aa2da947-3cb8-4050-aec7-860d24c4cdad",
+ "id": "c7664928",
"metadata": {},
"outputs": [
{
@@ -700,7 +723,7 @@
{
"cell_type": "code",
"execution_count": 35,
- "id": "95d0a7d6-751e-4e01-bae7-52ddd1efc40c",
+ "id": "b3cb84bc",
"metadata": {},
"outputs": [
{
@@ -723,7 +746,7 @@
{
"cell_type": "code",
"execution_count": 20,
- "id": "35bd0426-05c7-4ca5-ba47-9066ff5917d4",
+ "id": "5ae0f827",
"metadata": {},
"outputs": [
{
@@ -745,7 +768,7 @@
{
"cell_type": "code",
"execution_count": 21,
- "id": "0b71708b-2731-43e8-a0fb-fe1c61e5e27f",
+ "id": "aae72e9c",
"metadata": {},
"outputs": [],
"source": [
@@ -755,7 +778,7 @@
{
"cell_type": "code",
"execution_count": 22,
- "id": "2a1e1b63-fad0-4ca0-9a87-e14ac6d19d49",
+ "id": "ed45f9a8",
"metadata": {},
"outputs": [],
"source": [
@@ -775,7 +798,7 @@
{
"cell_type": "code",
"execution_count": 23,
- "id": "03c86d20-0110-4f7c-826f-4a6c2fedec22",
+ "id": "35139bb1",
"metadata": {},
"outputs": [
{
@@ -829,7 +852,7 @@
{
"cell_type": "code",
"execution_count": 24,
- "id": "2f3eea9a-b924-4c36-9c5b-d98f639b1b62",
+ "id": "f5534a95",
"metadata": {},
"outputs": [
{
@@ -850,7 +873,7 @@
{
"cell_type": "code",
"execution_count": 25,
- "id": "977932bc-ad5a-43c9-af05-4c9a8b7ac17e",
+ "id": "9f878429",
"metadata": {},
"outputs": [],
"source": [
@@ -922,7 +945,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "9679e0f3-8ad5-47b4-9dc5-daddd55e2ab4",
+ "id": "cd579ea4",
"metadata": {},
"outputs": [],
"source": []
@@ -930,7 +953,7 @@
{
"cell_type": "code",
"execution_count": 26,
- "id": "e20dac4d-1a4a-4cf8-a674-a32af03184d6",
+ "id": "380226cf",
"metadata": {},
"outputs": [],
"source": [
@@ -954,7 +977,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.2"
+ "version": "3.7.12"
}
},
"nbformat": 4,
diff --git a/binder/contrib/PostgresqlConnectionApacheSedona.ipynb b/binder/contrib/PostgresqlConnectionApacheSedona.ipynb
index 1e552b7d..4cf9f41e 100644
--- a/binder/contrib/PostgresqlConnectionApacheSedona.ipynb
+++ b/binder/contrib/PostgresqlConnectionApacheSedona.ipynb
@@ -1,265 +1,284 @@
{
- "cells": [
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "4130d5ae",
- "metadata": {
- "id": "4130d5ae",
- "outputId": "219ce729-58fd-49bf-e60f-a70d9cbec561"
- },
- "outputs": [
- {
- "name": "stderr",
- "output_type": "stream",
- "text": [
- "21/11/22 14:18:45 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\n",
- "Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties\n",
- "Setting default log level to \"WARN\".\n",
- "To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).\n"
- ]
- }
- ],
- "source": [
- "from pyspark.sql import SparkSession\n",
- "\n",
- "spark = SparkSession.builder.appName(\"db-connection-2\")\\\n",
- " .master(\"spark://spark-master:7077\")\\\n",
- " .config(\"spark.executor.memory\", \"10gb\")\\\n",
- " .config(\"spark.jars\", \"postgresql-42.2.24.jar\") \\\n",
- " .getOrCreate()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "f8c4f31c",
- "metadata": {
- "id": "f8c4f31c"
- },
- "outputs": [],
- "source": [
- "properties = {\"user\":\"\", \"password\":\"\", \"host\":\"\", \"port\":\"\", \"database\":\"\"}\n",
- "properties[\"url\"] = \"jdbc:postgresql://\"+properties[\"host\"]+\":\"+properties[\"port\"]+\"/\"+properties[\"database\"]"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "8926283f",
- "metadata": {
- "id": "8926283f"
- },
- "outputs": [],
- "source": [
- "jdbcDF = spark.read.format(\"jdbc\"). \\\n",
- "options(\n",
- " url=properties[\"url\"], # jdbc:postgresql://<host>:<port>/<database>\n",
- " dbtable='clima.t_indices_prec_cpc',\n",
- " user=properties[\"user\"],\n",
- " password=properties[\"password\"], \n",
- " driver=\"org.postgresql.Driver\") \\\n",
- ".load()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "5911da7c",
- "metadata": {
- "id": "5911da7c",
- "outputId": "a6d30ff5-fdc3-44b4-b8ab-4db922971a30"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "root\n",
- " |-- id_pk: integer (nullable = true)\n",
- " |-- codigo: integer (nullable = true)\n",
- " |-- ano: integer (nullable = true)\n",
- " |-- cdd: decimal(10,2) (nullable = true)\n",
- " |-- prcptot: decimal(10,2) (nullable = true)\n",
- " |-- sdii: decimal(10,2) (nullable = true)\n",
- " |-- r20mm: decimal(10,2) (nullable = true)\n",
- " |-- r30mm: decimal(10,2) (nullable = true)\n",
- " |-- r50mm: decimal(10,2) (nullable = true)\n",
- " |-- r80mm: decimal(10,2) (nullable = true)\n",
- " |-- r100mm: decimal(10,2) (nullable = true)\n",
- " |-- r150mm: decimal(10,2) (nullable = true)\n",
- " |-- rx1day: decimal(10,2) (nullable = true)\n",
- " |-- rx2day: decimal(10,2) (nullable = true)\n",
- " |-- rx5day: decimal(10,2) (nullable = true)\n",
- "\n"
- ]
- }
- ],
- "source": [
- "jdbcDF.printSchema()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "fb56bd25",
- "metadata": {
- "id": "fb56bd25",
- "outputId": "57c4a801-ab4e-4527-9407-11c679612581"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "1.15 ms ± 313 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
- ]
- }
- ],
- "source": [
- "%%timeit\n",
- "jdbcDF.filter(\"r100mm > 2000\")"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "2fea8349",
- "metadata": {
- "id": "2fea8349"
- },
- "outputs": [],
- "source": [
- "spark.stop()"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "87f5b066",
- "metadata": {
- "id": "87f5b066",
- "outputId": "d680d688-493b-46ba-8f6c-65c5621e58cf"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "Requirement already satisfied: psycopg2-binary in /usr/local/lib/python3.9/dist-packages (2.9.2)\n"
- ]
- }
- ],
- "source": [
- "! pip install psycopg2-binary"
- ]
- },
- {
- "cell_type": "code",
- "execution_count": null,
- "id": "ffa730a2",
- "metadata": {
- "id": "ffa730a2"
- },
- "outputs": [],
- "source": [
- "import psycopg2\n",
- "\n",
- "\n",
- "connection = psycopg2.connect(user=properties[\"user\"],\n",
- " password=properties[\"password\"],\n",
- " host=properties[\"host\"],\n",
- " port=properties[\"port\"],\n",
- " database=properties[\"database\"])"
- ]
- },
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "id": "68927e9f",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "2bb2e1bd",
+ "metadata": {
+ "id": "4130d5ae",
+ "outputId": "219ce729-58fd-49bf-e60f-a70d9cbec561"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "id": "b8321ee1",
- "metadata": {
- "id": "b8321ee1",
- "outputId": "8a78e901-0689-40df-e1d7-346eb4d8afaa"
- },
- "outputs": [
- {
- "name": "stdout",
- "output_type": "stream",
- "text": [
- "2.57 ms ± 125 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
- ]
- }
- ],
- "source": [
- "%%timeit\n",
- "cursor = connection.cursor()\n",
- "cursor.execute(\"SELECT * FROM clima.t_indices_prec_cpc t Where r100mm > 2000\")\n",
- "names = [ x[0] for x in cursor.description]\n",
- "result = cursor.fetchall()\n",
- "from pandas import DataFrame\n",
- "df = DataFrame(result, columns=names)"
- ]
- },
+ "name": "stderr",
+ "output_type": "stream",
+ "text": [
+ "21/11/22 14:18:45 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\n",
+ "Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties\n",
+ "Setting default log level to \"WARN\".\n",
+ "To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).\n"
+ ]
+ }
+ ],
+ "source": [
+ "from pyspark.sql import SparkSession\n",
+ "\n",
+ "spark = SparkSession.builder.appName(\"db-connection-2\")\\\n",
+ " .master(\"spark://spark-master:7077\")\\\n",
+ " .config(\"spark.executor.memory\", \"10gb\")\\\n",
+ " .config(\"spark.jars\", \"postgresql-42.2.24.jar\") \\\n",
+ " .getOrCreate()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "3127d4de",
+ "metadata": {
+ "id": "f8c4f31c"
+ },
+ "outputs": [],
+ "source": [
+ "properties = {\"user\":\"\", \"password\":\"\", \"host\":\"\", \"port\":\"\", \"database\":\"\"}\n",
+ "properties[\"url\"] = \"jdbc:postgresql://\"+properties[\"host\"]+\":\"+properties[\"port\"]+\"/\"+properties[\"database\"]"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "0981f97c",
+ "metadata": {
+ "id": "8926283f"
+ },
+ "outputs": [],
+ "source": [
+ "jdbcDF = spark.read.format(\"jdbc\"). \\\n",
+ "options(\n",
+ " url=properties[\"url\"], # jdbc:postgresql://<host>:<port>/<database>\n",
+ " dbtable='clima.t_indices_prec_cpc',\n",
+ " user=properties[\"user\"],\n",
+ " password=properties[\"password\"], \n",
+ " driver=\"org.postgresql.Driver\") \\\n",
+ ".load()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "9f813c79",
+ "metadata": {
+ "id": "5911da7c",
+ "outputId": "a6d30ff5-fdc3-44b4-b8ab-4db922971a30"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "id": "da2386c5",
- "metadata": {
- "id": "da2386c5"
- },
- "outputs": [],
- "source": [
- "connection.close()"
- ]
- },
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "root\n",
+ " |-- id_pk: integer (nullable = true)\n",
+ " |-- codigo: integer (nullable = true)\n",
+ " |-- ano: integer (nullable = true)\n",
+ " |-- cdd: decimal(10,2) (nullable = true)\n",
+ " |-- prcptot: decimal(10,2) (nullable = true)\n",
+ " |-- sdii: decimal(10,2) (nullable = true)\n",
+ " |-- r20mm: decimal(10,2) (nullable = true)\n",
+ " |-- r30mm: decimal(10,2) (nullable = true)\n",
+ " |-- r50mm: decimal(10,2) (nullable = true)\n",
+ " |-- r80mm: decimal(10,2) (nullable = true)\n",
+ " |-- r100mm: decimal(10,2) (nullable = true)\n",
+ " |-- r150mm: decimal(10,2) (nullable = true)\n",
+ " |-- rx1day: decimal(10,2) (nullable = true)\n",
+ " |-- rx2day: decimal(10,2) (nullable = true)\n",
+ " |-- rx5day: decimal(10,2) (nullable = true)\n",
+ "\n"
+ ]
+ }
+ ],
+ "source": [
+ "jdbcDF.printSchema()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "96e44ce9",
+ "metadata": {
+ "id": "fb56bd25",
+ "outputId": "57c4a801-ab4e-4527-9407-11c679612581"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "id": "cd53e548",
- "metadata": {
- "id": "cd53e548"
- },
- "outputs": [],
- "source": [
- ""
- ]
- },
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "1.15 ms ± 313 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
+ ]
+ }
+ ],
+ "source": [
+ "%%timeit\n",
+ "jdbcDF.filter(\"r100mm > 2000\")"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "8e6e1524",
+ "metadata": {
+ "id": "2fea8349"
+ },
+ "outputs": [],
+ "source": [
+ "spark.stop()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "1f1dce42",
+ "metadata": {
+ "id": "87f5b066",
+ "outputId": "d680d688-493b-46ba-8f6c-65c5621e58cf"
+ },
+ "outputs": [
{
- "cell_type": "code",
- "execution_count": null,
- "id": "3e1ba36d",
- "metadata": {
- "id": "3e1ba36d"
- },
- "outputs": [],
- "source": [
- ""
- ]
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "Requirement already satisfied: psycopg2-binary in /usr/local/lib/python3.9/dist-packages (2.9.2)\n"
+ ]
}
- ],
- "metadata": {
- "kernelspec": {
- "display_name": "Python 3 (ipykernel)",
- "language": "python",
- "name": "python3"
- },
- "language_info": {
- "codemirror_mode": {
- "name": "ipython",
- "version": 3
- },
- "file_extension": ".py",
- "mimetype": "text/x-python",
- "name": "python",
- "nbconvert_exporter": "python",
- "pygments_lexer": "ipython3",
- "version": "3.9.2"
- },
- "colab": {
- "name": "Example3 (3).ipynb",
- "provenance": []
+ ],
+ "source": [
+ "! pip install psycopg2-binary"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "dde87430",
+ "metadata": {
+ "id": "ffa730a2"
+ },
+ "outputs": [],
+ "source": [
+ "import psycopg2\n",
+ "\n",
+ "\n",
+ "connection = psycopg2.connect(user=properties[\"user\"],\n",
+ " password=properties[\"password\"],\n",
+ " host=properties[\"host\"],\n",
+ " port=properties[\"port\"],\n",
+ " database=properties[\"database\"])"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ae4910fa",
+ "metadata": {
+ "id": "b8321ee1",
+ "outputId": "8a78e901-0689-40df-e1d7-346eb4d8afaa"
+ },
+ "outputs": [
+ {
+ "name": "stdout",
+ "output_type": "stream",
+ "text": [
+ "2.57 ms ± 125 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
+ ]
}
+ ],
+ "source": [
+ "%%timeit\n",
+ "cursor = connection.cursor()\n",
+ "cursor.execute(\"SELECT * FROM clima.t_indices_prec_cpc t Where r100mm > 2000\")\n",
+ "names = [ x[0] for x in cursor.description]\n",
+ "result = cursor.fetchall()\n",
+ "from pandas import DataFrame\n",
+ "df = DataFrame(result, columns=names)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "7c8580f5",
+ "metadata": {
+ "id": "da2386c5"
+ },
+ "outputs": [],
+ "source": [
+ "connection.close()"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "ee849ad5",
+ "metadata": {
+ "id": "cd53e548"
+ },
+ "outputs": [],
+ "source": []
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "id": "a4015ad6",
+ "metadata": {
+ "id": "3e1ba36d"
+ },
+ "outputs": [],
+ "source": []
+ }
+ ],
+ "metadata": {
+ "colab": {
+ "name": "Example3 (3).ipynb",
+ "provenance": []
+ },
+ "kernelspec": {
+ "display_name": "Python 3 (ipykernel)",
+ "language": "python",
+ "name": "python3"
},
- "nbformat": 4,
- "nbformat_minor": 5
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.7.12"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
}
diff --git a/binder/contrib/VectorAnalisisApacheSedona.ipynb b/binder/contrib/VectorAnalisisApacheSedona.ipynb
index f1606590..873b49d9 100644
--- a/binder/contrib/VectorAnalisisApacheSedona.ipynb
+++ b/binder/contrib/VectorAnalisisApacheSedona.ipynb
@@ -1,9 +1,32 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "id": "3f8580a6",
+ "metadata": {},
+ "source": [
+ "```\n",
+ "Licensed to the Apache Software Foundation (ASF) under one\n",
+ "or more contributor license agreements. See the NOTICE file\n",
+ "distributed with this work for additional information\n",
+ "regarding copyright ownership. The ASF licenses this file\n",
+ "to you under the Apache License, Version 2.0 (the\n",
+ "\"License\"); you may not use this file except in compliance\n",
+ "with the License. You may obtain a copy of the License at\n",
+ " http://www.apache.org/licenses/LICENSE-2.0\n",
+ "Unless required by applicable law or agreed to in writing,\n",
+ "software distributed under the License is distributed on an\n",
+ "\"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
+ "KIND, either express or implied. See the License for the\n",
+ "specific language governing permissions and limitations\n",
+ "under the License.\n",
+ "```"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
- "id": "5b4f4a55-9ac4-4591-a925-ba032344662a",
+ "id": "216f0262",
"metadata": {},
"outputs": [],
"source": [
@@ -29,7 +52,7 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "caf280aa-c891-4729-900f-824141ab7c03",
+ "id": "4d87582c",
"metadata": {},
"outputs": [
{
@@ -115,7 +138,7 @@
{
"cell_type": "code",
"execution_count": 3,
- "id": "0b3b68fd-d9a4-4c95-bf45-8f8f2d995f7d",
+ "id": "bb49563e",
"metadata": {
"tags": []
},
@@ -163,7 +186,7 @@
{
"cell_type": "code",
"execution_count": 4,
- "id": "be17fe8a-17ce-4c78-8103-0e970fc630ef",
+ "id": "15f254fb",
"metadata": {},
"outputs": [
{
@@ -182,7 +205,7 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "c8fa4b42-5a7c-45c4-9b1f-129e4438b883",
+ "id": "e9d3a649",
"metadata": {},
"outputs": [
{
@@ -292,7 +315,7 @@
{
"cell_type": "code",
"execution_count": 6,
- "id": "b3b1560a-27b7-4074-8b6b-efb0cad05479",
+ "id": "4c886014",
"metadata": {},
"outputs": [
{
@@ -485,7 +508,7 @@
{
"cell_type": "code",
"execution_count": 7,
- "id": "fc3c7891-3474-49fd-b574-8200e2d93c5f",
+ "id": "f7cf4991",
"metadata": {},
"outputs": [
{
@@ -504,7 +527,7 @@
{
"cell_type": "code",
"execution_count": 8,
- "id": "4ef77f2d-136f-4881-97d4-c73d8c33491e",
+ "id": "4c6d1dcd",
"metadata": {
"tags": []
},
@@ -613,7 +636,7 @@
{
"cell_type": "code",
"execution_count": 9,
- "id": "78f6553a-61d7-4273-b027-675c6bd36cc4",
+ "id": "6699c6b5",
"metadata": {
"tags": []
},
@@ -640,7 +663,7 @@
{
"cell_type": "code",
"execution_count": 10,
- "id": "9032cb75-c459-49b8-a727-6946a068a0dd",
+ "id": "821ea4ea",
"metadata": {},
"outputs": [
{
@@ -840,7 +863,7 @@
{
"cell_type": "code",
"execution_count": 11,
- "id": "c59c3782-6b49-40f1-a1aa-6e54cacdf196",
+ "id": "71007b9b",
"metadata": {},
"outputs": [
{
@@ -1050,7 +1073,7 @@
{
"cell_type": "code",
"execution_count": 12,
- "id": "a918ed65-2778-4cb5-baf4-b9b8a74deb4c",
+ "id": "7fbee596",
"metadata": {},
"outputs": [
{
@@ -1099,7 +1122,7 @@
{
"cell_type": "code",
"execution_count": 13,
- "id": "0e9a7675-f771-4618-98f3-d15352dbc7ae",
+ "id": "aff07ad2",
"metadata": {},
"outputs": [],
"source": [
@@ -1109,7 +1132,7 @@
{
"cell_type": "code",
"execution_count": null,
- "id": "d280dc3d-6322-42df-82e7-0bd5cb0c0852",
+ "id": "9cb5f8a3",
"metadata": {},
"outputs": [],
"source": []
@@ -1131,7 +1154,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
- "version": "3.9.2"
+ "version": "3.7.12"
}
},
"nbformat": 4,