You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@sedona.apache.org by ji...@apache.org on 2021/09/21 07:44:55 UTC
[incubator-sedona] branch master updated: [SEDONA-31] Rename
sparklyr.sedona to apache.sedona (#541)
This is an automated email from the ASF dual-hosted git repository.
jiayu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-sedona.git
The following commit(s) were added to refs/heads/master by this push:
new 8d90799 [SEDONA-31] Rename sparklyr.sedona to apache.sedona (#541)
8d90799 is described below
commit 8d9079970941ad050b3beab934b31340186793c2
Author: Jia Yu <ji...@apache.org>
AuthorDate: Tue Sep 21 00:44:49 2021 -0700
[SEDONA-31] Rename sparklyr.sedona to apache.sedona (#541)
* Rename sparklyr.sedona to apache.sedona
* Fix a typo
* Enable fail-false
* Fix the path
---
.github/workflows/r.yml | 22 ++++++------
.github/workflows/scripts/install_r_deps.sh | 2 +-
.gitignore | 1 +
R/{sparklyr.sedona => }/.Rbuildignore | 0
R/{sparklyr.sedona => }/.gitignore | 0
R/{sparklyr.sedona => }/DESCRIPTION | 8 ++---
R/{sparklyr.sedona => }/NAMESPACE | 0
R/{sparklyr.sedona => }/R/bounding_box.R | 2 +-
R/{sparklyr.sedona => }/R/data_interface.R | 22 ++++++------
R/{sparklyr.sedona => }/R/dependencies.R | 2 +-
R/{sparklyr.sedona => }/R/imports.R | 0
R/{sparklyr.sedona => }/R/sdf_interface.R | 4 +--
R/{sparklyr.sedona => }/R/spatial_index_op.R | 2 +-
R/{sparklyr.sedona => }/R/spatial_join_op.R | 4 +--
R/{sparklyr.sedona => }/R/spatial_partitioner.R | 2 +-
R/{sparklyr.sedona => }/R/spatial_queries.R | 4 +--
R/{sparklyr.sedona => }/R/spatial_rdd.R | 8 ++---
R/{sparklyr.sedona => }/R/viz.R | 6 ++--
R/{sparklyr.sedona => }/README.md | 38 ++++++++++-----------
R/{sparklyr.sedona => }/docs/choropleth-map.png | Bin
R/{sparklyr.sedona => }/tests/testthat.R | 4 +--
.../tests/testthat/helper-initialize.R | 1 -
.../testthat/test-apply-spatial-partitioner.R | 0
.../tests/testthat/test-bounding-box.R | 0
.../tests/testthat/test-build-index.R | 0
.../tests/testthat/test-crs-transform.R | 0
.../tests/testthat/test-data-interface.R | 2 +-
.../tests/testthat/test-dbplyr-integration.R | 0
R/{sparklyr.sedona => }/tests/testthat/test-init.R | 0
.../tests/testthat/test-sdf-interface.R | 0
.../tests/testthat/test-spatial-join.R | 0
.../tests/testthat/test-spatial-queries.R | 0
.../testthat/test-spatial-rdd-aggregation-fns.R | 0
R/{sparklyr.sedona => }/tests/testthat/test-viz.R | 2 +-
R/{sparklyr.sedona => }/vignettes/.gitignore | 0
.../vignettes/connecting-to-spark.Rmd | 8 ++---
.../vignettes/images/choropleth-map.png | 0
.../vignettes/intro-to-spatial-rdd.Rmd | 10 +++---
R/{sparklyr.sedona => }/vignettes/overview.Rmd | 8 ++---
.../vignettes/spark-dataframes-and-beyond.Rmd | 6 ++--
R/{sparklyr.sedona => }/vignettes/viz.Rmd | 10 +++---
41 files changed, 89 insertions(+), 89 deletions(-)
diff --git a/.github/workflows/r.yml b/.github/workflows/r.yml
index 51703d1..f7c177a 100644
--- a/.github/workflows/r.yml
+++ b/.github/workflows/r.yml
@@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-18.04
strategy:
- fail-fast: false
+ fail-fast: true
matrix:
spark: [2.4.7, 3.0.2, 3.1.1]
scala: [2.11.8, 2.12.8]
@@ -70,7 +70,7 @@ jobs:
run: |
print(R.version)
install.packages("remotes")
- saveRDS(remotes::dev_package_deps("./R/sparklyr.sedona/", dependencies = TRUE), ".github/deps.Rds", version = 2)
+ saveRDS(remotes::dev_package_deps("./R/", dependencies = TRUE), ".github/deps.Rds", version = 2)
writeLines(sprintf("R-%i.%i", getRversion()$major, getRversion()$minor), ".github/R-version")
shell: Rscript {0}
- name: Get OS name
@@ -84,14 +84,14 @@ jobs:
uses: actions/cache@master
with:
path: ${{ env.R_LIBS_USER }}
- key: sparklyr.sedona-${{ steps.os-name.outputs.os-name }}-${{ hashFiles('.github/R-version') }}-${{ hashFiles('.github/deps.Rds') }}
- restore-keys: sparklyr.sedona-${{ steps.os-name.outputs.os-name }}-${{ hashFiles('.github/R-version') }}
+ key: apache.sedona-${{ steps.os-name.outputs.os-name }}-${{ hashFiles('.github/R-version') }}-${{ hashFiles('.github/deps.Rds') }}
+ restore-keys: apache.sedona-${{ steps.os-name.outputs.os-name }}-${{ hashFiles('.github/R-version') }}
- name: Cache Spark installations
if: runner.os != 'Windows'
uses: actions/cache@master
with:
path: ~/spark
- key: sparklyr.sedona-apache-spark-${{ steps.os-name.outputs.os-name }}-${{ env.SPARK_VERSION }}
+ key: apache.sedona-apache-spark-${{ steps.os-name.outputs.os-name }}-${{ env.SPARK_VERSION }}
- name: Install system dependencies
run: source ./.github/workflows/scripts/install_system_deps_for_r_build.sh
- name: Install R dependencies
@@ -112,22 +112,22 @@ jobs:
- name: Build R package
env:
WARNINGS_ARE_ERRORS: 1
- run: R CMD build --no-build-vignettes ./R/sparklyr.sedona
+ run: R CMD build --no-build-vignettes ./R
- name: Check
env:
_R_CHECK_FORCE_SUGGESTS_: false
WARNINGS_ARE_ERRORS: 1
run: |
- R CMD check --no-build-vignettes --no-manual --no-tests sparklyr.sedona*.tar.gz
- rm -f sparklyr.sedona*.tar.gz
- - name: Install sparklyr.sedona from source
- run: Rscript -e 'install.packages("./R/sparklyr.sedona/", repos = NULL, type = "source")'
+ R CMD check --no-build-vignettes --no-manual --no-tests apache.sedona*.tar.gz
+ rm -f apache.sedona*.tar.gz
+ - name: Install apache.sedona from source
+ run: Rscript -e 'install.packages("./R/", repos = NULL, type = "source")'
- name: Run tests
run: |
export SPARKLYR_LOG_FILE='/tmp/sparklyr.log'
source ./.github/workflows/scripts/prepare_sparklyr_sedona_test_env.sh
echo "Apache Sedona jar files: ${SEDONA_JAR_FILES}"
- cd ./R/sparklyr.sedona/tests
+ cd ./R/tests
NOT_CRAN='true' Rscript testthat.R
shell: bash
- uses: actions/upload-artifact@v1
diff --git a/.github/workflows/scripts/install_r_deps.sh b/.github/workflows/scripts/install_r_deps.sh
index 7959a65..5bc021a 100644
--- a/.github/workflows/scripts/install_r_deps.sh
+++ b/.github/workflows/scripts/install_r_deps.sh
@@ -8,7 +8,7 @@ SEP='"\\s+"'
R_REMOTES_NO_ERRORS_FROM_WARNINGS=true Rscript - <<_RSCRIPT_EOF_
if (!require(remotes))
install.packages("remotes")
- remotes::install_deps(pkgdir = "./R/sparklyr.sedona", dependencies = c("Imports"), upgrade = "always")
+ remotes::install_deps(pkgdir = "./R", dependencies = c("Imports"), upgrade = "always")
test_deps <- strsplit("$TEST_DEPS", $SEP)[[1]]
for (pkg in test_deps)
diff --git a/.gitignore b/.gitignore
index 2eee0df..b357a08 100644
--- a/.gitignore
+++ b/.gitignore
@@ -13,3 +13,4 @@
/.bloop/
/.metals/
/.vscode/
+.Rproj.user
diff --git a/R/sparklyr.sedona/.Rbuildignore b/R/.Rbuildignore
similarity index 100%
rename from R/sparklyr.sedona/.Rbuildignore
rename to R/.Rbuildignore
diff --git a/R/sparklyr.sedona/.gitignore b/R/.gitignore
similarity index 100%
rename from R/sparklyr.sedona/.gitignore
rename to R/.gitignore
diff --git a/R/sparklyr.sedona/DESCRIPTION b/R/DESCRIPTION
similarity index 81%
rename from R/sparklyr.sedona/DESCRIPTION
rename to R/DESCRIPTION
index b1a5529..8c936e5 100644
--- a/R/sparklyr.sedona/DESCRIPTION
+++ b/R/DESCRIPTION
@@ -1,7 +1,7 @@
Type: Package
-Package: sparklyr.sedona
-Title: Sparklyr Extension for Apache Sedona
-Version: 0.1.0
+Package: apache.sedona
+Title: R Interface for Apache Sedona
+Version: 1.1.0
Authors@R:
c(person(family = "Apache Sedona",
role = c("aut", "cre"),
@@ -11,7 +11,7 @@ Authors@R:
person(family = "RStudio",
role = "cph"))
Maintainer: Apache Sedona <pr...@sedona.apache.org>
-Description: This sparklyr extension is an R interface for Apache Sedona
+Description: This extension is an R interface for Apache Sedona built on sparklyr
(<https://sedona.apache.org>).
License: Apache License 2.0
Depends:
diff --git a/R/sparklyr.sedona/NAMESPACE b/R/NAMESPACE
similarity index 100%
rename from R/sparklyr.sedona/NAMESPACE
rename to R/NAMESPACE
diff --git a/R/sparklyr.sedona/R/bounding_box.R b/R/R/bounding_box.R
similarity index 98%
rename from R/sparklyr.sedona/R/bounding_box.R
rename to R/R/bounding_box.R
index 82e6b5c..eb2336c 100644
--- a/R/sparklyr.sedona/R/bounding_box.R
+++ b/R/R/bounding_box.R
@@ -27,7 +27,7 @@
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#' bb <- new_bounding_box(sc, -1, 1, -1, 1)
diff --git a/R/sparklyr.sedona/R/data_interface.R b/R/R/data_interface.R
similarity index 98%
rename from R/sparklyr.sedona/R/data_interface.R
rename to R/R/data_interface.R
index 7085327..4976a42 100644
--- a/R/sparklyr.sedona/R/data_interface.R
+++ b/R/R/data_interface.R
@@ -54,7 +54,7 @@ NULL
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -151,7 +151,7 @@ sedona_read_dsv_to_typed_rdd <- function(sc,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -187,7 +187,7 @@ sedona_read_shapefile_to_typed_rdd <- function(sc,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -232,7 +232,7 @@ sedona_read_geojson_to_typed_rdd <- function(sc,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -282,7 +282,7 @@ sedona_read_geojson <- function(sc,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -337,7 +337,7 @@ sedona_read_wkb <- function(sc,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect("spark://HOST:PORT")
#'
@@ -386,7 +386,7 @@ sedona_read_wkt <- function(sc,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -428,7 +428,7 @@ NULL
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -455,7 +455,7 @@ sedona_write_wkb <- function(x, output_location) {
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect("spark://HOST:PORT")
#'
@@ -482,7 +482,7 @@ sedona_write_wkt <- function(x, output_location) {
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -513,7 +513,7 @@ sedona_write_geojson <- function(x, output_location) {
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/R/dependencies.R b/R/R/dependencies.R
similarity index 99%
rename from R/sparklyr.sedona/R/dependencies.R
rename to R/R/dependencies.R
index 1148fdb..74c9773 100644
--- a/R/sparklyr.sedona/R/dependencies.R
+++ b/R/R/dependencies.R
@@ -27,7 +27,7 @@ spark_dependencies <- function(spark_version, scala_version, ...) {
}
packages <- c(
- "org.datasyslab:geotools-wrapper:geotools-24.0",
+ "org.datasyslab:geotools-wrapper:geotools-24.1",
"org.datasyslab:sernetcdf:0.1.0",
"org.locationtech.jts:jts-core:1.18.0",
"org.wololo:jts2geojson:0.14.3"
diff --git a/R/sparklyr.sedona/R/imports.R b/R/R/imports.R
similarity index 100%
rename from R/sparklyr.sedona/R/imports.R
rename to R/R/imports.R
diff --git a/R/sparklyr.sedona/R/sdf_interface.R b/R/R/sdf_interface.R
similarity index 98%
rename from R/sparklyr.sedona/R/sdf_interface.R
rename to R/R/sdf_interface.R
index f3c4631..4d8fa39 100644
--- a/R/sparklyr.sedona/R/sdf_interface.R
+++ b/R/R/sdf_interface.R
@@ -32,7 +32,7 @@ NULL
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -61,7 +61,7 @@ sdf_register.spatial_rdd <- function(x, name = NULL) {
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/R/spatial_index_op.R b/R/R/spatial_index_op.R
similarity index 98%
rename from R/sparklyr.sedona/R/spatial_index_op.R
rename to R/R/spatial_index_op.R
index 2398ad6..0dad96b 100644
--- a/R/sparklyr.sedona/R/spatial_index_op.R
+++ b/R/R/spatial_index_op.R
@@ -32,7 +32,7 @@
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/R/spatial_join_op.R b/R/R/spatial_join_op.R
similarity index 99%
rename from R/sparklyr.sedona/R/spatial_join_op.R
rename to R/R/spatial_join_op.R
index 959f1d4..e0797b6 100644
--- a/R/sparklyr.sedona/R/spatial_join_op.R
+++ b/R/R/spatial_join_op.R
@@ -57,7 +57,7 @@ NULL
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -122,7 +122,7 @@ sedona_spatial_join <- function(spatial_rdd,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/R/spatial_partitioner.R b/R/R/spatial_partitioner.R
similarity index 99%
rename from R/sparklyr.sedona/R/spatial_partitioner.R
rename to R/R/spatial_partitioner.R
index 2d6a077..e31b690 100644
--- a/R/sparklyr.sedona/R/spatial_partitioner.R
+++ b/R/R/spatial_partitioner.R
@@ -37,7 +37,7 @@
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/R/spatial_queries.R b/R/R/spatial_queries.R
similarity index 99%
rename from R/sparklyr.sedona/R/spatial_queries.R
rename to R/R/spatial_queries.R
index 004d350..e90fefc 100644
--- a/R/sparklyr.sedona/R/spatial_queries.R
+++ b/R/R/spatial_queries.R
@@ -50,7 +50,7 @@ NULL
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -135,7 +135,7 @@ sedona_knn_query <- function(rdd,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/R/spatial_rdd.R b/R/R/spatial_rdd.R
similarity index 97%
rename from R/sparklyr.sedona/R/spatial_rdd.R
rename to R/R/spatial_rdd.R
index 323201e..5d8d3c2 100644
--- a/R/sparklyr.sedona/R/spatial_rdd.R
+++ b/R/R/spatial_rdd.R
@@ -29,7 +29,7 @@
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -72,7 +72,7 @@ NULL
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -101,7 +101,7 @@ minimum_bounding_box <- function(x) {
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -135,7 +135,7 @@ approx_count <- function(x) {
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/R/viz.R b/R/R/viz.R
similarity index 99%
rename from R/sparklyr.sedona/R/viz.R
rename to R/R/viz.R
index 75406d8..5d2100e 100644
--- a/R/sparklyr.sedona/R/viz.R
+++ b/R/R/viz.R
@@ -59,7 +59,7 @@ NULL
#' @examples
#'
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -129,7 +129,7 @@ sedona_render_heatmap <- function(rdd,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
@@ -196,7 +196,7 @@ sedona_render_scatter_plot <- function(rdd,
#'
#' @examples
#' library(sparklyr)
-#' library(sparklyr.sedona)
+#' library(apache.sedona)
#'
#' sc <- spark_connect(master = "spark://HOST:PORT")
#'
diff --git a/R/sparklyr.sedona/README.md b/R/README.md
similarity index 84%
rename from R/sparklyr.sedona/README.md
rename to R/README.md
index c684ea1..7e85919 100644
--- a/R/sparklyr.sedona/README.md
+++ b/R/README.md
@@ -1,20 +1,20 @@
-# sparklyr.sedona
+# apache.sedona
## Overview
-Sparklyr.sedona is a [sparklyr](https://github.com/sparklyr/sparklyr)-based R interface for [Apache Sedona](https://sedona.apache.org).
+apache.sedona is a [sparklyr](https://github.com/sparklyr/sparklyr)-based R interface for [Apache Sedona](https://sedona.apache.org).
It presents what Apache Sedona has to offer through idiomatic frameworks and constructs in R
(e.g., one can build spatial Spark SQL queries using Sedona UDFs in conjunction with a wide range of `dplyr` expressions),
hence making Apache Sedona highly friendly for R users.
## Connecting to Spark
-To ensure Sedona serialization routines, UDTs, and UDFs are properly registered when creating a Spark session, one simply needs to attach `sparklyr.sedona` before
-instantiating a Spark conneciton. Sparklyr.sedona will take care of the rest. For example,
+To ensure Sedona serialization routines, UDTs, and UDFs are properly registered when creating a Spark session, one simply needs to attach `apache.sedona` before
+instantiating a Spark conneciton. apache.sedona will take care of the rest. For example,
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
spark_home <- "/usr/lib/spark" # NOTE: replace this with your $SPARK_HOME directory
sc <- spark_connect(master = "yarn", spark_home = spark_home)
@@ -24,7 +24,7 @@ will create a Sedona-capable Spark connection in YARN client mode, and
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
sc <- spark_connect(master = "local")
```
@@ -73,9 +73,9 @@ Spark SQL and through the `dplyr` interface of `sparklyr` (more on that later).
## Creating a SpatialRDD
NOTE: this section is largely based on https://sedona.apache.org/tutorial/rdd/#create-a-spatialrdd, except for examples have been
-written in R instead of Scala to reflect usages of `sparklyr.sedona`.
+written in R instead of Scala to reflect usages of `apache.sedona`.
-Currently `SpatialRDD`s can be created in `sparklyr.sedona` by reading a file in a supported geospatial format, or by extracting data from a
+Currently `SpatialRDD`s can be created in `apache.sedona` by reading a file in a supported geospatial format, or by extracting data from a
Spark SQL query.
For example, the following code will import data from [arealm-small.csv](https://github.com/apache/incubator-sedona/blob/master/binder/data/arealm-small.csv) into a `SpatialRDD`:
@@ -105,14 +105,14 @@ As one can see from the above, each record is comma-separated and consists of a
All other columns contain non-spatial attributes. Because column indexes are 0-based, we need to specify `first_spatial_col_index = 1` in the example above to
ensure each record is parsed correctly.
-In addition to formats such as CSV and TSV, currently `sparklyr.sedona` also supports reading files in WKT (Well-Known Text), WKB (Well-Known Binary), and GeoJSON formats.
-See `?sparklyr.sedona::sedona_read_wkt`, `?sparklyr.sedona::sedona_read_wkb`, and `?sparklyr.sedona::sedona_read_geojson` for details.
+In addition to formats such as CSV and TSV, currently `apache.sedona` also supports reading files in WKT (Well-Known Text), WKB (Well-Known Binary), and GeoJSON formats.
+See `?apache.sedona::sedona_read_wkt`, `?apache.sedona::sedona_read_wkb`, and `?apache.sedona::sedona_read_geojson` for details.
One can also run `to_spatial_rdd()` to extract a SpatailRDD from a Spark SQL query, e.g.,
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
library(dplyr)
sc <- spark_connect(master = "local")
@@ -141,11 +141,11 @@ will extract a spatial column named `"geom"` from the Sedona spatial SQL query a
As mentioned previously, data from `SpatialRDD` can be exported into a Spark dataframe and be queried and modified through
the `dplyr` interface of `sparklyr`. The example below shows how `sdf_register()`, a S3 generic that converts a lower-level
-object into a Spark dataframe object in `sparklyr`, can be applied to a `SpatialRDD` object created by `sparklyr.sedona`.
+object into a Spark dataframe object in `sparklyr`, can be applied to a `SpatialRDD` object created by `apache.sedona`.
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
sc <- spark_connect(master = "local")
polygon_rdd <- sedona_read_geojson(sc, location = "/tmp/polygon.json")
@@ -193,18 +193,18 @@ modified_polygon_sdf <- polygon_sdf %>%
Notice all of the above can open up many interesting possiblities. For example, one can extract ML features from geospatial
data in Spark dataframes, build a ML pipeline using `ml_*` family of functions in `sparklyr` to work with such features, and if
the output of a ML model happens to be a geospatial object as well, one can even apply visualization routines in
-`sparklyr.sedona` to visualize the difference between any predicted geometry and the corresponding ground truth
+`apache.sedona` to visualize the difference between any predicted geometry and the corresponding ground truth
(more on visualization later).
## Visualization
-It is worth mentioning an important part of `sparklyr.sedona` is its collection of R interfaces to Sedona visualization routines.
+It is worth mentioning an important part of `apache.sedona` is its collection of R interfaces to Sedona visualization routines.
For example, the following
is essentially the R equivalent of [this example in Scala](https://github.com/apache/incubator-sedona/blob/f6b1c5e24bdb67d2c8d701a9b2af1fb5658fdc4d/viz/src/main/scala/org/apache/sedona/viz/showcase/ScalaExample.scala#L142-L160).
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
sc <- spark_connect(master = "local")
@@ -256,6 +256,6 @@ It will create a scatter plot, and then overlay it on top of a choropleth map, a
<img src="docs/choropleth-map.png" width=800 />
-See `?sparklyr.sedona::sedona_render_scatter_plot`, `?sparklyr.sedona::sedona_render_heatmap`,
-and `?sparklyr.sedona::sedona_render_choropleth_map` for more details on R interfaces of
-Sedona visualization routines currently implemented by `sparklyr.sedona`.
+See `?apache.sedona::sedona_render_scatter_plot`, `?apache.sedona::sedona_render_heatmap`,
+and `?apache.sedona::sedona_render_choropleth_map` for more details on R interfaces of
+Sedona visualization routines currently implemented by `apache.sedona`.
diff --git a/R/sparklyr.sedona/docs/choropleth-map.png b/R/docs/choropleth-map.png
similarity index 100%
rename from R/sparklyr.sedona/docs/choropleth-map.png
rename to R/docs/choropleth-map.png
diff --git a/R/sparklyr.sedona/tests/testthat.R b/R/tests/testthat.R
similarity index 83%
rename from R/sparklyr.sedona/tests/testthat.R
rename to R/tests/testthat.R
index b23f51b..d378a50 100644
--- a/R/sparklyr.sedona/tests/testthat.R
+++ b/R/tests/testthat.R
@@ -1,5 +1,5 @@
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
library(testthat)
# increase timeout for downloading Apache Spark tgz files
@@ -18,5 +18,5 @@ if (identical(Sys.getenv("NOT_CRAN"), "true")) {
CheckReporter$new(),
SummaryReporter$new(show_praise = FALSE)
))
- test_check("sparklyr.sedona", filter = filter, reporter = reporter)
+ test_check("apache.sedona", filter = filter, reporter = reporter)
}
diff --git a/R/sparklyr.sedona/tests/testthat/helper-initialize.R b/R/tests/testthat/helper-initialize.R
similarity index 99%
rename from R/sparklyr.sedona/tests/testthat/helper-initialize.R
rename to R/tests/testthat/helper-initialize.R
index ea6d192..ed4a1bc 100644
--- a/R/sparklyr.sedona/tests/testthat/helper-initialize.R
+++ b/R/tests/testthat/helper-initialize.R
@@ -43,7 +43,6 @@ test_data <- function(file_name) {
"..",
"..",
"..",
- "..",
"core",
"src",
"test",
diff --git a/R/sparklyr.sedona/tests/testthat/test-apply-spatial-partitioner.R b/R/tests/testthat/test-apply-spatial-partitioner.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-apply-spatial-partitioner.R
rename to R/tests/testthat/test-apply-spatial-partitioner.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-bounding-box.R b/R/tests/testthat/test-bounding-box.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-bounding-box.R
rename to R/tests/testthat/test-bounding-box.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-build-index.R b/R/tests/testthat/test-build-index.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-build-index.R
rename to R/tests/testthat/test-build-index.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-crs-transform.R b/R/tests/testthat/test-crs-transform.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-crs-transform.R
rename to R/tests/testthat/test-crs-transform.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-data-interface.R b/R/tests/testthat/test-data-interface.R
similarity index 99%
rename from R/sparklyr.sedona/tests/testthat/test-data-interface.R
rename to R/tests/testthat/test-data-interface.R
index b9f8c4d..34127fd 100644
--- a/R/sparklyr.sedona/tests/testthat/test-data-interface.R
+++ b/R/tests/testthat/test-data-interface.R
@@ -17,7 +17,7 @@ test_rdd_with_non_spatial_attrs <- invoke_new(
1L, # numPartitions
sc$state$object_cache$storage_levels$memory_only
) %>%
- sparklyr.sedona:::new_spatial_rdd("point")
+ apache.sedona:::new_spatial_rdd("point")
expect_result_matches_original <- function(pt_rdd) {
expect_equal(
diff --git a/R/sparklyr.sedona/tests/testthat/test-dbplyr-integration.R b/R/tests/testthat/test-dbplyr-integration.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-dbplyr-integration.R
rename to R/tests/testthat/test-dbplyr-integration.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-init.R b/R/tests/testthat/test-init.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-init.R
rename to R/tests/testthat/test-init.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-sdf-interface.R b/R/tests/testthat/test-sdf-interface.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-sdf-interface.R
rename to R/tests/testthat/test-sdf-interface.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-spatial-join.R b/R/tests/testthat/test-spatial-join.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-spatial-join.R
rename to R/tests/testthat/test-spatial-join.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-spatial-queries.R b/R/tests/testthat/test-spatial-queries.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-spatial-queries.R
rename to R/tests/testthat/test-spatial-queries.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-spatial-rdd-aggregation-fns.R b/R/tests/testthat/test-spatial-rdd-aggregation-fns.R
similarity index 100%
rename from R/sparklyr.sedona/tests/testthat/test-spatial-rdd-aggregation-fns.R
rename to R/tests/testthat/test-spatial-rdd-aggregation-fns.R
diff --git a/R/sparklyr.sedona/tests/testthat/test-viz.R b/R/tests/testthat/test-viz.R
similarity index 97%
rename from R/sparklyr.sedona/tests/testthat/test-viz.R
rename to R/tests/testthat/test-viz.R
index dbd6510..434839f 100644
--- a/R/sparklyr.sedona/tests/testthat/test-viz.R
+++ b/R/tests/testthat/test-viz.R
@@ -68,7 +68,7 @@ test_that("sedona_render_choropleth_map() works as expected", {
TRUE,
TRUE
) %>%
- sparklyr.sedona:::new_spatial_rdd("pair_rdd")
+ apache.sedona:::new_spatial_rdd("pair_rdd")
sedona_render_choropleth_map(
pair_rdd,
diff --git a/R/sparklyr.sedona/vignettes/.gitignore b/R/vignettes/.gitignore
similarity index 100%
rename from R/sparklyr.sedona/vignettes/.gitignore
rename to R/vignettes/.gitignore
diff --git a/R/sparklyr.sedona/vignettes/connecting-to-spark.Rmd b/R/vignettes/connecting-to-spark.Rmd
similarity index 91%
rename from R/sparklyr.sedona/vignettes/connecting-to-spark.Rmd
rename to R/vignettes/connecting-to-spark.Rmd
index 39b09b5..44a5630 100644
--- a/R/sparklyr.sedona/vignettes/connecting-to-spark.Rmd
+++ b/R/vignettes/connecting-to-spark.Rmd
@@ -14,12 +14,12 @@ knitr::opts_chunk$set(
)
```
-To ensure Sedona serialization routines, UDTs, and UDFs are properly registered when creating a Spark session, one simply needs to attach `sparklyr.sedona` before
-instantiating a Spark conneciton. Sparklyr.sedona will take care of the rest. For example,
+To ensure Sedona serialization routines, UDTs, and UDFs are properly registered when creating a Spark session, one simply needs to attach `apache.sedona` before
+instantiating a Spark conneciton. apache.sedona will take care of the rest. For example,
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
spark_home <- "/usr/lib/spark" # NOTE: replace this with your $SPARK_HOME directory
sc <- spark_connect(master = "yarn", spark_home = spark_home)
@@ -29,7 +29,7 @@ will create a Sedona-capable Spark connection in YARN client mode, and
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
sc <- spark_connect(master = "local")
```
diff --git a/R/sparklyr.sedona/vignettes/images/choropleth-map.png b/R/vignettes/images/choropleth-map.png
similarity index 100%
rename from R/sparklyr.sedona/vignettes/images/choropleth-map.png
rename to R/vignettes/images/choropleth-map.png
diff --git a/R/sparklyr.sedona/vignettes/intro-to-spatial-rdd.Rmd b/R/vignettes/intro-to-spatial-rdd.Rmd
similarity index 83%
rename from R/sparklyr.sedona/vignettes/intro-to-spatial-rdd.Rmd
rename to R/vignettes/intro-to-spatial-rdd.Rmd
index 0a0347b..04b8382 100644
--- a/R/sparklyr.sedona/vignettes/intro-to-spatial-rdd.Rmd
+++ b/R/vignettes/intro-to-spatial-rdd.Rmd
@@ -24,9 +24,9 @@ dataframes, making them accessible through Spark SQL and through the `dplyr` int
## Creating a SpatialRDD
NOTE: this section is largely based on https://sedona.apache.org/tutorial/rdd/#create-a-spatialrdd, except for examples have been
-written in R instead of Scala to reflect usages of `sparklyr.sedona`.
+written in R instead of Scala to reflect usages of `apache.sedona`.
-Currently `SpatialRDD`s can be created in `sparklyr.sedona` by reading a file in a supported geospatial format, or by extracting data from a
+Currently `SpatialRDD`s can be created in `apache.sedona` by reading a file in a supported geospatial format, or by extracting data from a
Spark SQL query.
For example, the following code will import data from [arealm-small.csv](https://github.com/apache/incubator-sedona/blob/master/binder/data/arealm-small.csv) into a `SpatialRDD`:
@@ -56,14 +56,14 @@ As one can see from the above, each record is comma-separated and consists of a
All other columns contain non-spatial attributes. Because column indexes are 0-based, we need to specify `first_spatial_col_index = 1` in the example above to
ensure each record is parsed correctly.
-In addition to formats such as CSV and TSV, currently `sparklyr.sedona` also supports reading files in WKT (Well-Known Text), WKB (Well-Known Binary), and GeoJSON formats.
-See `?sparklyr.sedona::sedona_read_wkt`, `?sparklyr.sedona::sedona_read_wkb`, and `?sparklyr.sedona::sedona_read_geojson` for details.
+In addition to formats such as CSV and TSV, currently `apache.sedona` also supports reading files in WKT (Well-Known Text), WKB (Well-Known Binary), and GeoJSON formats.
+See `?apache.sedona::sedona_read_wkt`, `?apache.sedona::sedona_read_wkb`, and `?apache.sedona::sedona_read_geojson` for details.
One can also run `to_spatial_rdd()` to extract a SpatailRDD from a Spark SQL query, e.g.,
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
library(dplyr)
sc <- spark_connect(master = "local")
diff --git a/R/sparklyr.sedona/vignettes/overview.Rmd b/R/vignettes/overview.Rmd
similarity index 85%
rename from R/sparklyr.sedona/vignettes/overview.Rmd
rename to R/vignettes/overview.Rmd
index ff65d54..a580d70 100644
--- a/R/sparklyr.sedona/vignettes/overview.Rmd
+++ b/R/vignettes/overview.Rmd
@@ -1,8 +1,8 @@
---
-title: "Overview of sparklyr.sedona"
+title: "Overview of apache.sedona"
output: rmarkdown::html_vignette
vignette: >
- %\VignetteIndexEntry{Overview of sparklyr.sedona}
+ %\VignetteIndexEntry{Overview of apache.sedona}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
@@ -14,7 +14,7 @@ knitr::opts_chunk$set(
)
```
-[sparklyr.sedona](https://github.com/r-spark/sparklyr.sedona) is a [sparklyr](https://github.com/sparklyr/sparklyr)-based R interface for [Apache Sedona](https://sedona.apache.org).
+[apache.sedona](https://github.com/r-spark/apache.sedona) is a [sparklyr](https://github.com/sparklyr/sparklyr)-based R interface for [Apache Sedona](https://sedona.apache.org).
It presents what Apache Sedona has to offer through idiomatic frameworks and constructs in R (e.g., one can build spatial Spark SQL queries using Sedona UDFs in conjunction with a wide range of dplyr expressions), hence making Apache Sedona highly friendly for R users.
Generally speaking, when working with Apache Sedona, one choose between the following two modes:
@@ -26,7 +26,7 @@ While the former option enables more fine-grained control over low-level impleme
Because data from spatial RDDs can be imported into Spark dataframes as geometry columns and vice versa, one can switch between the abovementioned two modes fairly easily.
-At the moment `sparklyr.sedona` consists of the following components:
+At the moment `apache.sedona` consists of the following components:
* R interface for Spatial-RDD-related functionalities
* Reading/writing spatial data in WKT, WKB, and GeoJSON formats
diff --git a/R/sparklyr.sedona/vignettes/spark-dataframes-and-beyond.Rmd b/R/vignettes/spark-dataframes-and-beyond.Rmd
similarity index 88%
rename from R/sparklyr.sedona/vignettes/spark-dataframes-and-beyond.Rmd
rename to R/vignettes/spark-dataframes-and-beyond.Rmd
index 3250d87..0a5eab4 100644
--- a/R/sparklyr.sedona/vignettes/spark-dataframes-and-beyond.Rmd
+++ b/R/vignettes/spark-dataframes-and-beyond.Rmd
@@ -14,12 +14,12 @@ knitr::opts_chunk$set(
)
```
-In `sparklyr.sedona` , `sdf_register()`, a S3 generic from `sparklyr` converting a lower-level object to a Spark dataframe,
+In `apache.sedona` , `sdf_register()`, a S3 generic from `sparklyr` converting a lower-level object to a Spark dataframe,
can be applied to a `SpatialRDD` objects:
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
sc <- spark_connect(master = "local")
polygon_rdd <- sedona_read_geojson(sc, location = "/tmp/polygon.json")
@@ -68,4 +68,4 @@ modified_polygon_sdf <- polygon_sdf %>%
Notice all of the above can open up many interesting possiblities. For example, one can extract ML features from geospatial
data in Spark dataframes, build a ML pipeline using `ml_*` family of functions in `sparklyr` to work with such features, and if
the output of a ML model happens to be a geospatial object as well, one can even apply visualization routines in
-`sparklyr.sedona` to visualize the difference between any predicted geometry and the corresponding ground truth.
+`apache.sedona` to visualize the difference between any predicted geometry and the corresponding ground truth.
diff --git a/R/sparklyr.sedona/vignettes/viz.Rmd b/R/vignettes/viz.Rmd
similarity index 82%
rename from R/sparklyr.sedona/vignettes/viz.Rmd
rename to R/vignettes/viz.Rmd
index 137f0bc..2d46351 100644
--- a/R/sparklyr.sedona/vignettes/viz.Rmd
+++ b/R/vignettes/viz.Rmd
@@ -14,13 +14,13 @@ knitr::opts_chunk$set(
)
```
-An important part of `sparklyr.sedona` is its collection of R interfaces to Sedona visualization routines.
+An important part of `apache.sedona` is its collection of R interfaces to Sedona visualization routines.
For example, the following
is essentially the R equivalent of [this example in Scala](https://github.com/apache/incubator-sedona/blob/f6b1c5e24bdb67d2c8d701a9b2af1fb5658fdc4d/viz/src/main/scala/org/apache/sedona/viz/showcase/ScalaExample.scala#L142-L160).
``` r
library(sparklyr)
-library(sparklyr.sedona)
+library(apache.sedona)
sc <- spark_connect(master = "local")
@@ -72,6 +72,6 @@ It will create a scatter plot, and then overlay it on top of a choropleth map, a
<img src="images/choropleth-map.png" width=800 />
-See `?sparklyr.sedona::sedona_render_scatter_plot`, `?sparklyr.sedona::sedona_render_heatmap`,
-and `?sparklyr.sedona::sedona_render_choropleth_map` for more details on visualization-related
-R interfaces currently implemented by `sparklyr.sedona`.
+See `?apache.sedona::sedona_render_scatter_plot`, `?apache.sedona::sedona_render_heatmap`,
+and `?apache.sedona::sedona_render_choropleth_map` for more details on visualization-related
+R interfaces currently implemented by `apache.sedona`.