You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2022/02/17 06:59:10 UTC
[GitHub] [superset] Yann-J opened a new pull request #18778: Country map viz: geoboundary downloader
Yann-J opened a new pull request #18778:
URL: https://github.com/apache/superset/pull/18778
While contributing some new maps to the legacy country map visualization in #, I realized a few limitations, that this PR intends to address, i.e.:
- Support a new source to download high-quality geoJSON boundary files, the amazing [geoBoundaries](https://www.geoboundaries.org/index.html) - in particular, this source supports up to 4 levels of administrative divisions (whereas the current source only supports 1), thanks to a new python notebook
- Support for an attribution in the map display
- Better naming convention for map names, e.g. capitalizing every word in names with spaces
- A bunch of new maps coming from geoBoundaries relevant to my organization, with several administrative subdivision levels
### SUMMARY
Technical / design choices and caveats:
- The attributions comment displayed in the map is read from the GeoJSON source's `attribution` property at the top level of the `FeatureCollection`.
- __WARNING__: Many admin areas do not have an ISO code... The geoBoundaries datasource's ID is used instead, but it requires the user to know about these. Further improvements might be needed to allow the use of the area name instead, or make it easier to discover the expected codes...
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Sample map for Kenya Counties (notice the new attribution comment at the bottom):
Snapshot of all the added geojson maps:
### TESTING INSTRUCTIONS
Set up a data source with county/province-level data, including their ISO 3166-2 code, for instance uploading this sample random CSV:
```
ISO,name,value
ET-AA,Addis Ababa,58.24388052899692
ET-AF,Afar,82.39096125651662
ET-AM,Amhara,5.823758668250489
ET-BE,Beneshangul Gumu,4.906103785510196
ET-DD,Dire Dawa,64.04758252062041
ET-GA,Gambela,71.45936608261624
ET-HA,Hareri,6.278288599268822
ET-OR,Oromia,60.625174539397065
ET-SN,SNNPR,93.6721133500065
ET-SO,Somali,59.03513300561172
ET-TI,Tigray,34.66035278225454
```
Create a new Country Map chart with this dataset (in this example, for the 'Ethiopia Regions' country name).
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #18778: other(data): Country map viz: geoboundary downloader + more maps
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #18778:
URL: https://github.com/apache/superset/pull/18778#issuecomment-1042696476
# [Codecov](https://codecov.io/gh/apache/superset/pull/18778?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#18778](https://codecov.io/gh/apache/superset/pull/18778?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (85c0b30) into [master](https://codecov.io/gh/apache/superset/commit/9db6ed6ddd03a587d0b25bd3c19e72a94244f10d?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (9db6ed6) will **increase** coverage by `0.00%`.
> The diff coverage is `40.00%`.
[](https://codecov.io/gh/apache/superset/pull/18778?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #18778 +/- ##
=======================================
Coverage 66.32% 66.32%
=======================================
Files 1620 1620
Lines 63087 63091 +4
Branches 6372 6372
=======================================
+ Hits 41840 41844 +4
- Misses 19590 19591 +1
+ Partials 1657 1656 -1
```
| Flag | Coverage Δ | |
|---|---|---|
| javascript | `51.26% <40.00%> (+<0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/18778?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [.../legacy-plugin-chart-country-map/src/CountryMap.js](https://codecov.io/gh/apache/superset/pull/18778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LWNvdW50cnktbWFwL3NyYy9Db3VudHJ5TWFwLmpz) | `0.00% <0.00%> (ø)` | |
| [...s/legacy-plugin-chart-country-map/src/countries.ts](https://codecov.io/gh/apache/superset/pull/18778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvcGx1Z2lucy9sZWdhY3ktcGx1Z2luLWNoYXJ0LWNvdW50cnktbWFwL3NyYy9jb3VudHJpZXMudHM=) | `100.00% <100.00%> (ø)` | |
| [.../explore/components/controls/TextControl/index.tsx](https://codecov.io/gh/apache/superset/pull/18778/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL2V4cGxvcmUvY29tcG9uZW50cy9jb250cm9scy9UZXh0Q29udHJvbC9pbmRleC50c3g=) | `86.66% <0.00%> (+10.00%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/18778?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/18778?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [9db6ed6...85c0b30](https://codecov.io/gh/apache/superset/pull/18778?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #18778: other(data): Country map viz: geoboundary downloader + more maps
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #18778:
URL: https://github.com/apache/superset/pull/18778#issuecomment-1042707704
Thanks so much for this improvement! I'll check some countries that I'm familiar with and see what the quality is vs the current source. I know the current Finland map has a few errors and in some parts is very lacking in detail to the point of being unrecognizable vs a real map, so this could potentially be a great improvement. Also tagging @ktmud for a review as he wrote the initial notebook.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro edited a comment on pull request #18778: other(data): Country map viz: geoboundary downloader + more maps
Posted by GitBox <gi...@apache.org>.
villebro edited a comment on pull request #18778:
URL: https://github.com/apache/superset/pull/18778#issuecomment-1042707704
Thanks so much for this improvement @Yann-J ! I'll check some countries that I'm familiar with and see what the quality is vs the current source. I know the current Finland map has a few errors and in some parts is very lacking in detail to the point of being unrecognizable vs a real map, so this could potentially be a great improvement. Also tagging @ktmud for a review as he wrote the initial notebook.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] Yann-J commented on pull request #18778: other(data): Country map viz: geoboundary downloader + more maps
Posted by GitBox <gi...@apache.org>.
Yann-J commented on pull request #18778:
URL: https://github.com/apache/superset/pull/18778#issuecomment-1042805842
Thanks, and kudos to @ktmud as well on the original notebook... the new one is of course very heavily inspired from it, and I'm not python expert myself (and a total geopandas noob), so any feedback appreciated.
There are maybe 2 areas I'd draw some attention to:
- Regarding the quality of the maps, just like with the original script, the source files are a bit heavy and need to be down-sampled. There's clearly some room to tune the `simplify` parameters. The original algorithm used the (angular?) surface of the bounding box as a criteria, which I felt might favor large countries, so I've used its square root instead (i.e. the angular tolerance should be roughly proportional to the country's angular diameter)... Could be improved but the maps I generated for Africa all seemed to come out well...
- I've initially hit a major hurdle when loading the geojson files produced by geopandas... It took me many hours to figure out, because the files looked perfectly OK, and loaded without issues in all the geojson viewers I've tried, but kept giving a crazy centroid (in fact the antipode of the expected one) / bounding box (the entire world) when loaded in the web UI by D3, throwing off the projection entirely... It turns out that D3 (but apparently not geopandas or even the source files from geoBoundaries, or any geojson viewer I tried??) will respect the geoJSON convention on orienting the polygons, i.e. that the interior of the polygons is expected to be on the right side (or is it the other way?) of the line... If the points are provided in the opposite order, D3 will consider the interior of the polygons to be the rest of the world rather than the intended shape... This is the reason for the ugly hack loading the raw json and applying `reverse` on the `geometry` lists... There
might be a more elegant way to force that in geopandas / fiona but couldn't figure it out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org