geopandas: BUG: The dataset contains a mistake. Crimea should belong to Ukraine
-
[+] I have checked that this issue has not already been reported.
-
[+] I have confirmed this bug exists on the latest version of geopandas.
-
(optional) I have confirmed this bug exists on the main branch of geopandas.
Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.
Code Sample, a copy-pastable example
# Your code here
import pandas as pd
import geopandas
data = geopandas.read_file(geopandas.datasets.get_path('naturalearth_lowres'))
ukraine = data.loc[data['name'] == 'Ukraine']
ukraine.plot()
Problem description
Plotting map of Ukraine. Crimea isn’t displayed, despite being a part of Ukraine. It is annexed indeed, but it is a temporary issue, that is going to be resolved soon. No self-respecting countries recognized Crimea’s seperation. So it should be considered a part of Ukraine, obviously.
Expected Output
The map should show Crimea.
Output of geopandas.show_versions()
SYSTEM INFO
python : 3.7.12 (default, Jan 15 2022, 18:48:18) [GCC 7.5.0] executable : /usr/bin/python3 machine : Linux-5.4.144±x86_64-with-Ubuntu-18.04-bionic
GEOS, GDAL, PROJ INFO
GEOS : 3.6.2 GEOS lib : /usr/lib/x86_64-linux-gnu/libgeos_c.so GDAL : 3.4.1 GDAL data dir: /usr/local/lib/python3.7/dist-packages/fiona/gdal_data PROJ : 8.1.1 PROJ data dir: /usr/local/lib/python3.7/dist-packages/pyproj/proj_dir/share/proj
PYTHON DEPENDENCIES
geopandas : 0.10.2 pandas : 1.3.5 fiona : 1.8.21 numpy : 1.21.5 shapely : 1.8.1.post1 rtree : None pyproj : 3.2.1 matplotlib : 3.2.2 mapclassify: None geopy : 1.17.0 psycopg2 : 2.7.6.1 (dt dec pq3 ext lo64) geoalchemy2: None pyarrow : 6.0.1 pygeos : None /usr/local/lib/python3.7/dist-packages/psycopg2/init.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use “pip install psycopg2-binary” instead. For details see: http://initd.org/psycopg/docs/install.html#binary-install-from-pypi. “”")
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Reactions: 40
- Comments: 16 (6 by maintainers)
Hi all,
thank you for your input and push towards change. This issue has been discussed during the last community meetings and we propose the following:
gpd.read_file(gpd.datasets.get_path("naturalearth_lowres"))as you do today, you and anyone else will get the correct version.This solution should resolve not only this issue but also similar reported before (#1510).
Before we’ll go this direction, would this be a solution you would approve? If not, we can try to come up with another that would be community approved.
Thank you all and accept my personal apologies for our slow response time on this sensitive issue.
Hi, this is not the only area in the world that can be considered wrong (xref #1510).
We are using Natural Earth dataset. Crimea currently falls under the following disclaimer:
Though we decided not to do anything on this before and referred to Natural Earth, I have to admit that I wasn’t aware of this and it does not make me very happy…
Drawing boundaries is a sensitive issue and we don’t want to put such burden on the geopandas dev team, that is why we use NE as the data provider. This policy is new (or updated) and contains "optional “point-of-view” (POV) or “worldview” for administrative geographies." datasets, but only for 1:10 000 000 versions, not 1:110 000 000 we are using.
So I am not sure if we should actually do something here. I am afraid we would open a can of worms if we made an exception from the policy use default NE data and let them deal with boundaries.
Too late, by not showing the internationally recognized borders of Ukraine you already sided with russian genocidal war of conquest. You can either fix this mistake or face the consequences.
Hello there,
I’ve seen this map so this is why I resurrected my GitHub account to write this message,
Listen, we in Ukraine don’t care what dataset you’re using and what are their excuses.
To start with, this map is WRONG, not to mention that it is also deeply offensive and problematic and this sort of excuse “whoever controls the ground” is not valid for lots of reasons. Should I explain what reasons exactly?
There is a concept of internationally recognized borders, and everyone should respect it. If you’re not, or whoever owns that dataset is not - is a perpetrator of the war crimes the aggressor states are committing.
Like, do you show ISIS-controlled lands on your maps? Why not? Probably Al-Qaeda controlled? Hm?
Possible solutions to this problem:
We would really want this to be fixed, with no excuses.
Regards, Oleh
“Drawing boundaries is a sensitive issue”. Yes, it is a very sensitive issue for Ukrainian people which are dying right now for free will and our lives. But you made a simple and inappropriate decision. No excuse for this.
Hi @martinfleis, thank you for a quick response! I’m sure community will appreciate your willingness to fix this issue. I have some concern about the second point in your proposed plan, specifically:
I think I understand the motivation, but adding a preference to allow user to choose between a correct and a russian map is like adding a preference of whether you want a real map or a map of flat earth. Though in case of the russian map there’s also the added component of support for terrorism and genocide. I understand the motivation from a maintainer’s point of view, but for an unaware user the optics can be seriously damaging to project reputation.
I understand that you can’t prevent people from loading whatever map they want, due to the nature of open-source software, but adding a special option to do it seems unnecessary.
A similar issue was brought to my attention by @mourner (of Leaflet fame, who lives in Kyiv). My approach was to switch from Natural Earth data to GeoBoundaries data - see https://www.geoboundaries.org/
You folks might wanna consider this approach if maintaining a copy of the NE data proves too cumbersome.
Hi,
I am glad I had correctly understood the source of the issue, particularly that the NE dataset that comes with geopandas adopts the standard NE borders.
My suggestion is that of introducing a warning upon loading the dataset, pointing to the NE policy and to the fact that NE offers alternative views. In this way, the user is made aware of the potential issue and also of potential “solutions”. Hopefully this is also a minimal “fix” that does not open a can on worms on the geopandas side or requires lots of work/changes.
To be clear: the suggestion is not that of having a warning for Crimea in particular, rather about “disputed territories” in general; people may not think there is a dispute at all and criticize that choice of words&warning, but I’d say it is good enough for the relatively simple low-res NE dataset to highlight the issue of disputed territories/ disagreements and direct people to more information and possible fixes.
Thank you for your work.
@blacktaxi Fully understand this point. What this solution meant to do was to not only cover the Crimea issue but also all the other. To cover all areas that are marked as disputed by naturalearth.com on the map below.
At this point, I see only one solution and that is to remove datasets like this from geopandas compeletely. After all, geopandas is not an open data project but open source software project.
What I will do now is to create a pull request that manually changes the current boundaries and assigns Crimea back to Ukraine where it belongs. This fix will be included in the 0.12.2 release that will come out within a couple of days.
The final implemented solution will be further discussed.
Hi, how is fixing the issues with Crimea not being part of Ukraine map “opening a can of worms”?
Crimea is a part of Ukraine, it’s recognized as a part of Ukraine by international law, you being lazy and not wanting to fix it is completely inappropriate.
Map generation makes no sense at all if your maps don’t align with reality.
You refusing to fix it is the same as agreeing with the illegal occupation of the area and indirectly allowing russia’s crimes against humanity to continue in Crimea.
As per this discussion - https://github.com/nvkelso/natural-earth-vector/issues/812 - NaturalEarth does not seem to be willing to change anything.
@martinfleis we strongly suggest that you pick another provider for geo data which provides accurate and internationally accepted definitions of borders, since supporting the “de-facto” bullshit that NaturalEarth tries to spread will only degrade the usability of your library and potentially render it unusable.
They basically tell us they can change whatever they want based on their “observations” of “reality”, and this clearly doesn’t stand together with the spirit of OpenSource.
Set yourself free, support the freedom & humanity, use a different dataset!
You are messing Line of Control and Political border. Yes, you. Because you decided what data set should be used for your lib. And in this way you are supporting russia - state sponsor of terrorism. Internationally declared. There is an international law. You accept or deny it. There is no third option.
The fix including Crimea as a part of Russia is now at #2670. We hope to release the new version with these boundaries by the end of today.
The overall discussion on how to treat all disputed regions in GeoPandas will be moved to a separate issue and the solution will be implemented either fully or with a deprecation period in the next minor release (0.13).
Thank you all for the valuable feedback here and everywhere else.
@martinfleis THANK you for putting your time in on this.
We all appreciate you as a maintainer for geopandas who is doing their best. Thank you for your effort on geopandas. ❤️
Thank you for implementing a solution this fast ❤️