Note this advocacy page is 5 years old now. Not that a lot has changed since then. FlatGeobuf is new. Also GeoJSON has doubled down its commitment to really being WGS84 only, not supporting projected data. That makes it largely unsuitable for many GIS applications. Works great for simple maps on the web though!
In practice I tend to use GeoPackage, mostly because that’s what QGIS encourages. The only real drawback cited is that it doesn’t stream very well. Which is a problem but only in particular use cases, it’s great if you have a full copy of the file on your local disk already.
My understanding is that ESRI controls basically the entire market for GIS software. The format isn’t going to change unless ESRI decides it will change. It’s not too different from MS Office document formats, or AutoCAD DXF files. There’s a small community of people advocating for open GIS solutions, but it’s pretty tiny compared to the people who just rely on ESRI.
Per someone I know at Esri, Shapefiles are really only used for backwards compatibility and haven’t been recommended for years. It’s more habit than anything.
I agree, but I feel that it’s gonna take a long time, given that so many open data geo projects, especially once related to government or governmental organizations seem to use it as the go-to. And a lot of typical software it is used with expects it. I recently added support for it to a piece of software, because of this.
Also I am not so sure about whether GeoPackage is such a great idea in terms of parsing it. That is not using SQLite, which is more an implementation than a standard.
While the other options like GeoJSON have their own downsides, I’d really go for “container formats” for which one either have many implementations (JSON being natively implemented in almost every language) or something that is easy to parse to convert into whatever your software needs, not relying on a single major implementation and not having to essentially implement large parts of a database.
While I agree with parts of your point, IMO SQLite is so incredibly ubiquitous, support basically every platform and language you could image, that I think the point is moot. Whether we think about it or not, a very large amount of our software lives runs on SQLite. it’s incredibly high quality, and one of the best tested pieces of code in wide use. It’s also a format which is seeing a lot of other interesting uses in our industry right now (I keep thinking SQLite is so hot right now, as there’s one or two articles about it a week on lobste.rs at the moment).
There’s no reason we need to have The One True Format, but as a replacement for Shapefiles and transmitting static geospatial data, GeoPackage seems like an excellent place to start. When I was working on nationalmap.gov.au, shapefiles were the Bain of my existence; so often I’d get an upside down Australia made of dots over Japan, or a teeny weeny Australia sitting right next to null island. And promoting a non-proprietary format over anything from ESRI will always get support from me.
I completely agree on that. This was in no ways meant to be directed against SQLite (i love it and I wished people at large understood it better), but as consideration for long term standards for example in the context of governmental institutions releasing open data.
It’s certainly a better option than shapefiles for the reasons given in the article.
Also I am not so sure about whether GeoPackage is such a great idea in terms of parsing it. That is not using SQLite, which is more an implementation than a standard.
The database file format for sqlite hasn’t changed since 2006, is reasonably simple and several third party parsers exist. I don’t think this is likely to be a problematic format, even for people who for whatever reason don’t want to use the sqlite3 library.
I wouldn’t say it’s hard or problematic, but simply that sqlite is a database format and not a generic or GIS-specific format. Of course one can always use the other formats, but given that shape files currently are sadly often used as the only format I think being conservative here makes sense or else we might only end up with a short list of downsides. Or in other words, if you want to convince everyone to use a specific, widely used format you want to eliminate as much reasons to use something else as possible and going through SQLite first might be such a reason in certain situations.
I just hope it doesn’t cause a lot of headaches and weird situations should SQLite decide to change that format or being scared of doing so or have an old and a new format. Standards that everyone agrees on can have very long term consequences after all.
Note this advocacy page is 5 years old now. Not that a lot has changed since then. FlatGeobuf is new. Also GeoJSON has doubled down its commitment to really being WGS84 only, not supporting projected data. That makes it largely unsuitable for many GIS applications. Works great for simple maps on the web though!
In practice I tend to use GeoPackage, mostly because that’s what QGIS encourages. The only real drawback cited is that it doesn’t stream very well. Which is a problem but only in particular use cases, it’s great if you have a full copy of the file on your local disk already.
My understanding is that ESRI controls basically the entire market for GIS software. The format isn’t going to change unless ESRI decides it will change. It’s not too different from MS Office document formats, or AutoCAD DXF files. There’s a small community of people advocating for open GIS solutions, but it’s pretty tiny compared to the people who just rely on ESRI.
Per someone I know at Esri, Shapefiles are really only used for backwards compatibility and haven’t been recommended for years. It’s more habit than anything.
I agree, but I feel that it’s gonna take a long time, given that so many open data geo projects, especially once related to government or governmental organizations seem to use it as the go-to. And a lot of typical software it is used with expects it. I recently added support for it to a piece of software, because of this.
Also I am not so sure about whether GeoPackage is such a great idea in terms of parsing it. That is not using SQLite, which is more an implementation than a standard.
While the other options like GeoJSON have their own downsides, I’d really go for “container formats” for which one either have many implementations (JSON being natively implemented in almost every language) or something that is easy to parse to convert into whatever your software needs, not relying on a single major implementation and not having to essentially implement large parts of a database.
While I agree with parts of your point, IMO SQLite is so incredibly ubiquitous, support basically every platform and language you could image, that I think the point is moot. Whether we think about it or not, a very large amount of our software lives runs on SQLite. it’s incredibly high quality, and one of the best tested pieces of code in wide use. It’s also a format which is seeing a lot of other interesting uses in our industry right now (I keep thinking SQLite is so hot right now, as there’s one or two articles about it a week on lobste.rs at the moment).
There’s no reason we need to have The One True Format, but as a replacement for Shapefiles and transmitting static geospatial data, GeoPackage seems like an excellent place to start. When I was working on nationalmap.gov.au, shapefiles were the Bain of my existence; so often I’d get an upside down Australia made of dots over Japan, or a teeny weeny Australia sitting right next to null island. And promoting a non-proprietary format over anything from ESRI will always get support from me.
I completely agree on that. This was in no ways meant to be directed against SQLite (i love it and I wished people at large understood it better), but as consideration for long term standards for example in the context of governmental institutions releasing open data.
It’s certainly a better option than shapefiles for the reasons given in the article.
I’m not sure what you meant about the long term comment, isn’t SQLite one of the formats recommended by the US Library of Congress?
The database file format for sqlite hasn’t changed since 2006, is reasonably simple and several third party parsers exist. I don’t think this is likely to be a problematic format, even for people who for whatever reason don’t want to use the sqlite3 library.
I wouldn’t say it’s hard or problematic, but simply that sqlite is a database format and not a generic or GIS-specific format. Of course one can always use the other formats, but given that shape files currently are sadly often used as the only format I think being conservative here makes sense or else we might only end up with a short list of downsides. Or in other words, if you want to convince everyone to use a specific, widely used format you want to eliminate as much reasons to use something else as possible and going through SQLite first might be such a reason in certain situations.
I just hope it doesn’t cause a lot of headaches and weird situations should SQLite decide to change that format or being scared of doing so or have an old and a new format. Standards that everyone agrees on can have very long term consequences after all.
It’s an uncanny coincidence I unrelatedly used a shapefile (in JOSM) today for the first time in my life.