How can I efficiently export a large PostGIS (PostgreSQL) table to a GeoJSON file? How can I efficiently export a large PostGIS (PostgreSQL) table to a GeoJSON file? postgresql postgresql

How can I efficiently export a large PostGIS (PostgreSQL) table to a GeoJSON file?


Use seq to generate numbers, then change your query to query a range of id values.

I've changed $2 to $suffix in this example, and IRL you'll drop the echo once you're happy with the output to actually run the commands (or you could just copy/paste the output into another terminal). You'll want to change 2000 to something safely above the max ogc_fid (select ogc_fid from whatever order by 1 desc limit 1 to find it), and might need to change 1000 to some other chunk size depending on how large each row is. Don't forget to also change the 1000 in the WHERE clause if you do, though.

$ seq 0 500 1500050010001500$ for suffix in foo bar; do> for each in $(seq 0 1000 2000); do> echo ogr2ogr -f GeoJSON -progress nhd_$suffix.json "PG:dbname=$PG_DB host=$PG_HOST port=$PG_PORT user=$PG_USERNAME password=$PG_PASSWORD" \>  -sql "select resolution, geom from nhd_hr_$suffix where ogc_fid>=$each and ogc_fid < ($each + 1000)">  done> doneogr2ogr -f GeoJSON -progress nhd_foo.json PG:dbname= host= port= user= password= -sql select resolution, geom from nhd_hr_foo where ogc_fid>=0 and ogc_fid < (0 + 1000)ogr2ogr -f GeoJSON -progress nhd_foo.json PG:dbname= host= port= user= password= -sql select resolution, geom from nhd_hr_foo where ogc_fid>=1000 and ogc_fid < (1000 + 1000)ogr2ogr -f GeoJSON -progress nhd_foo.json PG:dbname= host= port= user= password= -sql select resolution, geom from nhd_hr_foo where ogc_fid>=2000 and ogc_fid < (2000 + 1000)ogr2ogr -f GeoJSON -progress nhd_bar.json PG:dbname= host= port= user= password= -sql select resolution, geom from nhd_hr_bar where ogc_fid>=0 and ogc_fid < (0 + 1000)ogr2ogr -f GeoJSON -progress nhd_bar.json PG:dbname= host= port= user= password= -sql select resolution, geom from nhd_hr_bar where ogc_fid>=1000 and ogc_fid < (1000 + 1000)ogr2ogr -f GeoJSON -progress nhd_bar.json PG:dbname= host= port= user= password= -sql select resolution, geom from nhd_hr_bar where ogc_fid>=2000 and ogc_fid < (2000 + 1000)

Heres the command in one line for easier copy/paste to your shell:

for suffix in foo bar; do for each in $(seq 0 1000 2000); do echo ogr2ogr -f GeoJSON -progress nhd_$suffix.json "PG:dbname=$PG_DB host=$PG_HOST port=$PG_PORT user=$PG_USERNAME password=$PG_PASSWORD"  -sql "select resolution, geom from nhd_hr_$suffix where ogc_fid>=$each and ogc_fid < ($each + 1000)";  done; done