block by veltman 2c79458b2226466920dbd601bf94551f

Geosupport w/ JS and node-ffi

Geocoding 10,000 addresses a second with NYC’s Geosupport library and Node FFI

Following on Chris Whong’s excellent writeup of how to make calls directly to NYC’s Geosupport client and this first attempt at generalizing it, here’s a way that let me geocode about 10,000 addresses a second on Ubuntu using Node FFI.

Note: this assumes Ubuntu - other Linux is probably fine but may need adjustments.

First, install the basics:

# Update, install Node and unzip (if needed)
sudo apt-get update --yes && sudo apt-get upgrade --yes
sudo apt-get install unzip nodejs npm

# Fix legacy node naming nonsense on Ubuntu (if needed)
sudo ln -s `which nodejs` /usr/bin/node

# Start a package (optional)
npm init

# Install node-ffi and proj4
# proj4 is optional, only for reprojecting coordinates to lng/lat
npm install --save ffi proj4

# Download Geosupport Desktop for Linux
wget http://www1.nyc.gov/assets/planning/download/zip/data-maps/open-data/gdelx_16c.zip

# Unzip the contents
unzip gdelx_16c.zip

# Set env variables to the full paths to lib/ and fls/
export LD_LIBRARY_PATH="$(pwd)/version-16c_16.3/lib/"
export GEOFILES="$(pwd)/version-16c_16.3/fls/"

That’s it! Now you’re ready to geocode using a script like the attached geocode.js, which wraps libgeo.so so you can call it directly by passing two Buffers.

That example assumes you have a JSON file, addresses.json, with addresses like so:

[
  {
    "HouseNumber": "86-54",
    "StreetName": "WINCHESTER BOULEVARD",
    "ZipCode": "11427",
    "BoroughCode": 4
  },
  ...
]

but you could use the same logic in any context.

This script also reprojects the results, which are in NY Long Island State Plane, into latitude/longitude using proj4.

To handle various error codes, or to use other Geosupport functions besides address geocoding (there are lots), you would only need to update the working area inputs (lines 25-26) and how you parse them afterwards (lines 45 - 55) according to the fields in the User guide (p. 570+).

Note: this example assumes you have a borough code rather than a borough name. NYC borough codes are:

Manhattan: 1
Bronx: 2
Brooklyn: 3
Queens: 4
Staten Island: 5

geocode.js