Far from the typical transfer of shapefiles and spreadsheets, a new era of web resources and APIs is coming. As GIS professionals, this has typically been a skillset not needed within our domain – but with the increase in web based technologies and a need to constantly pull resources from external sources, APIs are definitely something worth exploring.
What is an API
An API can vary greatly, depending on what it’s for. But put simply, it’s a set of functions and procedures, allowing the creation of applications that access the features or data of an operating system, application, or other service. These days pretty much any data service you’re accessing will have an API in order for you to query the system for data that you need.
What format can I access?
The format of an API’s data depends largely on the system you’re accessing, but JSON is likely to be the most common format you’ll come across. For this reason, we’ll explore a few API examples which return their data in JSON. The key challenge, is in turning a large chunk of text into tabular data, separated into distinct columns and records. Once you’re comfortable with this, using FME it’s really simple to write that data into a typical GIS format like a shapefile, or a File Geodatabase.
Example: Auckland Transport Developer Portal
The Auckland Transport Developer Portal is a great place to start exploring the use of APIs. In this example, we’re going to look at the Realtime Transit Feed (GTFS) which gives users the potential to query the locations of Auckland Transport’s vehicles in realtime.
Most developer portals have the option to try the API and return sample data in order for the user to verify it’s what they’re looking for. This is a great place to start, as users can make a sample request with correct syntax, and check out what data is returned really easily.
Here you can see with a very simple request, and using an API key unique to the user, you can return a lot of information – including spatial attributes. This response is formatted as JSON, which is returned as a single chunk of text.
Using FME to access an API
Within FME, you can replicate exactly the same request made above via the developer portal. Using the HTTP Caller transformer, you can make web requests to return data like above, and then use a series to unpack the data into the correct columns and rows.
This chain of transformers takes the nested JSON and flattens the attributes into legitimate attributes. Next it explodes the list into individual features, which you can see by the feature counts changing dramatically. At this step, one feature enters which is exploded into the 1150 features that the JSON text actually makes up.
Lastly, we use a Vertex Creator and Coordinate System Setter to create geometry using the attributes that the API returns, and sets the coordinate system in order to display it correctly on a map. The result, is that the JSON we originally started with has been split into individual records which can be displayed on a map and queried, like any other dataset. From here, it’s simple to output the data to any format that FME supports.
APIs are a great way to keep some of your datasets up to date, as you can build processes which are repeatable, run on a regular schedule, and require no manual processes due to the automatic transfer of data over the web. For the next dataset or data service you come across, why not try and master the art of the API!
Article Author: Kieran O’Donnell, FME Data Solutions, Locus
