Maps are both practical and political. They possess undoubtable utility for navigating the physical world and have a long history of being used to shape and reshape the our social and political conceptions of the world. The ability to mark a territory, carve up a continent (or remember one), count a people, or map our desires is a powerful one. Thus despite being one of the oldest visualization types we have, they remain one of the most popular ways of visualizing data today.
I recently became more interested in more deeply exploring web cartography, and in particular I had a chance to do a couple of things: read Axis Maps’ fantastic Guide to Thematic Mapping, which I highly recommend, and make some maps with a new map rendering library called Tangram.
Two things in particular drove my interest to learn more about making ‘slippy maps’:
- Patricio Gonzalez-Vivo gave a talk about WebGL based mapping at OpenVisConf 2016 and frankly, those demos blew me away. I wanted to better understand how to make these visually expressive maps.
- Practical needs in my day to day work got me interested in exploring new approaches to rendering data on slippy maps and the challenges we face around interactivity and layering when using a common technique of layering SVG on top of tile based maps.
This blog post, and the next one in the series, aim to share what I learned in exploring Tangram. This post will cover basic setup and concepts and also explore visualizing data on a choropleth map.
Part two will look at getting started with shaders and mouse interaction with Tangram—stay tuned for that.
What is Tangram?
Mapzen is a company working on an open web mapping stack, from tile servers to a rendering engine and other associated tools. Tangram is a rendering engine created by Mapzen that uses WebGL as its core rendering technology, this allows for inclusion of 3D geometry in maps as well as advanced shader-driven visual rendering options. Check out the demos on their home page to get a sense of what I am talking about.
Tangram is implemented as a leaflet.js plugin, leaflet.js is an open source library for making interactive maps on the web. Tangram allows customization of maps via a YAML configuration file. YAML might seem a strange choice at first if you are used to the dominance of JSON for driving all things web. But it ends up being a great configuration language and is more flexible and easier to work with than a JSON alternative would be. We’ll see examples of this later and you can see for yourself what a difference it makes.
Vector Tiles
Another important piece of this rendering stack is the use of vector tiles. Initially slippy maps used image tiles, which are sliced up pictures of the globe. A small image is made for each part of the world for every zoom level we want to support. As the user pans around the map new images appropriate for the zoom level are downloaded and rendered on screen.
Vector tiles replace the images with vector data about the world’s geometry (lines that describe countries or points that describe landmarks for example), and allow the client to have complete control over how that geometry is rendered. They also result in a more compact data transfer than there image based counterparts.
Your First Tangram Map
Lets see what it looks like to get set up with Tangram. The Tangram tutorial is a great resource for getting started, so we won’t duplicate it here. My hope is that this post can act as a companion to the excellent documentation on the tangram site; providing highlights, commentary and caveats as you get started, or a sneak peek if you are still looking from a distance.
So what does a basic map look like with tangram?
First some HTML
<!DOCTYPE html>
<head>
<meta charset="utf-8">
<!-- leaflet -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.0.0-rc.1/leaflet.js"></script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.0.0-rc.1/leaflet.css" />
<!-- Tangram library -->
<script src="https://mapzen.com/tangram/0.8/tangram.min.js"></script>
<style>
body { margin:0;position:fixed;top:0;right:0;bottom:0;left:0; }
#map { height: 100%; width: 100%; position: absolute; }
</style>
</head>
<body>
<div id="map"></div>
</body>
And some JavaScript
// Make a leaflet map object
var map = L.map('map');
// Create a tangram layer
var layer = Tangram.leafletLayer({
scene: 'scene.yaml',
attribution: '<a href="https://mapzen.com/tangram" target="_blank">Tangram</a> | © OSM contributors | <a href="https://mapzen.com/" target="_blank">Mapzen</a>'
});
// Add the tangram layer to the map.
layer.addTo(map);
// Lets go to Boston!
map.setView([42.364506, -71.038887], 15);
// Grab the live scene object and listen to events.
var scene = layer.scene;
scene.subscribe({
error: function (e) {
console.log('scene error:', e);
},
warning: function (e) {
console.log('scene warning:', e);
}
});
And here is the YAML configuration file we mentioned earlier. We’ll break this down a bit more below.
# Yay YAML, we can write comments!
# Each major part of the configuration is a key in a object/dictionary.
sources:
osm:
type: TopoJSON
url: https://vector.mapzen.com/osm/all/{z}/{x}/{y}.topojson?api_key=vector-tiles-HqUVidw
max_zoom: 16
cameras:
camera1:
type: perspective
lights:
light1:
visible: true
type: directional
direction: [0, 1, -.5]
diffuse: .4
ambient: 1
layers:
earth:
data: { source: osm }
draw:
polygons:
order: function() { return feature.sort_key; }
color: '#ffffff'
landuse:
visible: true
data: { source: osm }
draw:
polygons:
order: function() { return feature.sort_key; }
color: '#ffffff'
water:
data: { source: osm }
draw:
polygons:
order: function() { return feature.sort_key; }
color: '#96acba'
roads:
data: { source: osm }
filter:
not: { kind: ["path", "rail"] }
draw:
lines:
order: function() { return feature.sort_key; }
color: '#6e6e6e'
width: 8
cap: round
minor_road:
filter:
kind: minor_road
draw:
lines:
order: function() { return feature.sort_key; }
color: '#cfcfcf'
width: 5
buildings:
data: { source: osm }
draw:
polygons:
order: function() { return feature.sort_key; }
color: grey
3d-buildings:
filter: { $zoom: { min: 15 } }
draw:
polygons:
extrude: function () { return feature.height > 20 || $zoom >= 16; }
pois:
data: { source: osm }
draw:
points:
order: function() { return feature.sort_key; }
color: rgb(41, 34, 6)
size: 0
text:
text_source: name
font:
family: Arial
size: 14px
style: italic
weight: bold
fill: '#222222'
stroke: { color: white, width: 2 }
transform: uppercase
This should give you this
Link to live example with full source
A quick tour of the config
Many of the top level configuration options give an idea of what they are used to configure, camera for cameras and lights for lights etc. Lets look at a few of these sections in more detail.
Sources & Layers
Sources in Tangram describe providers of geometry or image data to be rendered and are the beginning or our pipeline. These can be vector tiles, as in the example shown below, which pulls in vector tiles for the world in TopoJSON format from Mapzen’s vector tile service. Sources can also be things like arbitrary GeoJSON/TopoJSON files, Mapbox Vector Tiles, and even raster image sources.
When specifying a source we need to give it a name so that we can refer to it later.
sources:
# lets give this source a name, it can be anything
# but we'll need the name later
osm:
type: TopoJSON
# Note that this url is templated the {z} {x} {y} allow tangram
# to put zoom levels and x, y positions into the actual url that gets requested.
url: https://vector.mapzen.com/osm/all/{z}/{x}/{y}.topojson?api_key=vector-tiles-HqUVidw
max_zoom: 16
Some sources have data that is exposed in various layers. For example Mapzen’s vector tile service is based on OpenStreetMap data and exposes common OpenStreetMap layers such as boundaries, buildings, roads, water and more. Each layer can be drawn using different styles and techniques based on what kind of geometry is in that layer.
You can have multiple sources in a Tangram map and we will see an example of that a bit later.
Lights and Colors
You can have multiple lights illuminating your map, a light is specified as follows
lights:
light1: # the light needs a name too. it can be anything
visible: true
type: directional #there are 3 different kinds of light
direction: [0, 1, -.5]
diffuse: .4
ambient: 1
The lighting model in Tangram is fairly flexible and sophisticated, if you have used other 3d graphics APIs it may seem familiar, and is based on the Blinn-Phong reflection model. If you haven’t used 3D graphics apis and are wondering who Blinn and Phong are, don’t worry. The docs give some great examples, and I made a small example to experiment with the lighting model that you can play with to get a feel for it.
However the main thing I want to mention here is that the color of a pixel on a given surface (say a building or street), is based on the relationship between the light properties and the material properties of the surface.
So if at some point you find that you set the color of something you are drawing to white (or some other color) and it doesn’t look that way in the final render; check your lights. They are likely be modifying the final result in a way you weren’t expecting. In particular, take a look at the ambient component of lights, if it is set too low the scene will look much darker.
Draw blocks and draw styles
Draw blocks and draw styles are the meat of the drawing operations in tangram (you could think of shaders as the spice!). For any given layer in a source you can specify a draw block and a define the style that should be used to draw it. Lets look at rendering all the water geometry from OpenStreetMap as an example:
layers: # All draw blocks are in a layer
# This is the name of the layer and should match the
# name of the layer in the source
water:
# This is the source that this layers geometry will be pulled from
data: { source: osm }
# This is the start of the draw block for this layer
draw:
# 'polygons' is the name of style used to draw this layer.
# it is a built in style
polygons:
# These are various options for the style.
# Note that you can have functions here
order: function() { return feature.sort_key; }
color: '#96acba'
Each draw block should be nested under a layer and should specify a built in draw style or a custom style. Custom styles are useful for reusing drawing parameters across different layers, but ultimately they must be based on one of the built in base draw styles. We won’t see custom styles in the examples in this post, but you will see them as you look at more of the documentation for Tangram.
In the example above, polygons is the draw style used to render the data from the water layer. The base style chosen must support the kind of geometry in the layer for it to work. For example, the POI layer in OpenStreetMap has points of interest, but no polygons are defined in it, thus the polygon’s base style wouldn’t have anything to render (we used the points style instead). You can find out which styles are compatible with what types of geometry in the documentation for the style.
Other sections
The above doesn’t cover all the necessary configuration of a tangram scene, in particular you need to define a camera and probably a few extra draw blocks for more layers.
The Rendering Model
You may be wondering, why use the YAML file in the first place? It might have an advantage of readability and maintainability over JSON, but why not just use JavaScript directly?
Under the scenes Tangram is doing a fair amount of work for you to convert this configuration into a set of programs in JavaScript and GLSL, the shader language that drives WebGL. The latter ultimately generates a compiled program that will be sent to your graphics card where it will do all the crunching and math to render those pretty pictures. The declarative configuration provides a tractable workflow to generate these programs while saving us from the underlying complexity.
As we will see below, you can interact with this configuration via JavaScript.
Visualizing data. Lets make a choropleth map
Ok, now that we have seen some of the basics, lets take a quick look at what visualizing some data might look like in Tangram. In particular, lets make a population choropleth of the world.
For this example I used population data that I downloaded from the World Bank and we will use D3 to help with figuring out the colors.
Our goal is to render the polygon for each country in a color representative of its population. The full source for this is linked at the bottom, but here is what we are tying to produce. Feel free to zoom around the map and even zoom in close to see cities.
Link to live example with full source
Step 1: Rendering the geometry
When I started doing this, I thought I’d just grab the appropriate layer from OpenStreetMap and give those polygons a dynamic color. But the first problem I ran into was that boundaries are not borders.
Repeat after me: Boundaries are not Borders
A caveat I want to share about the boundaries layer in OpenStreetMap is that boundaries at the country level are not the same as country borders, these boundaries, as one might suspect from the name, only represent the boundary between countries that are adjacent to each other, and are thus not closed polygons. This means there are no boundaries along the coast, and that Australia has no boundaries at all! This had me scratching my head for a quite while until I properly read the documentation for boundaries and what was going on.
To solve this issue I had to get another source of geometry for the country borders and overlay it onto the OpenStreetMap (OSM) data. So I grabbed a GeoJSON file with country outlines that I had lying around from previous projects. If you want to make one from scratch you can follow Mike Bostock’s tutorial.
This is what that ended up looking like.
sources:
osm: # Same layer as before
type: TopoJSON
url: https://vector.mapzen.com/osm/all/{z}/{x}/{y}.topojson?api_key=vector-tiles-jvpqPNW
max_zoom: 16
countryData: # New layer with country borders
type: GeoJSON
url: world.json # From a local file
Now that we have two sources loaded, we can render the the map as before but overlay the country polygons.
layers:
FeatureCollection: # This was the name of the layer with the country outlines in the GeoJSON file
visible: true
data:
source: countryData # Note we are using the new source
draw:
polygons:
order: 200 # like css z-index
color: 'light-blue'
lines: # Using the lines style effectively allows us to draw a stroke.
order: 201
color: 'blue'
width: 1px
# OSM layers below
earth:
visible: true
data: { source: osm }
draw:
polygons:
order: 101 # like css z-index
color: '#C4B295'
... more layers follow ...
That will render our new geometry (the country borders) above the OSM geometry because we also set the order to be higher that that set for the OSM layers. The order property is very much like z-index in CSS, with higher numbers rendering on top of lower numbers.
However that doesn’t seem too data-vizzy as all the countries are rendered in light blue. To get choropleth-y we are going to go to the JavaScript side and use our trusty D3 toolbelt to help us out.
Step 2: Preparing the data
Step one is to load the data. After the map is initialized we will kick off our data load
scene.subscribe({
error: function (e) {
console.log('scene error:', e);
},
warning: function (e) {
console.log('scene warning:', e);
},
load: function (e) {
console.log('scene load complete');
// While the scene should be loaded, my experience so far suggests
// waiting a bit longer before trying to modify the configuration.
setTimeout(function() {
loadData();
}, 500);
}
});
In loadData we use d3.csv to fetch and parse our CSV data file. We will then reshape the data a bit to fit our purpose. Then we will create a color scale, and finally precompute the country colors to pass along to Tangram.
function loadData() {
//Load some data and then update the scene.
d3.csv("world_population.csv", function(error, data) {
// Reshape the data, and extract the population counts
// for 2015.
var rows = data.map(function(d){
return {
'code': d['Country Code'],
'name': d['Country Name'],
'population': Number(d['2015'])
}
})
// Make a color scale
//
var lowCol = "hsl(62,100%,90%)";
var highCol = "hsl(222,30%,20%)";
var domain = d3.extent(rows, function(d) {
if (d.population > 0) {
return d.population
}
});
var color = d3.scaleLog()
.domain(domain)
.range([lowCol, highCol])
.interpolate(d3.interpolateHcl);
// So it looks like the color scale can't be correctly serialized
// across to the tangram renderer (which i think is running in a web
// worker), so we pre-compute the colors here.
var mapData = rows.reduce(function(memo, d) {
d.color = color(d.population);
memo[d.code] = d;
return memo
}, {});
attachData(scene, mapData);
});
}
Step 3: Updating the Scene
So at this stage we have the colors we want computed in our script, but we still need to get that over to the yaml configuration since that is what actually drives the rendering of the map. This is what the attachData function does.
function attachData(scene, mapData) {
// Attach the data and scale functions to the scene
var config = scene.config;
config.global.data = mapData;
config.layers.FeatureCollection.draw.polygons.color = function() {
var countryData = global.data[feature.adm0_a3];
if (countryData && countryData.color) {
return countryData.color;
} else {
return 'black'
}
};
updateConfig();
}
The scene object that we used to subscribe to events at the beginning also has a live version of the configuration loaded via YAML in a config property. If we update that property we can reload the configuration and update how our map is displayed (in fact we can completely configure a tangram map this way). We can also attach data and functions to the config that can be used when rendering our scene.
First we attach the data we prepared to a special global object in the configuration. This object is readable in all draw blocks.
config.global.data = mapData; // you can use any key name to attach your data to config.global
Then we attach a new color function to the draw block of our existing layer. Note that config.layers.FeatureCollection.draw.polygons.color matches the key used to attach the original color function that just drew everything in blue. This new function looks up the country’s target color in the global data that we attached previously.
We can then call updateConfig on the scene and it will parse the new configuration and redraw! Voila.
Benefits
So why use this approach render a choropleth over the tried and true method of overlaying an SVG on top of a leaflet slippy map?
One reason is performance. This is all rendered using the Canvas WebGL renderer, which is pretty fast, the more geometry you have the greater the performance benefit you would see from using this stack.
Another important reason is that now that all the parts are being rendered in the same technology, we have much more flexible control over layer ordering. If you look through that YAML file linked below and pay attention to the order property, you will see that we can render labels and even buildings above our choropleth data layer. You can see this in the screenshot below or on the live if you zoom in close on a city, the buildings show and aren’t tinted by the country color. With the SVG overlay approach all of the leaflet layers will always be below your SVG.
Some Implementation Notes
If looking closely at the code, a question one might have is why not attach the D3 scale function to the config and just use that directly? Why go through the bother of pre-computing the colors? That is indeed what I tried at first, but I found that the data and functions passed through in this manner need to be serializable, and the color scale was not. I would thus recommend against using functions that close over variables in your JavaScript scope. Ideally, pure functions that rely only on input and whats in global should be fine.
Another thing to note is that scene.updateConfig(); isn’t terribly fast, the entire config is re-parsed and the scene rebuilt, so its not suitable for fast animations and transitions. We will see an alternative in the next post.
Here is a link to the live example containing the full source for this example
Conclusion
Using Tangram was a lot of fun, though it required a bit of head scratching here and there, and it’s really exciting to see vector tiles in action and how accessible high performance geo-based rendering in the browser is. In addition to Mapzen, other providers like the ever reliable Mapbox also support vector tiles. I hope this has been a helpful guide as you get started with these tools and drool over those demos. If you are looking to experiment with Tangram yourself, I’d also suggest checking out Tangram Play which provides a nice online sandbox for experimentation without requiring any setup.
