Borders and Big Data


Fig 1 – Big Data Analytics is a lens, the data is a side effect of new media

I was reflecting on borders recently, possibly because of reading Cormac McCarthy’s The Border Trilogy. Borders come up fairly often in mapping:

  • Geography – national political borders, administrative borders
  • Cartography – border line styles, areal demarcation
  • Web Maps – pixel borders bounding polygonal event handlers
  • GIS – edges between nodes defining faces
  • Spatial DBs – Dimensionally Extended nine-Intersection Model (DE-9IM) 0,1 1,0 1,1 1,2 2,1

However, this is not about map borders – digital or otherwise.

McCarthy is definitely old school, if not Faulknerian. All fans of Neal Stephenson are excused. The Border Trilogy of course is all about a geographic border, the Southwest US Mexico border in particular. At other levels, McCarthy is rummaging about surfacing all sorts of borders: cultural borders, language borders (half the dialogue is Spanish), class borders, time borders (coming of age, epochal endings), moral borders with their many crossings. The setting is prewar 1930’s – 50’s, a pre-technology era as we now know it, and only McCarthy’s mastery of evocative language connects us to these times now lost.

A random excerpt illustrates:

“Because the outer door was open the flame in the glass fluttered and twisted and the little light that it afforded waxed and waned and threatened to expire entirely. The three of them bent over the poor pallet where the boy lay looked like ritual assassins. Bastante, the doctor said Bueno. He held up his dripping hands. They were dyed a rusty brown. The iodine moved in the pan like marbling blood. He nodded to the woman. Ponga el resto en el agua, he said. . . . “

The Crossing, Chapter III, p.24

Technology Borders
There are other borders, in our present preoccupation, for instance, “technology” borders. We’ve all recently crossed a new media border and are still feeling our way in the dark wondering where it may all lead. All we know for sure is that everything is changed. In some camps the euphoria is palpable, but vaguely disturbing. In others, change has only lately dawned on expiring regimes. Political realms are just now grappling with its meaning and consequence.

Big Data – Big Hopes
One of the more recent waves of the day is “Big Data,” by which is meant the collection and analysis of outlandishly large data sets, recently come to light as a side effect of new media. Search, location, communications, and social networks are all data gushers and the rush is on. There is no doubt that Big Data Analytics is powerful.

Disclosure: I’m currently paid to work on the periphery of a Big Data project, petabytes of live data compressed into cubes, pivoted, sliced, and doled out to a thread for visualizing geographically. My minor end of the Big Data shtick is the map. I am privy to neither data origins nor ends, but even without reading tea leaves, we can sense the forms and shadows of larger spheres snuffling in the night.

Analytics is used to learn from the past and hopefully see into the future, hence the rush to harness this new media power for business opportunism, and good old fashioned power politics. Big Data is an edge in financial markets where microseconds gain or lose fortunes. It can reveal opinion, cultural trends, markets, and social movements ahead of competitors. It can quantify lendibility, insurability, taxability, hireability, or securability. It’s an x-ray into social networks where appropriate pressure can gain advantage or thwart antagonists. Insight is the more benign side of Big Data. The other side, influence, attracts the powerful like bees to sugar.

Analytics is just the algorithm or lens to see forms in the chaos. The data itself is generated by new media gate keepers, the Googles, Twitters, and Facebooks of our new era, who are now in high demand, courted and feted by old regimes grappling for advantage.

Border Politics
Despite trenchant warnings by the likes of Nassim Taleb, “Beware the Big Errors of ‘Big Data’”, and Evgeny Morozov Net Delusion, the latest issue of “MIT Technology Review” declares in all caps:

“The mobile phone, the Net, and the spread of information —
a deadly combination for dictators”
MIT Tech Review


Dispelling the possibility of irony – feature articles in quick succession:

“A More Perfect Union”
“The definitive account of how the Obama campaign used big data to redefine politics.”
By Sasha Issenberg
“How Technology Has Restored the Soul of Politics”
“Longtime political operative Joe Trippi cheers the innovations of Obama 2012, saying they restored the primacy of the individual voter.”
By Joe Trippi
“Bono Sings the Praises of Technology”
“The musician and activist explains how technology provides the means to help us eradicate disease and extreme poverty.”
By Brian Bergstein

Whoa, anyone else feeling queasy? This has to be a classic case of Net Delusion! MIT Tech Review is notably the press ‘of technologists’, ‘by technologists’, and ‘for technologists’, but the hubris is striking even for academic and engineering types. The masters of technology are not especially sensitive to their own failings, after all, Google, the prima donna of new media, is anything but demure in its ambitions:

“Google’s mission is to organize the world’s information and make it universally accessible and useful.”
… and in unacknowledged fine print, ‘for Google’

Where power is apparent the powerful prevail, and who is more powerful than the State? Intersections of technologies often prove fertile ground for change, but change is transient, almost by definition. Old regimes accommodate new regimes, harnessing new technologies to old ends. The Mongol pony, machine gun, aeroplane, and nuclear fission bestowed very temporary technological advantage. It is not quite apparent what is inevitable about the demise of old regime power in the face of new information velocity.

What Big Data offers with one hand it takes away with the other. Little programs like “socially responsible curated treatment” or “cognitive infiltration” are only possible with Big Data analytics. Any powerful elite worthy of the name would love handy Ministry of Truth programs that steer opinion away from “dangerous” ideas.

“It is not because the truth is too difficult to see that we make mistakes… we make mistakes because the easiest and most comfortable course for us is to seek insight where it accords with our emotions – especially selfish ones.”

Alexander Solzhenitsyn

Utopian Borders
Techno utopianism, embarrassingly ardent in the Jan/Feb MIT Tech Review, blinds us to dangerous potentials. There is no historical precedent to presume an asymmetry of technology somehow inevitably biased to higher moral ends. Big Data technology is morally agnostic and only reflects the moral compass of its wielder. The idea that “… the spread of information is a deadly combination for dictators” may just as likely be “a deadly combination” for the naïve optimism of techno utopianism. Just ask an Iranian activist. When the bubble bursts, we will likely learn the hard way how the next psychopathic overlord will grasp the handles of new media technology, twisting big data in ways still unimaginable.

Big Data Big Brother?
Big Brother? US linked to new wave of censorship, surveillance on web
Forbes Big Data News Roundup
The Problem with Our Data Obsession
The Robot Will See You Now
Educating the Next Generation of Data Scientists
Moderated by Edd Dumbill (I’m not kidding)

Digital Dictatorship
Wily regimes like the DPRK can leverage primitive retro fashion brutality to insulate their populace from new media. Islamists master new media for more ancient forms of social pressure, sharia internet, fatwah by tweet. Oligarchies have co-opted the throttle of information, doling out artfully measured information and disinformation into the same stream. The elites of enlightened western societies adroitly harness new market methods for propagandizing their anaesthetized citizenry.

Have we missed anyone?

… and of moral borders
“The battle line between good and evil runs through the heart of every man”
The Gulag Archipelago, Alexander Solzhenitsyn


We have crossed the border. Everything is changed. Or is it?

Interestingly Cormac McCarthy is also the author of the Pulitzer Prize winning book, The Road, arguably about erasure of all borders, apparently taking up where techno enthusiasm left off.

Fig 2 – a poor man’s Big Data – GPU MapD – can you find your tweets?

Kind of 3D with D3

Fig 1 – NASA Near Earth Observation - Cloud Optical Thickness using Orthographic projection

Charts and Graphs

Growing up in the days before graphing calculators (actually before calculators at all), I’ve had a lingering antipathy toward graph and chart art. Painful evenings plotting interminable polynomials one dot at a time across blue lined graph paper left its inevitable scars. In my experience, maps are a quite different thing altogether. Growing up in Ohio at the very edge of Appalachia, I have quite pleasant memories of perusing road atlases, planning escapes to a dimly grasped world somewhere/anywhere else.

The “I’m outa here” syndrome of a mid-century youth, landed me in Colorado where I quickly found the compelling connection between USGS Topo maps and pleasant weekends climbing in Colorado’s extensive National Forests. I still fondly recall the USGS map desk at Denver’s Lakewood Federal Center where rows of flat metal cabinet drawers slid out to expose deep piles of 1:24000 scale topography maps with their rich linear contour features.

I bore you with this personal recollection to set the stage for my amazement at discovering d3.js. My enduring prejudice of charts and graphs had already begun to crack a bit after seeing Hans Rosling’s entertaining Ted lectures from 2006. (D3 version of Wealth and Health of Nations). Perhaps my pentient for the concrete posed a barrier to more abstract demands of statistical modeling. At any rate Hans Rosling’s engaging enthusiasm was able to make a dent in my prejudice, and I found myself wondering how his graphs were coded. Apart from the explicit optimism conveyed by Rosling’s energy, the visual effect of primary colored balloons rising in celebratory fashion is quite hopefully contradictory of the harsh Hobbesian dictum.

“the life of man, solitary, poor, nasty, brutish, and short.”

The revelation that my childhood experience of the 50s is recapitulated by the modern economic experience of children in Trinidad and Tobago was something of an eye opener. Although no maps are in sight, the dynamic visualization of normally depressing socio-economic statistics lent visual energy to a changing landscape of global poverty.

Fig 2 – Hans Rosling’s Wealth & Health of Nations implemented in D3

The application of technology to the human condition seems so inspiring that it’s worth the reminder that technology has its flaws. Can it really be that the computational power in hand on the streets of Lagos Nigeria now exceeds that of Redmond or Palo Alto? Will ready access to MIT courseware on Probability and Statistics actually mitigate the desperation of Nairobi’s slums? A dash of Realpolitik is in order, but the graphics are none the less beautiful and inspiring.

Fig 3 – Mobile phones transform Lagos

D3.Geo is an open source javascript library for data visualization. In d3, data enters, performs, and exits with a novel select pattern geared to dynamic rendering. The houselights dim, the stage is lit, data enters stage left, the data performance is exquisite, data exits stage right. This is a time oriented framework with data on the move. The dynamism of data is the key to compelling statistics and d3 examples illustrate this time after time.

With d3.js Economics may remain the Dismal Science, but its charts and graphs can now be a thing of beauty. D3 charts are visually interesting, but more than that, chart smarts are leaking into the spatial geography domain. Mike Bostock of NYT fame and Jason Davies are prolific contributors to

D3 version 3.0, released just a couple of weeks ago, added a wide range of projective geometry to d3.geo. Paths, Projections, and Streams combine to realize a rich html5 rendering library capable of animation effects rarely seen in GIS or on the web.

Visually these are not your grandfathers’ charts and maps.


There’s more. Sean Gillies recently remarked on the advent of TopoJSON, apparently a minor side project of mbostock, fully supported by D3.Geo.Paths. In the GIS world ESRI has long held the high ground on topological data (the Arc of GIS), and now some blokes from NYT charts and graphs have squashed it into an easy to use javascript library. Wow!

TopoJSON is still more or less a compressed transport mechanism between OGC Simple Features in a server DB and client side SVG, but I imagine there will shortly be server side conversions for PostGIS and SQL Server to take advantage of major low latency possibilities. Client side simplification using the Visvalingam’s algorithm are virtually instantaneous, so zoom reduction can occur client side quite nicely.

I think you get the idea. D3.js is powerful stuff and lots of fun.

Some Experiments

It’s the New Year’s holiday with spare time to fire up a favorite data source, NASA Neo, and try out a few things. The orthographic projection is an azimuthal projection with the advantage of reproducing the visual perspective of earth from a distant vantage point.

Simple as:

var projection = d3.geo.orthographic()

Fig 4 – Natural earth world data in orthographic projection

In addition to a graticule and circle edge paths, this takes advantage of TopoJSON formatted natural earth world data published on GitHUB by, you guessed it, mbostock.

        d3.json("topojson/world-110m.json", function (error, world) {
                .attr("class", "boundary")
                .attr("d", nasa.path);

Add some mouse event handlers to whirl the globe:

mousedown: function () {
        nasa.p0 = [d3.event.pageX, d3.event.pageY];
        nasa.context.clearRect(0, 0, nasa.width, nasa.height);
    mousemove: function () {
        if (nasa.p0) {
            nasa.p0 = [d3.event.pageX, d3.event.pageY];
            nasa.projection.rotate([nasa.λ(nasa.p0[0]), nasa.φ(nasa.p0[1])]);
            nasa.svg.selectAll("path").attr("d", nasa.path);
    mouseup: function () {
        if (nasa.p0) {
            nasa.context.clearRect(0, 0, nasa.width, nasa.height);
            nasa.p0 = null;

Interactive orthographic from mbostock has a nice set of handlers adapted for this experiment. Note there are still a few quirks regarding transposing mousedown event locations to be worked out in my experimental adaptation. (My holiday free time is running out so some things have to make do.)

With mouse action, d3’s orthographic projection becomes a globe. It responds much more smoothly in Chrome than IE, apparently due to a javascript performance advantage in Chrome.

Fig 5 – javascript performance testing

This ortho globe feels 3D all because D3 affords a fast refresh through a projection on the vector continent and graticule paths.

nasa.svg.selectAll("path").attr("d", nasa.path); 

Vectors are fast, but for deeper information content I turn to NASA’s Near Earth Observation data, available as imagery from this public WMS service. The beauty of this imagery is still compelling after all these years.

Fig 6 – NASA Land night temperature overlay

However, geodetic imagery needs to be transformed to orthographic as well. D3 has all the necessary plumbing. All I added was the NASA WMS proxy with a .net WCF service.

getOverlay: function (overlay) {
        if (overlay != null && overlay.length>0) {

            nasa.image = new Image;
            nasa.image.onload = nasa.onload;
            nasa.image.src = Constants.ServiceUrl + "/GetMap?url=" + overlay + "%26SRS=EPSG:4326%26BBOX=-180.0,-90,180,90%26width="+nasa.width+"%26height="+nasa.height+"%26format=image/jpeg%26Exceptions=text/xml";

    onload: function () {

        var dx = nasa.image.width,
            dy = nasa.image.height;

        nasa.context.drawImage(nasa.image, 0, 0, dx, dy);

        var sourceData = nasa.context.getImageData(0, 0, dx, dy).data,
            target = nasa.context.createImageData(nasa.width, nasa.height),
            targetData =;

        for (var y = 0, i = -1; y < nasa.height; ++y) {
            for (var x = 0; x < nasa.width; ++x) {
                var p = nasa.projection.invert([x, y]), λ = p[0], φ = p[1];
                if (λ > 180 || λ < -180 || φ > 90 || φ < -90) { i += 4; continue; }
                var q = ((90 - φ) / 180 * dy | 0) * dx + ((180 + λ) / 360 * dx | 0) << 2;
                targetData[++i] = sourceData[q];
                targetData[++i] = sourceData[++q];
                targetData[++i] = sourceData[++q];
                targetData[++i] = 255;
        nasa.context.clearRect(0, 0, nasa.width, nasa.height);
        nasa.context.putImageData(target, 0, 0);

Looping through 1,182,720 pixels in javascript is not the fastest, but just to be able to do it at all with only a dozen lines of javascript is very satisfying. There may be some server side options to improve performance and PostGIS Raster is worthy of investigation. However, html5 browsers with enhanced GPU access should eventually supply higher performance raster transforms.

Fig 7 – NASA Land night temperatures transformed with D3 Orthographic projection.invert

Selection JsTree

For this experiment I also made use of JsTree for the layer selections out of the NASA WMS GetCapabilities. Layer choices are extensive and even with a tree expander approach the options overflow the available div space of IE’s jQuery UI draggable accordion panel. Scroll bars work poorly with draggable divs. A future enhancement could be manually allocated multiple expanders following NASA’s Ocean, Atmosphere, Energy, Land, and Life categories. Unfortunately this would invalidate the nice GetCapabilities layer extraction feature of the proxy service. NASA’s WMS also provides LegendURL elements for better understanding of the color ranges, which should be added to the selection panel.

Since d3.geo offers projection.scale(), mousewheel events are a nice to have option that is easily bound to a browser window with jquery-mousewheel plugin.

$(window).mousewheel(function (event, delta, deltaX, deltaY) {
            var s = nasa.projection.scale();
            if (delta > 0) {
                nasa.projection.scale(s * 1.1);
            else {
                nasa.projection.scale(s * 0.9);
            nasa.svg.selectAll("path").attr("d", nasa.path);
            nasa.context.clearRect(0, 0, nasa.width, nasa.height);

Tilted Perspective
Even though NEO data resolution doesn’t really warrant it, using tilted perspective d3.geo.satellite projection affords a space station view point. Incorporating a steerable camera is a more complex project than time allows.

var projection = d3.geo.satellite()
    .rotate([76.00, -34.50, 32.12])
    .center([-2, 5])

Fig 8 – Tilted perspective d3.geo.satellite projection of NASA BlueMarbleNG-TB

Possible Extensions:
Time is the next target, since NASA Neo exposes Time extension features.
<Extent default=”2012-12-11″ name=”time”>


One approach would be a time series download as an array of image objects from the proxy service, with a step through display bound to a slider on the UI. Once downloaded and projected the image stack could be serially displayed as an animation sequence. NASA WMS Time domain is somewhat abbreviated with only about 12-14 steps available. For a longer animation affect some type of caching worker role might store images as a continuous set of PostGIS raster data types with an ongoing monthly update. PostGIS has the additional advantage of pre-projected imagery fetching for more efficient time lapse performance.

IIS requires adding a .json application/json mime type to make use of TopoJSON formatted data.


Mike Bostock and Jason Davies have done us all a great favor making the power of d3 freely available on GitHUB under BSD License. For sheer prodigality and examples can hardly be matched. Anyone need a Guyou Hemisphere projection or perhaps a Peirce Quincuncial tessellation? d3.geo has it ready to serve up in web friendly fashion.

James Fee sums up d3 quite succinctly in his recent post “Why D3 Will Change How We Publish Maps”. I agree. We’re all joining the blokes over in charts and graphs.

Interrupted Bonne
World Tour
Rotating EquiRect

.NET Rocks!
Road Trip Tracking

Fig 0 - Tracking a RoadTrip


Microsoft is on the move this fall. Win8 is the big news, but Visual Studio 2012, .Net 4.5, a revamped Azure, WP8, Office 2013, and even a first foray into consumer hardware, Surface Tablets (not tables), all see daylight this fall.

The Net Rocks duo, Carl Franklin and Richard Campbell, are also on the move. Carl and Richard head out this week for a whirl wind tour through 36 states in 72 days or roughly 1728 hours. The DNR Road Trip Tracking application,, keeps tabs on the .Net Rocks RV with Tweet encouragement for the road weary travelers. All are welcome to follow the DNR RV online and add Tweet comments at #dnrRoadTrip. The app gives event information and up to the minute updates of time to next show with tweets along the route. It even gives turn by turn directions for those inclined to catch the .Net Rocks RV and follow along in the real world – .NET Rocks stalking.

Technical Overview

Project Outline

Fig 1 – .Net Rocks Road Trip Tracking app project outline


SQL Server Azure is the key resource for the DNR tracking app. GPS feeds are pushed at 1 minute intervals from a commercial Airlink GPS unit to a Windows service listening for UDP packets. This Feed Service turns incoming UDP packets into Feed records stored in SQL Azure with a geography point location and datetime stamp.

On the same system, a Twitter Query service is checking for Tweets on a 30 second interval using the Twitter REST API. Tweets are also turned into Feed records in SQL Azure. However, the geography point locations for Tweets are pulled from the latest GPS record so they are associated in time with the location of the GPS unit in the DNR RV.


Fig2 – Windows 8 IE10 browser showing stop and routes

On the front end, an Azure WebRole provides the UI and WCF service for communicating with the SQL Azure Feed data. In order to handle the widest spectrum of devices, this UI leverages jQuery Mobile sitting on top of HTML5. jQuery Mobile Supported Platforms

Inside the app divs (jQuery Mobile <div data-role=”page”..> ) are maps leveraging Bing Maps Ajax v7 API. The UI client also accesses Bing Geocode and Bing Route services through a proxy.

Fig 3 – IE9 on a laptop with GPS dots and Tweet icons along the way

Some more details:


Since there are several thousand points for each ‘event to event’ route, these are stored in SQL Azure as geography LineStrings. Using SQL Server spatial functions, the routes can be simplified on query for improved performance in both network latency and map rendering. SQL Azure’s geography Reduce(factor) function is a thinning algorithm that reduces the number of vertices of geography features while maintaining shape integrity. In this app the reduce factor is tied to zoomlevels of the map, thinning the number of points returned to the UI.

The map viewport is used as a bounding box STIntersect so only the visible routes are returned. Obviously Carl and Richard may find reasons to take a different route so the GPS breadcrumbs may wander from the Bing generated routes.


Fig 4 – WebMatrix iPad simulator

The Twitter REST API queries are simple search by hashtag queries:

To avoid returning lots of duplicate Tweets the search is limited by the last since_id in the Feed table. There are some caveats to REST Twitter searches:
“Search is focused on relevance and not completeness. This means that some
Tweets and users may be missing from search results

Fig 5 - webmatrix WP7 emulator

Fig 6 - iPhone simulator

GPS points

GPS points are generated every 60 seconds while the RV GPS unit is powered on. When the vehicle is off, and power is scarce, the unit still sends a packet once every 4 hours. Carl and Richard will be driving a lot of hours and there will be lots of points generated over the next 72 days and roughly 1728 hours. Assuming a 25% driving time over the duration, there could be as many as 1728/4 *60 = 25,920 GPS locations. Even Bing Maps Ajax v7 will choke trying to render this many locations.

In order to keep things more reasonable, there is another thinning algorithm used in the GPS query service. This is again tied to zoomlevel . At lower zoom levels points are thinned using a type of decimation – every 20th, 10th, 5th point, etc is kept depending on the zoomlevel. In addition only points required by the viewport bounding box are returned. Once the map is zoomed to higher resolution (zoom > 11) all of the points will be returned.

GPS map locations include a rollover infobox with time and detected speed at the location. We can all check up on Carl’s driving (moving: 86.99mph) and keep track of coffee stops (moving: 0.0 mph).

Bing Routes

Routing services are provided for user position to the latest GPS location and Stop venues selected on the map or from the Stop list. In addition to the route map a turn by turn directions list is provided as a page option. The GeoLocation API is used for identifying a user’s location for these routing requests. Geolocation API is an opt in API, so users need to allow location sharing to have their location automatically available. If allowed, getCurrentPosition returns a latitude, longitude which is run through the Bing reverse geocoding service to get an address used as the ‘from’ field for routing requests.

Fig 7 - Stop Detail with Maps.Themes.BingTheme()

Fig 8 - Bing Route Denver to Seattle Stop

Fig 9 - Bing Turn by Turn directions

jQuery Mobile

jQuery Mobile is a javascript library for abstracting away some of the complexity of supporting a large number of devices. WP7, Win8 tablets, iPads, iPhones, and Android devices are multiplying while traditional laptop and desktop systems have a number of browser choices and versions. jQuery Mobile is not perfect but it is a great help in a project that had to be ready in about ten days from start to finish.

One interesting feature of jQuery Mobile is the page transition effect. These are based on CSS3 and are not supported by all browsers. It adds a little pizazz to see slide, flip, and pop effects for page transitions.

JQuery Mobile apps do not have access to device native sensors such as accelerometer, compass, gyrometer, etc , so jQuery Mobile webapps will not have the full range of capabilities found in custom apps for Win8, WP7, iOS, and Android. However, just one web UI for all is an enticing benefit, while deployment is ordinary webapp rather than a series of more complex app store submissions. This approach allows easy updates over the course of the tour.

Fig 10 – Microsoft Way on an Android AVD emulator

Collecting some locations always leads to the possibility of heatmaps. These are value gradients which are helpful for analyzing geospatial data.

Fig 11 – Tweet heatmap along tour route from Seattle to Wyoming

Maybe it’s pretty obvious where Tweets are concentrated, but how about locations of app users who share their location. Australia is hot, India is not. Guess who listens to .NetRocks! Or, at least who’s less cautious about sharing location with GeoLocation API. User heatmaps bring to mind some intriguing possibilities for monitoring use, markets, and promotion effectiveness.

Fig 12 - GeoLocation users world wide


On the Road Again

Windows 8 RTM and Bing Maps

Fig 1 Windows 8 Grid App template

Windows 8 RTM was released to developers last week. Win8 with its “Metro” style UI promises to be a major reset to both the Windows OS and the Microsoft developer community. No longer called “Metro” due to some potential trademark issues, the new Win8 style is very much aimed at touch devices. It targets the full spectrum of platforms: smart phones, tablets, laptops, desktops, and servers, but touch is obviously the preferred interface.

The goal appears to be an OS that easily transitions between desktop and mobile. A worthy goal for multi device consumers who in a Microsoft world would no longer need to shift gears dramatically from one device to the next. OSs are like musical instruments, the fingerings differ from one to the next requiring an annoying shift of gestalt gears. Now that smartphones are firmly entrenched, consumers are forced to master alternative OSs and leakage out of the Microsoft OS corral is growing. Microsoft is concerned enough to take the risk of a Windows reset with Win8 Metro.

In the conventional desktop world we have Windows, Apple OS X, and Linux focused on screen, mouse, and keyboard interfaces. In the mobile world we have iOS, Android, BlackBerry OS, and WP7 focused on screen, touch, voice, and a plethora of sensors pertinent to motion. Win8 puts both into one OS, desktop and mobile, but mobile is the front man. WP8 and Win8 appear to be on a merging trajectory.

Win8 and Maps
From a mapping perspective there are again two paths, desktop and mobile (Metro)
… oh and plain old internet.

IE10 forks browser web apps as well:
(with some nice html5, css additions, … and websockets)
1) Win8 desktop IE10 – compatible desktop experience including plugins like Silverlight
2) Win8 mobile IE10 – plugin free and touch complete. Silverlight is out for mobile IE10, as is any sanctioned 3D.

There are also cosmetic differences that make desktop a compatibility version and mobile optimized for smaller form factors and touchy gestures. Bing Maps ajax v7 is slightly upgraded with a new Bing Theme module. As far as IE10 browsers, Bing Maps webapps using generic html5, css, jQuery Mobile with Bing Maps Ajax v7 will work with both IE10 branches and most any device, as well as avoiding Windows Store distribution.

Win8 Desktop – old stuff
Not much to say here. Everything I tried worked the same as usual. Bing Maps ajax v7, Bing Maps Silverlight, and Bing Maps WPF controls are all supported. Normal ClickOnce install works as before. VS2012 continues to evolve to accommodate Azure and mobile target platforms – ARM joins x86 and x64. Everything is familiar once you get to the desktop, but there can be some discovery process getting there.

Win8 Mobile – new stuff
A lot to explore here. – some media info on user experience for mobile Win8 Bing.

From a Bing Maps developer perspective there are again two paths. (Lots of dualism in the picture) ref Windows API for Metro apps
Both paths support distribution through the new Windows Store.

1) Bing Maps SDK for JavaScript – Windows 8 JavaScript API is the same as the existing Bing Maps ajax v7 API and makes transitions easy for existing ajax v7 map applications. Starting a project in VS2012 allows selection of a template with all the appropriate boilerplate. The map action is exactly the same as Bing Maps ajax v7 API.

Sample html for Bing Maps JavaScript SDK:

<!DOCTYPE html>
    <meta charset="utf-8">
    <meta name="viewport" content="width=1024, height=768" />

    <!-- WinJS references -->
    <link href="//Microsoft.WinJS.1.0/css/ui-dark.css" rel="stylesheet" />
    <script src="//Microsoft.WinJS.1.0/js/base.js"></script>
    <script src="//Microsoft.WinJS.1.0/js/ui.js"></script>

    <!-- BingMaps-JavaScript-SDK references -->
    <script type="text/javascript" src="ms-appx:///Bing.Maps.JavaScript//js/veapicore.js"></script>
    <script type="text/javascript" src="ms-appx:///Bing.Maps.JavaScript//js/veapiModules.js"></script>

    <!-- application specific -->
    <link href="/css/default.css" rel="stylesheet">
    <script type="text/javascript"  src="/js/default.js"></script>

<div id="content">
    <div id="header"> </div>
    <div id="mapdiv"> </div>

Fig 2 – Windows 8 JavaScript API sample

Javascript SDK event types are just the same as the existing ajax v7 API and do not include all the new sensor capabilities. However, the more interesting parts of Win8 are the multiple sensor support that requires Win8 Bing Maps SDK for Metro.

2) XAML Bing Maps control – This is the most comprehensive SDK with access to all core platform features. Bing Maps SDK for Metro style apps is still RTM Beta with final build due at the end of September. For the present Maps will be decorated with a large BETA band.

First there are some special Bing Map Keys available:

  1. Basic Windows Metro style app – Application is a Windows Metro style app that is publicly available without restriction and does not exceed 500,000 transactions of any type within a 12 month period.
  2. Enterprise Windows Metro style app – Application is a Windows Metro style app that is a public or a private application

The distinguishing capability in Bing Maps SDK for Metro style is access to sensors when available on a target platform:

Sensor classes:

  •  Accelerometer – This sensor returns G-force values with respect to the x, y, and z axes.
  • Compass – This sensor returns a heading with respect to True North and, possibly, Magnetic North.
  • Gyrometer – This sensor returns angular velocity values with respect to the x, y, and z axes
  • Inclinometer -This sensor returns pitch, roll, and yaw values that correspond to rotation angles around the x, y, and z axes, respectively.
  • LightSensor – This sensor returns the ambient-light reading as a LUX value.
  • OrientationSensor – This sensor returns a rotation matrix and a Quaternion that can be used to adjust the user’s perspective in a game application.
  • SimpleOrientationSensor – This sensor detects the current quadrant orientation of the specified device as well as its face-up or face-down status.

GPS is a subset of GeoLocation which can use fall back locations such as ip or WiFi triangulation as well as gps.

Fig 3 - Example GeoLocation lookup in Bing Maps SDK for C# .

Fig 4 Win8

Fig 4 – Win8 Bing Maps SDK Beta

Example XAML Page


    <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">
            <ColumnDefinition Width="350" />
            <ColumnDefinition Width="*" />
            <RowDefinition Height="*" />
        <StackPanel Orientation="Vertical" VerticalAlignment="Top" HorizontalAlignment="Center">
            <Button x:Name="btn" Click="btnClicked" Content="Get Location" FontSize="26"/>
            <Grid x:Name="Output" HorizontalAlignment="Left" VerticalAlignment="Top" Grid.Row="1">
                        <ColumnDefinition Width="Auto" />
                        <ColumnDefinition />
                        <RowDefinition Height="Auto" />
                        <RowDefinition Height="Auto" />
                        <RowDefinition Height="Auto" />
                    <TextBlock TextWrapping="Wrap" Grid.Row="0" Grid.Column="0" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Left" Text="Latitude: " />
                    <TextBlock TextWrapping="Wrap" Grid.Row="1" Grid.Column="0" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Left" Text="Longitude: " />
                    <TextBlock TextWrapping="Wrap" Grid.Row="2" Grid.Column="0" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Left" Text="Accuracy: " />
                    <TextBlock x:Name="ScenarioOutput_Latitude" TextWrapping="Wrap" Grid.Row="0" Grid.Column="1" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Right" Text="No data" />
                    <TextBlock x:Name="ScenarioOutput_Longitude" TextWrapping="Wrap" Grid.Row="1" Grid.Column="1" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Right" Text="No data" />
                    <TextBlock x:Name="ScenarioOutput_Accuracy" TextWrapping="Wrap" Grid.Row="2" Grid.Column="1" Style="{StaticResource BasicTextStyle}"  FontSize="24" HorizontalAlignment="Right" Text="No data" />

        <bm:Map Grid.Column="1" x:Name="map" Width="Auto" Height="Auto" MapType="Road" ZoomLevel="12" Credentials="<Bing Key Here>" >
                <bm:Location Latitude="47.62" Longitude="-122.19" />

C# code behind

using Bing.Maps;
using System;
using System.Threading;
using System.Threading.Tasks;
using Windows.Devices.Geolocation;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Navigation;

namespace BingMap1

    public sealed partial class MainPage : Page
        private Geolocator _geolocator = null;
        private CancellationTokenSource _cts = null;

        public MainPage()

        /// <summary>
        /// Invoked when this page is about to be displayed in a Frame.
        /// </summary>
        /// <param name="e">Event data that describes how this page was reached.  The Parameter
        /// property is typically used to configure the page.</param>
        protected override void OnNavigatedTo(NavigationEventArgs e)
            _geolocator = new Geolocator();

        private void btnClicked(object sender, RoutedEventArgs e)
            GetGeolocation(sender, e);

        async private void GetGeolocation(object sender, RoutedEventArgs e)
                _cts = new CancellationTokenSource();
                CancellationToken token = _cts.Token;

                Geoposition pos = await _geolocator.GetGeopositionAsync().AsTask(token);
		  // update text
                ScenarioOutput_Latitude.Text = pos.Coordinate.Latitude.ToString();
                ScenarioOutput_Longitude.Text = pos.Coordinate.Longitude.ToString();
                ScenarioOutput_Accuracy.Text = pos.Coordinate.Accuracy.ToString();
		   // update map location
                map.SetView(new Location(pos.Coordinate.Latitude, pos.Coordinate.Longitude), 12);
            catch (System.UnauthorizedAccessException ue)
                ScenarioOutput_Latitude.Text = "No data";
                ScenarioOutput_Longitude.Text = "No data";
                ScenarioOutput_Accuracy.Text = "No data";
            catch (TaskCanceledException te)
                _cts = null;

Fig 5 – Metro style tiles including the above BingMaps1 project

Fig 6 – Win8 Bing Maps SDK with GeoLocation and BirdsEye view

The Windows Store is a prominent tile in the Start page. Win8 applications are vetted through a Microsoft analysis and certification process before distribution through the public Windows Store. Of course any app created in VS2012 is available as a tile in the Start Page of the development system, but getting an app to clients happens through the Windows Store. Enterprise LOB applications do have a SideLoading process to bypass the public Windows Store.

Fig 7 – Windows Store for Win8 app delivery (Sideloading is possible for Enterprise licensees)

At present Win8 is still a future affair. Developers can play, while consumers wait. Microsoft dominance of desktops will guarantee a desktop market going forward. Since neither phone nor tablet will be available to consumers with Win8 until later 4th quarter, Win8 is an OS focused on devices that may or may not generate a market. The risk is alienating a relatively loyal base of desktop users in the hope of gaining traction in the fast growing mobile market. Without a market, developers will not devote much effort to all the new stuff.

Win8 is a “do or die” gambit for Balmer, Sinofsky, and friends. Mobile is moving fast and Microsoft’s future in mobile remains in serious question. If the mobile market is entirely lost, Microsoft is headed toward a long slow Gibbonesqe decline. However, Win8 is potentially the lever required to pry Microsoft into at least a place at the mobile table. We won’t know much for a few more quarters. In the meantime steep learning curves ahead for Bing Maps devs.

Fig 8 - NASA Avg Day Temperatures over past month

Mobile Maps with WebMatrix2 and jQuery Mobile

Fig 1 – WebMatrix 2 iPhone Simulator

Some changes have been rolling out of Redmond recently. Perhaps the realization that mobile is truly capable of tipping the Windows canoe is sinking in. The underdog viewpoint is relatively new to Microsoft. The newest Azure rollout includes more versatile VM instances and lots of languages hooks.

Fig 2 – the newest Azure SDK has lots of language hooks

Github:windows plays nice with Windows Azure Websites and Microsoft offers a free website development IDE tool, WebMatrix2. This is not a replacement for Visual Studio, but it’s a great little tool for javascript UI development.

Going Mobile

There are a few ways to approach mobile development. You can develop for each of the popular platforms, but that takes a bit of effort getting the tools for iOS and Android as well as learning enough Objective-C and Android Java to be productive.

There are some tools for cross platform development like Titanium Appcelerator, which provide a runtime for translating controls written in a javascript format to native controls on the device. This requires learning new controls, IDE tools, as well as adding the runtime to your app.

Both of the above require registration and familiarity with various app store submission processes along with a fee.

3rd Way Mobile

One of the easiest ways to make a mobile map app is to just use the familiar web platform approach leveraging HTML5. Expose some data through a custom service layer or hook up to a REST API, and then use jQuery Mobile to make UI apps with HTML5 targeted to mobile clients. This won’t require an app store submission or their requisite fees.

WebMatrix adds some nice features for website development. Just install the tool and open a directory project to get started.

Fig 3 – WebMatrix2 with a MobileLocator project

WebMatrix makes it easy to check a design in most of the common browsers and mobile platforms.

Fig 4 – WebMatrix Run Control exposes multiple browser and simulator options

Mobile simulators for iPhone, iPad, and WP7 are not installed by default. Click on Gallery to check for available online extensions to WebMatrix.

Fig 5 – WebMatrix Gallery button for extensions

For mobile development a couple of “need to have” extensions are the iPhone and iPad Simulators.

Fig 6 – WebMatrix gallery options iPad and iPhone Simulators

A WP7 emulator is also available.

Fig 7 – WebMatrix Windows Phone 7 Emulator

WebMatrix extensions are not available for Android, probably due to some sort of license restriction. However, Android emulators are part of the free Android SDK install under the Android AVD manager. Android testing requires a separate emulator setup without the convenience of the WebMatrix Run list.

Fig 8 – Android emulators are part of the Android SDK install

Once the Android AVD is available, testing is just a matter of typing in the local url of a webapp site and checking for visual integrity.

jQuery Mobile

jQuery Mobile offers HTML5 javascript controls specialized for mobile devices. It abstracts away many of the variations in html5 implementation.
(See supported platform list)

jQuery Mobile makes use of specialized javascript metadata to create jQuery type controls from a single html file. Element attributes like data-role, data-rel, and data-transition add html5 objects and behaviors suited to mobile devices. The data-transition attribute adds animated transition effects. (Note, not all browsers support data-transition but fallback is benign)

<a href="index.html" data-transition="slidefade">Transition to page</a>

jQuery Mobile Theme Roller is a styling mechanism with readymade themes. This is a bit different from jQuery UI themes. Using data theme attributes ‘a’ through ‘e’, called swatches, styling propagates to all contained elements:

<div data-role="page" data-theme='b' id="foo">

Sample Mobile App
With jQuery Mobile the basic UI tools are available for controlling a map application such as this store finder with driving directions. Using HTML5 has the additional benefit of support across both mobile and desktop browsers. Now a useful Mobile UI can be added to a map finder service layer by just adding a page link to a consumer facing business portal.

In this sample, location points are viewed as a map or a list. Bing route and geocode services are used to get basic polyline route and turn by turn directions. Not rocket science, and with jQuery Mobile this is an easy extension geared to mobile device clients.

Fig 9 – Sample app shown in the iPad and iPhone WebMatrix simulators

Fig 10 – iPad Simulator showing a Bing generated Route

Fig 11 – iPad simulator showing the turn by turn directions view

Fig 12 – WP7 WebMatrix Emulator


The last 6 months have seen a dramatic uptick in mobile phone clients at corporate websites. Adding a mobile option lets customers access simple map finders and directions while on the road. Customers en route typically do not want to install a web app just to find nearby coffee, gas, or grocery stores carrying a particular brand of pretzels. So there is a place for leveraging HTML5 to escape the app store corral, and jQuery Mobile makes it easy.

TerraServer RIP

“Two roads diverged in a yellow wood”

Fig 1 MapSavvy WMS referenced in AutoCAD Map 3D

A Brief History of WMS

Open Geospatial Consortium, OGC, has been an influential geospatial standards body since the late 90’s. OGC standards are at the root of SQL spatial databases, commercial GIS tools, as well as most open source mapping projects found at OSGeo. OGC standards have been adopted at all levels from Microsoft SQL Server, Oracle Spatial, ESRI, and AutoCAD to PostGIS, GeoServer, and Quantum GIS. OGC standards are part of the DNA of modern GIS and internet mapping.

One of the more popular OGC standards, first published at the turn of the millennium, is Web Mapping Service. WMS is a map distribution standard for showing arbitrary BBox map areas in common internet browsers.

One of the early significant WMS services was a partnership between Microsoft Research and the USGS called Microsoft TerraServer, which was in continuous operation from 1998 until today, 5/1/2012. More recently the name was changed to MSR Maps. Microsoft TerraServer pioneered free availability of geospatial data in an OGC compliant service. At the same time, TerraServer research established the viability of quad tile storage, but then merged tiles into arbitrary BBox chunks for WMS GetMap requests.

TerraServer was grandfather to two forks of web mapping, quad tile pyramids and WMS open data services. May 1st 2012 marks the end of this pioneering service.

requiescat in pace

How WMS works

WMS defines a standard method of publishing capabilities to the web and furnishing images of requested areas. A WMS client first asks what layers are available, in what formats, styles, and projections. Some decisions are made depending on need and availability. These decisions along with a specific bounding box are sent back to the WMS server. The WMS server builds the requested image and returns it as a rectangular png, jpeg, tiff etc. Behind the WMS server is some type of data repository, either vector data or imagery, but the WMS server handles all requests for specific parts of this data resource.

Fig2 - OGC Web Mapping Service

This was an ambitious scheme in 2000, attempting to smooth the distribution of a multitude of proprietary formats and display methods. The concept was simply to make access to spatial data a “write once, use anywhere architecture.” Momentum built slowly at first, but by this time, May of 2012, there are many choices for WMS servers, and WMS clients are built into a multitude of mapping tools and websites.

KaBoom! Two roads

In 2005 this whole approach was upended with the introduction of web maps and a new tile serving slippy map architecture. The problem WMS ran into was the limit of CPU resources attempting to build individual map requests for millions of users. Servers eventually bogged down trying to fulfill requests for arbitrary WMS viewports. The new slippy map architecture doesn’t build anything. Tiles flow to clients simply as requested, directly from storage or cache. The browser client is responsible for rendering, so web mapping suddenly became massively parallel. Map pan and zoom went from request, pause, display, to continuous uninterrupted motion, quite magical at its first introduction.

This approach was a recapitulation of TerraServer storage but dropped the WMS at the end of the server pipe. Tile pyramids became just static image stores streaming into a map client as needed by pan and zoom.

WMS was fine for the technical folks and enterprise focus on proprietary assets, but it just didn’t scale into the internet consumer realm. Web mapping forked at this point. WMS continued to grow as a basic function of every desktop map tool and internet client, while web mapping adopted vast tile pyramids prebuilt from vector features and aerial imagery. Microsoft Bing Maps, MapQuest, and Google Maps are the glamorous handmaidens of search, serving up map tiles to the masses. OGC services haul water behind the scenes, with the pocket protector crowd.

Between Two Roads

The two architectures coexist in the internet realm. OGC services are work horses in engineering cubicles, but the mobile generation is all about slippy tiles. Many web apps merge one of the world scale base map tile pyramids with layers of OGC services on top. This approach marshals world level extent together with locally maintained assets. Example Fig 3

Fig 3 – Bing Maps behind Census Bureau TIGER WMS/WFS layers

Against the Stream

WMS layers can live on top of popular web maps like Bing, but what about the opposite direction? Can WMS clients make use of worldwide tile pyramids?

“A dead thing can go with the stream, but only a living thing can go against it.”
G.K. Chesterton

OnTerra MapSavvy

Far down stream from Jim Gray and Tom Barclay, OnTerra created a simple WMS service to bridge the fork in the road between OGC and Bing Maps tiles, but in the opposite direction. MapSavvy is a WMS service that simply exposes Bing Maps Aerial and Road imagery to existing users of WMS clients. Subscribe to the service and imagery for the entire world is available inside your mapping tool of choice.

There are thousands of man years of development tied up in tools that perform very specific business functions. These tools are not especially public facing. They perform the required task, but the task is of no particular interest to the rest of us. Companies have to pay people to use tools like this. In cases of business focused WMS tools, wouldn’t it be nice to access the vast resources of tile imagery abounding on the internet?

For example, Microsoft is just finishing a program to capture most of the US and Western Europe at 30cm pixel resolution and serving this in Bing Maps Aerial. This kind of resolution at national scales is a very useful resource. Designers, engineers, defense agencies, and GIS analysts can always use higher resolution imagery available for large parts of the planet. Lower resolution imagery fills in context for the remainder of the world. Ref: Bing Maps resolution chart

Bing Maps 30cm Aerial: “The project is scheduled for completion by June 2012 at which point a refresh cycle will begin to update much of what will already have been collected, with priority placed on locations that have been more subject to change (as opposed to, say, the Mojave Dessert).” See Fig 4

Fig 4 – Global Mapper WMS client using MapSavvy - Mojave Desert 0.30 m per pixel

Here is a short list from the multitude of WMS clients:

• AutoCAD Map 3D – Fig 1
• Bentley MicroStation V8i
• Gaia
• Global Mapper – Fig 4
• Tableau
• Smallworld GeoSpatial
• gvSig – Fig 5
• Quantum GIS – Fig 6
• MapInfo
• TatukGIS Viewer
• TopoFusion

Fig 5 – gvSIG WMS client using MapSavvy - Coors Field in Denver

Fig 6 – QGIS WMS client - Detail of Coors Field Blake Street entrance - 0.30 m per pixel

Some Details

Bing Maps, like all web mapping tile services, leverages the symmetry of a Mercator world projection, which is technically defined as epsg:3857. In order to support the widest range of WMS clients MapSavvy publishes both epsg:3857 and epsg:4326. The second projection has less value, as it distorts the imagery, but, in cases where a WMS client cannot make use of the more accurate epsg:3857, epsg:4326 is helpful. This is especially useful, at higher resolutions where the distortion is reduced. MapSavvy helps resolve some of the distortion issues of epsg:4326 by using transformed images for zoom levels at the top of the tile pyramid.

Unlike Google Maps, which explicitly excludes use of Google tiles in a WMS service, Bing Maps wants customers to use their tile servers. OnTerra licenses Microsoft Bing Map tiles for use in a WMS. MapSavvy subscriptions include a restricted Microsoft sub-license for use of Bing road and imagery tiles in the client, which simplifies life for small business users.

Some TerraServer Alternatives

TerraServer has historical interest but in the past five years there hasn’t been much demand for DOQ, Urban Ortho, or DRG Topo maps. The demise of TerraServer may go largely unnoticed. For those left orphaned, National Atlas publishes several WMS endpoints. However, Bing Maps Aerial is a much improved replacement for USGS DOQ and USGS Urban Ortho. MapSavvy WMS can replace TerraServer DOQ and Urban Ortho layers directly in existing WMS clients.

If you’re looking for alternative USGS Topo contours, Ahlzen’s TopOSM is a good place to start.

TopOSM Contours

Fig 7 TopOSM contours over Bing Maps


Microsoft TerraServer was the starting point for a lot of what we take for granted in 2012. Its passing is understandable but leaves a twinge of regret.

Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;

- Robert Frost

“When you arrive at a fork in the road, take it.”
- Yogi Berra

Leaflet and Census TIGERweb

Fig 1 – Sample Leaflet interface to the US Census Bureau’s new TIGERweb WMS service with Bing Maps background


US Bureau of the Census – some background

1990 Census
The 1990 Census line data made a revolutionary leap from GBF-DIME format to TIGER, Topologically Integrated Geographic Encoding and Referencing, and also adopted CD-ROM media distribution. In the early 90′s I wrote a few TIGER translation programs for AutoCAD users and users of a popular (now defunct) desktop GIS package called Atlas*GIS. TIGER was useful for its linear data, but was more often used for the boundary polygons that could be tied to Census population demographic data. The line features of the 1990 TIGER were based on the older 1980 GBF-DIME enhanced with USGS 1:100,000 DLG features. GBF DIME was notable for the tell tale zigzag streets in urban areas.

2000 Census
By the mid 90’s Open GIS Consortium (OGC) was actively working on OpenGIS standards. As the internet continued ramping up, OGC released the OpenGIS Web Map Server (WMS) Specification in 2000. Even though Census data releases are a year or so after the actual census, it was several more years before the Census Bureau started investigating alternative WMS services for distributing data. In the meantime boundaries were available for internet and ftp download in ESRI shp and E00 formats. This was convenient, but requires self hosting web solutions.

2010 Census
Ever since the 2000 WMS specification there were hopes for a service distribution of census TIGER/Line. Not until the end of 2011 was this belatedly realized as TIGERweb. But of course by the mid 2000s internet mapping began changing radically with the popularity of Google Maps, GPS, Bing Maps, Open Street Maps, and now ubiquitous mobile devices. The Census Bureau will need to continue evolving to meet the challenges of these rapid changes.

TIGERweb 2011
So why not use the TIGERweb for an experimental look at some more recent web tools? TIGERweb Viewer UI is based on the ESRI Silverlight APIs. Silverlight is a powerful UI technology, but it is destined to be supplanted by html5. As browser market fragmentation and mobile devices force adoption of standards based technologies, proprietary plugins are just not making the transition, at least for public facing web apps.

Fig 2 – TIGERweb Viewer based on ESRI Silverlight API

TIGERweb Viewer seems oddly designed. Rather than the clean, spare look popular today, TIGERweb Viewer seems to be a throwback to an earlier internet. Unusual matrice animations with rotating panels and creepy crawling boundary selectors seem a bit out of place in the modern web. Even though technically possible, and simple to implement in Silverlight, such distractions do nothing to make the UI endearing. After the first blink most users will inevitably recall early internet flashing text and PowerPoint decks filled with animated gifs. Anyone remember the irritation of Clippy?

However, TIGERweb WMS Service is a wide open door for standards based javascript libraries. One of the more elegant available tools is LeafletJS. This is a “lightweight” js map library that works well with jQuery and abstracts away some of the complexity of supporting a fragmenting mobile browser world.

Leaflet is Open Source BSD-licensed with an active contributing community and available on github. It uses newer html5 technologies available in A-grade browsers, but includes fall back for older browsers. It’s nice to have touch events and the geolocation api.

The Leaflet api provides a convenient L.TileLayer.WMS object that transforms a WMS endpoint into a slippy tile map. Performance isn’t as good as a static tile pyramid, but it isn’t bad and simplifies life immensely.

  var censusTracts = new L.TileLayer.WMS("",
    layers: 'Census Tracts',
    format: 'image/png',
    transparent: true,
    attribution: TigerAttribution

Bing Maps
Unlike Google Maps, Microsoft Bing allows licensed access to tile servers. For development, non-profit, and education uses, this is just a matter of registering for a Bing key. Adding a Bing Maps layer with Leaflet is possible using Once this Leaflet Bing extension javascript is referenced into a page, adding Bing Map styles is simply a matter of adding L.TileLayer.Bing.

var BingAerialWithLabels = new L.TileLayer.Bing(bingkey, "AerialWithLabels",
  maxZoom: 21,
  attribution: BingAttribution

Fig 3 – Leaflet Census TIGER WMS with Bing Maps Road style

TIGER/Line 2010 accuracy has been upgraded substantially, but you can occasionally see some discrepancies against Bing Aerial.

Fig 4 – Leaflet Census TIGER WMS with Bing Maps Aerial with Labels style

Once all of the WMS and Bing L.TileLayers are instantiated, a new L.Map with the desired L.TileLayer set is declared this way:

            map = new L.Map('mapDiv',
                        center: new L.LatLng(39.748, -104.981),
                        zoom: 8,
                        layers: [censusStates, censusCounties, censusTracts, censusBlockGrps, censusBlocks]

Leaflet also provides a simple L.Control.Layers for a user layer control panel:

            var baseMaps = {
                "BingRoads": BingRoad,
                "BingAerial": BingAerial,
                "BingAerialWithLabels": BingAerialWithLabels

            var overlayMaps = {
                "States": censusStates,
                "Counties": censusCounties,
                "Tracts": censusTracts,
                "BlockGroups": censusBlockGrps,
                "Blocks": censusBlocks,
                "Lakes": censusArealHydrography,
                "Rivers Streams": censusLinearHydrography,
                "LocalRoads": censusLocalRoads,
                "SecondaryRoads": censusSecondaryRoads,
                "PrimaryRoads": censusPrimaryRoads,
                "Nexrad": nexrad

            var layersControl = new L.Control.Layers(baseMaps, overlayMaps,
                collapsed: !L.Browser.touch

Fig 5 – Leaflet L.Control.Layers

TIGERweb WMS GetFeatureInfo is available for queries. For example:,37.75,-122.25,38.0&width=1000&height=1000&QUERY_LAYERS=Counties&INFO_FORMAT=text/xml&X=100&Y=100

<?xml version="1.0"?>
<FeatureInfoResponse xmlns="" xmlns:esri_wms="">
  HU100="111214" --HousingUnits
  NAME="Marin County"

jQuery makes XMLHttpRequest communication easy, but TIGERweb WMS doesn’t expose an open ClientAccessPolicy.xml, so a self hosted proxy is required.

                type: "GET",
                url: getInfoURL,
                dataType: "xml",
                success: GetFeatureInfo

Fig 6 – Leaflet Popup panel with the results of TIGERweb WMS GetFeatureInfo request

Some Caveats

1) The TIGERweb WMS furnishes images for all the TIGER boundary and line features. However, there doesn’t seem to be a WMS layer for labels like those on the TIGERweb Viewer. There is an undocumented label service that might be useful.

2) WMS GetFeatureInfo requests can provide information from the polygon stack, but the lack of a cross-domain policy file on the TIGERweb WMS server means that a pass through proxy service needs to be hosted on the same server as the Leaflet web app. This proxy service simply provides the same GetFeatureInfo result to the client while avoiding cross-domain issues.

1) Bing javascript
The Bing javascript extension seemed to run into a problem with intersect at this line:
if (zoom = coverage.zoomMin && coverage.bbox.intersects(bounds))

Removing the intersect(bounds) clause seemed to get things working without noticeable problems:
if (zoom = coverage.zoomMin )

2) Leaflet seems to have problems with TileLayer ordering. Bing TileLayers can mask other features if directly added to the map layer set.

3) There are some odd behaviors on Amazon Kindle Fire, possibly due to the Silk web browser’s EC2 caching.

4) Leaflet L.Control.Layers doesn’t work consistently in the mobile browsers I tested. The collapsed layer control icon never opened for iPhone or Kindle Fire. Adding the suggested option collapsed: !L.Browser.touch keeps the layer control open instead of closing when untouched. This wouldn’t be a problem except that mobile devices don’t have the screen space for extra panels.

5) The default attribution for Bing gets quite lengthy, especially when displaying Aerial vendor meta data. Turning this off and deferring to the Bing Maps Logo frees up more screen space which is at a premium on mobile devices.


Census Bureau’s new TIGERweb is a great step forward for users of TIGER data. Hopefully the momentum will continue forward to include demographic data. SF1 data joined to TIGER polygons would be a more useful service. Adding an OGC WFS service would be the ideal, but WFS always has problems with potentially very large result sets, so I’m not sure this is practical on a public OGC service.

Since the first time I saw Leaflet I’ve been intrigued with its elegance and the potential for abstracting away some of the issues surrounding web apps targeted at mobile and touch clients. In spite of a few minor issues Leaflet is one of the best javascript map apis I’ve used. I believe the issues I ran into are on a bug list and could be resolved by next release.

This bare bones CensusTest project took only a few hours over the weekend to write using Leaflet.

Fig 7 – Lower Manhattan

Extraterrestrial Map Kinections


Fig 1 – LRO Color Shaded Relief map of moon – Silverlight 5 XNA with Kinect interface


Silverlight 5 was released after a short delay, at the end of last week.
Just prior to exiting stage left, Silverlight, along with all plugins, shares a last aria. The spotlight now shifts abruptly to a new diva, mobile html5. Backstage the enterprise awaits with a bouquet of roses. Their concourse will linger long into the late evening of 2021.

The Last Hurrah?

Kinect devices continue to generate a lot of hacking interest. With the release of an official Microsoft Kinect beta SDK for Windows, things get even more interesting. Unfortunately, Kinect and the web aren’t exactly ideal partners. It’s not that web browsers wouldn’t benefit by moving beyond the venerable mouse/keyboard events. After all, look at the way mobile touch, voice, inertia, gyro, accelerometer, gps . . . have all suddenly become base features in mobile browsing. The reason Kinect isn’t part of the sensor event farmyard may be just a lack of portability and an ‘i’ prefix. Shrinking a Kinect doesn’t work too well as stereoscopic imagery needs a degree of separation in a Newtonian world.

[The promised advent of NearMode (50cm range) offers some tantalizing visions of 3D voxel UIs. Future mobile devices could potentially take advantage of the human body’s bi-lateral symmetry. Simply cut the device in two and mount one half on each shoulder, but that isn’t the state of hardware at present. ]


Fig 2 – a not so subtle fashion statement OmniTouch


For the present, experimenting with Kinect control of a Silverlight web app requires a relatively static configuration and a three-step process: the Kinect out there, beyond the stage lights, and the web app over here, close at hand, with a software piece in the middle. The Kinect SDK, which roughly corresponds to our visual and auditory cortex, amplifies and simplifies a flood of raw sensory input to extract bits of “actionable meaning.” The beta Kinect SDK gives us device drivers and APIs in managed code. However, as these APIs have not been compiled for use with Silverlight runtime, a Silverlight client will by necessity be one step further removed.

Microsoft includes some rich sample code as part of the Kinect SDK download. In addition there are a couple of very helpful blog posts by David Catuhe and a codeplex project, kinect toolbox.

Step 1:

The approach for using Kinect for this experimental map interface is to use the GestureViewer code from Kinect Toolbox to capture some primitive commands arising from sensory input. The command repertoire is minimal including four compass direction swipes, and two circular gestures for zooming, circle clockwise zoom in, and circle counter clockwise zoom out. Voice commands are pretty much a freebie, so I’ve added a few to the mix. Since GestureViewer toolbox includes a learning template based gesture module, you can capture just about any gesture desired. I’m choosing to keep this simple.

Step 2:

Once gesture recognition for these 6 commands is available, step 2 is handing commands off to a Silverlight client. In this project I used a socket service running on a separate thread. As gestures are detected they are pushed out to local port 4530 on a tcp socket service. There are other approaches that may be better with final release of Silverlight 5.

Step 3:

The Silverlight client listens on port 4530, reading command strings that show up. Once read, the command can then be translated into appropriate actions for our Map Controller.


Fig 3 – Kinect to Silverlight architecture

Full Moon Rising


But first, instead of the mundane, let’s look at something a bit extraterrestrial, a more fitting client for such “extraordinary” UI talents. NASA has been very busy collecting large amounts of fascinating data on our nearby planetary neighbors. One data set that was recently released by ASU, stitches together a comprehensive lunar relief map with beautiful color shading. Wow what if the moon really looked like this!


Fig 4 – ASU LRO Color Shaded Relief map of moon

In addition to our ASU moon USGS has published a set of imagery for Mars, Venus, Mercury, as well as some Saturn and Jupiter moons. Finally, JPL thoughtfully shares a couple of WMS services and some imagery of the other planets:

This type of data wants to be 3D so I’ve brushed off code from a previous post, NASA Neo 3D XNA, and adapted it for planetary data, minus the population bump map. However, bump maps for depicting terrain relief are still a must have. A useful tool for generating bump or normal imagery from color relief is SSBump Generator v5.3 . The result using this tool is an image that encodes relative elevation of the moon’s surface. This is added to the XNA rendering pipeline to combine a surface texture with the color relief imagery, where it can then be applied to a simplified spherical model.


Fig 5 – part of normal map from ASU Moon Color Relief imagery

The result is seen in the MoonViewer client with the added benefit of immediate mode GPU rendering that allows smooth rotation and zoom.

The other planets and moons have somewhat less data available, but still benefit from the XNA treatment. Only Earth, Moon, Mars, Ganymede, and Io have data affording bump map relief.

I also added a quick WMS 2D viewer html using OpenLayers against the JPL WMS servers to take a look at lunar landing sites. Default OpenLayers isn’t especially pretty, but it takes less than 20 lines of js to get a zoomable viewer with landing locations. I would have preferred the elegance of Leaflet.js, but EPSG:4326 isn’t supported in L.TileLayer.WMS(). MapProxy promises a way to proxy in the planet data as EPSG:3857 tiles for Leaflet consumption, but OpenLayers offers a simpler path.


Fig 6 – OpenLayer WMS viewer showing lunar landing sites

Now that the Viewer is in place it’s time to take a test drive. Here is a ClickOnce installer for GestureViewer modified to work with the Silverlight Socket service:

Recall that this is a Beta SDK, so in addition to a Kinect prerequisite, there are some additional runtime installs required:

Using the Kinect SDK Beta

Download Kinect SDK Beta 2:

Be sure to look at the system requirements and the installation instructions further down the page. This is Beta still, and requires a few pieces. The release SDK is rumored to be available the first part of 2012.

You may have to download some additional software as well as the Kinect SDK:

Finally, we are making use of port 4530 for the Socket Service. It is likely that you will need to open this port in your local firewall.

As you can see this is not exactly user friendly installation, but the reward is seeing Kinect control of a mapping environment. If you are hesitant to go through all of this install trouble, here is a video link that will give you an idea of the results.

YouTube video demonstration of Kinect Gestures


Voice commands using the Kinect are very simple to add so this version adds a few.

Here is the listing of available commands:

       public void SocketCommand(string current)
            switch (command)
                    // Kinect voice commands
                case "mercury-on": { MercuryRB.IsChecked = true; break; }
                case "venus-on": { VenusRB.IsChecked = true; break; }
                case "earth-on": { EarthRB.IsChecked = true; break; }
                case "moon-on": { MoonRB.IsChecked = true; break;}
                case "mars-on": { MarsRB.IsChecked = true; break;}
                case "marsrelief-on": { MarsreliefRB.IsChecked = true; break; }
                case "jupiter-on": { JupiterRB.IsChecked = true; break; }
                case "saturn-on": { SaturnRB.IsChecked = true; break; }
                case "uranus-on": { UranusRB.IsChecked = true; break; }
                case "neptune-on": { NeptuneRB.IsChecked = true; break; }
                case "pluto-on": { PlutoRB.IsChecked = true; break; }

                case "callisto-on": { CallistoRB.IsChecked = true; break; }
                case "io-on": { IoRB.IsChecked = true;break;}
                case "europa-on": {EuropaRB.IsChecked = true; break;}
                case "ganymede-on": { GanymedeRB.IsChecked = true; break;}
                case "cassini-on": { CassiniRB.IsChecked = true; break; }
                case "dione-on":  {  DioneRB.IsChecked = true; break; }
                case "enceladus-on": { EnceladusRB.IsChecked = true; break; }
                case "iapetus-on": { IapetusRB.IsChecked = true;  break; }
                case "tethys-on": { TethysRB.IsChecked = true; break; }
                case "moon-2d":
                        MoonRB.IsChecked = true;
                        Uri uri = Application.Current.Host.Source;
                        System.Windows.Browser.HtmlPage.Window.Navigate(new Uri(uri.Scheme + "://" + uri.DnsSafeHost + ":" + uri.Port + "/MoonViewer/Moon.html"), "_blank");
                case "mars-2d":
                        MarsRB.IsChecked = true;
                        Uri uri = Application.Current.Host.Source;
                        System.Windows.Browser.HtmlPage.Window.Navigate(new Uri(uri.Scheme + "://" + uri.DnsSafeHost + ":" + uri.Port + "/MoonViewer/Mars.html"), "_blank");
                case "nasaneo":
                        EarthRB.IsChecked = true;
                        System.Windows.Browser.HtmlPage.Window.Navigate(new Uri(""), "_blank"); break;
                case "rotate-east": {
                        RotationSpeedSlider.Value += 1.0;
                        tbMessage.Text = "rotate east";
                case "rotate-west":
                        RotationSpeedSlider.Value -= 1.0;
                        tbMessage.Text = "rotate west";
                case "rotate-off":
                        RotationSpeedSlider.Value = 0.0;
                        tbMessage.Text = "rotate off";
                case "reset":
                        RotationSpeedSlider.Value = 0.0;
                        orbitX = 0;
                        orbitY = 0;
                        tbMessage.Text = "reset view";

                //Kinect Swipe algorithmic commands
                case "swipetoleft":
                        orbitY += Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                        tbMessage.Text = "orbit left";
                case "swipetoright":
                        orbitY -= Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                        tbMessage.Text = "orbit right";
                case "swipeup":
                        orbitX += Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                        tbMessage.Text = "orbit up";
                case "swipedown":
                        orbitX -= Microsoft.Xna.Framework.MathHelper.ToRadians(15);
                        tbMessage.Text = "orbit down";

                //Kinect gesture template commands
                case "circle":

                        if (scene.Camera.Position.Z > 0.75f)
                            scene.Camera.Position += zoomInVector * 5;
                        tbMessage.Text = "zoomin";
                case "circle2":
                        scene.Camera.Position += zoomOutVector * 5;
                        tbMessage.Text = "zoomout";

Possible Extensions

After posting this code, I added an experimental stretch vector control for zooming and 2 axis twisting of planets. These are activated by voice: ‘vector twist’, ‘vector zoom’, and ‘vector off.’ The Map control side of gesture commands could also benefit from some easing function animations. Another avenue of investigation would be some type of pointer intersection using a ray to indicate planet surface locations for events.


Even though Kinect browser control is not prime time material yet, it is a lot of experimental fun! The MoonViewer control experiment is relatively primitive. Cursor movement and click using posture detection and hand tracking is also feasible, but fine movement is still a challenge. Two hand vector controlling for 3D scenes is also promising and integrates very well with SL5 XNA immediate mode graphics.

Kinect 2.0 and NearMode will offer additional granularity. Instead of large swipe gestures, finger level manipulation should be possible. Think of 3D voxel space manipulation of subsurface geology, or thumb and forefinger vector3 twisting of LiDAR objects, and you get an idea where this could go.

The merger of TV and internet holds promise for both whole body and NearMode Kinect interfaces. Researchers are also adapting Kinect technology for mobile as illustrated by OmniTouch.

. . . and naturally, lip reading ought to boost the Karaoke crowd (could help lip synching pop singers and politicians as well).


Fig 7 – Jupiter Moon Io

Paradise Lost


Fig 1 – web mapping utopia, location uncertain

“Not surprisingly, in a culture in which information was becoming standardized and repeatable, mapmakers began to exclude “paradise” from their charts on the grounds that its location was too uncertain.”

—Neil Postman, Technopoly (1992)

I recently spent a few hours re-reading some old books on the shelf. Neil Postman’s Technopoly triggered some reflection on the present state of web mapping. As a technophile, I tend to look forward to the next version of whatever with anticipation. Taking the longer view, however, can be a useful exercise.

Progress – yes, no, maybe?

Postman’s critique asserts that Technology is in the vanguard of illusory “progress.” That the impact of technology shapes deep things in a culture with unimagined side effects. The core technology of our current cultural turnover is electronics and the key utility is “Information,” its storage and flow. Information volume and velocity apparently grow exponentially over time loosely tracking the famous Moore’s Law time curve. Our web mapping subset of Technology is embedded in this Information ramp up, and we are still grappling with the confusion of Information and Knowledge, resulting from an accelerating information glut.

The principle of Information is popping up all over, from Claude Shannon’s Information Theory to Hawking-Berkenstein’s Black hole solution which surprisingly showed that the total Information content of a Black hole is proportional to the Planck square surface area of its event horizon (as opposed to a cubic volume relation). Naturally the concept of Information leaks into the soft sciences as well. Bit obsession is now woven into Western economic fabric with the assumption of continuous progress.

Sociologically, this all leads to a breakup of previous generations of “knowledge monopolies” and a hyper-sensitivity to initial conditions, (the old Ray Bradbury crushed butterfly scenario at the root of chaos theory). The upheaval is met with a kind of assumed optimism, an unquestioned utopian view of progress around the benefits of modern technology. Information is good, right? More information faster is even better! Web maps contribute more information faster, so web mapping is on the side of the Good, the True, and the Beautiful. Naturally web maps can be quite beautiful, whatever that means subjectively.

The age of typography, ushered into Western culture by Gutenberg, had profound and enduring effects on history and culture, affecting everything – from the Reformation and the rise of Democratic Nationalism to our educational bureaucracy and even common definitions of truth and knowledge. But not until Marshall McLuhan were the typographic origins of these effects popularly visible, and only in retrospect. The electronic age, in its current internet iteration, is undoubtedly creating similarly profound dislocations, whose consequences are not at all apparent at present. Unintended side effects are just that, unintended. Consequences are unintended, because they are unknowable.

“Technology solved the problem of information scarcity, the disadvantages of which were obvious. But it gave no warnings about the danger of information glut, the disadvantages of which were not seen so clearly, the long range result – information chaos.”


“The world has never before been confronted with information glut and has hardly had time to reflect on its consequences.”

—Neil Postman, Technopoly (1992)

Whither web mapping?

So the question at hand revolves around the smaller microcosm of mapping in the internet era. That an ever growing amount of this information flood is geospatial is indisputable. Computing mobility adds terrestrial location to all business and social enterprise – Tweets to Facebook, Fleet Tracking to Risk Analysis are increasingly tethered to spatial attributes. Maps take all of these streams into a symbolic spatial representation, which filters for location. To a mapmaker everything is a map and terra incognita has long since vanished.

Web mapping adds an element of exploration, with zoom and pan flight through these abstract spaces, that mirrors movement in our physical world. Our community is also wont to add controls affording endless tinkering with the form, as if one more contribution to universal “Choice fatigue” will add value. But Google glommed on to the real deal. In a state of information overload, the key to riches is meaningful information reduction. Simplification is the heart of search. Web mapping is one more filtering approach reducing information along the axis of spatial proximity.

Do interactive maps add seriously to comprehension or just to entertainment, as simply a novelty? Cartographers relish pointing out this quandary to web developers and other mere mortals. In the web mapping community proliferating means can easily be confused with progress. Doesn’t it seem peculiar, for example, to attach any meaning whatsoever to charts of Kindergartener’s DIBELS scores, let alone median DIBELS scores charted on national school district polygons. Does it lead to anything but a bureaucratic illusion of some control over chaos? No child is better off for the graph (unless distant employment potential as an educational bureaucrat is considered relevant), and, “no child left behind” slogans to the contrary, the overthrow of typography based education proceeds apace in a confusing melee of winners and losers. The application of numeracy to every conceivable problem like this elevates modeling to mythic proportions.

“When a technology becomes mythic, it is always dangerous because it is then accepted as it is, and is therefore not easily susceptible to modification or control.”

—Neil Postman, Technopoly (1992)


What is a map but a modeling technology, a symbolic abstraction of space to visualize a formulaic concept? Maps are all entangled with mathematical models, ellipsoids, surfaces, and transforms, but are we guilty of mythically inflating the power of maps to communicate something of the Good, the True and the Beautiful? Does the proliferation of web maps, for instance, alter the injustice of Atanas Entchev’s incarceration for the so called “crime“of immigration? Likely not, and in fact, may contribute to the irony of an ICE database assigning the Entchev family a spatial attribute coinciding with some “Community Education Center” in Newark, NJ.

“Our inventions are wont to be pretty toys, which distract our attention from serious things. They are but improved means to an unimproved end, an end which it was already but too easy to arrive at.”

—Henry David Thoreau, Walden (1854), p. 42

Another pillar of cyber utopian thought is the inevitability of improved community with improved connectivity. The meteoric rise of Facebook exemplifies this confusion. For in fact Facebook is faceless, not anonymous, but prone to carefully crafted pseudonymous identities. Unintended messages ripple across unknowable communities, and histories, whether wanted or unwanted, retain regrettably long tails. Twitter, as well, champions brevity while fragmenting communication across a ghostly crowd of undisclosed persons.

“The great communication that we have today can lead to complete depersonalization. Then one is just swimming in a sea of communication and no longer encounters persons at all.”

—Benedict XVI “Light of the World” p59 facebook

Ray Kurzweil’s anticipation of the Coming Singularity includes human/non-human relationships via digital intermediaries, a thought with somewhat disturbing implications. Knowledge of location across an intermediary network matters little to human relationships and adding non-human intelligence to the mix is only disturbing. Real human relationships have deeper currents than high velocity information or spatial attribution. Studies in the educational community have repeatedly shown, for instance, that the presence of a real teacher is overwhelmingly more effective than video or online classes. To state the obvious, face to face renders location aware apps irrelevant and leaves artificial intelligence firmly anchored in the creepy category.


Robot Teacher?

Assumptions of all goodness by Cyber Utopians are not at all justified as remarked by Evgeny Morozov in The Internet in Society: Empowering and Censoring Citizen? Enduring histories on Facebook, subject to examination by Iranian intelligence for hardly promising ends, should be unsettling to all but the grimmest of Marxist utopians. Doubtless a few stray Stalinist at Duke and Zuccotti Park are taking notes on new media and the social web.


Evgeny’s passing point about "cyber captivity" underlines a growing problem of lost opportunity. The prescience of Alduous Huxley’s Soma comes to mind. Does obsessive gaming, for example, reduce higher value opportunities for learning, productivity, and human relations? Do vast iTune libraries subtract from the net benefit of personal mastery of a musical instrument? These calculations are impossible to quantify and really revolve around deeper questions of spiritual significance, sub specie aeternitatis.

More information faster is not necessarily a net positive in another sense. Proliferating conspiracy theories only corroborate Neil Postman’s shuffled deck analogy, that a disintegrating information context conditions perception to credulity. Anything is believable because the next card is experienced as random. Unwarranted credulity paves the way to tyranny as recent history has shown all too tragically. The Rwandan genocide, less than 10 years ago, was incited by incredulous claims aired to a credulous public in creepy Goebbels fashion. The future specter of OsGeo web maps delineating the boundaries of inyenzi (cockroach in Kinyarwandan), does little to encourage optimism.

“The fact is, there are few political, social, and especially personal problems that arise because of insufficient information. Nonetheless, as incomprehensible problems mount, as the concept of progress fades, as meaning itself becomes suspect, the Technopolist stands firm in believing that what the world needs is yet more information.”

—Neil Postman, Technopoly (1992)


These may all be rather marginally Luddite issues. The genuine “terra incognita” of information technology, and consequently web mapping, is the tectonic plate of culture. What kind of global cultural, economic, and political earthquakes have been set in motion? What tidal wave of changing perceptual process is yet to be hailed from the yardarm?

Kind of exciting to think about.

Paradise Lost? Please report to Lost and Found.

FOSS4G 2011

I was privileged to spend a few days at FOSS4G in Denver. It’s quite the opportunity to have an international convergence of GIS developers handily just a bus ride up the highway. I could walk among the current luminaries of OsGeo, Arnulf Christl, Paul Ramsey, Frank Warmerdam, Simone Giannecchini, and so on and on, and trade a handshake with the hard working front range folks, Peter Batty, Brian Timoney (way to go Brian!), and Matt Kruzemark too. The ideas are percolating in every corner, but it strikes me in an odd way that this world has changed irrevocably. The very technology that makes Geo FOSS possible is also making it, of all things, mundane.

Open Standards

It wasn’t that long ago that OGC standards were brand new. Each release anticipated a new implementation for discovery. Now implementations proliferate helter skelter. OGC services? Take your pick. Or, slice, dice, and merge with MapProxy. Map Tiling engines line up like yogurt brands at Whole Foods. GeoFOSS has progressed a long way from naive amazement over OGC WMS connections. WFS and WCS have already passed their respective moments of glorious novelty.

This is the year of WPS, Web Processing Service, and the hope of constructing webs of analysis chains from disparate nodes. Each node adds a piece toward a final solution. Anyone with some experience using FME Transformers has some idea of what this looks like on the desktop. WPS moves analytic workflows into open standards and onto the web. Ref: and GeoServer Ext

FME Workbench
Fig2 – example of FME Workbench processing chain

On the floor at FOSS4G 2011

In a different vein, the next release of PostGIS will push raster analysis right into SQL. This is new territory for PostGIS and elevates a raster datatype to the same level as geometry. This PostGIS Raster Functions page gives some flavor for the raster functions soon to be available when 2.0 releases. Ever needed a raster reprojection?

raster ST_Transform(raster rast, integer srid, double precision scalex, double precision scaley, text algorithm=NearestNeighbor, double precision maxerr=0.125);

Algorithm options are: ‘NearestNeighbor’, ‘Bilinear’, ‘Cubic’, ‘CubicSpline’, and ‘Lanczos.’

Wow ST_MapAlgebra! And on the roadmap MapAlgebra raster on raster! MySQL and SQL Server have quite the feature curve to follow as PostGIS forges ahead.

(See even the slightly jaded can catch some excitement at FOSS4G.)

Numerous workshops dealt with javascript map frameworks OpenLayers, MapQuery, GeoExt, Leaflet. The html5 trend is underlined by news flashes from the concurrent Microsoft Build Conference.

“For the web to move forward and for consumers to get the most out of touch-first browsing, the Metro style browser in Windows 8 is as HTML5-only as possible, and plug-in free. The experience that plug-ins provide today is not a good match with Metro style browsing and the modern HTML5 web.” Steven Sinofsky

CouchDB/GeoCouch was present although “eventually consistent” NoSQL seems less pressing.

Although Oracle is trying hard to break Java, it is still a popular platform among FOSS projects. The Javascript platform is a growth industry with the help of backend tools like Node.js, html5 WebSockets, and Asynchronous WebWorkers. C/C++ takes away wins in the popular performance shootout.

3D WebGL made an appearance with WebGLEarth, while Nasa World Wind still has an occasional adherent.

Open Software

With over 900 attendees, the small world of FOSS seems all of a sudden crowded, jostling through accelerating growth. The Ordnance Survey was represented at a plenary session, “We have to learn to embrace open source. We want to use OS. We want to encourage OS.” Who’d of ever thought? opens their kimono in a big way with data transparency and OS APIs. Work shop topics include such things as: “Open Source Geospatial Software Powering Department of Defense Installation and Environment Business Systems,” or catch this, “The National Geospatial-Intelligence Agency OSS Challenge.” Is this the new Face of FOSS?

Open Bureaucracy?

Such bureaucracies know little of agile. The memorable phrase “embrace, extend, extinguish” comes to mind. But who embraces whom? Is the idea of FOSS and its larger www parent, a trend that eventually extinguishes bureaucracy or does the ancient ground of bureaucratic organization trump connection? Byzantine bureaucracy has been around since … well the Byzantine Empire, and given human nature is likely enduring. But, what does a “flat” Byzantine Bureaucracy look like? How about crowd sourced Byzantia? Would an aging Modernity mourn the loss of “Kafkaesque” as an adjective?

Assuming growth continues, will success reduce the camaraderie of community as a motivation? Just this year we hear of OSM’s Steve Coast sidling into Microsoft, followed a little later by GDAL’s Frank Warmerdam beaming up into the Google mother ship. The corporate penalty, of course, is the loss of personal intellectual property. In other words, will Steve Coast’s imaginative ideas now fall under the rubric, “all your base are belong to us,” and Frank’s enduring legacy recede into the patent portfolio of “Do no evil?” FOSS4G halls are still filled with independent consultants and academics, but a significant corporate representation is evident by this apparent oxymoron, a presentation by ESRI, “Open Source GIS Solutions.”

Some Perspective:

The GIS nervous system grows connections around the world faster than a three year old’s brain. OK, maybe I exaggerate, after all, “During the first three years, your child’s brain establishes about 1,000 trillion nerve connections!” Really, how many connections are there in the World Wide Web. For all its impact on life, our beloved internet encompasses the equivalent of what, a few cubic centimeters of a single infant’s anatomy.

Open Geospatial and FOSS are just a small part of this minor universe, but it’s easy to forget that even ten years ago this all hardly existed. “The first Interoperability Program testbed (Web Mapping Testbed) appeared in 1999.” About the same time Frank Warmerdam started GDAL and the GFOSS engines started.

I still recall the wonder of downloading Sol Katz utilities from BLM’s ftp. The novelty of all this data free to use from the USGS was still fresh and amazing. Sol sadly died before seeing his legacy, but what a legacy. The Sol Katz award this year went to Java Topology Suite’s well deserving Martin Davis.