Territorial Turf

Fig 1 – Territories from Census Zip Code Tabulation Areas (ZCTAs) on Bing Maps

An interesting GIS development over the years has been the evolution from monolithic applications to multiple open source plug and play tools. Considering GIS as a three tier system with back end storage, a controller in the middle, and a UI display out front, more and more of the middle tier is migrating to either end.

SQL DBs, such as SQL Server, Oracle Spatial, and especially PostGIS, now implement a multitude of GIS functions originally considered middle tier domain. On the other end, the good folks at turf.js and proj4js continue to push atomic GIS functions out to javascript, where they can fit into the client side UI. The middle tier is getting thinner and thinner as time goes on. Generally the middle is what costs money, so rolling your own middle with less work is a good thing. As a desirable side effect, instead of kitchen sink GIS, very specific user tools can be cobbled together as needed.

Looking for an excuse to experiment with turf.js, I decided to create a Territory Builder utilizing turf.js on the client and some of the US Census Bureau Services on the backend.

US Census Bureau does expose some “useful” services at TigerWeb. I tend to agree with Brian Timoney that .gov should stick to generating data exposed in useful ways. Apps and presentation are fast evolving targets and historically .gov can’t really hope to keep up. Although you can use the census custom TigerWeb applications for some needs, there are many other occasions when you would like to build something less generic. For example a Territory builder over Bing Maps.

Here is the simplified POC work flow:

1. Use Census WMS GetMap to show polygons on a canvas over Bing Maps.
2. Use Census WMS GetFeatureInfo to return a GeoID of selected polygon(s).
3. Use Census REST service and GeoID to return selected polygon vertices.
4. Use Turf.js merge function to merge selected polygons into a single Territory polygon
5. Display territory polygon using Bing Maps Ajax v7 Microsoft.Maps.AdvancedShapes polygon

Fig 2 – Territory polygon with a demographic overlay on top of Bing Maps

Note: Because there doesn’t appear to be an efficient way to join demographic data to geography with current TigerWeb service APIs, the demographic tab of this app uses a custom WMS PostGIS backend, hooking SF1 to TIGER polygons.

1. Census WMS GetMap over Bing Maps

WMS GetMap requests simply return an image. In order to overlay the image on a Bing Map this Territory app uses an html5 canvas and context app.context.drawImage(imageObj, 0, 0); The trick is to stack the canvas in between the Bing Map and the Bing Map navigation, scale, and selector tools. TigerWeb WMS conveniently exposes epsg:3857 which correctly aligns with Bing Map tiles.

    function showCensus() {
        if (app.canvas == null) {
            app.canvas = document.createElement('canvas');
            app.canvas.id = 'censuscanvas';
            app.canvas.style.position = 'relative';
            var mapDiv = app.map.getRootElement();
            //position this canvas under Bing navigation and over map tiles
            mapDiv.insertBefore(app.canvas, mapDiv.childNodes[3]);
            app.context = app.canvas.getContext("2d");
        }
        //match canvas size to map size
        app.canvas.height = app.map.getHeight();
        app.canvas.width = app.map.getWidth();

        var imageObj = new Image();
        //image onload function draws image on context
        imageObj.onload = function () {
            app.canvas.style.opacity = app.alpha / 100;
            app.context.drawImage(imageObj, 0, 0);
        };
        //prepare a TigerWeb WMS GetMap request
        //use proj4.js to transform ll to mercator bbox
        var b = app.map.getBounds();
        var epsg3857 = "+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs";
        var sw = proj4(epsg3857).forward([b.getWest(), b.getSouth()]);
        var ne = proj4(epsg3857).forward([b.getEast(), b.getNorth()]);
        bbox = sw[0] + "," + sw[1] + "," + ne[0] + "," + ne[1];

        var url = "";
        if (tab == "Territory") {
            app.alpha = 100;
            // GetMap request
            url = "http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetMap&SERVICE=WMS&VERSION=1.3.0&LAYERS=" + app.censusPolygonType + "&STYLES=&FORMAT=image/png&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&CRS=EPSG:3857&BBOX=" + bbox + "&WIDTH=" + app.canvas.width + "&HEIGHT=" + app.canvas.height;
        }
        imageObj.src = url;
    }

2. Census WMS GetFeatureInfo for obtaining GeoID

Unfortunately the WMS GetMap request has no IDs available for polygons, even if requested format is image/svg+xml. SVG is an xml format and could easily contain associated GeoID values for joining with other data resources, but this is contrary to the spirit of OGC WMS service specs. Naturally we must obey OGC Law, which is too bad. Adding a GeoID attribute would allow options such as choropleth fills directly on existing svg paths. For example adding id = “80132″ would allow fill colors by zipcode with a bit of javascript.

http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetMap&SERVICE=WMS&VERSION=1.3.0 &LAYERS=2010 Census ZIP Code Tabulation Areas&STYLES=&FORMAT=image/svg+xml&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&CRS=EPSG:3857 &BBOX=-11679625.942909468,4709198.547476525,-11645573.246808422,4737900.651597611&WIDTH=891&HEIGHT=751

<g transform="matrix(1.3333333 0 0 -1.3333333 445.5 375.5)">
<path id="80132" d="M300.50809 -258.78592 L302.54407 -257.91829 L303.17977 -257.55608 L304.45554 -256.98889
                                .
				.
L280.30116 -268.55133 L280.7184 -268.34637 L298.33448 -259.95117 L300.50809 -258.78592 "
  stroke="#99454A" fill="none" stroke-opacity="1" stroke-width="1.5"
 stroke-linecap="round" stroke-linejoin="round" stroke-miterlimit="10" />

Instead TigerWeb WMS exposes GetFeatureInfo requests. Using a click event we calculate a lat,lon location and send a GetFeatureInfo request to find a GeoID for the enclosing polygon.

Request:
http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetFeatureInfo&SERVICE=WMS&VERSION=1.3.0&CRS=EPSG:4326 &BBOX=13.6972399985862,-128.010213138288,52.2800368298775,-53.444917258507&WIDTH=1061&HEIGHT=549 &INFO_FORMAT=text/xml&QUERY_LAYERS=2010 Census ZIP Code Tabulation Areas&X=388&Y=190

Response:
<?xml version=”1.0″ encoding=”UTF-8″?>
<FeatureInfoResponse xmlns=”http://www.esri.com/wms” xmlns:esri_wms=”http://www.esri.com/wms”>
<FIELDS INTPTLON=”-101.2099848″ INTPTLAT=”+38.9263824″ CENTLON=”-101.2064010″ CENTLAT=”+38.9226272″ STGEOMETRY=”Polygon” OBJECTID=”16553″ AREAWATER=”163771″ AREALAND=”1327709219″ FUNCSTAT=”S” ZCTA5CC=”B5″ MTFCC=”G6350″ NAME=”ZCTA5 67764″ LSADC=”Z5″ BASENAME=”67764″ GEOID=”67764″ ZCTA5=”67764″ OID=”221404258331221″/>
</FeatureInfoResponse>

Since cross browser restrictions come into play it’s necessary to add a bit of server side proxy code to actually return the xml.

Some MVC controller magic:

 function GetCensusFeature(e) {
    var x = e.pageX - $("#map").offset().left;
    var y = e.pageY - $("#map").offset().top;
    var FeatureUrl = {
        url: "http://tigerweb.geo.census.gov/arcgis/services/TIGERweb/tigerWMS_Current/MapServer/WmsServer?REQUEST=GetFeatureInfo&SERVICE=WMS&VERSION=1.3.0&LAYERS=" + app.censusPolygonType + "&STYLES=&FORMAT=image/png&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&CRS=EPSG:3857&BBOX=" + bbox + "&WIDTH=" + app.canvas.width + "&HEIGHT=" + app.canvas.height + "&INFO_FORMAT=text/xml&QUERY_LAYERS=" + app.censusPolygonType + "&X=" + x + "&Y=" + y
    }
        app.postit('/api/Operation/GetFeatureInfo', {
            data: JSON.stringify(FeatureUrl),
            success: function (response) {
                if (response.length > 0) {
                    var parser = new DOMParser();
                    var xmlDoc = parser.parseFromString(response, "text/xml");
				          .
				          .

Eventally gets here to a C# Controller which can proxy the GetFeatureInfo request:

        /// <summary>
        /// GetFeatureInfo
        /// </summary>
        /// <param name="FeatureUrl"></param>
        /// <returns>xml doc</returns>
        [HttpPost]
        public async Task<IHttpActionResult> GetFeatureInfo(Models.FeatureUrl feature)
        {
            var wc = new HttpClient();
            var getFeatureInfoRequest = new Uri(feature.url);
            var response = await wc.GetAsync(getFeatureInfoRequest);
            response.EnsureSuccessStatusCode();
            var content = await response.Content.ReadAsStreamAsync();
            var result = "";
            using (var reader = new StreamReader(content))
            {
                while (!reader.EndOfStream)
                {
                    result += reader.ReadLine();
                }
            }
            return Ok(result);
        }

Finally, on success, javascript can parse the xml for GeoID and add a canvas context box with some text information:

     success: function (response) {
              if (response.length > 0) {
                    var parser = new DOMParser();
                    var xmlDoc = parser.parseFromString(response, "text/xml");
                    if (tab == "Territory" && xmlDoc.getElementsByTagName("FeatureInfoResponse")[0].hasChildNodes()) {
                        app.context.beginPath();
                        app.context.rect(x, y, 125, 25);
                        app.context.fillStyle = 'white';
                        app.context.fill();
                        app.context.lineWidth = 1;
                        app.context.strokeStyle = 'black';
                        app.context.stroke();
                        app.context.fillStyle = 'black';
                        app.context.font = '11pt Calibri';
                        var fields = xmlDoc.getElementsByTagName("FIELDS")[0];
                        app.context.fillText(fields.getAttribute('NAME'), x + 5, y + 15);
                        var geoid= fields.getAttribute('GEOID')
					            .
					            .

Fig 3 – building territories from Census Block Groups on Bing Maps

3. Census REST to obtain vertices

The GEOID retrieved from our proxied GetFeatureInfo request allows us to grab vertices with another TigerWeb service, Census REST.

This spec is a little more proprietary and requires some detective work to unravel.
FeatureUrl.url = “http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/PUMA_TAD_TAZ_UGA_ZCTA/MapServer/1/query?where=GEOID%3D” + geoid + “&geometryPrecision=6&outSR=4326&f=pjson”;

There are different endpoint urls for the various polygon types. In this case Zip Codes are found in PUMA_TAD_TAZ_UGA_ZCTA. We don’t need more than 1 meter resolution so precision 6 is good enough and we would like the results in epsg:4326 to avoid a proj4 transform on the client.

This follows the same process as previously sending the REST request through a proxy controller. You can see the geojson geometry result with this sample request:
http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/PUMA_TAD_TAZ_UGA_ZCTA/MapServer/1/query?where=GEOID%3D80004&geometryPrecision=6&outSR=4326&f=pjson

Census REST doesn’t appear to offer a simplify parameter so the coordinates returned are at the highest resolution. Highly detailed polygons can easily return several thousand vertices, which is a problem for performance, but the trade-off is eliminating hosting data ourselves.

4. turf.js merge function to Build Territory

Finally the interesting part, where we get to use turf.js to handle merging multiple polygons into a single territory polygon.

var FeatureUrl = {
url: "http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/PUMA_TAD_TAZ_UGA_ZCTA/MapServer/1/query?where=GEOID%3D" + geoid + "&geometryPrecision=6&outSR=4326&f=pjson";
}
app.postit('/api/Operation/GetFeatureGeo', {
    data: JSON.stringify(FeatureUrl),
    success: function (response) {
        var polygon = JSON.parse(response);
        if (polygon.features.length == 0 || polygon.features[0].geometry.rings.length == 0) {
            $('#loading').addClass('hide');
            alert("No Geo Features returned.")
            return;
        }
        if (app.territoryFeatures == null) {
            app.territoryFeatures = new Array();
            app.territory = turf.polygon(polygon.features[0].geometry.rings);
            app.territoryFeatures.push(app.territory);
        }
        else {
            var tpolygon = turf.polygon(polygon.features[0].geometry.rings);
            app.territoryFeatures.push(tpolygon);
            var fc = turf.featurecollection(app.territoryFeatures);
            try {
                app.territory = turf.merge(fc);
            }
            catch (err) {
                $('#loading').addClass('hide');
                alert("turf.js error: " + err.message);
                return;
            }
        }
        if (app.territory.geometry.coordinates.length > 0) {
            displayTerritory();
            $('#totalDivID').removeClass('hide');
        }

        $('#loading').addClass('hide');
    },
    error: function (response) {
        alert("TigerWeb error:" + response);
    }
});

5. Display Territory Bing Maps Ajax polygon

The final rendering just uses Bing Maps Ajax v7 Microsoft.Maps.AdvancedShapes to add the new territory to the map.

function displayTerritory() {
    app.territoryLayer.clear();
    var coordLen = app.territory.geometry.coordinates.length;
    if (app.territory.geometry.type == 'Polygon') {
        var rings = new Array();
        for (var i = 0; i < coordLen; i++) {
            var vertices = new Array();
            for (var j = 0; j < app.territory.geometry.coordinates[i].length; j++) {
                vertices.push(new Microsoft.Maps.Location(app.territory.geometry.coordinates[i][j][1], app.territory.geometry.coordinates[i][j][0]));
            }
            rings.push(vertices);
        }
        var polygon = new Microsoft.Maps.Polygon(rings, { fillColor: new Microsoft.Maps.Color(100, 100, 0, 100) });
        app.territoryLayer.push(polygon);
    }
    else if (app.territory.geometry.type == 'MultiPolygon') {
        var multi = new Array();
        for (var i = 0; i < coordLen; i++) {
            var ringslen = app.territory.geometry.coordinates[i].length;
            for (var j = 0; j < ringslen; j++) {
                var vertices = new Array();
                for (var k = 0; k < app.territory.geometry.coordinates[i][j].length; k++) {
                    vertices.push(new Microsoft.Maps.Location(app.territory.geometry.coordinates[i][j][k][1], app.territory.geometry.coordinates[i][j][k][0]));
                }
                multi.push(vertices)
            }
        }
        var polygon = new Microsoft.Maps.Polygon(multi, { fillColor: new Microsoft.Maps.Color(100, 100, 0, 100) });
        app.territoryLayer.push(polygon);
    }
    else {
        $('#loading').addClass('hide');
        alert("geometry type is " + app.territory.geometry.type + ". Territory requires a Polygon or MultiPolygon.")
        return;
    }
}

Fig 4 – Territory Total for P0040003 Latino Origin – Map: Quantile population classifier Census Block Groups on Bing Map

Summary

TigerWeb offers some useful data access. With TigerWeb WMS and REST api, developers can customize apps without hosting and maintaining a backend SQL store. However, there are some drawbacks.

Some potential improvements:
1. Adding an svg id=GeoID would really improve the usefulness of TigerWeb WMS image/svg+xml, possibly eliminating steps 2 and 3 of the workflow.

2. Technically it’s possible to use the TigerWeb REST api to query geojson by area, but practically speaking the results are too detailed for useful performance. A helpful option for TigerWeb REST would be a parameter to request simplified polygons and avoid lengthy vertice transfers.

turf.js is a great tool box, however, occasionally the merge function had trouble with complex polygons from TigerWeb.

Fig 6 - turf merge had some rare difficulties with complex polygons from TigerWeb

In the end, territories are useful for query aggregates of SF1 demographic data.

Fig 5 – Territory Total for P0120025 Male 85 Years and older – Map: Jenks population classifier Census Block Group on Bing Map

Visualizing Large Data Sets with Bing Maps Web Apps

Fig 1 - MapD Twitter Map 80M tweets Oct 19 - Oct 30

Visualizing large data sets with maps is an ongoing concern these days. Just ask the NSA, or note this federal vehicle tracking initiative reported at the LA Times. Or, this SPD mesh network for tracking any MAC address wandering by.

“There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all of the time. But at any rate they could plug in your wire whenever they wanted to.”

George Orwell, 1984

On a less intrusive note, large data visualization is also of interest to anyone dealing with BI or just fascinated with massive public data sets such as twitter universe. Web maps are the way to go for public distribution and all web apps face the same set of issues when dealing with large data sets.

1. Latency of data storage queries, typically SQL
2. Latency of services for mediating queries and data between the UI and storage.
3. Latency of the internet
4. Latency of client side rendering

All web map javascript APIs have these same issues whether it’s Google, MapQuest, Nokia Here, or Bing Maps. This is a Bing Maps centric perspective on large data mapping, because Bing Maps has been the focus of my experience for the last year or two.

Web Mapping Limitation

Bing Maps Ajax v7 is Microsoft’s javascript API for web mapping applications. It offers typical Point (Pushpin), Polyline, and Polygon vector rendering in the client over three tile base map styles, Road, Aerial, and AerialWithLabels. Additional overlay extensions are also available such as traffic. Like all the major web map apis, vector advantages include client side event functions at the shape object level.

Although this is a powerful mapping API rendering performance degrades with the number of vector entities in an overlay. Zoom and pan navigation performs smoothly on a typical client up to a couple of thousand points or a few hundred complex polylines and polygons. Beyond these limits other approaches are needed for visualizing geographic data sets. This client side limit is necessarily fuzzy as there is a wide variety of client hardware out there in user land, from older desktops and mobile phones to powerful gaming rigs.

Large data Visualization Approaches

1) Tile Pyramid – The Bing Maps Ajax v7 API offers a tileLayer resource that handles overlays of tile pyramids using a quadkey nomenclature. Data resources are precompiled into sets of small images called a tile pyramid which can then be used in the client map as a slippy tile overlay. This is the same slippy tile approach used for serving base Road, Aerial, and AerialWithLabels maps, similar to all web map brands.

Fig 2 - example of quadkey png image names for a tile pyramid

Pro:· Fast performance

  • Server side latency is eliminated by pre-processing tile pyramids
  • Internet streaming is reduced to a limited set of png or jpg tile images
  • Client side rendering is reduced to a small set of images in the overlay

Con: Static data – tile pyramids are pre-processed

  • data cannot be real time
  • Permutations limited – storage and time limitations apply to queries that have large numbers of permutations
  • Storage capacity – tile pyramids require large storage resources when provided for worldwide extents and full 20 zoom level depth

2) Dynamic tiles – this is a variation of the tile pyramid that creates tiles on demand at the service layer. A common approach is to provide dynamic tile creation with SQL or file based caching. Once a tile has been requested it is then available for subsequent queries directly as an image. This allows lower levels of the tile pyramid to be populated only on demand reducing the amount of storage required.

Pro:

  • Can handle larger number of query permutations
  • Server side latency is reduced by caching tile pyramid images (only the first request requires generating the image)
  • Internet streaming is reduced to a limited set of png tile images
  • Client side rendering is reduced to a small set of images in the overlay

Con:

  • Static data – dynamic data must still be refreshed in the cache
  • Tile creation performance is limited by server capability and can be a problem with public facing high usage websites.

3) Hybrid - This approach splits the zoom level depth into at least two sections. The lowest levels with the largest extent contain the majority of a data set’s features and is provided as a static tile pyramid. The higher zoom levels comprising smaller extents with fewer points can utilize the data as vectors. A variation of the hybrid approach is a middle level populated by a dynamic tile service.

Fig 3 – Hybrid architecture

Pro:

  • Fast performance – although not as fast as a pure static tile pyramid it offers good performance through the entire zoom depth.
  • Allows fully event driven vectors at higher zoom levels on the bottom end of the pyramid.

Con:

  • Static data at larger extents and lower zoom levels
  • Event driven objects are only available at the bottom end of the pyramid

Example:
sample site and demo video

tile layer sample

Fig 4 - Example of a tileLayer view - point data for earthquakes and Mile Markers

Fig 5 - Example of same data at a higher zoom using vector data display

4) Heatmap
Heatmaps refer to the use of color gradient or opacity overlays to display data density. The advantage of heat maps is the data reduction in the aggregating algorithm. To determine the color/opacity of a data set at a location the data is first aggregated by either a polygon or a grid cell. The sum of the data in a given grid cell is then applied to the color gradient dot for that cell. If heatmaps are rendered client side they have good performance only up to the latency limits of service side queries, internet bandwidth, and local rendering.

Fig 6 - Example of heatmap canvas over Bing Maps rendered client side

Grid Pyramids – Server side gridding
Hybrid server side gridding offers significant performance advantages when coupled with pre-processed grid cells. One technique of gridding processes a SQL data resource into a quadkey structure. Each grid cell is identified by its unique quadkey and contains the data aggregate at that grid cell. A grid quadkey sort by length identifies all of the grid aggregates at a specific quadtree level. This allows the client to efficiently download the grid data aggregates at each zoom level and render locally on the client in an html5 canvas over the top of a Bing Maps view. Since all grid levels are precompiled, resolution of cells can be adjusted by Zoom Level.

Pro:

  • Efficient display of very large data sets at wide extents
  • Can be coupled with vector displays at higher zoom levels for event driven objects

Con: gridding is pre-processed

  • real time data cannot be displayed
  • storage and time limitations apply to queries that have large numbers of permutations

Fig 7 – Grid Pyramid screen shot of UI showing opacity heatmap of Botnet infected computers

5) Thematic
Thematic maps use spatial regions such as states or zipcodes to aggregate data into color coded polygons. Data is aggregated for each region and color coded to show value. A hierarchy of polygons allows zoom levels to switch to more detailed regions at closer zooms. An example hierarchy might be Country, State, County, Sales territory, Zipcode, Census Block.

Pro:

  • Large data resources are aggregated into meaningful geographic regions.
  • Analysis is often easier using color ranges for symbolizing data variation

Con:

  • Rendering client side is limited to a few hundred polygons
  • Very large data sets require pre-processing data aggregates by region

Fig 8 - thematic map displaying data aggregated over 210 DMA regions using a quantized percentile range

6) Future trends
Big Data visualization is an important topic as the web continues to generate massive amounts of data useful for analysis. There are a couple of technologies on the horizon that help visualization of very large data resources.

A. Leverage of client side GPU

Here is an example WebGL http://www.web-maps.com/WebGLTest using CanvasLayer. ( only Firefox, chrome, IE11 *** Cannot be viewed in IE10 ***)

This sample shows speed of pan zoom rendering of 30,000 random points which would overwhelm typical js shape rendering. Data performance is good up to about 500,000 points per Brendan Kenny. Complex shapes need to be built up from triangle primitives. Tessellation rates for polygon generation approaches 1,000,000 triangles per 1000ms using libtess. Once tessellated the immediate mode graphic pipeline can navigate at up to 60fps. Sample code is available on github.

This performance is achieved by leveraging the client GPU. Because immediate mode graphics is a powerful animation engine, time animations can be used to uncover data patterns and anomalies as well as making some really impressive dynamic maps like this Uber sample. Unfortunately all the upstream latency remains: collecting the data from storage and sending it across the wire. Since we’re talking about larger sets of data this latency is more pronounced. Once data initialization finishes, client side performance is amazing. Just don’t go back to the server for new data very often.

Pro:

  • Good client side navigation performance up to about 500,000 points

Con:

  • requires a webgl enabled browser
  • requires GPU on the client hardware
  • subject to latency issues of server query and internet streaming
  • WebGL tessellation triangle primitives make display of polylines and polygons complex

Fig 9 – test webGL 30,000 random generated points (requires WebGL enabled browser – Firefox, Chrome, IE11)

Note: IE11 added WebGL capability which is a big boost for the web. There are still some glitches, however, and gl_PointSize in shader is broken for simple points like this sample.

Fig 10 – Very interesting WebGL animations of shipping GPS tracks using WebGL Canvas –courtesy Brendan Kenny

B. Leverage of server side GPU
MapD – Todd Mostak has developed a GPU based spatial query system called MapD (Massively Parallel Database)

MapD Synopsis:
  • MapD is a new database in development at MIT, created by Todd Mostak.
  • MapD stands for “massively parallel database.”
  • The system uses graphics processing units (GPUs) to parallelize computations. Some statistical algorithms run 70 times faster compared to CPU-based systems like MapReduce.
  • A MapD server costs around $5,000 and runs on the same power as five light bulbs.
  • MapD runs at between 1.4 and 1.5 teraflops, roughly equal to the fastest supercomputer in 2000.
  • uses SQL to query data.
  • Mostak intends to take the system open source sometime in the next year.
  • Bing Test: http://onterrawms.blob.core.windows.net/bingmapd/index.htm

    Bing Test shows an example of tweet points over Bing Maps and illustrates the performance boost from the MapD query engine. Each zoom or pan results in a GetMap request to the MapD engine that queries millions of tweet point records (81 million tweets Oct 19 – Oct 30), generating a viewport png image for display over Bing Map. The server side query latency is amazing considering the population size of the data. Here are a couple of screen capture videos to give you the idea of the higher fps rates:

    MapDBingTestAerialYellow50ms.wmv
    MapDBingHeatTest.wmv

    Interestingly, IE and FireFox handle cache in such a way that animations up to 100fps are possible. I can set a low play interval of 10ms and the player appears to do nothing. However, 24hr x12 days = 288 images are all being downloaded in just a few seconds. Consequently the next time through the play range the images come from cache and animation is very smooth. Chrome handles local cache differently in Windows 8 and it won’t grab from cache the second time. In the demo case the sample runs at 500ms or 2fps which is kind of jumpy but at least it works in Windows 8 Chrome with an ordinary internet download speed of 8Mbps

    Demo site for MapD: http://mapd.csail.mit.edu/

    Pro:

    • Server side performance up to 70x
    • Internet stream latency reduced to just the viewport image overlay
    • Client side rendering as a single image overlay is fast

    Con:

    • Source code not released, and there may be proprietary license restrictions
    • Most web servers do not include GPU or GPU clusters – especially cloud instances

    Note: Amazon AWS offers GPU Clusters but not cheap.

    Cluster GPU Quadruple Extra Large 22 GiB memory, 33.5 EC2 Compute Units, 2 x NVIDIA Tesla “Fermi” M2050 GPUs, 1690 GB of local instance storage, 64-bit platform, 10 Gigabit Ethernet( $2.100 per Hour)

    NVidia Tesla M2050 – 448 CUDA Cores per GPU and up to 515 Gigaflops of double-precision peak performance in each GPU!

    Fig 11 - Demo displaying public MapD engine tweet data over Bing Maps

    C. Spatial Hadoophttp://spatialhadoop.cs.umn.edu/
    Spatial Hadoop applies the parallelism of Hadoop clusters to spatial problems using the MapReduce technique made famous by Google. In the Hadoop world a problem space is distributed across multiple CPUs or servers. Spatial Hadoop adds a nice collection of spatial objects and indices. Although Azure Hadoop supports .NET, there doesn’t seem to be a spatial Hadoop in the works for .NET projects. Apparently MapD as open source would leap frog Hadoop clusters at least for performance per dollar.

    D. In Memory database (SQL Server 2014 Hekatron in preview release) – Microsoft plans to enhance the next version of SQL Server with in-memory options. SQL server 2014 in-memory options allows high speed queries for very large data sets when deployed to high memory capacity servers.

    Current SQL Server In-Memory OLTP CTP2

    Creating Tables
    Specifying that the table is a memory-optimized table is done using the MEMORY_OPTIMIZED = ON clause. A memory-optimized table can only have columns of these supported datatypes:

    • Bit
    • All integer types: tinyint, smallint, int, bigint
    • All money types: money, smallmoney
    • All floating types: float, real
    • date/time types: datetime, smalldatetime, datetime2, date, time
    • numeric and decimal types
    • All non-LOB string types: char(n), varchar(n), nchar(n), nvarchar(n), sysname
    • Non-LOB binary types: binary(n), varbinary(n)
    • Uniqueidentifier”

    Since geometry and geography data types are not supported with the next SQL Server 2014 in-memory release, spatial data queries will be limited to point (lat,lon) float/real data columns. It has been previously noted that for point data, float/real columns have equivalent or even better search performance than points in a geography or geometry form. In-memory optimizations would then apply primarily to spatial point sets rather than polygon sets.

    Natively Compiled Stored Procedures The best execution performance is obtained when using natively compiled stored procedures with memory-optimized tables. However, there are limitations on the Transact-SQL language constructs that are allowed inside a natively compiled stored procedure, compared to the rich feature set available with interpreted code. In addition, natively compiled stored procedures can only access memory-optimized tables and cannot reference disk-based tables.”

    SQL Server 2014 natively compiled stored procedures will not include any spatial functions. This means optimizations at this level will also be limited to float/real lat,lon column data sets.

    For fully spatialized in-memory capability we’ll probably have to wait for SQL Server 2015 or 2016.

    Pro:

    • Reduce server side latency for spatial queries
    • Enhances performance of image based server side techniques
      • Dynamic Tile pyramids
      • images (similar to MapD)
      • Heatmap grid clustering
      • Thematic aggregation

    Con:

    • Requires special high memory capacity servers
    • It’s still unclear what performance enhancements can be expected from spatially enabled tables

    D. Hybrids

    The trends point to a hybrid solution in the future which addresses the server side query bottleneck as well as client side navigation rendering bottleneck.

    Server side –
    a. In-Memory spatial DB
    b. Or GPU based parallelized queries

    Client side – GPU enhanced with some version of WebGL type functionality that can makes use of client GPU

    Summary

    Techniques are available today that can accommodate large date resources in Bing Maps. Trends indicate that near future technology can really increase performance and flexibility. Perhaps the sweet spot for Big Data map visualization over the next few years will look like a MapD or a GPU Hadoop engine on the server communicating to a WebGL UI over 1 gbps fiber internet.

    Orwell feared that we would become a captive audience. Huxley feared the truth would be drowned in a sea of irrelevance.

    Amusing Ourselves to Death, Neil Postman

    Of course, in America, we have to have the best of both worlds. Here’s my small contribution to irrelevance:

    Fig 12 - Heatmap animation of Twitter from MapD over Bing Maps (100fps)

    .NET Rocks!
    Road Trip Tracking

    Fig 0 - Tracking a RoadTrip


    Overview

    Microsoft is on the move this fall. Win8 is the big news, but Visual Studio 2012, .Net 4.5, a revamped Azure, WP8, Office 2013, and even a first foray into consumer hardware, Surface Tablets (not tables), all see daylight this fall.

    The Net Rocks duo, Carl Franklin and Richard Campbell, are also on the move. Carl and Richard head out this week for a whirl wind tour through 36 states in 72 days or roughly 1728 hours. The DNR Road Trip Tracking application, http://www.web-maps.com/RoadTrip2012/, keeps tabs on the .Net Rocks RV with Tweet encouragement for the road weary travelers. All are welcome to follow the DNR RV online and add Tweet comments at #dnrRoadTrip. The app gives event information and up to the minute updates of time to next show with tweets along the route. It even gives turn by turn directions for those inclined to catch the .Net Rocks RV and follow along in the real world – .NET Rocks stalking.

    Technical Overview

    Project Outline

    Fig 1 – .Net Rocks Road Trip Tracking app project outline

    Backend:

    SQL Server Azure is the key resource for the DNR tracking app. GPS feeds are pushed at 1 minute intervals from a commercial Airlink GPS unit to a Windows service listening for UDP packets. This Feed Service turns incoming UDP packets into Feed records stored in SQL Azure with a geography point location and datetime stamp.

    On the same system, a Twitter Query service is checking for Tweets on a 30 second interval using the Twitter REST API. Tweets are also turned into Feed records in SQL Azure. However, the geography point locations for Tweets are pulled from the latest GPS record so they are associated in time with the location of the GPS unit in the DNR RV.

    Frontend:

    Fig2 – Windows 8 IE10 browser showing stop and routes

    On the front end, an Azure WebRole provides the UI and WCF service for communicating with the SQL Azure Feed data. In order to handle the widest spectrum of devices, this UI leverages jQuery Mobile sitting on top of HTML5. jQuery Mobile Supported Platforms

    Inside the app divs (jQuery Mobile <div data-role=”page”..> ) are maps leveraging Bing Maps Ajax v7 API. The UI client also accesses Bing Geocode and Bing Route services through a proxy.

    Fig 3 – IE9 on a laptop with GPS dots and Tweet icons along the way

    Some more details:

    Routes

    Since there are several thousand points for each ‘event to event’ route, these are stored in SQL Azure as geography LineStrings. Using SQL Server spatial functions, the routes can be simplified on query for improved performance in both network latency and map rendering. SQL Azure’s geography Reduce(factor) function is a thinning algorithm that reduces the number of vertices of geography features while maintaining shape integrity. In this app the reduce factor is tied to zoomlevels of the map, thinning the number of points returned to the UI.

    The map viewport is used as a bounding box STIntersect so only the visible routes are returned. Obviously Carl and Richard may find reasons to take a different route so the GPS breadcrumbs may wander from the Bing generated routes.

    Tweets

    Fig 4 – WebMatrix iPad simulator

    The Twitter REST API queries are simple search by hashtag queries:
    http://search.twitter.com/search.json?q=%23dnrroadtrip

    To avoid returning lots of duplicate Tweets the search is limited by the last since_id in the Feed table. There are some caveats to REST Twitter searches:
    “Search is focused on relevance and not completeness. This means that some
    Tweets and users may be missing from search results

    Fig 5 - webmatrix WP7 emulator

    Fig 6 - iPhone simulator

    GPS points

    GPS points are generated every 60 seconds while the RV GPS unit is powered on. When the vehicle is off, and power is scarce, the unit still sends a packet once every 4 hours. Carl and Richard will be driving a lot of hours and there will be lots of points generated over the next 72 days and roughly 1728 hours. Assuming a 25% driving time over the duration, there could be as many as 1728/4 *60 = 25,920 GPS locations. Even Bing Maps Ajax v7 will choke trying to render this many locations.

    In order to keep things more reasonable, there is another thinning algorithm used in the GPS query service. This is again tied to zoomlevel . At lower zoom levels points are thinned using a type of decimation – every 20th, 10th, 5th point, etc is kept depending on the zoomlevel. In addition only points required by the viewport bounding box are returned. Once the map is zoomed to higher resolution (zoom > 11) all of the points will be returned.

    GPS map locations include a rollover infobox with time and detected speed at the location. We can all check up on Carl’s driving (moving: 86.99mph) and keep track of coffee stops (moving: 0.0 mph).

    Bing Routes

    Routing services are provided for user position to the latest GPS location and Stop venues selected on the map or from the Stop list. In addition to the route map a turn by turn directions list is provided as a page option. The GeoLocation API is used for identifying a user’s location for these routing requests. Geolocation API is an opt in API, so users need to allow location sharing to have their location automatically available. If allowed, getCurrentPosition returns a latitude, longitude which is run through the Bing reverse geocoding service to get an address used as the ‘from’ field for routing requests.

    Fig 7 - Stop Detail with Maps.Themes.BingTheme()

    Fig 8 - Bing Route Denver to Seattle Stop

    Fig 9 - Bing Turn by Turn directions

    jQuery Mobile

    jQuery Mobile is a javascript library for abstracting away some of the complexity of supporting a large number of devices. WP7, Win8 tablets, iPads, iPhones, and Android devices are multiplying while traditional laptop and desktop systems have a number of browser choices and versions. jQuery Mobile is not perfect but it is a great help in a project that had to be ready in about ten days from start to finish.

    One interesting feature of jQuery Mobile is the page transition effect. These are based on CSS3 and are not supported by all browsers. It adds a little pizazz to see slide, flip, and pop effects for page transitions.

    JQuery Mobile apps do not have access to device native sensors such as accelerometer, compass, gyrometer, etc , so jQuery Mobile webapps will not have the full range of capabilities found in custom apps for Win8, WP7, iOS, and Android. However, just one web UI for all is an enticing benefit, while deployment is ordinary webapp rather than a series of more complex app store submissions. This approach allows easy updates over the course of the tour.

    Fig 10 – Microsoft Way on an Android AVD emulator

    Heatmaps
    Collecting some locations always leads to the possibility of heatmaps. These are value gradients which are helpful for analyzing geospatial data.

    Fig 11 – Tweet heatmap along tour route from Seattle to Wyoming

    Maybe it’s pretty obvious where Tweets are concentrated, but how about locations of app users who share their location. Australia is hot, India is not. Guess who listens to .NetRocks! Or, at least who’s less cautious about sharing location with GeoLocation API. User heatmaps bring to mind some intriguing possibilities for monitoring use, markets, and promotion effectiveness.

    Fig 12 - GeoLocation users world wide

    Summary

    On the Road Again

    Windows 8 RTM and Bing Maps

    Fig 1 Windows 8 Grid App template

    Windows 8 RTM was released to developers last week. Win8 with its “Metro” style UI promises to be a major reset to both the Windows OS and the Microsoft developer community. No longer called “Metro” due to some potential trademark issues, the new Win8 style is very much aimed at touch devices. It targets the full spectrum of platforms: smart phones, tablets, laptops, desktops, and servers, but touch is obviously the preferred interface.

    The goal appears to be an OS that easily transitions between desktop and mobile. A worthy goal for multi device consumers who in a Microsoft world would no longer need to shift gears dramatically from one device to the next. OSs are like musical instruments, the fingerings differ from one to the next requiring an annoying shift of gestalt gears. Now that smartphones are firmly entrenched, consumers are forced to master alternative OSs and leakage out of the Microsoft OS corral is growing. Microsoft is concerned enough to take the risk of a Windows reset with Win8 Metro.

    In the conventional desktop world we have Windows, Apple OS X, and Linux focused on screen, mouse, and keyboard interfaces. In the mobile world we have iOS, Android, BlackBerry OS, and WP7 focused on screen, touch, voice, and a plethora of sensors pertinent to motion. Win8 puts both into one OS, desktop and mobile, but mobile is the front man. WP8 and Win8 appear to be on a merging trajectory.

    Win8 and Maps
    From a mapping perspective there are again two paths, desktop and mobile (Metro)
    … oh and plain old internet.

    Internet
    IE10 forks browser web apps as well:
    (with some nice html5, css additions, … and websockets)
    1) Win8 desktop IE10 – compatible desktop experience including plugins like Silverlight
    2) Win8 mobile IE10 – plugin free and touch complete. Silverlight is out for mobile IE10, as is any sanctioned 3D.

    There are also cosmetic differences that make desktop a compatibility version and mobile optimized for smaller form factors and touchy gestures. Bing Maps ajax v7 is slightly upgraded with a new Bing Theme module. As far as IE10 browsers, Bing Maps webapps using generic html5, css, jQuery Mobile with Bing Maps Ajax v7 will work with both IE10 branches and most any device, as well as avoiding Windows Store distribution.

    Win8 Desktop – old stuff
    Not much to say here. Everything I tried worked the same as usual. Bing Maps ajax v7, Bing Maps Silverlight, and Bing Maps WPF controls are all supported. Normal ClickOnce install works as before. VS2012 continues to evolve to accommodate Azure and mobile target platforms – ARM joins x86 and x64. Everything is familiar once you get to the desktop, but there can be some discovery process getting there.

    Win8 Mobile – new stuff
    A lot to explore here. – some media info on user experience for mobile Win8 Bing.

    From a Bing Maps developer perspective there are again two paths. (Lots of dualism in the picture) ref Windows API for Metro apps
    Both paths support distribution through the new Windows Store.

    1) Bing Maps SDK for JavaScript – Windows 8 JavaScript API is the same as the existing Bing Maps ajax v7 API and makes transitions easy for existing ajax v7 map applications. Starting a project in VS2012 allows selection of a template with all the appropriate boilerplate. The map action is exactly the same as Bing Maps ajax v7 API.

    Sample html for Bing Maps JavaScript SDK:

    <!DOCTYPE html>
    <html>
    <head>
        <meta charset="utf-8">
        <meta name="viewport" content="width=1024, height=768" />
        <title>BingMaps-JavaScript-SDK</title>
    
        <!-- WinJS references -->
        <link href="//Microsoft.WinJS.1.0/css/ui-dark.css" rel="stylesheet" />
        <script src="//Microsoft.WinJS.1.0/js/base.js"></script>
        <script src="//Microsoft.WinJS.1.0/js/ui.js"></script>
    
        <!-- BingMaps-JavaScript-SDK references -->
        <script type="text/javascript" src="ms-appx:///Bing.Maps.JavaScript//js/veapicore.js"></script>
        <script type="text/javascript" src="ms-appx:///Bing.Maps.JavaScript//js/veapiModules.js"></script>
    
        <!-- application specific -->
        <link href="/css/default.css" rel="stylesheet">
        <script type="text/javascript"  src="/js/default.js"></script>
    
    </head>
    <body>
    <div id="content">
        <div id="header"> </div>
        <div id="mapdiv"> </div>
     </div>
    </body>
    </html>

    Fig 2 – Windows 8 JavaScript API sample

    Javascript SDK event types are just the same as the existing ajax v7 API and do not include all the new sensor capabilities. However, the more interesting parts of Win8 are the multiple sensor support that requires Win8 Bing Maps SDK for Metro.

    2) XAML Bing Maps control – This is the most comprehensive SDK with access to all core platform features. Bing Maps SDK for Metro style apps is still RTM Beta with final build due at the end of September. For the present Maps will be decorated with a large BETA band.

    First there are some special Bing Map Keys available:

    1. Basic Windows Metro style app – Application is a Windows Metro style app that is publicly available without restriction and does not exceed 500,000 transactions of any type within a 12 month period.
    2. Enterprise Windows Metro style app – Application is a Windows Metro style app that is a public or a private application

    The distinguishing capability in Bing Maps SDK for Metro style is access to sensors when available on a target platform:

    Sensor classes:

    •  Accelerometer – This sensor returns G-force values with respect to the x, y, and z axes.
    • Compass – This sensor returns a heading with respect to True North and, possibly, Magnetic North.
    • Gyrometer – This sensor returns angular velocity values with respect to the x, y, and z axes
    • Inclinometer -This sensor returns pitch, roll, and yaw values that correspond to rotation angles around the x, y, and z axes, respectively.
    • LightSensor – This sensor returns the ambient-light reading as a LUX value.
    • OrientationSensor – This sensor returns a rotation matrix and a Quaternion that can be used to adjust the user’s perspective in a game application.
    • SimpleOrientationSensor – This sensor detects the current quadrant orientation of the specified device as well as its face-up or face-down status.

    GeoLocation:
    GPS is a subset of GeoLocation which can use fall back locations such as ip or WiFi triangulation as well as gps.

    Fig 3 - Example GeoLocation lookup in Bing Maps SDK for C# .

    Fig 4 Win8

    Fig 4 – Win8 Bing Maps SDK Beta

    Example XAML Page

    <Page
        x:Class="BingMap1.MainPage"
        IsTabStop="false"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:local="using:BingMap1"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        mc:Ignorable="d"
        xmlns:bm="using:Bing.Maps">
    
        <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}">
            <Grid.ColumnDefinitions>
                <ColumnDefinition Width="350" />
                <ColumnDefinition Width="*" />
            </Grid.ColumnDefinitions>
            <Grid.RowDefinitions>
                <RowDefinition Height="*" />
            </Grid.RowDefinitions>
            <StackPanel Orientation="Vertical" VerticalAlignment="Top" HorizontalAlignment="Center">
                <Button x:Name="btn" Click="btnClicked" Content="Get Location" FontSize="26"/>
                <Grid x:Name="Output" HorizontalAlignment="Left" VerticalAlignment="Top" Grid.Row="1">
                    <Grid>
                        <Grid.ColumnDefinitions>
                            <ColumnDefinition Width="Auto" />
                            <ColumnDefinition />
                        </Grid.ColumnDefinitions>
                        <Grid.RowDefinitions>
                            <RowDefinition Height="Auto" />
                            <RowDefinition Height="Auto" />
                            <RowDefinition Height="Auto" />
                        </Grid.RowDefinitions>
                        <TextBlock TextWrapping="Wrap" Grid.Row="0" Grid.Column="0" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Left" Text="Latitude: " />
                        <TextBlock TextWrapping="Wrap" Grid.Row="1" Grid.Column="0" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Left" Text="Longitude: " />
                        <TextBlock TextWrapping="Wrap" Grid.Row="2" Grid.Column="0" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Left" Text="Accuracy: " />
                        <TextBlock x:Name="ScenarioOutput_Latitude" TextWrapping="Wrap" Grid.Row="0" Grid.Column="1" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Right" Text="No data" />
                        <TextBlock x:Name="ScenarioOutput_Longitude" TextWrapping="Wrap" Grid.Row="1" Grid.Column="1" Style="{StaticResource BasicTextStyle}" FontSize="24" HorizontalAlignment="Right" Text="No data" />
                        <TextBlock x:Name="ScenarioOutput_Accuracy" TextWrapping="Wrap" Grid.Row="2" Grid.Column="1" Style="{StaticResource BasicTextStyle}"  FontSize="24" HorizontalAlignment="Right" Text="No data" />
                    </Grid>
                </Grid>
            </StackPanel>
    
            <bm:Map Grid.Column="1" x:Name="map" Width="Auto" Height="Auto" MapType="Road" ZoomLevel="12" Credentials="<Bing Key Here>" >
                <bm:Map.Center>
                    <bm:Location Latitude="47.62" Longitude="-122.19" />
                </bm:Map.Center>
            </bm:Map>
        </Grid>
    </Page>

    C# code behind

    using Bing.Maps;
    using System;
    using System.Threading;
    using System.Threading.Tasks;
    using Windows.Devices.Geolocation;
    using Windows.UI.Xaml;
    using Windows.UI.Xaml.Controls;
    using Windows.UI.Xaml.Navigation;
    
    namespace BingMap1
    {
    
        public sealed partial class MainPage : Page
        {
            private Geolocator _geolocator = null;
            private CancellationTokenSource _cts = null;
    
            public MainPage()
            {
                this.InitializeComponent();
            }
    
            /// <summary>
            /// Invoked when this page is about to be displayed in a Frame.
            /// </summary>
            /// <param name="e">Event data that describes how this page was reached.  The Parameter
            /// property is typically used to configure the page.</param>
            protected override void OnNavigatedTo(NavigationEventArgs e)
            {
                _geolocator = new Geolocator();
            }
    
            private void btnClicked(object sender, RoutedEventArgs e)
            {
                GetGeolocation(sender, e);
            }
    
            async private void GetGeolocation(object sender, RoutedEventArgs e)
            {
                try
                {
                    _cts = new CancellationTokenSource();
                    CancellationToken token = _cts.Token;
    
                    Geoposition pos = await _geolocator.GetGeopositionAsync().AsTask(token);
    		  // update text
                    ScenarioOutput_Latitude.Text = pos.Coordinate.Latitude.ToString();
                    ScenarioOutput_Longitude.Text = pos.Coordinate.Longitude.ToString();
                    ScenarioOutput_Accuracy.Text = pos.Coordinate.Accuracy.ToString();
    		   // update map location
                    map.SetView(new Location(pos.Coordinate.Latitude, pos.Coordinate.Longitude), 12);
                }
                catch (System.UnauthorizedAccessException ue)
                {
                    ScenarioOutput_Latitude.Text = "No data";
                    ScenarioOutput_Longitude.Text = "No data";
                    ScenarioOutput_Accuracy.Text = "No data";
                }
                catch (TaskCanceledException te)
                {
                }
                finally
                {
                    _cts = null;
                }
            }
        }
    }

    Fig 5 – Metro style tiles including the above BingMaps1 project

    Fig 6 – Win8 Bing Maps SDK with GeoLocation and BirdsEye view

    Distribution
    The Windows Store is a prominent tile in the Start page. Win8 applications are vetted through a Microsoft analysis and certification process before distribution through the public Windows Store. Of course any app created in VS2012 is available as a tile in the Start Page of the development system, but getting an app to clients happens through the Windows Store. Enterprise LOB applications do have a SideLoading process to bypass the public Windows Store.

    Fig 7 – Windows Store for Win8 app delivery (Sideloading is possible for Enterprise licensees)

    Summary
    At present Win8 is still a future affair. Developers can play, while consumers wait. Microsoft dominance of desktops will guarantee a desktop market going forward. Since neither phone nor tablet will be available to consumers with Win8 until later 4th quarter, Win8 is an OS focused on devices that may or may not generate a market. The risk is alienating a relatively loyal base of desktop users in the hope of gaining traction in the fast growing mobile market. Without a market, developers will not devote much effort to all the new stuff.

    Win8 is a “do or die” gambit for Balmer, Sinofsky, and friends. Mobile is moving fast and Microsoft’s future in mobile remains in serious question. If the mobile market is entirely lost, Microsoft is headed toward a long slow Gibbonesqe decline. However, Win8 is potentially the lever required to pry Microsoft into at least a place at the mobile table. We won’t know much for a few more quarters. In the meantime steep learning curves ahead for Bing Maps devs.

    Fig 8 - NASA Avg Day Temperatures over past month

    Mobile Maps with WebMatrix2 and jQuery Mobile

    Fig 1 – WebMatrix 2 iPhone Simulator

    Some changes have been rolling out of Redmond recently. Perhaps the realization that mobile is truly capable of tipping the Windows canoe is sinking in. The underdog viewpoint is relatively new to Microsoft. The newest Azure rollout includes more versatile VM instances and lots of languages hooks.

    Fig 2 – the newest Azure SDK has lots of language hooks

    Github:windows plays nice with Windows Azure Websites and Microsoft offers a free website development IDE tool, WebMatrix2. This is not a replacement for Visual Studio, but it’s a great little tool for javascript UI development.

    Going Mobile

    There are a few ways to approach mobile development. You can develop for each of the popular platforms, but that takes a bit of effort getting the tools for iOS and Android as well as learning enough Objective-C and Android Java to be productive.

    There are some tools for cross platform development like Titanium Appcelerator, which provide a runtime for translating controls written in a javascript format to native controls on the device. This requires learning new controls, IDE tools, as well as adding the runtime to your app.

    Both of the above require registration and familiarity with various app store submission processes along with a fee.

    3rd Way Mobile

    One of the easiest ways to make a mobile map app is to just use the familiar web platform approach leveraging HTML5. Expose some data through a custom service layer or hook up to a REST API, and then use jQuery Mobile to make UI apps with HTML5 targeted to mobile clients. This won’t require an app store submission or their requisite fees.

    WebMatrix adds some nice features for website development. Just install the tool and open a directory project to get started.

    Fig 3 – WebMatrix2 with a MobileLocator project

    WebMatrix makes it easy to check a design in most of the common browsers and mobile platforms.

    Fig 4 – WebMatrix Run Control exposes multiple browser and simulator options

    Mobile simulators for iPhone, iPad, and WP7 are not installed by default. Click on Gallery to check for available online extensions to WebMatrix.

    Fig 5 – WebMatrix Gallery button for extensions

    For mobile development a couple of “need to have” extensions are the iPhone and iPad Simulators.

    Fig 6 – WebMatrix gallery options iPad and iPhone Simulators

    A WP7 emulator is also available.

    Fig 7 – WebMatrix Windows Phone 7 Emulator

    WebMatrix extensions are not available for Android, probably due to some sort of license restriction. However, Android emulators are part of the free Android SDK install under the Android AVD manager. Android testing requires a separate emulator setup without the convenience of the WebMatrix Run list.

    Fig 8 – Android emulators are part of the Android SDK install

    Once the Android AVD is available, testing is just a matter of typing in the local url of a webapp site and checking for visual integrity.

    jQuery Mobile

    jQuery Mobile offers HTML5 javascript controls specialized for mobile devices. It abstracts away many of the variations in html5 implementation.
    (See supported platform list)

    jQuery Mobile makes use of specialized javascript metadata to create jQuery type controls from a single html file. Element attributes like data-role, data-rel, and data-transition add html5 objects and behaviors suited to mobile devices. The data-transition attribute adds animated transition effects. (Note, not all browsers support data-transition but fallback is benign)

    <a href="index.html" data-transition="slidefade">Transition to page</a>

    jQuery Mobile Theme Roller is a styling mechanism with readymade themes. This is a bit different from jQuery UI themes. Using data theme attributes ‘a’ through ‘e’, called swatches, styling propagates to all contained elements:

    <div data-role="page" data-theme='b' id="foo">

    Sample Mobile App
    http://www.web-maps.com/MobileFinder
    With jQuery Mobile the basic UI tools are available for controlling a map application such as this store finder with driving directions. Using HTML5 has the additional benefit of support across both mobile and desktop browsers. Now a useful Mobile UI can be added to a map finder service layer by just adding a page link to a consumer facing business portal.

    In this sample, location points are viewed as a map or a list. Bing route and geocode services are used to get basic polyline route and turn by turn directions. Not rocket science, and with jQuery Mobile this is an easy extension geared to mobile device clients.

    Fig 9 – Sample app shown in the iPad and iPhone WebMatrix simulators

    Fig 10 – iPad Simulator showing a Bing generated Route

    Fig 11 – iPad simulator showing the turn by turn directions view

    Fig 12 – WP7 WebMatrix Emulator

    Summary

    The last 6 months have seen a dramatic uptick in mobile phone clients at corporate websites. Adding a mobile option lets customers access simple map finders and directions while on the road. Customers en route typically do not want to install a web app just to find nearby coffee, gas, or grocery stores carrying a particular brand of pretzels. So there is a place for leveraging HTML5 to escape the app store corral, and jQuery Mobile makes it easy.

    TerraServer RIP

    “Two roads diverged in a yellow wood”

    Fig 1 MapSavvy WMS referenced in AutoCAD Map 3D

    A Brief History of WMS

    Open Geospatial Consortium, OGC, has been an influential geospatial standards body since the late 90’s. OGC standards are at the root of SQL spatial databases, commercial GIS tools, as well as most open source mapping projects found at OSGeo. OGC standards have been adopted at all levels from Microsoft SQL Server, Oracle Spatial, ESRI, and AutoCAD to PostGIS, GeoServer, and Quantum GIS. OGC standards are part of the DNA of modern GIS and internet mapping.

    One of the more popular OGC standards, first published at the turn of the millennium, is Web Mapping Service. WMS is a map distribution standard for showing arbitrary BBox map areas in common internet browsers.

    One of the early significant WMS services was a partnership between Microsoft Research and the USGS called Microsoft TerraServer, which was in continuous operation from 1998 until today, 5/1/2012. More recently the name was changed to MSR Maps. Microsoft TerraServer pioneered free availability of geospatial data in an OGC compliant service. At the same time, TerraServer research established the viability of quad tile storage, but then merged tiles into arbitrary BBox chunks for WMS GetMap requests.

    TerraServer was grandfather to two forks of web mapping, quad tile pyramids and WMS open data services. May 1st 2012 marks the end of this pioneering service.

    requiescat in pace

    How WMS works

    WMS defines a standard method of publishing capabilities to the web and furnishing images of requested areas. A WMS client first asks what layers are available, in what formats, styles, and projections. Some decisions are made depending on need and availability. These decisions along with a specific bounding box are sent back to the WMS server. The WMS server builds the requested image and returns it as a rectangular png, jpeg, tiff etc. Behind the WMS server is some type of data repository, either vector data or imagery, but the WMS server handles all requests for specific parts of this data resource.

    Fig2 - OGC Web Mapping Service

    This was an ambitious scheme in 2000, attempting to smooth the distribution of a multitude of proprietary formats and display methods. The concept was simply to make access to spatial data a “write once, use anywhere architecture.” Momentum built slowly at first, but by this time, May of 2012, there are many choices for WMS servers, and WMS clients are built into a multitude of mapping tools and websites.

    KaBoom! Two roads

    In 2005 this whole approach was upended with the introduction of web maps and a new tile serving slippy map architecture. The problem WMS ran into was the limit of CPU resources attempting to build individual map requests for millions of users. Servers eventually bogged down trying to fulfill requests for arbitrary WMS viewports. The new slippy map architecture doesn’t build anything. Tiles flow to clients simply as requested, directly from storage or cache. The browser client is responsible for rendering, so web mapping suddenly became massively parallel. Map pan and zoom went from request, pause, display, to continuous uninterrupted motion, quite magical at its first introduction.

    This approach was a recapitulation of TerraServer storage but dropped the WMS at the end of the server pipe. Tile pyramids became just static image stores streaming into a map client as needed by pan and zoom.

    WMS was fine for the technical folks and enterprise focus on proprietary assets, but it just didn’t scale into the internet consumer realm. Web mapping forked at this point. WMS continued to grow as a basic function of every desktop map tool and internet client, while web mapping adopted vast tile pyramids prebuilt from vector features and aerial imagery. Microsoft Bing Maps, MapQuest, and Google Maps are the glamorous handmaidens of search, serving up map tiles to the masses. OGC services haul water behind the scenes, with the pocket protector crowd.

    Between Two Roads

    The two architectures coexist in the internet realm. OGC services are work horses in engineering cubicles, but the mobile generation is all about slippy tiles. Many web apps merge one of the world scale base map tile pyramids with layers of OGC services on top. This approach marshals world level extent together with locally maintained assets. Example Fig 3

    Fig 3 – Bing Maps behind Census Bureau TIGER WMS/WFS layers

    Against the Stream

    WMS layers can live on top of popular web maps like Bing, but what about the opposite direction? Can WMS clients make use of worldwide tile pyramids?

    “A dead thing can go with the stream, but only a living thing can go against it.”
    G.K. Chesterton

    OnTerra MapSavvy

    Far down stream from Jim Gray and Tom Barclay, OnTerra created a simple WMS service to bridge the fork in the road between OGC and Bing Maps tiles, but in the opposite direction. MapSavvy is a WMS service that simply exposes Bing Maps Aerial and Road imagery to existing users of WMS clients. Subscribe to the service and imagery for the entire world is available inside your mapping tool of choice.

    There are thousands of man years of development tied up in tools that perform very specific business functions. These tools are not especially public facing. They perform the required task, but the task is of no particular interest to the rest of us. Companies have to pay people to use tools like this. In cases of business focused WMS tools, wouldn’t it be nice to access the vast resources of tile imagery abounding on the internet?

    For example, Microsoft is just finishing a program to capture most of the US and Western Europe at 30cm pixel resolution and serving this in Bing Maps Aerial. This kind of resolution at national scales is a very useful resource. Designers, engineers, defense agencies, and GIS analysts can always use higher resolution imagery available for large parts of the planet. Lower resolution imagery fills in context for the remainder of the world. Ref: Bing Maps resolution chart

    Bing Maps 30cm Aerial: “The project is scheduled for completion by June 2012 at which point a refresh cycle will begin to update much of what will already have been collected, with priority placed on locations that have been more subject to change (as opposed to, say, the Mojave Dessert).” See Fig 4

    Fig 4 – Global Mapper WMS client using MapSavvy - Mojave Desert 0.30 m per pixel

    Here is a short list from the multitude of WMS clients:

    • AutoCAD Map 3D – Fig 1
    • Bentley MicroStation V8i
    • Gaia
    • Global Mapper – Fig 4
    • Tableau
    • Smallworld GeoSpatial
    • gvSig – Fig 5
    • Quantum GIS – Fig 6
    • MapInfo
    • TatukGIS Viewer
    • TopoFusion

    Fig 5 – gvSIG WMS client using MapSavvy - Coors Field in Denver

    Fig 6 – QGIS WMS client - Detail of Coors Field Blake Street entrance - 0.30 m per pixel

    Some Details

    Bing Maps, like all web mapping tile services, leverages the symmetry of a Mercator world projection, which is technically defined as epsg:3857. In order to support the widest range of WMS clients MapSavvy publishes both epsg:3857 and epsg:4326. The second projection has less value, as it distorts the imagery, but, in cases where a WMS client cannot make use of the more accurate epsg:3857, epsg:4326 is helpful. This is especially useful, at higher resolutions where the distortion is reduced. MapSavvy helps resolve some of the distortion issues of epsg:4326 by using transformed images for zoom levels at the top of the tile pyramid.

    Unlike Google Maps, which explicitly excludes use of Google tiles in a WMS service, Bing Maps wants customers to use their tile servers. OnTerra licenses Microsoft Bing Map tiles for use in a WMS. MapSavvy subscriptions include a restricted Microsoft sub-license for use of Bing road and imagery tiles in the client, which simplifies life for small business users.

    Some TerraServer Alternatives

    TerraServer has historical interest but in the past five years there hasn’t been much demand for DOQ, Urban Ortho, or DRG Topo maps. The demise of TerraServer may go largely unnoticed. For those left orphaned, National Atlas publishes several WMS endpoints. However, Bing Maps Aerial is a much improved replacement for USGS DOQ and USGS Urban Ortho. MapSavvy WMS can replace TerraServer DOQ and Urban Ortho layers directly in existing WMS clients.

    If you’re looking for alternative USGS Topo contours, Ahlzen’s TopOSM is a good place to start.


    TopOSM Contours

    Fig 7 TopOSM contours over Bing Maps

    Summary

    Microsoft TerraServer was the starting point for a lot of what we take for granted in 2012. Its passing is understandable but leaves a twinge of regret.

    Two roads diverged in a yellow wood,
    And sorry I could not travel both
    And be one traveler, long I stood
    And looked down one as far as I could
    To where it bent in the undergrowth;

    .
    .
    - Robert Frost

    “When you arrive at a fork in the road, take it.”
    - Yogi Berra

    Leaflet and Census TIGERweb


    Fig 1 – Sample Leaflet interface to the US Census Bureau’s new TIGERweb WMS service with Bing Maps background

    Ref: http://silverlight.onterrasys.com/MapSavvyLeaflet/

    US Bureau of the Census – some background

    1990 Census
    The 1990 Census line data made a revolutionary leap from GBF-DIME format to TIGER, Topologically Integrated Geographic Encoding and Referencing, and also adopted CD-ROM media distribution. In the early 90′s I wrote a few TIGER translation programs for AutoCAD users and users of a popular (now defunct) desktop GIS package called Atlas*GIS. TIGER was useful for its linear data, but was more often used for the boundary polygons that could be tied to Census population demographic data. The line features of the 1990 TIGER were based on the older 1980 GBF-DIME enhanced with USGS 1:100,000 DLG features. GBF DIME was notable for the tell tale zigzag streets in urban areas.

    2000 Census
    By the mid 90’s Open GIS Consortium (OGC) was actively working on OpenGIS standards. As the internet continued ramping up, OGC released the OpenGIS Web Map Server (WMS) Specification in 2000. Even though Census data releases are a year or so after the actual census, it was several more years before the Census Bureau started investigating alternative WMS services for distributing data. In the meantime boundaries were available for internet and ftp download in ESRI shp and E00 formats. This was convenient, but requires self hosting web solutions.

    2010 Census
    Ever since the 2000 WMS specification there were hopes for a service distribution of census TIGER/Line. Not until the end of 2011 was this belatedly realized as TIGERweb. But of course by the mid 2000s internet mapping began changing radically with the popularity of Google Maps, GPS, Bing Maps, Open Street Maps, and now ubiquitous mobile devices. The Census Bureau will need to continue evolving to meet the challenges of these rapid changes.

    TIGERweb 2011
    So why not use the TIGERweb for an experimental look at some more recent web tools? TIGERweb Viewer UI is based on the ESRI Silverlight APIs. Silverlight is a powerful UI technology, but it is destined to be supplanted by html5. As browser market fragmentation and mobile devices force adoption of standards based technologies, proprietary plugins are just not making the transition, at least for public facing web apps.


    Fig 2 – TIGERweb Viewer based on ESRI Silverlight API

    TIGERweb Viewer seems oddly designed. Rather than the clean, spare look popular today, TIGERweb Viewer seems to be a throwback to an earlier internet. Unusual matrice animations with rotating panels and creepy crawling boundary selectors seem a bit out of place in the modern web. Even though technically possible, and simple to implement in Silverlight, such distractions do nothing to make the UI endearing. After the first blink most users will inevitably recall early internet flashing text and PowerPoint decks filled with animated gifs. Anyone remember the irritation of Clippy?

    However, TIGERweb WMS Service is a wide open door for standards based javascript libraries. One of the more elegant available tools is LeafletJS. This is a “lightweight” js map library that works well with jQuery and abstracts away some of the complexity of supporting a fragmenting mobile browser world.

    Leaflet is Open Source BSD-licensed with an active contributing community and available on github. It uses newer html5 technologies available in A-grade browsers, but includes fall back for older browsers. It’s nice to have touch events and the geolocation api.

    The Leaflet api provides a convenient L.TileLayer.WMS object that transforms a WMS endpoint into a slippy tile map. Performance isn’t as good as a static tile pyramid, but it isn’t bad and simplifies life immensely.

      var censusTracts = new L.TileLayer.WMS("http://tigerweb.geo.census.gov/ArcGIS/services/tigerWMS/MapServer/WMSServer",
      {
        layers: 'Census Tracts',
        format: 'image/png',
        transparent: true,
        attribution: TigerAttribution
      });

    Bing Maps
    Unlike Google Maps, Microsoft Bing allows licensed access to tile servers. For development, non-profit, and education uses, this is just a matter of registering for a Bing key. Adding a Bing Maps layer with Leaflet is possible using https://gist.github.com/1221998. Once this Leaflet Bing extension javascript is referenced into a page, adding Bing Map styles is simply a matter of adding L.TileLayer.Bing.

    var BingAerialWithLabels = new L.TileLayer.Bing(bingkey, "AerialWithLabels",
    {
      maxZoom: 21,
      attribution: BingAttribution
    });

    Fig 3 – Leaflet Census TIGER WMS with Bing Maps Road style

    TIGER/Line 2010 accuracy has been upgraded substantially, but you can occasionally see some discrepancies against Bing Aerial.

    Fig 4 – Leaflet Census TIGER WMS with Bing Maps Aerial with Labels style

    Once all of the WMS and Bing L.TileLayers are instantiated, a new L.Map with the desired L.TileLayer set is declared this way:

                map = new L.Map('mapDiv',
                        {
                            center: new L.LatLng(39.748, -104.981),
                            zoom: 8,
                            layers: [censusStates, censusCounties, censusTracts, censusBlockGrps, censusBlocks]
                        });

    Leaflet also provides a simple L.Control.Layers for a user layer control panel:

                var baseMaps = {
                    "BingRoads": BingRoad,
                    "BingAerial": BingAerial,
                    "BingAerialWithLabels": BingAerialWithLabels
                };
    
                var overlayMaps = {
                    "States": censusStates,
                    "Counties": censusCounties,
                    "Tracts": censusTracts,
                    "BlockGroups": censusBlockGrps,
                    "Blocks": censusBlocks,
                    "Lakes": censusArealHydrography,
                    "Rivers Streams": censusLinearHydrography,
                    "LocalRoads": censusLocalRoads,
                    "SecondaryRoads": censusSecondaryRoads,
                    "PrimaryRoads": censusPrimaryRoads,
                    "Nexrad": nexrad
                };
    
                var layersControl = new L.Control.Layers(baseMaps, overlayMaps,
                {
                    collapsed: !L.Browser.touch
                });
                map.addControl(layersControl);

    Fig 5 – Leaflet L.Control.Layers

    TIGERweb WMS GetFeatureInfo is available for queries. For example:

    http://tigerweb.geo.census.gov/ArcGIS/services/tigerWMS/MapServer/WMSServer?Service=WMS&Version=1.1.1&Request=GetFeatureInfo&Layers=Counties&SRS=EPSG:4326&BBOX=-122.5,37.75,-122.25,38.0&width=1000&height=1000&QUERY_LAYERS=Counties&INFO_FORMAT=text/xml&X=100&Y=100
    
    <?xml version="1.0"?>
    <FeatureInfoResponse xmlns="http://www.esri.com/wms" xmlns:esri_wms="http://www.esri.com/wms">
    <FIELDS
      HU100="111214" --HousingUnits
      POP100="252409"
      INTPTLON="-122.7459738"
      INTPTLAT="+38.0518169"
      CENTLON="-122.7485752"
      CENTLAT="+38.0551413"
      STGEOMETRY="Polygon"
      UR="Null"
      STATE="06"
      OID="27590357299439"
      NAME="Marin County"
      BASENAME="Marin"
      MTFCC="G4020"
      METDIV="41884"
      LSADC="06"
      GEOID="06041"
      FUNCSTAT="A"
      CSA="488"
      COUNTY="041"
      COUNTYCC="H1"
      CBSA="41860"
      AREAWATER="797420433"
      AREALAND="1347585521"
      COUNTYNS="00277285"/>
    </FeatureInfoResponse>

    jQuery makes XMLHttpRequest communication easy, but TIGERweb WMS doesn’t expose an open ClientAccessPolicy.xml, so a self hosted proxy is required.

                $.ajax({
                    type: "GET",
                    url: getInfoURL,
                    dataType: "xml",
                    success: GetFeatureInfo
                });

    Fig 6 – Leaflet Popup panel with the results of TIGERweb WMS GetFeatureInfo request

    Some Caveats

    TIGERweb
    1) The TIGERweb WMS furnishes images for all the TIGER boundary and line features. However, there doesn’t seem to be a WMS layer for labels like those on the TIGERweb Viewer. There is an undocumented label service that might be useful.

    http://tigerweb.geo.census.gov/ArcGIS/rest/services/Labels/MapServer/export

    2) WMS GetFeatureInfo requests can provide information from the polygon stack, but the lack of a cross-domain policy file on the TIGERweb WMS server means that a pass through proxy service needs to be hosted on the same server as the Leaflet web app. This proxy service simply provides the same GetFeatureInfo result to the client while avoiding cross-domain issues.

    Leaflet
    1) Bing javascript
    The Bing javascript extension seemed to run into a problem with intersect at this line:
    if (zoom = coverage.zoomMin && coverage.bbox.intersects(bounds))

    Removing the intersect(bounds) clause seemed to get things working without noticeable problems:
    if (zoom = coverage.zoomMin )

    2) Leaflet seems to have problems with TileLayer ordering. Bing TileLayers can mask other features if directly added to the map layer set.

    3) There are some odd behaviors on Amazon Kindle Fire, possibly due to the Silk web browser’s EC2 caching.

    4) Leaflet L.Control.Layers doesn’t work consistently in the mobile browsers I tested. The collapsed layer control icon never opened for iPhone or Kindle Fire. Adding the suggested option collapsed: !L.Browser.touch keeps the layer control open instead of closing when untouched. This wouldn’t be a problem except that mobile devices don’t have the screen space for extra panels.

    5) The default attribution for Bing gets quite lengthy, especially when displaying Aerial vendor meta data. Turning this off and deferring to the Bing Maps Logo frees up more screen space which is at a premium on mobile devices.

    Summary

    Census Bureau’s new TIGERweb is a great step forward for users of TIGER data. Hopefully the momentum will continue forward to include demographic data. SF1 data joined to TIGER polygons would be a more useful service. Adding an OGC WFS service would be the ideal, but WFS always has problems with potentially very large result sets, so I’m not sure this is practical on a public OGC service.

    Since the first time I saw Leaflet I’ve been intrigued with its elegance and the potential for abstracting away some of the issues surrounding web apps targeted at mobile and touch clients. In spite of a few minor issues Leaflet is one of the best javascript map apis I’ve used. I believe the issues I ran into are on a bug list and could be resolved by next release.

    This bare bones CensusTest project took only a few hours over the weekend to write using Leaflet.

    Fig 7 – Lower Manhattan

    WebBIM? hmmm .. Bing BIM?

    BIM or Building Information Modeling isn’t exactly news. I guess I tend to think of it as a part of AEC, starting back in the early generations of CAD, as a way to manage a facility in the aftermath of design and construction. BIM is after all about Buildings, and Buildings have been CAD turf all these many years. Since those early days of FM, CAD has expanded into the current BIM trend with building lifecycle management that is swallowing up larger swathes of surrounding environments.

    BIM models are virtual worlds, ‘mirror lands’ of a single building or campus. As BIM grows the isolated BIM models start to aggregate and bump up against the floor of web mapping, the big ‘Mirror Land’. One perspective is to look at a BIM model as a massively detailed Bill of Material (BIM<=>BOM . . bing) in which every component fitted into the model is linked to additional data for specification, design, material, history, approval chains, warranties, and on and on. BIM potentially becomes one massively connected network of hyperlinks with a top level 3D model that mimics the real world.

    Sound familiar? – BIM is a sub-internet on the scale of a single building with an interface that has much in common with web mapping. Could this really be yet another reincarnation of Ted Nelson’s epic Xanadu Project, the quixotic precursor of today’s internet?

    Although of relatively recent origins, BIM has already spawned its own bureaucratic industry with the likes of NBIMS replete with committees, charters, and governance capable of seriously publishing paragraphs like this:

    “NBIM standards will merge data interoperability standards, content values and taxonomies, and process definitions to create standards which define “business views” of information needed to accomplish a particular set of functions as well as the information exchange standards between stakeholders.”

    No kidding, “taxonomies”? I’m tempted to believe that ‘Information’ was cleverly inserted to avoid the embarrassing eventuality of an unadorned “Building Model.” Interesting how Claude Shannon seems to crop up in a lot of acronyms these days: BIM, GIS, NBIMS, ICT, even IT?

    BIM has more recently appeared on the GIS radar with a flurry of discussion applying GIS methods to BIM. Here, for example are a couple of posts with interesting discussion: SpatialSustain, GeoExpressions, and Vector One. Perhaps this is just another turf battle arising in the CAD versus GIS wars. I leave that for the GISCIers and NBIMSers to decide.

    My interest is less in definition and more an observation that buildings too are a part of the rapidly growing “Mirror Land” we call web maps. Competing web maps have driven resolution down to the region of diminishing returns. After all, with 30cm commonly available, is 15cm that much more compelling? However, until recently, Mirror Land has been all about maps and the wide outside world. Even building models ushered in with SketchUp are all about exteriors.

    The new frontier of web mapping is interior spaces, “WebBIM,” “Bing BIM” ( Sorry, I just couldn’t resist the impulse). Before committing to national standards, certifications, and governance taxonomies perhaps we need to just play with this a bit.

    We have Bing Maps Local introducing restaurant photoscapes. Here’s an example of a restaurant in Boston with a series of arrow connected panoramas for virtual exploration of the interior.

    And another recent Bing Maps introduction, mall maps. Who wants to be lost in a mall, or are we lost without our malls?

    And then in the Google World Art Projects explore museums. Cool, Streetside Inside!

    It’s not obvious how far these interior space additions will go in the future, but these seem to be trial balloons floated for generating feedback on interior extensions to the web map mirror world. At least they are not introduced with full fledged coverage.

    “Real” BIM moves up the dimension chain from 2D to 3D and on to 4 – 5D, adding time and cost along the way. Mirror Land is still caught in 2-3D. The upcoming Silverlight 5 release will boost things toward the 3-4D. Multi-verse theories aside (now here’s a taxonomy to ponder – the Tegmark cosmological taxonomy of universes), in the 3-4D range full WebBIM can hit the streets. In the meantime the essential element of spatially hyperlinked data is already here for the curious to play with.

    So what’s a newbie Web BIMMER to do? The answer is obvious, get a building plan and start trying a few things. Starting out in 2D, here is an approach: get a building floorplan, add it to a Bing Maps interface, and then do something simple with it.

    Step 1 – Model a building

    CAD is the place to start for buildings. AEC generates floorplans by the boatload and there are even some available on line, but lacking DWG files the next possibility is using CAD as a capture tool. I tried both approaches. My local grocery store has a nice interior directory that is easily captured in AutoCAD by tracing over the image:

    King Soopers Store Directory
    Fig 5 – King Soopers Store Directory (foldout brochure)

    As an alternative example of a more typical DWG source, the
    University of Alaska
    has kindly published their floor plans on the internet.

    In both scenarios the key is getting the DWG into something that can readily be used in a web map application. Since I’m fond of Bing Silverlight Map Control, the goal is DWG to XAML. Similar things can be done with SVG, and will be as HTML5 increases its reach, and probably even in KML for the Googler minded. At first I thought this would be as easy as starting up Safe Software’s FME, but XAML is not in their writers list, perhaps soon. Next, I fell back to the venerable DXF text export with some C# code to turn it into XAML. This was actually fairly easy with the DXF capture of my local grocery store. I had kept the DWG limited to simple closed polylines and text, separated by layer names.

    Here is the result:

    Now on to more typical sources, DWG files which are outside of my control. Dealing with arcs and blocks was more than I wanted, so I took an alternative path. FME does have an SVG writer. SVG is hauntingly similar to XAML (especially haunting to W3C), and writing a simple SVG to XAML translator in C# was easier than any other approach I could think of. There are some XSLT files for SVG to XAML, but instead I took the quick and dirty route of translating SVG text to XAML text in my own translator giving me some more control.

    Here is the result:

    Step 2 – Embed XAML into a Bing Map Control interface

    First I wrote a small webapp that allows me to zoom Bing maps aerial to a desired building and draw a polyline around its footprint. This polyline is turned into a polygon XAML snippet added to a TextBox suitable for cut/paste as a map link in my Web BIM experiment.

     <m:MapPolygon Tag="KingSoopers2" x:Name="footprint_1"
     Fill="#FFfdd173" Stroke="Black" StrokeThickness="1" Opacity="0.5"
    MouseLeftButtonUp="MapPolygon_MouseLeftButtonUp">
    <m:MapPolygon.Locations>
    39.05779345,-104.84322060
    39.05772368,-104.84321926
    39.05770910,-104.84302480
    39.05771014,-104.84290947
    39.05772159,-104.84277536
    39.05776116,-104.84277804
    39.05776429,-104.84243204
    39.05833809,-104.84248434
    39.05833288,-104.84283303
    39.05836204,-104.84284510
    39.05835996,-104.84313880
    39.05832872,-104.84313880
    39.05832663,-104.84340836
    39.05825478,-104.84340568
    39.05825374,-104.84354113
    39.05821000,-104.84353979
    39.05820792,-104.84369670
    39.05779137,-104.84367792
    39.05779345,-104.84322060
    </m:MapPolygon.Locations>
    </m:MapPolygon>
    

    As a demonstration, it was sufficient to simply add this snippet to a map control. The more general method would be to create a SQL table of buildings that includes a geography column of the footprint suitable for geography STIntersects queries. Using a typical MainMap. ViewChangeEnd event would then let the UI send a WCF query to the table, retrieving footprints falling into the current viewport as a user navigates in map space. However, the real goal is playing with interior plans and I left the data connector feature for a future enhancement.

    In order to find buildings easily, I added some Geocode Service calls for an Address finder. The footprint polygon with its MouseLeftButtonUp event leads to a NavigationService that moves to the desired floor plan page. Again generalizing this would involve keeping these XAML floor plans in a SQL Azure Building table for reference as needed. A XAML canvas containing the floor plans would be stored in a BLOB column for easy query and import to the UI. Supporting other export formats such as XAML, SVG, and KML might best be served by using a GeometryCollection in the SQL table with translation on the query response.

    Step 3 – Do something simple with the floorplans

    Some useful utilities included nesting my floorplan XAML inside a <local:DragZoomPanel> which is coded to implement some normal pan and zoom functions: pan with left mouse, double click zoom in, and mouse wheel zoom +/-. Mouse over text labeling helps identify features as well. In addition, I was thinking about PlaneProjections for stacking multiple floors so I added some slider binding controls for PlaneProjection attributes, just for experimentation in a debug panel.

    Since my original King Soopers image is a store directory an obvious addition is making the plan view into a store directory finder.

    I added the store items along with aisles and shelf polygon id to a table accessed through a WCF query. When the floorplan is initialized a request is made to a SQL Server table with this directory item information used to populate a ListBox. You could use binding, but I needed to add some events so ListBoxItems are added in code behind.

    Mouse events connect directory entries to position polygons in the store shelves. Finally a MouseLeftButtonUp event illustrates opening a shelf photo view which is overlaid with a sample link geometry to a Crest product website. Clicks are also associated with Camera Icons to connect to some sample Photosynthes and Panoramas of store interior. Silverlight 5 due out in 2011 promises to have Silverlight integration of Photosynthe controls as well as 3D.

    Instead of a store directory the UAA example includes a simple room finder which moves to the corresponding floor and zooms to the room selected. My attempts at using PlaneProjection as a multi floor stack were thwarted by lack of control over camera position. I had hoped to show a stack of floor plans at an oblique view with an animation for the selected floor plan sliding it out of the stack and rotating to planar view. Eventually I’ll have a chance to revisit this in SL5 with its full 3D scene graph support.

    Where are we going?

    You can see where these primitive experiments are going: Move from a Bing Map to a Map of interior spaces and now simple Locators can reach into asset databases where we have information about all our stuff. The stuff in this case is not roads, addresses, and shops, but smaller stuff, human scale stuff that we use in our everyday consumer and corporate lives. Unlike rural cultures, modern western culture is much more about inside than outside. We spend many more hours inside the cube than out, so a Mirror Land where we live is certainly a plausible extension, whether we call it CAD, GIS, BIM, FM, or BINGBIM matters little.

    It’s also noteworthy that this gives vendors a chance to purchase more ad opportunities. After all, our technology is here to serve a consumer driven culture and so is Mirror Land.

    Interior spaces are a predictable part of Mirror Land and we are already seeing minor extensions. The proprietary and private nature of many interior spaces is likely to leave much out of public mapping. However, retail incentives will be a driving force extending ad opportunities into personal scale mapping. Eventually Mobile will close the loop on interior retail space, providing both consumer location as well as local asset views. Add some mobile camera apps, and augmented reality will combine product databases, individualized coupon links, nutritional content, etc to the shelf in front of you.

    On the enterprise side, behind locked BIM doors, Silverlight with its rich authentication framework, but more limited mobile reach, will play a part in proprietary asset management which is a big part of FM, BM, BIM ….. Location of assets is a major part of the drive to efficiency and covers a lot of ground from inventory, to medical equipment, to people.

    Summary:

    This small exercise will likely irk true NBIMSers who will not see much “real” BIM in a few floor plans. So I hasten to add this disclaimer, I’m not really a Web BIMer or even a Bing BIMer, but I am looking forward to the extension of Mirror Land to the interior spaces I generally occupy.

    Whether GIS analysis reaches into web mapped interiors is an open question. I’m old enough to remember when there were “CAD Maps” and “Real GIS”, and then “Web Maps” and “Real GIS.” Although GIS (real, virtual, or otherwise) is slowly reaching deeper into Mirror Land, we are still a long way from NBIMS sanctioned “Real” WebBIM with GIS analysis. But then that means it’s still fun, right?

    GPS Tracking with a Twitter Twist

    It has been awhile since I had time to post here. Ive been busy on a whole raft of POCs using Silverlight Map Control. Silverlight seems to be a popular way to add map services to enterprise applications, at least it is generating a lot of POC traffic. Most are impressed with its flexibility. Today I wanted to take a couple of minutes and share a peek at a fun project I was working on in between real work. Its a tracking application with a Twitter twist. The vehicle tracking is tied into a Tweet community along a vehicle route.

    DotNet Rocks is an online radio type show that does interviews with technical folks. They are primarily interested in .NET topics so you won’t find a lot of open source, or even GIS for that matter. Along with the recent release of Visual Studio 2010 and Silverlight 4, Richard Campbell and Carl Franklin, the hosts of the .NETRocks program, are taking a US tour in an RV. They will be hitting a dozen or so major cities around the country while doing their show on the road. They just had a stop in Phoenix last night and as I write this are heading out to Houston.

    The DNR RoadTrip project uses a SQL Server 2008 DB for maintaining the GPS history and caching Tweets. On the server we have a couple of one minute polling services that check for any new GPS positions or Tweets. There are some static FixedLocation Feeds that maintain venue locations. The GPS track feeds go into the DB directly. The Tweet feeds often have no geo tags so their datetime stamps are used to look up the GPS record closest in time. The location of that GPS position is then applied to the Tweet record location. Here is an edmx view of the DB, but notice geography data types are still not supported so the Location field is missing.

    dnr_roadtrip screen shot
    Fig 3 – .edmx view of the DB (does not show geography Location field)
    dnr_roadtrip screen shot
    Fig 4 – MSSMS Database Diagrams (shows Location geography field)

    The Tweet feeds are queried by the polling service using a #hashtag Twitter search. The hashtag for this Road Trip project, #dnr_roadtrip, lets anyone interested conversevia twitter. The first Tweet at a track location is available as a rollover. If additional Tweets come in at the same location they are made available by an additional Feed Service call:

    // Pull the ATOM feed from the Twitter API using the TweetSharp.com library.
    var search = FluentTwitter.CreateRequest()
        .AuthenticateAs("Twitter account", "Twitter key")
        .Search().Query()
        .ContainingHashTag(TwitterHashTag)  // TwitterHashTag set in the config file.
        .Since(m_SinceId)
        .AsAtom();
    

    The polling service makes use of the “since_id” as a place holder. This way only newer Tweets since the previous since_id need to be processed.

    While these polling services are running on the server, the Silverlight Bing Maps UI is communicating with the DB using a straightforward WCF basicHttpBinding. The WCFRoadShow provides the DB queries for populating the UI. There four primary Feed Service WCF calls:

    • GetCurrentLocation() – renders the most recent vehicle location
    • GetFixedLocation() – renders venue stop icons from the DB
    • GetRoute(string updatetype) – renders the GPS tracking icons
    • GetTweet()- renders the Tweet icons related to GPS records

    There is one additional Feed Service call which is used to return all Tweets at a given location. This last WCF Service call is triggered by a click event from one of the Tweet icons and is the most complex feed query. It makes use of a SQL Server 2008 geography data type proximity search. The returned Tweet Feed records are then used in a Tweet ListBox using Data Binding.

    List tweets = new List();
    XNamespace ns = "http://www.w3.org/2005/Atom";
    foreach (FeedServiceReference.FeedItem item in e.Result.FeedItems)
    {
      if (item.FeedContent != null)
      {
         // Note: namespace required
         // These are the entry xml elements from Twitter
         //   saved in RoadTrip2010 DB as a varbinary FeedItem
      MemoryStream m = new System.IO.MemoryStream(item.FeedContent);
      XDocument Xitem = XDocument.Load(m);
         foreach (var entry in Xitem.Elements(ns + "entry"))
         {
            string tweeturi = entry.Elements(ns + "link").ElementAt(0).Attribute("href").Value;
            string iconuri = entry.Elements(ns + "link").ElementAt(1).Attribute("href").Value;
            string title = entry.Element(ns + "title").Value;
            string author = entry.Element(ns + "author").Element(ns + "name").Value;
            tweets.Add(new TweetItem(tweeturi, iconuri, title, author));
          }
       }
    }
    ((ListBox)_LayoutRoot.FindName("twitterItems")).ItemsSource = tweets;
    ((Border)_LayoutRoot.FindName("twitterPanelBorder")).Visibility = Visibility.Visible;
    

    You can see from the above code snippet that the rss xml feed item for the tweets is returned and then using Binding is attached to a ListBox’s ItemTemplate DataTemplate. This is a handy way of rendering the indeterminate list length of Tweet records found at a given location. Also note from the code snippet that it was necessary to prefix the Atom namespace for accessing feed Elements and Attributes. This is because the returned Atom wrapper was discarded and only entry elements are stored as a varbinary FeedItems.

    In addition to the DB Tweet feed cache, I decided to make use of Twitters newer geotagging capabilities. This is still relatively new and not used very frequently, but there is a simple Twitter location query that I hooked to a double click map event:

    string twitterurl = "http://search.twitter.com/search.atom?geocode=";
    twitterurl += string.Format(CultureInfo.InvariantCulture, "{0,10:0.000000},{1,11:0.000000},
     1mi", loc.Latitude, loc.Longitude).Replace(" ", "");

    This takes the click map location in latitude longitude and searches for the nearest geotagged Tweets that fall within a 1mi radius.

    Adding geotag Tweet queries injects some interest because unrelated geotagged Tweets can be viewed anywhere in the world. It is surprising to me how many people are unabashed about sharing their location with the world. I was also surprised how much street capitalism occupies the lowly Twitter world. Go to any urban area and a few clicks will reveal the whole spectrum of street talk, the good, the bad, and the ugly.

    Since a similar Flickr query is available it was simple to add a geotagged Flickr photo query as well. Double clicking on the map will bring up ListBoxes for all Tweets and Flickr photos that are geotagged and fall within 1mile of the click spot.

    string flickrurl = "http://api.flickr.com/services/rest/?
    format=rest&method=flickr.photos.search&api_key={0}&safe_search=1
    &radius_units=mi&radius=20&lat={3,10:0.000000}&lon={4,11:0.000000}";

    This Flickr query returns a group of photo xml entries. Each of these have an id that can be use to obtain the actual photo using a second lookup.
    string src = “http://farm{0}.static.flickr.com/{1}/{2}_{3}.jpg”;

    In both cases the returned entries are used to populate ListBox’s ItemTemplate DataTemplate.

    Another interesting aspect of this project is the donation of a PreEmptive’s Dotfuscator. This is an injection type tool for monitoring traffic and obtaining detailed analytics about a UI. The Dotfuscator tool is easy to use, especially since PreEmptive was kind enough to provide a detailed config.xml.

    Using the tool creates a new xap file of the UI which holds event triggers for different aspects of the code. These events trigger messages back to the PreEmptive’s Runtime Intelligence Service where they are aggregated and then made available for a nice display. You can click on the PreEmptive icon to take a look at the analytics generated on this project. I was impressed with how easy it was to use and how nice it was to get analytics at a highly granular level, down to clicks on individual elements in the xaml.

    Since some interested viewers kept a viewer open I also added a timer refresh to update currentPosition, Tweets, and GPS locations:

    • refreshTimer = new DispatcherTimer();
    • refreshTimer.Interval = new TimeSpan(0, 0, 0, 0, 120000); // 120000 Milliseconds 2 min
    • refreshTimer.Tick += new EventHandler(Refresh_Tick);
    • refreshTimer.Start();

    The key control for using any of the queries for GPS and Tweets is the DateTime Slider. This is a handy control found over in the DeepEarth codeplex project.
    There are plenty of other useful controls available in the DeepEarth Bing Maps Toolkit. This DateSlider encapsulates selection of DateTime ranges across a min/max range:

    dateRange.Minimum = DateTime.Parse("4/16/2010 00:00:00", CultureInfo.InvariantCulture);
    dateRange.RangeStart = DateTime.Now.AddDays(-2);
    dateRange.RangeEnd = DateTime.Now.AddHours(1);
    dateRange.Maximum = DateTime.Parse("5/9/2010 00:00:00", CultureInfo.InvariantCulture);
    startdatetext.Text = dateRange.RangeStart.ToString();
    enddatetext.Text = dateRange.RangeEnd.ToString();
    

    In the process of deploying this project I learned about CultureInfo. Surprisingly, for a RoadTrip in the USA there were a number of folks from other countries who wanted to watch Richard and Carl roam across North America. I had to go back through the application adding CultureInfo.InvariantCulture Globalization to get things to work for the non ‘en-US’ followers. Adjusting for at least working even without fully accommodating other languages turned out to be surprisingly simple. This is one of the pluses for working in the Microsoft world.

    Another big plus is browser compatibility. A quick check with browsers I happen to have on hand verified that all of these work fine:

    • Firefox v3.0.17
    • Chrome 4.1.249.1069
    • Safari 4.0.4
    • Opera 10.51
    • IE of course

    Don’t even try this with an SVG project.

    Summary

    Twitter is fun and Twitter with maps is a GIS social network. Some claimed to find it mildly addicting. The internet world is over flowing with connectable APIs, SDKs, etc and Bing Maps Silverlight makes hooking these all together relatively statraighforward. Remember it’s really just me and a few weeks working part time to put together a pretty interesting Twitter Map. I know there are lots of more sophisticated Twitter mapping applications already floating around, but of course the attraction to Silverlight is ease of ownership.

    Map Clipping with Silverlight



    Fig 1 – Clip Map Demo

    Bing Maps Silverlight Control has a lot of power, power that is a lot of fun to play with even when not especially practical. This weekend I was presented with a challenge to find a way to show web maps, but restricted to a subset of states, a sub-region. I think the person making the request had more in mind the ability to cut out an arbitrary region and print it for reporting. However, I began to think why be restricted to just the one level of the pyramid. With all of this map data available we should be able to provide something as primitive as coupon clipping, but with a dynamic twist.

    Silverlight affords a Clip masking mechanism and it should be simple to use.

    1. Region boundary:

    The first need is to compile the arbitrary regional boundary. This would be a challenge to code from scratch, but SQL Server spatial already has a function called “STUnion.” PostGIS has had an even more powerful Union function for some time, and Paul Ramsey has pointed out the power of fast cascaded unions. Since I’m interested in seeing how I can use SQL Serrver, though, I reverted to the first pass SQL Server approach. But, as I was looking at STUnion it was quickly apparent that this is a simple geo1.STUnion(geo2) function and what is needed is an aggregate union. The goal is to union more than just two geography elements at a time, preferably the result of an arbitrary query.

    Fortunately there is a codeplex project, SQL Server Tools, which includes the very thing needed, along with some other interesting functions. GeographyAggregateUnion is the function I need, Project/Unproject and AffineTransform:: will have to await another day. This spatial tool kit consists of a dll and a register.sql script that is used to import the functions to an existing DB. Once in place the functions can be used like this:

    SELECT dbo.GeographyUnionAggregate(wkb_geometry)) as Boundary
    FROM [Census2009].[dbo].[states]
    WHERE NAME = ‘Colorado’ OR NAME = ‘Utah’ or NAME = ‘Wyoming’

    Ignoring my confusing choice of geography column name, “wkb_geometry,” this function takes a “geography” column result and provides the spatial union:

    Or in my case:


    Fig 3 – GeographyUnionAggregate result in SQL Server

    Noting that CO, WY, and UT are fairly simple polygons but the result is 1092 nodes I tacked on a .Reduce() function.
    dbo.GeographyUnionAggregate(wkb_geometry).Reduce(10) provides 538 points
    dbo.GeographyUnionAggregate(wkb_geometry).Reduce(100) provides 94 points
    dbo.GeographyUnionAggregate(wkb_geometry).Reduce(100) provides 19 points

    Since I don’t need much resolution I went with the 19 points resulting from applying the Douglas-Peuker thinning with a tolerance factor of 1000.

    2. Adding the boundary

    The next step is adding this union boundary outlining my three states to my Silverlight Control. In Silverlight there are many ways to accomplish this, but by far the easiest is to leverage the builtin MapPolygon control and add it to a MapLayer inside the Map hierarchy:

    <m:MapLayer>
      <m:MapPolygon x:Name=”region”
      Stroke=”Blue” StrokeThickness=”5″
        Locations=”37.0003960382868,-114.05060006067 37.000669,-112.540368
        36.997997,-110.47019 36.998906,-108.954404
             .
            .
            .
        41.996568,-112.173352 41.99372,-114.041723
         37.0003960382868,-114.05060006067 “/>
    </m:MapLayer>


    Now I have a map with a regional boundary for the combined states, CO, WY, and UT.

    3. The third step is to do some clipping with the boundary:

    UIElement.Clip is available for every UIElement, however, it is restricted to Geometry clipping elements. Since MapPolygon is not a geometry it must be converted to a geometry to be used as a clip element. Furthermore PathGeometry is very different from something as straight forward as MapPolygon, whose shape is defined by a simple LocationCollection of points.

    PathGeometry in XAML:


    <Canvas.Clip>
      <PathGeometry>
        <PathFigureCollection>
          <PathFigure StartPoint=”1,1″>
            <PathSegmentCollection>
              <LineSegment Point=”1,2″/>
              <LineSegment Point=”2,2″/>
              <LineSegment Point=”2,1″/>
              <LineSegment Point=”1,1″/>
            </PathSegmentCollection>
          </PathFigure>
        </PathFigureCollection>
      </PathGeometry>
    </Canvas.Clip>


    The easiest thing then is to take the region MapPolygon boundary and generate the necessary Clip PathGeometry in code behind:

      private void DrawClipFigure()
      {
        if (!(MainMap.Clip == null))
        {
         &nbspMainMap.ClearValue(Map.ClipProperty);
        }
        PathFigure clipPathFigure = new PathFigure();
        LocationCollection locs = region.Locations;
        PathSegmentCollection clipPathSegmentCollection = new PathSegmentCollection();
        bool start = true;
        foreach (Location loc in locs)
        {
          Point p = MainMap.LocationToViewportPoint(loc);
          if (start)
          {
           clipPathFigure.StartPoint = p;
           start = false;
         }
         else
         {
          LineSegment clipLineSegment = new LineSegment();
          clipLineSegment.Point = p;
          clipPathSegmentCollection.Add(clipLineSegment);
         }
        }
        clipPathFigure.Segments = clipPathSegmentCollection;
        PathFigureCollection clipPathFigureCollection = new PathFigureCollection();
        clipPathFigureCollection.Add(clipPathFigure);

        PathGeometry clipPathGeometry = new PathGeometry();
        clipPathGeometry.Figures = clipPathFigureCollection;
        MainMap.Clip = clipPathGeometry;
      }

    This Clip PathGeometry can be applied to the m:Map named MainMap to mask the underlying Map. This is easily done with a Button Click event. But when navigating with pan and zoom, the clip PathGeometry is not automatically updated. It can be redrawn with each ViewChangeEnd:
    private void MainMap_ViewChangeEnd(object sender, MapEventArgs e)
    {
      if (MainMap != null)
      {
        if ((bool)ShowBoundary.IsChecked) DrawBoundary();
        if ((bool)ClipBoundary.IsChecked) DrawClipFigure();
      }
    }


    This will change the clip to match a new position, but only after the fact. The better way is to add the redraw clip to the ViewChangeOnFrame:

    MainMap.ViewChangeOnFrame += new EventHandler<MapEventArgs>(MainMap_ViewChangeOnFrame);

    private void MainMap_ViewChangeOnFrame(object sender, MapEventArgs e)
    {
      if (MainMap != null)
      {
        if ((bool)ShowBoundary.IsChecked) DrawBoundary();
        if ((bool)ClipBoundary.IsChecked) DrawClipFigure();
      }
    }


    In spite of the constant clip redraw with each frame of the navigation animation, navigation is smooth and not appreciably degraded.

    Summary:

    Clipping a map is not terrifically useful, but it is supported with Silverlight Control and provides another tool in the webapp mapping arsenal. What is very useful, are the additional functions found in SQL Server Tools. Since SQL Server spatial is in the very first stage of its life, several useful functions are not found natively in this release. It is nice to have a superset of tools like GeographyAggregateUnion, Project/Unproject,
    and AffineTransform::.

    The more generalized approach would be to allow a user to click on the states he wishes to include in a region, and then have a SQL Server query produce the boundary for the clip action from the resulting state set. This wouldn’t be a difficult extension. If anyone thinks it would be useful, pass me an email and I’ll try a click select option.



    Fig 4 – Clip Map Demo