Alice in Mirrorland – Silverlight 5 Beta and XNA

“In another moment Alice was through the glass, and had jumped lightly down into the Looking-glass room”

Silverlight 5 Beta was released into the wild at MIX 11 a couple of weeks ago. This is a big step for mirror land. Among many new features is the long anticipated 3D capability. Silverlight 5 took the XNA route to 3D instead of the WPF 3D XAML route. XNA is closer to the GPU with the time tested graphics rendering pipeline familiar to Direct3D/OpenGL developers, but not so familiar to XAML developers.

The older WPF 3D XAML aligns better with X3D, the ISO sanctioned XML 3D graphics standard, while XNA aligns with the competing WebGL javascript wrapper for OpenGL. Eventually XML 3D representations also boil down to a rendering pipeline, but the core difference is that XNA is immediate mode while XML 3D is kind of stuck with retained mode. Although you pick up recursive control rendering with XML 3D, you lose out when it comes to moving through a scene in the usual avatar game sense.

From a Silverlight XAML perspective, mirror land is largely a static machine with infrequent events triggered by users. In between events, the machine is silent. XAML’s retained mode graphics lacks a sense of time’s flow. In contrast, enter XNA through Alice’s DrawingSurface, and the machine whirs on and on. Users occasionally throw events into the machine and off it goes in a new direction, but there is no stopping. Frames are clicking by apace.

Thus time enters mirror land in frames per second. Admittedly this is crude relative to our world. Time is measured out in the proximate range of 1/20th to 1/60th a second per frame. Nothing like the cusp of the moment here, and certainly no need for the nuance of Dedekind’s cut. Time may be chunky in mirror land, but with immediate mode XNA it does move, clicking through the present moment one frame at a time.

Once Silverlight 5 is released there will be a continuous XNA API across Microsoft’s entire spectrum: Windows 7 desktops, Windows 7 phones, XBox game consoles, and now the browser. Silverlight 5 and WP7 implementations are a subset of the full XNA game framework available to desktop and XBox developers. Both SL5 and WP7 will soon have merged Silverlight XNA capabilities. For symmetry sake XBox should have Silverlight as apparently announced here. It would be nice for a web browsing XBox TV console.

WP7 developers will need to wait until the future WP7 Mango release before merging XNA and Silverlight into a single app. It’s currently an either/or proposition for the mobile branch of XNA/SL.

At any rate, with SL5 Beta, Silverlight and 3D XNA now coexist. The border lies at the <DrawingSurface> element:

<DrawingSurface Draw="OnDraw" SizeChanged="DrawingSurface_SizeChanged" />

North of the border lies XML and recursive hierarchies, a largely language world populated with “semantics” and “ontologies.” South of the border lies a lush XNA jungle with drums throbbing in the night. Yes, there are tropical white sands by an azure sea, but the heart of darkness presses in on the mind.

XAML touches the academic world. XNA intersects Hollywood. It strikes me as one of those outmoded Freudian landscapes so popular in the 50’s, the raw power of XNA boiling beneath XAML’s super-ego. I might also note there are bugs in paradise, but after all this is beta.

Merging these two worlds causes a bit of schizophrenia. Above is Silverlight XAML with the beauty of recursive hierarchies and below is all XNA with its rendering pipeline plumbing. Alice steps into the DrawingSurface confronting a very different world indeed. No more recursive controls beyond this border. Halt! Only immediate mode allowed. The learning curve south of the border is not insignificant, but beauty awaits.

XNA involves tessellated models, rendering pipelines, vertex shaders, pixel shaders, and a high level shading language, HLSL, accompanied by the usual linear algebra suspects. Anytime you run across register references you know this is getting closer to hardware.

…a cry that was no more than a breath: “The horror! The horror!”

sampler2D CloudSampler : register(s0);
static const float3 AmbientColor = float3(0.5f, 0.75f, 1.0f);
static const float3 LampColor = float3(1.0f, 1.0f, 1.0f);
static const float AmbientIntensity = 0.1f;
static const float DiffuseIntensity = 1.2f;
static const float SpecularIntensity = 0.05f;
static const float SpecularPower = 10.0f;
			.
			.

Here is an overview of the pipeline from Aaron Oneal’s MIX talk:

So now that we have XNA it’s time to take a spin. The best way to get started is to borrow from the experts. Aaron Oneal has been very kind to post some nice samples including a game engine called Babylon written by David Catuhe.

The Silverlight 5 beta version of Babylon uses Silverlight to set some options and SL5 DrawingSurface to host scenes. Using mouse and arrow keys allows the camera/avatar to move through the virtual environment colliding with walls etc. For those wishing to get an idea of what XNA is all about this webcafe model in Babylon is a good start.

The models are apparently produced in AutoCAD 3DS and are probably difficult to build. Perhaps 3D point clouds will someday help, but you can see the potential for navigable high risk complex facility modeling. This model has over 60,000 faces, but I can still walk through exploring the environment without any difficulty and all I’m using is an older NVidia motherboard GPU.

Apparently, SL5 XNA can make a compelling interactive museum, refinery, nuclear facility, or WalMart browser. This is not a stitched pano or photosynth interior, but a full blown 3D model.

You’ve gotta love that late afternoon shadow affect. Notice the camera is evidently held by a vampire. I checked carefully and it casts no shadow!

But what about mapping?

From a mapping perspective the fun begins with this solar wind sample. It features all the necessary models, and shaders for earth, complete with terrain, multi altitude atmosphere clouds, and lighting. It also has examples of basic mouse and arrow key camera control.

Solar Wind Globe
Fig 4 – Solar Wind SL5 XNA sample

This is my starting point. Solar Wind illustrates generating a tessellated sphere model with applied textures for various layers. It even illustrates the use of a normal (bump) map for 3D effects on the surface without needing a tessellated surface terrain model. Especially interesting is the use of bump maps to show a population density image as 3D.

My simple project is to extend this solar wind sample slightly by adding layers from NASA Neo. NASA Neo conveniently publishes 45 categories and 129 layers of a variety of global data collected on a regular basis. The first task is to read the Neo GetCapabilities XML and produce the TreeView control to manage such a wealth of data. The TreeView control comes from the Silverlight Toolkit project. Populating this is a matter of reading through the Layer elements of the returned XML and adding layers to a collection which is then bound to the tree view’s ItemsSource property.

    private void CreateCapabilities111(XDocument document)
    {
        //WMS 1.1.1
        XElement GetMap = document.Element("WMT_MS_Capabilities").Element("Capability")
            .Element("Request").Element("GetMap").Element("DCPType")
            .Element("HTTP").Element("Get").Element("OnlineResource");
        XNamespace xlink = "http://www.w3.org/1999/xlink";
        getMapUrl = GetMap.Attribute(xlink + "href").Value;
        if (getMapUrl.IndexOf("?") != -1) getMapUrl =
                  getMapUrl.Substring(0, getMapUrl.IndexOf("?"));

        ObservableCollection layers = new ObservableCollection();
        foreach (XElement element in
document.Element("WMT_MS_Capabilities").Element("Capability")
                .Element("Layer").Descendants("Layer"))
        {
            if (element.Descendants("Layer").Count() > 0)
            {
                WMSLayer lyr0 = new WMSLayer();
                lyr0.Title = (string)element.Element("Title");
                lyr0.Name = "header";
                foreach (XElement element1 in element.Descendants("Layer"))
                {
                    WMSLayer lyr1 = new WMSLayer();
                    lyr1.Title = (string)element1.Element("Title");
                    lyr1.Name = (string)element1.Element("Name");
                    lyr0.sublayers.Add(lyr1);
                }

                layers.Add(lyr0);
            }
        }
        LayerTree.ItemsSource = layers;
    }

Once the tree is populated, OnSelectedItemChanged events provide the trigger for a GetMap request to NASA Neo returning a new png image. I wrote a proxy WCF service to grab the image and then write it to png even if the source is jpeg. It’s nice to have an alpha channel for some types of visualization.

The difficulty for an XNA novice like myself is understanding the hlsl files and coming to terms with the rendering pipeline. Changing the source image for a Texture2D shader requires dropping the whole model, changing the image source, and finally reloading the scene model and pipeline once again. It sounds like an expensive operation but surprisingly this re-instantiation seems to take less time than receiving the GetMap request from the WMS service. In WPF it was always interesting to put a Video element over the scene model, but I doubt that will work here in XNA.

The result is often a beautiful rendering of the earth displaying real satellite data at a global level.

Some project extensions:

  • I need to revisit lighting which resides in the cloud shader hlsl. Since the original cloud model is not real cloud coverage, it is usually not an asset to NASA Neo data. I will need to replace the cloud pixel image with something benign to take advantage of the proper lighting setup for daytime.
  • Next on the list is exploring collision. WPF 3D provided a convenient RayMeshGeometry3DHitTestResult. In XNA it seems getting a point on the earth to trigger a location event requires some manner of collision or Ray.Intersects(Plane). If that can be worked out the logical next step is grabbing DEM data from USGS for generating ground level terrain models.
  • There is a lot of public LiDAR data out there as well. Thanks to companies like QCoherent, some of it is available as WMS/WFS. So next on the agenda is moving 3D LiDAR online.
  • The bump map approach to displaying variable geographic density as relief is a useful concept. There ought to be lots of global epidemiology data that can be transformed to a color density map for display as a relief bump map.

Lots of ideas, little time or money, but Silverlight 5 will make possible a lot of very interesting web apps.

Helpful links:
Silverlight 5 Beta: http://www.silverlight.net/getstarted/silverlight-5-beta/
Runtime: http://go.microsoft.com/fwlink/?LinkId=213904

Silverlight 5 features:
http://i1.silverlight.net/content/downloads/silverlight_5_beta_features.pdf?cdn_id=1
“Silverlight 5 now has built-in XNA 3D graphics API”

XNA: http://msdn.microsoft.com/en-us/aa937791.aspx
Overview

NASA Neo: http://localhost/NASA-Neo/publish.htm

Babylon Scenes: Michel Rousseau, courtesy of Bewise.fr

Babylon Engine: David Catuhe / Microsoft France / DPE

Summary:

“I am real!” said Alice, and began to cry.

“You won’t make yourself a bit realler by crying,” Tweedledee remarked: “there’s nothing to cry about.”

“If I wasn’t real,” Alice said – half-laughing through her tears, it all seemed so ridiculous – “I shouldn’t be able to cry.”

Silverlight WMS Viewer


Silverlight MapControl OWS Viewer
Fig 1 – Silverlight MapControl CTP used as a Simple WMS Viewer

In the past couple of blogs I’ve looked at using SQL Server and PostGIS overlays on top of the Silverlight VE MapControl CTP. I’ve also looked at using PostGIS mediated by Geoserver as cached tile sources with GeoWebCache. As these investigations have shown, map layers can be provided using basic shape overlays directly from spatial database tables or as image tile sources such as GeoWebCache.

There is another possibility: using an Image element with its Source pointed at a WMS service. Although this approach is not as interactive as vector shape overlays or as efficient as a tile source, it does have one big advantage. It can be used to access public WMS services. Currently the most common OWS services in the public realm are WMS. WFS has never really seemed to take off as an OWS service and WCS is still a bit early to tell. Using a public WMS means conforming to the service as exposed by someone else. Many, probably most, WMS services do not have a tile caching capability, which means access to a public WMS in Silverlight MapControl requires a Silverlight Image element like this:

     <Image x:Name="wmsImage" Source="the WMS Source"></Image>

where ‘the WMS Source’ looks similar to this:
    http://localhost/geoserver/wms?SERVICE=WMS&Version=1.1.1
    &REQUEST=GetMap&LAYERS=topp:states&STYLES=population
    &TRANSPARENT=TRUE&SRS=EPSG:4326
    &BBOX=-84.2103683719356, 24.8162723396119,
      -78.1019699344356, 28.2556356649882
    &FORMAT=image/png&EXCEPTIONS=text/xml&WIDTH=1112&HEIGHT=700

If we use a Silverlight Image element to display a WMS GetMap request we have a basic WMS Viewer. However, there are some hurdles to overcome.

WMS servers provide several url encoded requests, GetCapabilities, GetMap, and for layers that are queryable GetFeatureInfo. The workflow is generally to look at a GetCapabilities result to discover the exposed layers and information about their extent, style, and query capabilities. In addition there is usually information about supported formats, coordinates systems, and possibly even scale ranges. After examining the GetCapabilities result, the next step is configuring a GetMap request, which is then placed in the Silverlight xaml hierarchy as an Image element. Once that is accomplished an optional third step is to use the map layer for GetFeatureInfo requests to look at attribute data for a selected feature. There is another request, DescribeLayer, which is not needed for this project.

GetCapabilities

The first hurdle in creating a Silverlight WMS viewer is accessing the XML returned from a GetCapabilities request and processing it into a useable Silverlight control. First there is a cross domain issue since our Silverlight viewer wants to show WMS map views from a different server. As in previous projects this is done by providing a WCF proxy service on the Silverlight server to provide access to different WMS services at the server rather than incurring the wrath of a client cross domain access.

Here is an example of a client initialization:
    svc = GetServiceClient();
    svc.GetCapabilitiesCompleted += svc_GetCapabilitiesCompleted;
    svc.GetCapabilitiesAsync(“http://localhost/geoserver/wms?
    Service=WMS&Version=1.1.1&Request=GetCapabilities”);

and here is WMSService.svc GetCapabilities, which simply takes a GetCapabilities url encoded query and feeds back out to the svc_GetCapabilitiesCompleted the result as an XML string.

[OperationContract]
public string GetCapabilities(string url)
{
  HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
  using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)
  {
    StreamReader reader = new StreamReader(response.GetResponseStream());
    return (reader.ReadToEnd());
  }
}

On the xaml client side we then pick up the resulting xml using XLinq to Parse into an XDocument.

private void svc_GetCapabilitiesCompleted(object sender, GetCapabilitiesCompletedEventArgs e)
{
  if (e.Error == null)
  {
    layers.Children.Clear();
          .
          .  
    XDocument document = XDocument.Parse(e.Result, LoadOptions.None);
    foreach (XElement element in document.Element
("WMT_MS_Capabilities").Element("Capability").Element("Layer").Descendants("Layer"))
    {
            .
            .
    }
  }
}

XLinq is very easy to use for manipulating XML, much easier in C# than using DOM commands. The intent of XLinq XML is similar to Java JDOM in providing an access syntax more appropriate to a language like C#. For example, document.Element(“WMT_MS_Capabilities”).Element(“Capability”).Element(“Layer”).Descendants(“Layer”), provides a collection of all the layer elements descending from the head layer in our GetCapabilities result. Using the layer sub elements we can build a simple selection menu from CheckBox and ComboBox elements. The main sub elements of interest are:

<Name>topp:states</Name>

<SRS>EPSG:4326</SRS>

<LatLonBoundingBox minx=”-124.731422″ miny=”24.955967″ maxx=”-66.969849″ maxy=”49.371735″/>

<Style>
    .
    .
</Style>

Here is an example of the simplicity of XLinq’s query capability:

  XElement layer = document.Element("WMT_MS_Capabilities").Element("Capability").Element("Layer");
  var testsrs = from srs in layer.Elements("SRS") select (string)srs;
  if (testsrs.Contains("EPSG:900913")) EPSG900913.IsEnabled = true;
  else EPSG900913.IsEnabled = false;

XLinq is a pleasure to use combining aspects of JDOM, JAXB, and SQL, and making life easier for developers.

The final selection menu looks like this:

Silverlight MapControl Selection menu
Fig 1 – Silverlight Layer Selection Menu

An opacity slider is easy to add and lets users see through opaque layers. Not all WMS services support ‘Transparent=True’ parameters.

private void Slider_ValueChanged(object sender, RoutedPropertyChangedEventArgs e)
{
    OpacityText.Text = string.Format("{0:F2}", e.NewValue);

    foreach (UIElement item in layers.Children)
    {
        if (item is CheckBox)
        {
            if ((bool)(item as CheckBox).IsChecked)
            {
                MapLayer lyr = VEMap.FindName((item as CheckBox).Content.ToString()) as MapLayer;
                lyr.Opacity = e.NewValue;
            }
        }
    }
}

GetMap

Now come a few more hurdles for GetMap requests. A layer selection or view change triggers a call to SetImage(MapLayer lyr, string style)

  private void SetImage(MapLayer lyr, string style)
  {
    if (lyr.Children.Count > 0) lyr.Children.Clear();// clear previous layer
    int aw = (int)VEMap.ActualWidth;
    int ah = (int)VEMap.ActualHeight;
    LocationRect bounds = VEMap.GetBoundingRectangle();
    Point sw = new Point(bounds.Southwest.Longitude, bounds.Southwest.Latitude);
    Point ne = new Point(bounds.Northeast.Longitude, bounds.Northeast.Latitude);
    string wmsurl = wmsServer + "?SERVICE=WMS&Version=1.1.1&REQUEST=GetMap&LAYERS=" +
    lyr.Name + "&STYLES=" + style + "&TRANSPARENT=TRUE&SRS=EPSG:4326&BBOX=" +
    sw.X + "," + sw.Y + "," + ne.X + "," + ne.Y +
    "&FORMAT=image/png&EXCEPTIONS=text/xml&WIDTH=" + aw + "&HEIGHT=" + ah;
    Image image = new Image();
    image.Source = new BitmapImage(new Uri(wmsurl, UriKind.RelativeOrAbsolute));
    lyr.AddChild(image, new Location(bounds.North, bounds.West));
    image.MouseLeftButtonDown += Image_MouseClick;
  }

Quite simple, however, there is a problem with this, which can be seen below.

Coordinate system problem
Fig 3 – Coordinate system problem

EPSG:4326 is versatile, since it’s supported by every WMS service I’ve run across, but this geographic coordinate projection system does not match Virtual Earth base maps, or Google, Yahoo, and OSM for that matter. All of the web mapping services use a tiled spherical mercator projection. Fortunately they all use the same one. At this point there is a relatively new official EPSG designated 3785. And, here is an interesting discussion including some commentary on EPSG politics: EPSG and 3785.

However, for awhile now Geoserver has included a pseudo EPSG number, “900913,” to do the same thing and it is already incorporated into my PostGIS spatial_ref_sys tables and the Geoserver version I am using. Unfortunately MapServer chose a different pseudo EPSG, “54004,” which illustrates the problem of an existing standards body getting miffed and dragging their feet. <SRS>EPSG:900913</SRS>, <SRS>EPSG:54004</SRS>, and EPSG:3785 are similar mathematically speaking, but it will take a few iterations of existing software before EPSG:3785 is distributed widely. In the meantime this is the definition distributed with Geoserver:

    900913=PROJCS["WGS84 / Simple Mercator", GEOGCS["WGS 84",
         DATUM["WGS_1984", SPHEROID["WGS_1984", 6378137.0, 298.257223563]],
         PRIMEM["Greenwich", 0.0], UNIT["degree", 0.017453292519943295],
         AXIS["Longitude", EAST], AXIS["Latitude", NORTH]],
         PROJECTION["Mercator_1SP_Google"],
         PARAMETER["latitude_of_origin", 0.0], PARAMETER["central_meridian", 0.0],
         PARAMETER["scale_factor", 1.0], PARAMETER["false_easting", 0.0],
         PARAMETER["false_northing", 0.0], UNIT["m", 1.0], AXIS["x", EAST],
         AXIS["y", NORTH], AUTHORITY["EPSG","900913"]]

In order to make a GetMap Image match a spherical Mercator we make a couple of modifications. First change “EPSG:4326″ to “EPSG:900913″ or “EPSG:54004.” Then provide a conversion like this for the BBox parameters (more details -Mercator projection)

  private Point Mercator(double lon, double lat)
  {
    /* spherical mercator for Google, VE, Yahoo etc
     * epsg:900913 R= 6378137
     * x = long
     * y= R*ln(tan(pi/4 +lat/2)
     */
    double x = 6378137.0 * Math.PI / 180 * lon;
    double y = 6378137.0 * Math.Log(Math.Tan(Math.PI/180*(45+lat/2.0)));
    return new Point(x,y);
  }

Note that VEMap.GetBoundingRectangle(); guarantees a bounds that will never reach outside the 85,-85 latitude limits. (more precisely -85.05112877980659, 85.05112877980659) Using the above to calculate a BBox in spherical coordinates results in:

  sw = Mercator(bounds.Southwest.Longitude, bounds.Southwest.Latitude);
  ne = Mercator(bounds.Northeast.Longitude, bounds.Northeast.Latitude);

This makes the necessary correction, but not all WMS services publish support for EPSG:900913, EPSG:54004, or EPSG:3785. How to work around this dilemma? Unfortunately I have not yet found a solution to this problem, short of doing an image warp with each view change. Since some WMS services do not support any of the spherical Mercator projections, I’m left with a default EPSG:4326 that will not overlay correctly until zoomed in beyond ZoomLevel 7.

higher zoom
Fig 4 – Coordinate system problem disappears at higher zoom

One more hurdle in GetMap is the problem with a dateline wrap or antimeridian spanning. This occurs when a view box spans the longitude -180/180 break. After checking around a bit, the best solution I could come up with is splitting the image overlay into west and east parts.
if (bounds.West > bounds.East) I know I will need more than one tile:

  LocationRect boundsE = new LocationRect(bounds.North, bounds.West, bounds.South, 180.0);
  LocationRect boundsW = new LocationRect(bounds.North, -180.0, bounds.South, bounds.East);

Although this works for a 360 degree viewport, zoom levels > 2.5 are actually greater than 360 degrees and I need to work out some sort of Modular Arithmetic for more than two parts. ZoomLevel 1 will require 4 separate tiles. My problem currently is that detecting the actual number of degrees in a viewport is problematic. VEMap.GetBoundingRectangle() will return East and West Extents but this will not indicate width in actual degrees for a clock arithmetic display that repeats itself. For now I’m taking the simplest expedient of turning off the overlay for ZoomLevels less than 2.5.

Multiple earths at ZoomLevel 1
Fig 5 – Virtual Earth ZoomLevel 1 with viewport > 360deg

GetFeatureInfo

GetFeatureInfo requests are used to send a position back to the server to look up a list of table attributes for feature(s) intersecting that position. Not all WMS layers expose GetFeatureInfo as indicated in the GetCapabilities <Layer queryable=”0″>.

The basic approach is to handle a click event which then builds the necessary WMS GetFeatureInfo Request. There is a mild complication since UIElements do not possess MouseClick events. Substituting a MouseLeftButtonDown event is a work around to this lack: image.MouseLeftButtonDown += Image_MouseClick; There are other possibilities such as using SharpGIS.MouseExtensions. However, the Silverlight MapControl seems to mask these extensions. Until I can work out that issue I defaulted to the simple MouseLeftButtonDown event.

   private void Image_MouseClick(object sender, MouseButtonEventArgs e)
  {
      Image img = sender as Image;
      Point pt = e.GetPosition(img);
      var location = VEMap.ViewportPointToLocation(e.GetPosition(VEMap));
      BitmapImage bi = img.Source as BitmapImage;
      string info = bi.UriSource.AbsoluteUri.ToString();
      int beg = info.IndexOf("LAYERS=")+7;
      int end = info.IndexOf('&',beg);
      string layer = info.Substring(beg, end-beg);
      info = info.Replace("GetMap", "GetFeatureInfo");
      info += "&QUERY_LAYERS=" + layer + "&INFO_FORMAT=application/vnd.ogc.gml
      &X=" + pt.X + "&Y=" + pt.Y;
      svc.GetFeatureInfoAsync(info);

      FeatureInfo.Children.Clear();
      InfoPanel.Visibility = Visibility.Collapsed;
      InfoPanel.SetValue(MapLayer.MapPositionProperty, location);
      MapLayer.SetMapPositionOffset(InfoPanel, new Point(10, -150));
  }

By attaching the event to each image element, I’m able to make use of bi.UriSource, just modifying the request parameter and adding the additional GetFeatureInfo parameters. There are some variations in the types of Info Formats exposed by different WMSs. I selected the gml which is supported by GeoServer and text/xml for use with USGS National Atlas.

private void svc_GetFeatureInfoCompleted(object sender, GetFeatureInfoCompletedEventArgs e)
{
  if (e.Error == null)
  {
    XDocument document = XDocument.Parse(e.Result, LoadOptions.None);
    XNamespace wfs = "http://www.opengis.net/wfs";
    XNamespace gml = "http://www.opengis.net/gml";
    XNamespace geo = "http://www.web-demographics.com/geo";
    if (document.Element("ServiceExceptionReport") == null)
    {
      if (document.Element("FeatureInfoResponse") == null)
      {
        XElement feature = document.Element(wfs + "FeatureCollection").Element(gml + "featureMember");
        if (feature != null)
        {
          XElement layer = (XElement)feature.FirstNode;
          if (layer != null)
          {
            TextBlock tb = new TextBlock();
            tb.Foreground = new SolidColorBrush(Colors.White);
            tb.Text = layer.Name.ToString().Substring(layer.Name.ToString().IndexOf('}') + 1);
            FeatureInfo.Children.Add(tb);
            InfoPanel.Visibility = Visibility.Visible;
            IEnumerable de = layer.Descendants();
            foreach (XElement element in de)
            {
              string name = element.Name.ToString();
              name = name.Substring(name.IndexOf('}') + 1);
              XNamespace ns = element.Name.Namespace;
              string prefix = element.GetPrefixOfNamespace(ns);
              if (!prefix.Equals("gml"))
              {
                tb = new TextBlock();
                tb.Foreground = new SolidColorBrush(Colors.White);
                tb.Text = name + ": " + element.Value.ToString();
                FeatureInfo.Children.Add(tb);
              }
            }
          }
        }
      }
    }
  }
}

Summary

There are some minor problems using Silverlight VE MapControl as a generic OWS Viewer.

  1. Lack of general support for EPSG:3785 in many public WMS servers
  2. Detecting viewport extents for ZoomLevels displaying more than 360 degrees
  3. Missing MouseClick and double click events for UIElements

None of these are show stoppers and given a couple more days I’d probably work out solutions for 2 and 3. The biggest hurdle is lack of support for spherical Mercator coordinate systems. As long as you have control of the WMS service this can be addressed, but it will be some time before EPSG:3785 propagates into all the WMS services out in the wild.

On the plus side, Silverlight’s sophistication makes adding nice design and interaction features relatively easy. Again having C# code behind makes life a whole lot easier than dusting off javascript. I’m also really liking XLinq for manipulating small xml sources.

Next on the project list will be adding some nicer animation to the information and menu panels. Further in the future will be adding WFS view capability.


Silverlight MapControl OWS Viewer
Fig 1 – Silverlight MapControl CTP WMS Viewer showing USGS Atlas

X3D and Virtual Scene Graphs

x3d earth
Fig 1 – XJ3D Viewer showing an x3d earth model.

Virtual Worlds and Scene Graphs

Well I confess, I’m no gamer! I’ve never handled a Wii or blown away monsters on a PS3. It doesn’t hold my interest that much. Guitar Hero makes me feel slightly nauseous. (When will they introduce Symphony Conductor for the Wii?) I suppose I’m an anachronism of the old paperback book era. My kids groan, but I can still get excited about the E.E. Smith Lensman series. However, I’m a bit chagrined to admit that I was oblivious to X3D until this past week. It’s just that gaming and GIS have not overlapped in my world yet.

X3D has been around quite awhile. Some of the specs date back as far as 2003 and then there is the VRML97 ancestory so this is not new stuff. It reminds me a bit of the early SVG days, circa 2000. The novelty of event driven graphics in the browser was overpowering then, but it wasn’t until years later that Flash and Silverlight actually made xml graphics generally available to web app interfaces, almost ten years later. Now of course, SVG even has native support in most of the non microsoft browsers. I can imagine in the year 2018 that 3D graphics XML will be ubiquitous, but it will probably be some derivative of ChromeFlash, XNA, or WPF, while X3D may still languish in the outer reaches of civilization.

X3D has a similar feel in that it’s a powerful anticipating specification with beautiful XML scene graph capability, but only a few implementations far from the mainstream. I won’t say I’m a sucker for XML, but I really, really like to look at a file and understand what it says. Nothing beats a good old text editor for playing with an XML graphics file. Sorry, my text bias is showing again.

The Java3D API is a parallel enabling technology. The scene graph approach implemented in Java3D’s api enabled implementations of free X3D viewers. There are several viewers, but the one that actually worked on the geospatial component of X3D was Xj3D.

What is X3D?

Directly from the x3d site:

“X3D is a scalable and open software standard for defining and communicating real-time, interactive 3D content for visual effects and behavioral modeling. It can be used across hardware devices and in a broad range of applications including CAD, visual simulation, medical visualization, GIS, entertainment, educational, and multimedia presentations.”

“X3D provides both the XML-encoding and the Scene Authoring Interface (SAI) to enable both web and non-web applications to incorporate real-time 3D data, presentations and controls into non-3D content.”

and here is an example of what it looks like:
  
   <?xml version=”1.0″ encoding=”UTF-8″?>
   <!DOCTYPE X3D PUBLIC “ISO//Web3D//DTD X3D 3.2//EN”
   “http://www.web3d.org/specifications/x3d-3.2.dtd”>
   <X3D
   profile=”Immersive”
   version=”3.2″
   xmlns:xsd=”http://www.w3.org/2001/XMLSchema-instance”
   xsd:noNamespaceSchemaLocation=”http://www.web3d.org/specifications/x3d-3.2.xsd”>
      <head>
         <component name=”Geospatial” level=”1″/>
      </head>
      <Scene>
         <Group>
            <PointLight location=”10000000000 0 0″ radius=”100000000000000″/>
            <PointLight location=”-10000000000 0 0″ radius=”100000000000000″/>
            <PointLight location=”0 10000000000 0″ radius=”100000000000000″/>
            <PointLight location=”0 -10000000000 0″ radius=”100000000000000″/>
            <PointLight location=”0 0 10000000000″ radius=”100000000000000″/>
            <PointLight location=”0 0 -10000000000″ radius=”100000000000000″/>
  
            <GeoViewpoint
             position=”39.755092 -104.988123 60000″
             orientation=”1.0 0.0 0.0 -1.57″
             description=”Denver”
             headlight=”true”
             fieldOfView=”1.57″
             geoSystem=’”GDC”‘>
             <GeoOrigin DEF=”ORIGIN”
             geoCoords=”0.0 0.0 -6378137″
             geoSystem=”GDC” />
            </GeoViewpoint>
            <Inline url=”tiles/0/tile_0.x3d”/>
         </Group>
      </Scene>
   </X3D>

After the normal XML declaration follows the root element:
   <X3D
   profile=”Immersive”
   version=”3.2″
   xmlns:xsd=”http://www.w3.org/2001/XMLSchema-instance”
   xsd:noNamespaceSchemaLocation=”http://www.web3d.org/specifications/x3d-3.2.xsd”>

In this case using version 3.2 of the X3D spec and a profile, “Immersive”. Profiles are used to break the X3D spec into smaller implementation areas. Consequently, with less effort, an implementor can meet a profile of the X3D spec suitable to the need instead of the entire “Full” specification.

X3D Profiles
Fig 2 – X3D Profiles

Curiously, the X3D Full profile lists the geospatial component, but the Immersive profile seemed to implement the geospatial elements of interest in my particular case. Perhaps geospatial was later moved to the Full profile. Here is a list of the xml node elments found in the geospatial component:

  • GeoCoordinate
  • GeoElevationGrid
  • GeoLocation
  • GeoLOD
  • GeoMetadata
  • GeoOrigin
  • GeoPositionInterpolator
  • GeoProximitySensor
  • GeoTouchSensor
  • GeoTransform
  • GeoViewpoint

You can see a ‘GeoViewpoint’ element and a ‘GeoOrigin’ element in the small x3d listing above. The Immersive/Full profile is meant to imply a virtual world immersion experience. This means that a scene graph populates a world with context including lighting and a viewpoint that navigates through the virtual world space. Now this is of interest to a GIS developer in the sense that the virtual world space can mimic a real world space.

X3D Earth provides some nice examples of using X3D with earth data. Not all of the examples are in working order using the latest build of XJ3D Browser. These are older samples and may not reflect the current configuration of the X3D specification.

Another example of a virtual world space mimicing the actual world is Google’s Earth API. You can use a flying viewpoint to navigate through a model of the Rocky Mountains and get something like the experience of a bird or fighter jet. The mimicry is still coarse but the effect is recognizable. This is the horizon end point of mapping. The map abstraction eventually disappears into a one to one mathematical mapping with some subtle abstraction available as overlays and heads up displays.

This is where the online mapping world eventually merges with the gaming world. Only for mapping, the imaginary part of the world is de-emphasized in favor of an immersive real world experience as an aid to navigating in external physical space. Thinking about these kinds of worlds and their overlaps has proven fascinating, as shown by the popularity of the Matrix movie series. Are we the Ghost in the Machine or the other way around? But that is best left to philosophy types.

Let’s move on to a couple of other useful X3D scene elements. GeoLOD geospatial Level Of Detail element is an interesting element for use in geospatial worlds. By recursively referencing deeper levels of x3d GeoTerrain tiles you have an affect similar to DeepEarth tile pyramids, but with the addition of 3D Terrain:

<?xml version=”1.0″ encoding=”UTF-8″?>
<!DOCTYPE X3D PUBLIC “ISO//Web3D//DTD X3D 3.2//EN”
“http://www.web3d.org/specifications/x3d-3.2.dtd”>
<X3D
profile=”Immersive”
version=”3.2″
xmlns:xsd=”http://www.w3.org/2001/XMLSchema-instance”
xsd:noNamespaceSchemaLocation=”http://www.web3d.org/specifications/x3d-3.2.xsd”>
   <head>
      <component name=”Geospatial” level=”1″/>
      </head>
      <Scene>
         <GeoLOD
            center=”-61.875 -168.75 0.0 ”
            range=’2507443.095376239′
            geoSystem=”GDC”
            child1Url=’../6/tile_0000100.x3d’
            child2Url=’../6/tile_0000101.x3d’
            child3Url=’../6/tile_0000102.x3d’
            child4Url=’../6/tile_0000103.x3d’
         >
         <GeoOrigin DEF=”ORIGIN”
            geoCoords=”0.0 0.0 -6378137.0 ”
            geoSystem=”GDC” />
         <Group containerField=’rootNode’>
            <Shape>
               <Appearance>
                  <Material diffuseColor=”0.3 0.4 0.3 “/>
                  <ImageTexture
                     url=”../../images/5/img_000010.png”
                     repeatS=”false”
                     repeatT=”false”/>
               </Appearance>
                     <GeoElevationGrid
                        ccw=’true’
                        colorPerVertex=’false’
                        creaseAngle=’0′
                        geoGridOrigin =”-67.5 -180.0 0.0 ”
                        yScale =’1.0′
                        geoSystem=’”GDC”‘
                        normalPerVertex=’false’
                        solid=’true’
                        xDimension=’37′
·                     xSpacing=’0.625′
                        zDimension=’37′
                        zSpacing=’0.3125′
                        height=”
                        -1618 -3574 -3728 -3707 -3814 -3663 -3455 -3089
                         -3258 -3457 -3522 -3661 -3519 -3757 -3412 -3379
                         -3674 -3610 -3503 -3679 -3521 -3537 -3397 -3790
                         -3795 -3891 -3883 -3864
                                             .
                                             .
                         -5086 -5138 -5089 -5229 -4965 -5021 -5064 -5099
                         -5062 -4952 -4955 -4810 -4635 -4686 -4599 -4408
                         -4470 -5407 -4482 -4600 -4486 -3942 -4202 -4100
                         -4306 -4293 -4010 -4091
                         “>
                      <GeoOrigin USE=”ORIGIN” />
                  </GeoElevationGrid>
               </Shape>
            </Group>
         </GeoLOD>
   </Scene>
</X3D>

In the x3d snippet above you can see the GeoLOD that references down to level 6 of the terrain tile hierarchy. This is a recursive reference as each tile lower in the pyramid references the four tiles of the next level that make up the current tile, basically a quad tree approach found in most tile setups. The element attributes include a center, range, geosystem, and the ChildURL references. Inside the GeoLOD is a <Group> <Shape> element containing the current tile’s Appearance Material with a reference to the draping tile image and a GeoElevationGrid element with an array of elevations relative to the GeoOrigin.

For this to work there are two pyramids, one containing the draping images and one with the x3d GeoTerrainGrids. The GeoTerrainGrid is the same as x3d ElevationGrid except it introduces the concept of a geospatial reference system. This is all very clever and useful, but it requires the Xj3D Browser viewer to play with it. In order to author this kind of world scene you also need tools to tile both the underlying terrain models as well as the overlaid imagery.

x3d earth
Fig 3 – x3d earth model tile pyramid

Over in Microsoft I took a quick look at XNA Game Studio, which is the current development tool for XBox games. The XNA serializer supports a type of XML <XnaContent>, but there doesn’t seem to be much information about the XML XnaContent schema. I wasn’t able to find an X3D importer for XNA. Too bad, since an XNA importer would be another way to experiment with Terrain grids.

Summary:

As far as the future, it is easy to see that the WPF <MeshGeometry3D> can be used to perform the duties of a an ElevationGrid element. This means, at least in WPF, the basic building blocks are available to display and manipulate a scene in 3D. What is lacking is the concept of LOD, Level of Detail. After first seeing the DeepZoom MultiScaleImage element in action, I fell in love with the very natural navigation through its 2D space.

It is not much of a stretch to see this being extended to a “MultiScaleMeshGeometry3D” element. Again WPF appears to be a look at the future of Silverlight, so it is not impossible to imagine this type of capability in browsers by 2018. In the non-Microsoft world X3D will play a role once implementations make their way into Chrome, FireFox, and Opera. In some form or other, XML scene graphs will be in the future of GIS, and yes, we will all be pro gamers!

BMNG SilverLight 2.0 Beta Deep Zoom

Well somebody had to do it! Im willing to give it a try.

What is it?

One of the many announcements at the MIX08 conference is the availability of Deep Zoom technology for Silverlight 2.0 Beta 1. This results from an R&D program in Microsoft called Sea Dragon. Sea Dragon was evidently a Microsoft acquisition awhile back. Reminiscent of Keyhole(now google earth), Sea Dragon is a tool for smooth viewing of very large image resources. The novelty is to have it useable inside a SilverLight browser view. http://labs.live.com/Seadragon.aspx

For those so inclined to load beta stuff there is a demo available here with a nice code behind zoom pan capability: http://visitmix.com/blogs/Joshua/Hard-Rock-Cafe/

The idea is that behind the viewport is a large image in PNG, JPEG, TIFF, BMP format which is fed to the screen resolution using a MultiScaleImage element in a SilverLight 2.0 page. Actually just jpeg as it turns out.

Well of course this is extremely interesting to geospatial people since all of the WMS, WCS, WFS, KML, tile caching, image pyramids etc are aimed at exactly this functionality.

How is the user experience?

Holy Mackerel, finally a reason to use Vista! But wait this is SilverLight it should work in FireFox and Safari too. By the way,if any Sea Dragon big Tuna comes trolling by this post I already have my map hacker wishlist for 2.0 at the end of this entry.

OK, let’s try it.

First I grabbed a full resolution BMNG image off of BitTorrent: http://www.geotorrent.org/torrents/61.torrent

Chris Holmes has detailed instructions on this part here: http://docs.codehaus.org/display/GEOSDOC/Load+NASA+Blue+Marble+Data

The eventual result will be twelve large(267Mb) ecw files, “world-topo-bathy-200408-3x86400x43200.ecw,” one for each month of 2004. Ecw is a wavelet compression format for imagery http://en.wikipedia.org/wiki/ECW_(file_format), but to use the image we need it in a different format, Gdal to the rescue. The easiest approach is to take advantage of the FWtools bin directory to run a command line translation like this:

“C:\Program Files\FWTools2.1.0\bin\gdal_translate” -of GTiff world-topo-bathy-200401-3x86400x43200.ecw BMNG.tiff

After a few minutes the result is a tiff image of 86400×43200 of about 11Gb. Now it is time to use the Deep Zoom Composer (actually a decomposer) to process this into a MultiScaleImage info.bin
http://blogs.msdn.com/expression/archive/2008/03/05/download-the-preview-of-the-deep-zoom-composer.aspx

When I attempted an import of this 11Gb tiff into Deep Zoom Composer, the Mermaid.exe choked after a few minutes. I guess we aren’t ready for geospatial scale exactly yet. Note to self: do this with -o Tiff, since mermaids may not like GTiff.

So I went back to a smaller downsample to start my experiment. This time I chose a 3600×1800 png at 4.154Mb. This was rather instant success. Now there is a BMNG test pyramid on my hard drive. The pyramid is 13 levels and each subdirectory contains the necessary images in jpg. Deep Zoom Composer rather handily chunks the tiles in each level, even calculating non square tiling.


Fig 1 Example of the a pyramid level resulting from Deep Zoom Image Composer

After playing around a bit the export spits out what we need for a Silverlight MultiScaleImage. Remember this is an element type introduced with SilverLight2.0 Beta so you can’t really see this unless you want to do a beta Silverlight 2.0 install.

Here are some links on other neat things in R&D over at Microsoft labs.live: http://labs.live.com/photosynth/whatis/

SilverLight works a lot better with an IIS server, but I am using an Apache server, so I created a javascript Silverlight project. Using the default project, I modified the Scene.xaml and associated scene.js to make use of the new MultiScaleImage element:

<MultiScaleImage      
x:Name=msi“      
ViewportWidth=1.0“      
ViewportOrigin=0,0“      
Source=/BMNGZoom/BMNG/info.bin>

This worked pretty well to get the image object in the browser with a sort of spring loaded entre. Perhaps the springy annoyance can be turned off by setting UseSpringsProperty=”off.” However, adding zoom and pan are bit more problematic. I am brand new to Silverlight but oddly there seem to be very few events available:
MouseMove, MouseEnter, MouseLeave, MouseLeftButtonDown, MouseLeftButtonUp

If you want MouseRight, Keyboard, MouseWheel etc, you need to have some code behind. Since I didn’t really have time to figure out all of the code behind tricks for getting this to serve from Apache, I took a primitive approach. By attaching an event to MouseLeftButtonUp I can simulate a click event. Then connecting this click event to the MultiScaleImage ViewportWidth *= 0.9; I could make a one way zoom in without too much effort. Not very useful, but good enough to get a feel for the interaction, which by the way is very impressive. The zooming is familiar to anyone used to VE or GE type of continuous zoom. Pretty nifty for a browser interface.

There is even an ‘old view’ to ‘new view’ animation effect, which cleverly distracts the user while the new tiles are streaming in over the old tiles. Local tile cache makes revisiting very smooth. I will have to try this on an older tier 0 GPU system so I can watch the tiles move slowly into place.

http://www.web-maps.com/BMNGZoom/ >


Fig 2- primitive test of a Deep Zoom BMNG

Now that I had this working on a relatively small 4.1 Mb image, my next step was to step up the size and see what the effect would be. I already knew 11Gb Gtiff was not going to work. Dividing the full Blue Marble into 8 tiles and using PNG output seemed like a possibility. This gives 8 files at 256Mb each.

However, I noticed that the pyramid files are jpeg so why not start with 8 jpeg files instead:
“C:\Program Files\FWTools2.1.0\bin\gdal_translate” -of JPEG
-projwin -180 90 -90 0 world-topo-bathy-200401-3x86400x43200.ecw BMNG1.jpg
Input file size is 86400, 43200
Computed -srcwin 0 0 21600 21600 from projected window.
0…10.

After a few minutes I could open the Deep Zoom Image Composer again and do an import on the full eight tile collection. The composer did not show anything in the view window with these larger jpg images so I was working blind on composition. I did the export anyway out of curiosity.
I’ll post the results next week since it will take a good bit of uploading time.
The result of this bit of experiment was quite surprising. The pyramid building goes fairly smoothly in the Deep Zoom Composer and is painless compared to manually building pyramids in Java. Actually Geotools ImagePyramid has some advantages like iteration over multiple images and command line capability. But the resulting tile pyramid doesn’t have the client side streaming.The MultiScaleImage element hides the complexity of an ajax slippy map interface in a single element. On the down side adding functionality seems to be aimed at IIS ASP type servers. I imagine with a bit of time I can work out the details of a pan and MouseWheel zoom. SilverLight includes RenderTransform matrix capability, it just requires code behind to make it useful with mouse and keyboard functions.

The question is “how does this work?” Of course the answer is “I dont know,” but that doesn’t stop some speculation. The pyramid is obvious. The fact that it works on both a linux or a windows box eliminates a stub socket on the server side. It appears to be an object downloaded to the client which orchestrates things in an ajax mode. Of course clr won’t work with Firefox on Linux or Safari so there must be a plugin object which can be duplicated cross platform.

Wishlist for Deep Zoom 2.0 from a map hacker

1. Can we scale this puppy to use extremely large image sets? I imagine DCOG is going to want to see their 6″ aerials for the Denver metro area in Deep Zoom. Why should they have to come up with a pyramid tile set of MultiScaleImage elements?

2. How about MultiScaleImage elements for WPF xaml/xbap? I would like to use it with a more comprehensive set of code behind tools as an xbap.

3. Once it’s in WPF how about using some OO magic and add MultiScaleImageBrush for draping on a 3D mesh?

Lets extrapolate a couple more steps

4. Why stop at images? How about extending to MultiScale3DMesh. Then my spare time project for an AJAX LiDAR viewer won’t require much work.

5. Don’t stop there, lets have MultiScaleImageBrush on MultiScale3DMesh.

Now sharpen your patent pen

6. Why not MultiScaleVideo? Sooner or later all of the bifocaled baby boomers will be downloading movies to their iPhone, oops ZunePhone. How else are we going to see anything on those miniscule screens. Besides talk about “immersive,” movies could really be interactive. Imax resolution on a phone, why not!

3D Flood Plain Models

WPF 3D Flood simulations:
http://www.web-demographics.com/FloodPlain/FloodPlain.xbap

Fig 1 – Terrain with a planar sweep simulating water levels
  
After last week’s import of SketchUp building models into WPF xbap, I decided it was time to work with some of the higher resolution DEM data available from the USGS. The USGS has been upgrading a number of quads to 10m DEMs. These are scattered over the US and provide 9x number of points for building TIN models for select areas than the older 30m DEM. Much of the updating is in the western states which have a good deal of relief to model.

  
I downloaded the USGS 10m DEM for Colorado Springs, CO. The data consists of about 1.5 million points in a 10mx10m grid over the USGS 1:24000 quadrangle for Colorado Springs. The data I downloaded was in the SDTS format. I used a couple of existing tools to first convert the data to the older and more accessible USGS format ASCII DEM, sdts2dem.exe. The DEM format is setup in column dominant profiles but because the profiles are clipped to the geographic boundaries of the 1:24000 quadrangle the points are not in a symmetrical grid structure. Profiles on the east and west edges of the quadrangle will be only partial profiles which does complicate life when building a TIN.

  
Another tool takes these DEM profiles and converts them to simple x,y,z ascii, DEM2XYZN.EXE. From that point I was on my own. I had already created a small utility for converting JPL SRTM30 png files into TIN models. The JPL data is row dominant so it took just a little bit of editing to change to column dominant. The data I wanted was far from the quad edges so I could make the simplification of assuming a symmetric col x row matrix and calculate triangles from adjacent columns without accounting for the partial profiles at the edges.

  
I did not want to create an enormous Geometry for the model so I cut out the portion of interest surrounding the previous Broadmoor Hotel model. This 1.86km x 2.18km subset became a context terrain model for adding to the building model. Vertical relief for this smaller area ranged from 1844.6m to 2052.9m. The resulting TIN model was left in the UTM-13 metric coordinate projection.

   

Next I took the resulting TIN Geometry and added it to my project as a Page Resource MeshGeometry3D. I could then bind the mesh to a GeometryModel3D element called “surface” inside the Model3DGroup.Children used to hold the Broadmoor model. The Broadmoor model was previously translated from DXF which produced a WPF Geometry3D model normalized to 0,0,0. If I had written a KMZ translator I could have had the model in georeferenced coordinate Lat,Long which could then be transformed to a UTM projection, but the normalizing would still be required in WPF to make it visible inside the Canvas viewport. Since the surface TIN I am adding is in UTM I also had to add a Model3D.Transform, specific to the TIN Model3D, to scale and translate the surface under the building. The translation takes the UTM center point of the Broadmoor location in UTM and translates that center point to 0,0,0. In addition I noticed that AutoCAD models are in architectural inches rather than metric units. This made it necessary to add a uniform scale of 39.36999 inches per meter. The set of transform parameters was not easily obtained and did require some playing with the translation parameters to get a good fit. Fortunately I did have a DOQ and a DRG material surfaces that I could use to compare locations and make adjustment.

  
The result places the Broadmoor model in a terrain context with a 10m resolution. I kept the same set of scroll tools for rotation and scaling which I used to experiment with viewing my new model. I soon discovered that the rendering quickly broke down as the scaling moved in closer to the building. It appears that z-fighting is a problem with this model when inside the bounds of the terrain surface. I played with the camera near plane and far plane parameters but these did not seem to make any difference to the z-fighting breakup. I then assumed that it was best to leave the WPF to calculate its own near and far plane distances.

  
I also tried setting the camera Position and LookDirection to zoom the scene but again z-fighting occurred at positions close to the building. I finally did get a helpful clue with a post to the WPF forum. It was suggested I use the FieldOfView parameter to adjust the zoom value rather than scaling the entire scene. The only zoom method that seemed to ameliorate the z-fighting turned out to be the FieldOfView. This was unfortunate since ultimately I was hoping to develop some type of interface to fly the camera into and through the scene. Using camera parameters such as Position LookDirection, Rotate, and Tilt would let me fly the camera through the scene space while also adjusting its LookDirection with some type of trackball wrapped camera icon. I believe I can still do this but will need to substitute a combined FieldOfView variable and scene rotation for the position 3D location of the camera. For example I can rotate and tilt the camera as a substitute for the LookDirection vector while narrowing the FieldOfView and rotating the scene to simulate camera position changes. Although this works without the z-fighting affect I hope that future releases of WPF will do better with the z-fighting problem using actual camera position.

  
So now I have a Broadmoor Hotel building model placed on a USGS 10m Terrain with material selections of USGS DOQ or USGS DRG. At the cost of scroll bar proliferation, the added camera controls let the client move around and through the scene. This is clumsy but valuable for experimenting. The unfortunate workaround for z-fighting limits the usefulness of actually entering inside the model. Interior lighting and textures would help with orientation inside, however, the small relative scale of the interior along with the extremely narrow FieldOfView required to get to that zoom level, make minor adjustments have large results. I think this is nearly the definition of chaos and means it is easy to get lost inside. Moving from outside to inside may require two discontinuous models, with an abrupt model switch at the entrance. These are areas obviously well worked out in gaming tools but WPF is a simpler set of tools at this stage and I doubt that sophisticated 3D browser gaming will result from WPF 1.0.

  
Once I had this terrain to play with I decided to take it a step further. My first thought was to make a simple line of site mapping leveraging the “RayMeshGeometry3DHitResult” (a terrifyingly long name for a method) for a series of rays emanating from the viewer eye height. This would not provide a full view shed for the scene but might be useful for a simplified view analysis. However, I was distracted by another opportunity. In order to display the line of sight view map I needed to add a simple plane to the scene for overlaying my view calculations.

  
I discovered that adding a plane to the scene would let me play with a planar sweep and simulate a water level. Of course at 6000 feet above sea level this is not a dramatically realistic addition but it is fun. I simply added a geometry plane with a blue color and a fractional opacity. By binding yet another scrollbar to this plane’s associated TranslateTransform3D OffsetZ parameter I am able to raise and lower the water level as desired.

   

Fig 2 – Water rising at the Broadmoor

  
There are some scenarios where this might actually be useful. In the past I have translated ENC charts for use in a web application with SVG. ENC charts are detailed 2D charts of navigable water ways. NOAA and the Army Corps of Engineers have ENC chart data available for areas such as harbors and rivers, http://chartmaker.noaa.gov/MCD/enc/index.htm. I had previously used ENC charts for the Mississippi up to Baton Rouge and around Galveston harbor. These charts are highly symbolized in 2 dimensions. Transferring to 3D and using a water plane above a bathymetry/terrain model would add some realism. In addition it would be interesting to fly around a navigation chart scene and see the affect of tidal variation (connected to the server clock with a tide table http://tidesandcurrents.noaa.gov/) and ship draft on navigation in well known channels. Of course the standard FEMA flood plain models might also make interesting subjects for this type of planar sweep.

Fig 3 – SVG ENC chart for Mississippi channel near New Orleans
   

Fig 4 – Water rising at the Broadmoor view from above

Google skp files

I went to a little promotional seminar in Denver for Google Enterprise. The seminar was mostly marketing Power Points, but the coffee was good and the demonstration pods afterwards were interesting. I have not played much with Sketchup since it was first introduced. The demo overview got me interested again. Once I got back to the office I downloaded the most recent trial version for some experimenting. Sketchup is really an easy to learn tool, at least compared to AutoCAD. It would probably take a week or so to get very proficient but that is really a short time considering the hastle of entering 3D models.

http://www.web-demographics.com/Broadmoor/Broadmoor.xbap

Fig 1 – WPF XBAP version of a Broadmoor Sketchup Model

   

Sketchup is oriented to the Architectural community more than tools like Blender or MS Expression Blend which appeal more to graphics designers and animators.

  
Sketchup is Google’s tool for populating their globe data spaces with building models. It has many more uses than that, but I think they were hoping to get a community going to populate Google Earth with models. I notice Microsoft in usual form doesn’t attempt to develop a community but invests money to model whole cities en mass. Google’s recently announced licensing of auto modelling algorithms may help get their city models going faster. Darpa also seems interested in harnessing drive through point LIDAR for creating virtual cityscapes in a more or less automated fashion.

  
It is always interesting to see how GIS, CAD, and Facility Management tend to blur at the edges. These building and city models are at the crude edge of the technology curve. I predict it won’t be too long before we start seeing full models with interior spaces and walk through capability available on the web.

  
Sketchup has a generous set of import/export options. I was able to load a model of the Broadmoor Hotel here in Colorado Springs. I then exported to several of the different formats available, including Collada DAE, KMZ , and DXF. DXF is the most accessible and I quickly converted the model to a XAML version to see what could be done. DXF does not preserve the rich material textures and photo overlays in the Sketchup model so the result is rather drab by comparison. However, the fun is that it can be done at all. Once in XAML the usual WPF tools are available to do a bit of manipulation.

  
Collada DAE appears to be a richer xml based interchange format that includes references to all of the jpg photo and material overlays. The kmz format also identifies these jpg/png overlays but does not automatically export them for reference. Somehow a KMZ/KML to XAML translator has more appeal since it would be useful in other areas. However it would require getting hold of all the jpgs from a different export. I will need to look at the skp apis and see what they offer for helping with various exports.

  
The building model can be part of a WPF enhanced browser which makes it a part of the web instead of a parallel world like Google Earth or Sketchup. The Google Earth viewer is more focused and in a narrow way much richer, while WPF is a broader approach and gives more freedom to the developer. I have wondered what Google might come up with if they created their own browser. It could obviously include capabilities like Google Earth and Sketchup as well as the usual html/javascript. I’m sure it is something Google has thought about.

  
It was fun to see what can be done with these kinds of models and Google’s openness to innovation is refreshing. It reminds me of the earlier John Walker days of Autodesk. At that time Autodesk was very community friendly creating portals to their basic CAD functions through DXF, AutoLisp etc. I think it was 1988 when I made my first wire frame earth globe with AutoLisp functions using the old World Data Bank I vectors. A lot has changed in the intervening decades.

Some more XBAP

This week I experimented with a WMS interface using XAML. The idea was to see what could be done with NASA’s new NEO services. These are largely global environmental satellite imagery sets updated on a regular schedule, so there are things like global reflectance and sea temperatures etc. In addition NASA has a series of animations on their SVS Scientific Visualization site. All of these lend themselves to global coverage so I decided to use a globe interface rather than the more traditional 2D.

http://www.web-demographics.com/Earth3D-Neo/Earth3D-Neo.xbap

  
XAML lets the developer create 3D scenes so it is not difficult to create a tin model of a sphere. In this case I used some code to generate a sphere with 2162 triangles which provides a fairly smooth spherical surface. Directional lighting was added with a camera focused on the North American quadrant. Then a series of tools were added to create DiffuseMaterial overlays. The base overlays are fairly standard global images from JPL and NASA including the ubiquitous Blue Marble. These can be interchanged with a selection from a ListBox added to a ToolTray in the top DockPanel. In addition a ListBox of example overlays was added to verify the use of a local GeoServer WMS service. WMS is the easiest to incorporate as a simple linked ImageSource reference to an ImageBrush for the DiffuseMaterial overlays. These can be stacked nicely as long as a transparent png is returned from the selected WMS layer.


Fig 1 – WPF XBAP interface for NASA NEO WMS site
   

Leveraging WPF animation triggers lets the client add some spin to the globe, while a TrackballDecorator tag lets the user rotate the globe and scale using left and right mouse gestures. Binding allows a scrollbar to connect to scaling transforms. To add a little more movement a ComboBox filled with NASA SVS references lets the client view thumbnails and select mpegs for playing over the sphere’s material sets. This demonstrates some of the media flexibility in WPF. Since some of these mpgs are opaque I also set the opacity to .75 to allow the sublayers to partially show through.

   

Images are useful but there are a lot of vectors in the mapping world. As an experiment I also added a few DrawingBrush.Drawing elements as material overlays. A RectangleGeometry patch indicates my area of the globe. There may be a better approach but I used a 360×360 square and added the rectangle in its x,y coordinates which means the latitude had to be adjusted to the exaggerated north/south dimension with a 2x scalar. Some more experimentation with DrawingBrush Stretch might improve this bit of hack.

   

Using a VisualBrush let me add a Canvas and TextBlock label which is updated with the surface selection, making an earth conforming label. Finally a more complex compass figure was floated above the globe. This involved a simple rectangular surface with a set of DrawingGroup elements added for the compass segments. The complexity here is correctly setting the transforms to allow the compass to show but also let it scale and rotate with the globe.

   

The NEO WMS site has a wealth of services described in its GetCapabilities request: http://neowms.sci.gsfc.nasa.gov/wms/wms?version=1.1.1&service=WMS&request=GetCapabilities

   

This allowed me to experiment with a more complex binding to a TreeView element using an XmlDataProvider. This is a great tool that allows Xpath selections to be fed directly into TreeViewItems through a HierarchicalDataTemplate. I did not connect directly to NEO’s getCapability request since there is a problem with using an outside link to another site. The returned xml references an external DTD. This reference evidently triggers a security failure when the xml reader attempts to verify the dtd. As a shortcut I simply added a copy of the GetCapabilities XML with the offending dtd reference removed. This bit of cheating can be fixed but it would involve creating a server proxy servlet to read the GetCapabilities and remove the offending line before feeding it to the XmlDataProvider. The end result is a nice but static tree view selection list of the layers exposed by NEO’s WMS service.

   

VisualBrush could be used to add WFS GML or for that matter KML. This would involve adding a couple of servlets to translate GML and/or KML to VisualBrush GeometryDrawing elements that become another DiffuseMaterial overlay. I leave that to experimentation down the road.

   

I should also comment on the SpecularMaterial. This is added to give a bit more of a dimensional quality to the sphere. It adds a bit of specular reflectance which unfortunately is not especially smooth due to the coarse nature of the TIN model. I left it in as an interesting artifact and it doesn’t detract too much except for the darker overlays. It also requires a bit of manipulation on the overlays since it needs to be last in the list.

   

The coordinate mouseover required some digging. The 3D aspects of XAML are nice for modeling and viewing but the interactive capabilities are a bit restricted. I was able to use RayMeshGeometry3DHitTestResult to get the 3D point of the ray intersect of my globe. Adding a bit of code to transform the 3D point to Lat,Long gives the coordinate readout, but I have yet to actually get access to any of the Drawing elements at that intersection. In other words I didn’t find a way to query into the scene for a particular element at the ray intersection which would be very useful for external queries to the backing server.

   

The lat,long does give me a way to utilize a GetFeatureInfo query to WMS servers but I would have also liked to access drawing elements from WFS overlays directly in the client. A little more research may reveal a way to do this. I am aware that the 3DTool kit provides a way to fool the viewer by adding a transparent surface that tracks the 3D underneath. User interaction with the 3D scene is simulated by actual interaction with the 2D surface. I don’t quite see how this will work with a globe once it starts spinning.

   

Having lat,long coordinates did allow me to add one additional function. Using Ctrl+Mouse Left Click lets me capture a lat,long point and using NavigationService to activate a GetTerrain servlet. This servlet takes advantage of the JPL DEM WMS site. JPL has kindly added a WMS extension “styles=feet_short_int” to allow elevation pngs. The 16bit grayscale provides elevation values which the GetTerrain servlet uses to create a TIN model. Obviously these require a big chunk of memory so this approach probably doesn’t scale too well. The TIN model is streamed back to the client as a loose XAML 3D terrain model. Using a 1/3 degree patch results in about a 12Mb xaml. This is helped a bit with the Apache compression but is still a cumbersome download unless the client has a high bandwidth connection.

   

Since the source is global the GetTerrain used the best possible DEM which has global extent, the SRTM3. This provides 3arc global DEM coverage. I use the NASA BMNG as the surface overlay since it also has global coverage and a fair resolution. There is much better coverage in the USA with 30m DEM or 10m DEM and a number of possible overlays such as USGS DRG, USGS DOQ, and Hi Res Urban imagery. The public imagery services don’t come too close to the universal offerings of Google, Live Maps, or MapQuest but hopefully the government will keep adding additional resolution over time. I was tempted to sneak in a set of Google tiles as overlays but beyond the legal challenges the projection requirements require a bit of customization I didn’t care to reproduce.

   

The resulting Terrain WPF model is loose xaml. This means that it cannot take advantage of code behind C# and is somewhat restricted in its capabilities. Some bindings can be added for scaling and rotating but the TrackballDecorator, for example, requires C#. Unfortunately the C# code behind can only be created using a compiler to xbap. There doesn’t appear to be a dynamic way to serve xbap from a Tomcat server. It may be possible in some manner using an IIS server but that is beyond my expertise presently. Another possibility would be to dynamically create the necessary XAML with its C# code behind and then execute a command line build. This might work for small sites but building a new version with each change of selection on a real website would not scale at all. The revision increments could easily climb out of the possibly shortint space allotted to the Microsoft compiler. This seems to be a real restriction to WPF in the browser and will prohibit its adoption in many situations.

   

WPF/E uses JavaScript and could operate more like a rich browser client, however, at this time it is restricted to 2D elements. One possible work around is to utilize AJAX techniques to load loose xaml with an XML reader and create dynamic xaml inside the original download. This is a bit complex but was an approach often used in SVG. The code behind C# for the entire application is compiled and downloaded initially. Then as selections are made instead of doing the normal REST link to new content, content is requested from the server as xml and inserted into the existing DOM. I imagine this will be the solution of choice for any more complex xbap application.

   

This experiment turned out to be a fair exercise of WPF and shows the real utility of the WPF XAML approach. It is not too hard to imagine Google Earth like interfaces created using interoperable OGC sites and WPF browser solutions. This in fact would seem to be a more enterprise friendly approach to in house services than dependence on a Google or Microsoft branded mapping service.


Fig 2 – Night in the Rocky Mountains

A quick look at GoGrid


Fig 1 a sample ASP .NET 3.5 website running on a GoGrid server instance

GoGrid is a cloud service similar to AWS.( http://www.gogrid.com ) Just like Amazon's AWS EC2, the user starts a virtual server instance from a template and then uses the instance like a dedicated server. The cost is similar to AWS, starting at about $0.10 per hourfor a minimal server. The main difference from a user perspective is the addition of Windows servers and an easy to use control panel. The GoGrid control panel provides point and click setup of server clusters with even a hardware load balancer .

The main attraction for me is the availability of virtual Windows Servers. There are several Windows 2003 configuration templates as well as sets of RedHat or CentOS Linux templates:
· Windows 2003 Server (32 bit)/ IIS
· Windows 2003 Server (32 bit)/ IIS/ASP.NET/SQL Server 2005 Express Edition
· Windows 2003 Server (32 bit)/ SQL Server 2005 Express Edition
· Windows 2003 Server (32 bit)/ SQL Server 2005 Workgroup Edition
· Windows 2003 Server (32 bit)/ SQL Server 2005 Standard Edition

The number of templates is more limited than EC2 and I did not see a way to create custom templates. However, this limitation is offset by ease of management.
For my experiment I chose the Windows 2003 Server (32 bit)/ IIS/ASP.NET/SQL Server 2005Express Edition. This offered the basics I needed to serve a temporary ASP web application.

After signing up, I entered my GoGrid control panel. Here I can add a service by selecting from the option list.


Fig 2- GoGrid Control Panel

Filling out a form with the basic RAM, OS, and Image lets me add a WebbApp server to my panel. I could additionally add several WebAPP servers and configure a LoadBalancer along with a Backend Database server by similarly filling out Control Panel forms.This appears to take the AWS EC2 service a step further by letting typical scaling workflows be part of the front end GUI. Although scaling in this manner can be done in AWS it requires installation of a software Load Balancer on one of the EC2 instances and a manual setup process.


Fig 3 – example of a GoGrid WebAPP configuration form

Once my experimental server came on line I was able to RemoteDesktop into the server and begin configuring my WebAPP. I first installedthe Microsoft .NET 3.5 framework so I could make use of some of its new features. I then copied up a sample web application showing the use of a GoogleMap Earth mode control in a simple ASP interface. This is a display interface which is connected to a different database server for displaying GMTI results out of a PostGIS table.

Since I did not want to point a domain at this experimental server, I simply assigned the GoGrid IP to my IIS website. I ran into a slight problem here because the sample webapp was created using .NET 3.5System.Web.Extensions. The webapp was not able to recognize the extension configurations in my WebConfig file. I tried copying the System.Web.Extensions.dlls into my webapp bin file. However, I was still getting errors. I then downloaded the ASP Ajax control and installed it on the GoGrid server but still was unable to get the website to display. Finally I went back to Visual Studio and remade the webapp using the ASP.NET Web App template without the extensions. I was then able to upload to my GoGrid server and configure IIS to see my website as the default http service.

There was still one more problem. I could see the website from the local GoGrid system but not from outside. After contacting GoGrid support I was quickly in operation with a pointer to the Windows Firewall which GoGrid Support kindly fixed for me. The problem was that theWindows 2003 template I chose does not open port 80 by default. I needed to use the Firewall manager to open port 80 for the http service. For those wanting to use ftp the same would be required for port 21.

I now had my experimental system up and running. I had chosen a 1Gb memory server so my actual cost on the server is $0.19/hour which is a little less for your money than the AWS EC2:

$0.10Small Instance (Default)
1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform

But again, running ASP .NET 3.5 is much more complex on EC2, requiring a Mono installation on a Linux base. I have not yet tried that combination and somehow doubt that it would work with a complex ASP .NET 3.5 website, especially with Ajax controls.

The GoogleMap Control with the Earth mode was also interesting. I had not yet embedded this into an ASP website. It proved to be fairly simple. I just needed to add a <asp:ScriptManager ID=”mapscriptmanager” runat=”server”/> to my site Master page and then the internal page javascript used to create the GoogleMap Control worked as normal.

I had some doubts about accessing the GMTI points from the webapp since often there are restrictions using cross domain xmlhttpRequests. There was no problem. My GMTI to KML servlet produces kml mime type "application/vnd.google-earth.kml+xml" which is picked up in the client javascript using the Google API:·
geoXml = new GGeoXml(url);

Evidently cross domain restrictions did not apply in this case, which made me happy, since I didn't have to write a proxy servlets just to access the gmti points on a different server.

In Summary GoGrid is a great cloud service which finally opens the cloud to Microsoft shops. The GUI control panel is easy to use and configuring a fully scalable load balanced cluster can be done right from the control panel. GoGrid fills a big hole in the cloud computing world.

Census data

I have been busy for awhile now working with demographic data. The US Census Bureau collects an extravagant amount of population data in the decennial census which last occurred in 2000. They are currently preparing for the 2010 census. http://www.census.gov/ The challenge in my case is to provide an interface to the demographic data in a thematic form for use against an OWS background. I am using an SVG client so I have full access to vector polygonal entities.

My original thought was to use WMS SLD <FeatureTypeStyle> to color fill polygons by calculated population range. Although this is possible with simple queries it becomes cumbersome when dealing with table JOIN sql queries. The way the census demographic data is organized provides several tables of statistical fields described in a catalog matrix. The front part of each record includes the geography header which keys to a special table of geography records. The geography table includes the geography hierarchy fields but not the actual geometry. An additional JOIN is required from concatenated geography fields to a polygon geometry table. In essence a double JOIN is used to get from a population statistic in the catalog matrix to the actual points of the associated geometry.

Here is a simple PostGIS SQL select on a single matrix element, H004002, at the county level of geography within a spatial bbox for Summary File SF1 Census demographics:

SELECT AsSVG(g.the_geom,1,6) as geom, sf1.”H004002″ as field, g.”NAME” as label, geo.”STATE”||geo.”COUNTY” as id FROM “SF10037″ sf1
JOIN “SF1geo” geo ON geo.”LOGRECNO”=sf1.”LOGRECNO” JOIN “county” g ON geo.”STATE”||geo.”COUNTY”=g.”STATE”||g.”COUNTY” WHERE geo.”SUMLEV”=’050′ AND geo.”GEOCOMP”=’00′ AND g.the_geom && GeometryFromText(‘LINESTRING(-107.11538505554199 38.072115898132324,
-104.59134674072266 40.524038314819336)’,4269)

Resulting in this view

The difficulty in using an OWS service from something like GeoServer is that the database query is automatically built from an xml filter definition. I did not see a way to build the complex join query using the current filter implementation. Short of diving into the GeoServer code, I could not use an OGC standard approach to my query and decided to bypass the OWS approach and go directly against the PostgreSQL/PostGIS. There is, however, an effort in the GeoServer community to add a complex feature capability which may address this type of issue. http://docs.codehaus.org/display/GEOTOOLS/Community+Schema+Support+and+Complex+Types
Census geography follows a tree hierarchy which subdivides geography into smaller and smaller polygonal entities. Unfortunately the tree is not homogeneous across all of the US. For example, ‘Native American Lands’, ‘Populated Places’, ‘Metropolitan Areas’ etc each have a different tree. The vast majority of the US, though, follows a simple ‘state -> county -> tract -> blockgroup -> block’ hierarchy.

The census does not necessarily collect/publish data to the full block depth for all demographic data. For example, using Summary File SF2 with a characteristic iterator of ‘Pakistani alone’ and total population in occupied housing units, no records are available below the county geography. On the other hand using SF1 vacancy status for housing, records are available all the way down to the block level:

The thematic legend color scheme was chosen using the excellent information by Cynthia Brewer for sequential data representation: http://www.personal.psu.edu/faculty/c/a/cab38/ColorBrewer/ColorBrewer.html

I kept the class categories and color range simpler by implementing with a small 5 step interval across the data range. The interesting problem with any type of sequential thematics is choosing appropriate class ranges. The simplest approach is to take the total range and simply divide it by the selected number of interval to obtain the class extents. This approach can often result in large numbers of homogeneously colored polygons, since many polygons have zero or small populations while a few have a large set of population. The majority of polygons will then fall into the same color class with only a few color coded further up the range. In fact if the range spread is significant there may be several classes with no polygons at all.

Another approach is to sort the polygons by population and divide the total number of polygons by interval into classes, which are then assigned colors. This approach classifies across the number of polygons represented instead of across the population range. The result is a more pleasing gradation of polygon color. However immediate problems occur when the data clusters at one end of the range. The result of this simplistic approach is that many of the classes fall into the same interval while only one or two classes contain all of the clustered data falling at one end of the histogram. This can be seen above where all four of the lower classes are taken up in values between 0 and 1 while only the last class picks up the high end of the population data range. The maps may be more pleasing but the information conveyed is again skewed.

The last approach can be tweaked a bit by subdividing only the polygons containing populations above zero. By lumping all of the zero range into a single class and then subdividing the remainder of the data, where there are interesting things going on, the view is a bit less skewed for zero dominant characteristics. Of course the data may be dominated by some value slightly above 0 in which case the tweaking is not especially helpful

The issues of chloropleth mapping are quite complex and even of interest to mathematicians. There are some in depth mathematical texts analyzing the intricacies of the problem which I am avoiding for the moment. My ultimate goal is to take advantage of SVG to render the actual histogram chart of the polygon selection sorted into population order. This chart would be overlaid with a set of simplistic intervals as low opacity elements with draggable edges. The idea is to allow the user to carefully select non symmetric classes from the histogram view. The histogram with its class selection is retained as an enhancement to the legend so the viewer knows exactly the class extents shown in the thematic view. In essence this passes the complexity of classification off to the user, but also retains a record for the viewer.

The need for a user profile and a way to save view setups is also apparent since the idea ultimately is to create some interesting analysis and make it available to other viewers anywhere in range of the internet.

Here is a screen shot of a similar interface developed a while ago for imagery clamp and thresholding using an interactive range setting on the rgb histograms.