Andrew Trice

Monday February 02, 2009

Lazy-Loading Hierarchical Data

In theme with some of my recent posts regarding how to manage large data sets (here and here), this post will examine how to load large hierarchical data sets into a Flex application.

Unlike my previous posts, this post has very little to do with visualizing a large hierarchical data set. Instead, it examines an approach that can be used to dynamically load massive hierarchies into a Flex or AIR interface, without having to wait or perform computationally intensive tasks.

A common navigational interaction with hierarchical data is drilling-down into data. Unless you are performing some kind of search on the data, you typically start at the top-most level and browse through the hierarchy until you find what you are looking for. When working with XML data in Flex, you would think that the entire data hierarchy would need to be present in the client runtime, right?

Well... no, not really. You can lazy-load hierarchical data sources with a relatively low level of effort and low level of complexity. Lazy-loading is a technique where you only load data as you need it, as opposed to loading an entire data structure initially or before it is actually needed (eager loading). This is a benefit because it can reduce overhead overhead and increase speed when the hierarchical data structure is requested (less computing required to query the data, faster network throughput b/c of smaller data size, and faster object parsing & serialization).

With drilling-down into hierarchical data, you would only need to request visible levels of the hierarchy as they are necessary. Take a look at the following example; It is a Flex AdvancedDataGrid, which is showing a hierarchical data structure. When the application initializes, it requests the first-level children of the hierarchy from the server. As each node is expanded, the children of that node are then requested from the server. Yes, there is a slight delay the first time a node is expanded. However, this is only on the initial data load. Each subsequent request uses the hierarchy that has already been loaded into memory.

click on a node to expand it

Now, let's examine how this works. This example is using XML data structures, however this method works with strongly-typed object collections as well.

Every time that a node is expanded, the "itemOpen" event is dispatched, which invokes the onItemOpen() function. The onItemOpen() function evaluates the children of the currently expanded node and determines if more data should be requested from the server. By default, each node is populated with a "dummy" xml node that has the attribute "loadData" with a value "true". If loadData is encountered, then it requests new data from the server.

You can also have simultaneous service requests working at the same time. In case you're wondering how this works, a reference to the target XML node for each service request is maintained in the StatefulResponder class. This way, the result handler always knows which node to populate when a service result returns from the server.

main.mxml

<?xml version="1.0" encoding="utf-8"?>
<mx:Application 
  xmlns:mx="http://www.adobe.com/2006/mxml" 
  layout="absolute"
  creationComplete="onCreationComplete()">
  
  <mx:Script>
    <![CDATA[
      import mx.collections.XMLListCollection;
      import mx.controls.Alert;
      import mx.rpc.events.FaultEvent;
      import mx.rpc.events.ResultEvent;
      import mx.rpc.AsyncToken;
      import flash.utils.setTimeout;
      import mx.events.AdvancedDataGridEvent;
      import mx.collections.HierarchicalData;
      
      [Bindable] private var xmlData : XML = <data name="root" depth="0" ><node name='loading...' loadData='true' /></data>;
      
      private function onCreationComplete() : void
      {
        //load initial dataset 
        requestData( xmlData );
      }
      
      private function requestData( targetNode : XML ) : void
      {
        //The stateful responder holds on to the current node
        var responder : StatefulResponder = new StatefulResponder( onResult, onFault, targetNode ); 
        
        //request data from the server
        var params : Object = { depth:targetNode.@depth.toString() };
        var call : AsyncToken = httpService.send( params );
        call.addResponder( responder );
      }
      
      private function onResult( event : ResultEvent ) : void
      {
        //retrieve the target node that was referenced in the StatefulResponder
        var targetNode : XML = (event.token.responders[0] as StatefulResponder).targetNode;
        var resultXML : XML = new XML( event.result.toString() );
        
        //delete exsiting children
        delete targetNode.node;
        
        //loop over result collection and append to target node
        for each ( var node : XML in resultXML.children() )
        {
          targetNode.appendChild( node );
        }
      }
      
      private function onFault( event : FaultEvent ) : void
      {
        Alert.show( event.message.toString(), "ERROR" );  
      }
      
      private function onItemOpen( event : AdvancedDataGridEvent ) : void
      {
        //each time a node is expanded, request the children from the server, if you have not alredy retrieved them
        var children : XMLList = XML( event.itemRenderer.data ).children();
        if ( children.length() == 1 && children[0].@loadData == "true" )
        {
          requestData( event.itemRenderer.data as XML );
        }
      }
    ]]>
  </mx:Script>
  
  <mx:HTTPService 
    id="httpService"
    url="data.jsp" 
    showBusyCursor="true" 
    resultFormat="text" />
  
  <mx:AdvancedDataGrid 
    width="100%" height="100%"
    dataProvider="{new HierarchicalData(xmlData)}" 
    itemOpen="onItemOpen( event )">
        <mx:columns>
            <mx:AdvancedDataGridColumn dataField="@name"/>
        </mx:columns>
    </mx:AdvancedDataGrid>  
  
</mx:Application>

Stateful Responder

package
{
  import mx.rpc.Responder;

  public class StatefulResponder extends Responder
  {
    public var targetNode : XML;
    
    public function StatefulResponder(result:Function, fault:Function, targetNode : XML)
    {
      super(result, fault);
      this.targetNode = targetNode;
    }
  }
}

... and yes, this is querying the server for live data every single time it is loaded. It makes a request to data.jsp, which is just a simple jsp that generates XML data. It outputs the depth and node name attributes in the XML based on the url parameters that are passed into it.

data.jsp

<data>
<% for ( int i = 0; i < 25; i++ ) { %>
  <node name='Node <%= request.getParameter("depth") %>.<%= i %>' depth='<%= request.getParameter("depth") %>.<%= i %>'><node name='loading...' loadData='true' /></node>
<% } %>
</data>

It's as easy as that. You can easily manage massive hierarchies without impact the user's experience or degraded performance.