Friday, June 1, 2007

Building a Data Driven Mashup with Microsoft Popfly

Regular readers of my blog know that I have been looking at Enterprise Mashups for a while now. Because I am an engineer on the WebLogic Portal product, I have focused on solutions involving that product. However, in this blog I will cover a new product called Popfly from Microsoft that takes an interesting approach to Mashups. I will walk you through how to build a Mashup in Popfly step by step, including the programming of a data service. The Mashup will display a slideshow that contains picture from Flickr of each of the US cities that dev2dev Tech Days visited.

NOTE: this blog entry was originally posted June 1st, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

The World of Enterprise Mashups

A common question people ask is: "what's a Mashup?". That question is usually followed by another question: "why would I want one in my enterprise?" We are covering this topic in detail with a traveling road show called dev2dev Tech Days. The topic of the seminar is "Mashup the Enterprise", and we answer these questions. In the last few days, you have seen others blogging about that event on dev2dev such as James Bayer and Sean Boiling.

For those unable to attend the seminar, I can offer a few pointers to help get you started.

  • AP News - my favorite sample mashup from the web; it uses an AP News wire feed, Yahoo Geocoding, and Google Maps
  • dev2dev Tech Days 2007 - Mashup the Enterprise - register for a FREE seminar from BEA, available in 45 cities
  • Webinar: Enterprise Mashups with WebLogic Portal 9.2 - I presented an introduction to the Enterprise Mashup space
  • Building a Simple Ajax Powered Google Maps Mashup - a tutorial to build a simple Hello World mashup example
  • Picasa and Google Maps Mashup - James Bayer shows how to use Workshop to build a Picasa Mashup

BEA and Enterprise Mashups

Before we head into Microsoft land, I need to quickly cover how BEA is addressing the Mashup space. BEA offers several approaches to building a Mashup. The first approach is to use a portal product like WebLogic Portal or AquaLogic Portal. A second approach is a new product that will launch soon called AquaLogic Ensemble that specifically targets the Mashup space.

Enter Microsoft Popfly

Popfly is a new offering from Microsoft, currently just out the door as an Alpha release. The best way to get a feel for Popfly is to watch their 15 minute screencast here. Interesting eh? Let me summarize:

  • Popfly is a web based composition engine for assembling components into workable Mashups
  • The tooling intends to allow non-programmers to accomplish this
  • A Popfly Mashup is composed of "Blocks" which can either be data driven or presentation driven

It may seem odd for a BEA employee to cover a Microsoft product. But it all makes sense - Mashups are similar to Service Oriented Architectures (SOA) in that they demand interop between heterogeneous systems. While this particular blog entry will not show BEA WebLogic Portal integrating with Popfly, you should expect that I am heading in that direction. I have been showing how WLP portlets can be surfaced in iGoogle for nearly a year now, and customers have loved to see how WLP interops with other Mashups. Next, in a follow up blog I will show artifacts from WLP Content Management being surfaced in Popfly. Cool! But before we can show WLP+Popfly, I need to first introduce you to Popfly development which is what this blog entry is all about.

Your First Glimpse at a Popfly Mashup

To give you a better idea, see the iframe below which contains the completed demo that I will explain in this blog. I know it doesn't look like much, but I will show how this widget was assembled using 3 Popfly Blocks, including one that I built myself.

PJL revision:
Readers complained that just viewing my blog page required installing the Silverlight plugin because of the iframe. I am therefore just linking it here now. dev2dev Tech Days Popfly Mashup
/PJL revision:

Warning: the Flickr Block is presenting a random picture associated with each city. All apologies if a picture turns up that is not office-friendly. I just saw a picture of a guy in underwear. Yikes.

Here is what it is doing:

  • There already exists a hosted data feed (returning JSON) listing some of the locations that dev2dev Tech Days will visit this year
  • I built a Popfly Data Block that consumes that JSON feed, bringing the list of cities into my Popfly project
  • The dev2dev Data Block is wired into a pre-built Flickr Data Block, so that for each city, a single picture is retrieved using the city name as the image query
  • Finally, the Flickr Data Block is wired into a pre-built Slideshow Presentation Block to display the pictures
popfly_demo_endState

Starting with Popfly

Currently, Popfly is in a managed Alpha release. You must apply for a Popfly account and then wait. I applied last week and just got mine today, so perhaps it will go quickly for you.

Once you get your account, you will be able to start building Mashups. A developers kit is available for download once you have a login. There are a few hundred Blocks already available for your use, with more being added daily. You will also need to download a new browser plugin called Silverlight to use the toolset (you will be prompted). For this demo, you will also need to configure a Flickr API Key, available here.

Step 1: Creating the Data Service Block

As I mentioned previously, I assembled an Ajax powered Google Maps Mashup for my dev2dev article. The demo is hosted at http://ajaxmashup.googlepages.com, and includes a data feed that returns a list of dev2dev Tech Days cities. That data feed returns a format called JSON. If JSON is unfamiliar to you, I will add a link to my article that explains it as soon as it is posted.

To consume the JSON feed in Popfly, I first had to create a new Block project. A Block once again is a Mashup component that either works with data or generates presentation (a single Block can do both, but it shouldn't). My Block will retrieve the JSON, translate it into the appropriate Javascript object, and pass the object to the next Block. From what I have seen, Blocks are written in Javascript, although the intro screencast shows some interesting Visual Basic use cases.

Once a the Block project has been created, you must supply two files:

  • Block descriptor: an XML file that describes your block; this information is used to power the visual wireup capabilities
  • Javascript file: a single Javascript file that includes your Block implementation

The Block descriptor is self explanatory, and well documented. It enumerates the inputs and outputs of your Block so that a non-programmer can wireup the Block into a Mashup. Popfly defines a set of standard data types, including url, userName, title, etc. so that the tool can make intelligent default guesses on how to wire two Blocks together. The following is our Block descriptor. Notice how we have defined a single operation (method) on this Block: getLocations. This operation has no inputs, and its output is an array of Dev2DevLocation objects.

<?xml version="1.0" encoding="utf-8"?>
<block class="Dev2DevDays">
<providerName>Dev2DevDays</providerName>
<providerUrl>
http://ajaxmashup.googlepages.com/
</providerUrl>
<providerLogoUrl>
http://wlp.bea.com/images/bea_logo.gif
</providerLogoUrl>
<blockIconUrl>
http://wlp.bea.com/images/bea_logo.gif
</blockIconUrl>
<operations>
<operation name="getLocations">
<description>
Get the locations of BEA dev2dev Tech Days
2007, "Mashup the Enterprise".
</description>
<inputs />
<outputs>
<output isArray="true" type="custom"
object="Dev2DevLocation" />
</outputs>
</operation>
</operations>
<objects>
<object name="Dev2DevLocation">
<field name="city" type="city" isArray="false"/>
<field name="venue" type="name" isArray="false" />
<field name="address" type="location"
isArray="false"/>
<field name="date" type="date" isArray="false"/>
</object>
</objects>
</block>








The second piece of our Block is the implementation itself. I implemented this in Javascript, and it is quite simple. It is essentially a function call that marshals the JSON feed into an object array that can be used in Popfly.









// create our namespace for the Block
function Dev2DevDays() { }

Dev2DevDays.prototype.getLocations = function () {

// set the URL to our JSON service, hosted on googlepages.com
url = "http://ajaxmashup.googlepages.com/getD2DSites.html";

// get the JSON list of locations
var resultJSON = environment.getText(url);
var responseObject = eval("(" + resultJSON + ")");

// convert to the Array type we want
var resultsArray = new Array();
for(i=0;i < responseObject.locations.location.length; i++) {
resultsArray[i] = new Dev2DevLocation(
responseObject.locations.location[i].city,
responseObject.locations.location[i].location,
responseObject.locations.location[i].address,
responseObject.locations.location[i].date);
}
return resultsArray;
};

// Define the output payload, must match the XML descriptor
function Dev2DevLocation(city, venue, address, date)
{
this.city = city;
this.venue = venue;
this.address = address;
this.date = date;
}
Dev2DevLocation.prototype.toString = function(){
return this.venue+", "+this.address;
};








For those that have implemented Mashups in a browser, you are aware that browser security would normally restrict access to the googlepages.com JSON feed. The Single Origin Policy prevents browsers from crossing multiple network domains. Popfly provides a workaround: it includes a server side proxy accessible via the getText() or getXml() Javascript calls.









Wiring the dev2dev Block to the Flickr Block








The next step is to take the output of the dev2dev Data Block, which is an array of locations, and wire it into the Flickr Block. The Flickr block is provided out of the box by Popfly, so you do not need to build it. The Flickr block needs a query string to retrieve a photo. In this case, I wire that using the "city" field from the JSON feed. This can be done using the visual UI.









popfly_demo_wireup









Wiring the Flickr Block to the Slideshow Block








The last step is to take the output of the Flickr Data Block, which is an array of Photo objects, and wire it into the Slideshow Block. This can be done visually by just connecting the two blocks, as the default wireup will work just fine. Now the Mashup is done, how is easy is that!









Workaround for Popfly Alpha








OK, so for now it's not quite as easy as I said. I found I need one workaround to make this all hang together. The Flickr Block and Slideshow Block have a slight mismatch for their data contract because of the way I built the Mashup. Ordinarily, the Flickr Block is given a single string (city, in our case) to query on. But in our case, we are handing it a list of strings.









In this case, the Flickr Block will output an array of arrays - the outer array having a cell for each city, and the contents of each cell being an array of Photo objects. The Slideshow Block unfortunately expects a single dimension array of Photo objects. Therefore, the visual wireup will not succeed in building a working Mashup. Fortunately, there is a way to manually intervene in the wireup by editing the Javascript code for the wireup.









This is done by selecting the "Advanced Settings" when configuring the source Block, which is the Flickr Block in this case. In this case, the code needs to be altered to create a single concatenated array instead of a multi-dimensional array.









popfly_demo_advancedWireup









The code below is the replacement code to use, which was derived from the default code provided by Popfly. Note the addition of the concat() function to assemble one contiguous array. Also, the call to call_slideshow() has been taken out of the loop and is now called just once using the single array.









data["flickr"] = new Array();
for (var i=0; i<data["d2d Demo Block"].length; i++)
{
try
{
data["flickr"] = data["flickr"].concat(
flickr.getPhotos(data["d2d Demo Block"][i].city, 1));

}
catch (ex)
{
environment.reportErrorInLoop(i, ex);
}
}
call_slideshow();








Testing and Deploying the Mashup









Now that the Mashup is complete, it is a simple matter to put it to the test. By clicking on the Preview button, you can immediately deploy the Mashup and make sure it works.









After, you can navigate to the Project Manager and "Share" your dev2dev Block and Mashup. This allows other users to see your work.









Another Look at this Demo









I have recorded a cheesy movie that shows how the UI works for assembling this Mashup. I made the movie very late at night, so please excuse the tired voiceover. You will need to download the free FLV player to view them.

















The Road Ahead









While cool, this demo may not have clear relevance to BEA customers. But hopefully you can see that we have established the basic working principles for building a data driven Mashup with Popfly. The next step: exposing enterprise data from a WebLogic Portal application into Popfly. Stay tuned!











No comments: