Laird OnDemand: June 2007

Tuesday, June 26, 2007

BEA WebLogic Portal + Swivel.com + Excel Spreadsheet = Enterprise Data Mashups

IT works hard to build web applications and other infrastructure to support the needs of information workers within the enterprise. However, it is commonly found that significant portions of the business are managed using ad hoc processes based around email and spreadsheets. While we have been tackling the issue of collaboration via email for years, we should also be looking at the role of spreadsheets and the amount of data locked up inside of them. In this blog entry I will explore an approach that allows spreadsheet data to be managed by IT, and then visually mashed up with other data sets. This solution combines the Content Management capabilities of WebLogic Portal with the data mashup features of Swivel.com.

NOTE: this blog entry was originally posted June 26th, 2007 on my previous blogging system (dev2dev.bea.com).

Mashups in the Enterprise

While mashups started in the consumer space, mashups are beginning to migrate into the enterprise space as well, as noted by technologists such as Dion Hinchliffe. The promise of cheap, quick and easy applications that tie together existing assets is appealing to both IT and the business. Enterprise mashups don't intend to solve every problem, but they do deliver rapid results with enough functionality to draw in the information workers. In previous blog entries and a webinar, I have been showing how to build enterprise mashups using web technologies. This entry will diverge and show how a non-web technology, namely the spreadsheet, can power an enterprise data mashup.

The enterprise data mashup will be built using two products that will be combined to help IT manage spreadsheets within the enterprise. First, I will use the Content Management capabilities of WebLogic Portal to manage the workflow of spreadsheets. Second, I will show you how a new tool called Swivel can extract valuable information out of spreadsheets, visualize it, and then combine in with other data to produce an enterprise data mashup. Finally, the output of Swivel will be fed back into WebLogic Portal Content Management, completing the lifecycle.

Managing File Based Assets using WebLogic Portal's Content Management

As a developer, you can probably appreciate the peril in not using a source code control system. Some of us have worked at places where the central code repository for an application is some guy's machine. The integration environment has a name, like "Larry" or "Bob", and is powered by bagels and coffee. With the widespread use of source code control systems, hopefully you haven't seen this in quite some time.

The same should apply to the non-developer community as well. Instead of source code repositories, file based documents should be managed with a content management system. Without belaboring the point, here is a list of features that make a proper content management system a critical component to deploy in the enterprise:

Versioning - roll back that bad checkin from 3 AM
Locking - ever had to hand merge two spreadsheets? A checkout mechanism can do wonders
Auditing - nothing is more fun than playing the blame game when the train wreck happens
Workflow - because who can keep track of who needs to sign off on that press release
Search - philosophically, if a document cannot be found, does it really exist?
Security - spreadsheets with salary information have a habit of just turning up on public shared volumes; this is bad
Virtualization - there will never be only one, so WLP serves as a virtualization layer across multiple repositories
Reuse - documents stored on a worker's laptop are not available for other uses

Because this blog entry is ultimately about enterprise mashups, the last point it especially relevent. Before you can pursue an enterprise mashup project for the enterprise data in these documents, they must be accessible. A content management system is the natural tool for that job, and therefore the first step in this type of project.

Using Swivel to Mashup Enterprise and Public Data

Swivel (www.swivel.com) is a startup funded by Minor Ventures in the bay area. Swivel offers a hosted service that aims to be the YouTube of spreadsheets. It allows the general public to upload spreadsheets from all corners of life, and provides mashup capabilities to combine those data sets to produce meaningful graphs. The intent of this service is to, in their words:

Swivel's mission is to liberate the world's data and make it useful so new insights can be discovered and shared.

What lies underneath that high-level statement is a rich web interface to a powerful data mashup engine. To get right to the point, the workflow to have in mind when using Swivel is as follows:

Upload one of your spreadsheets
View the graph of the data from that spreadsheet
Find a public data set that may correlate to your data (world temperatures, Dow Jones average, GDP by nation, etc)
Mash them together to create a combined graph - which may show interesting correlations

To take a short tour of the service, visit the Swivel Tour. Or, just visit the site and start clicking around, it is fairly intuitive. Here is a screen shot of the tool in action:

And the result of a mashup - this graph is measuring the sales of Avitek Air Conditioners against world temperature fluctuations. There appears to be some correlation:

The graph above is a sample enterprise data mashup. It shows how sensitive Avitek sales are to weather, which could affect projections and planning for the company's future. This seems valuable to me. In the real world, perhaps you will find some interesting trends that have not yet been identified in your company's performance. It's hard to really say until you give it a try.

But I know what your first question will be, because it always the first question no matter what the topic is. What about security? Who in their right mind would publish sensitive enterprise data to a hosted service? First, ever hear of Salesforce.com? Second, Swivel will offer a private data service in the near future that will allow you to keep your uploads private.

The remainder of this blog entry will cover an integration that I built that combines the Content Management features of WebLogic Portal with the data mashup capabilities of Swivel.

A Look at the WebLogic Portal and Swivel Integration

Note: please don't get the wrong idea about the status of this integration. I built this integration as a working demonstration of the combination of two great products. The code will be made available on dev2dev Code Share, and is not faked in any way. The integration uses only public BEA APIs, and those APIs are supported. But the integration itself is not an officially supported BEA product or solution.

The integration consists of a portlet that is deployed into WebLogic Portal. This portlet is custom designed to manage your Swivel graphs. It also provides an easy user interface to publish new spreadsheets to Swivel in order to create new graphs.

The integration is based on the following workflow:

A spreadsheet is created in Excel
The spreadsheet is managed in WLP Content Management, with features such as versioning, workflow, and security
The Swivel Portlet is used to locate and publish the spreadsheet to Swivel as a new data set
The data is mashed together with other Swivel data to create a new interesting graph
The graph is published back to WLP Content Management, to complete the lifecycle

Watch, Learn, and Swivel!

Sometimes the best path towards understanding is to see a demo, and I think this is one of those times. To support this, I have recorded a video of the integration, showing all of the important parts and how they hang together.

Instructional Videos:

To view the demonstration videos, navigate to the video page, or click on one of the links below:

Part 1: adding the Swivel portlet to WebLogic Portal
Part 2: navigating the library of spreadsheets and graphs in WebLogic Portal
Part 3: publishing a spreadsheet from WLP Content Management to Swivel
Part 4: using the Swivel user interface to create compelling enterprise data mashups from spreadsheets
Part 5: completing the round trip, bringing the Swivel graph back into WLP Content Management

Hosted Demo:

To try the integration hands-on, follow these steps:

Navigate to the WLP 10 Playground at wlp.bea.com
Create a user account
Add a Page by clicking on the Add Page tab
Add the Swivel portlet by clicking on the Add Portlet link and dragging the Swivel portlet to your page
Start Swivelling!

CUFS - Measuring the Potential of Swivel

Swivel aims to empower information workers to build data mashups, but is it on target? I have covered other mashup tools in previous blog entries. Some of the tools I have blogged about are developer focused, and I also covered Schmapplets which is a mashup tool targeted at non-technical users. With the Schmapplet blog entry, I proposed 4 metrics for determining if a non-technical user will find success with a particular tool. Those 4 metrics, abbreviated as CUFS, are as follows along with how Swivel measures up to each:

Comfortable: spreadsheets are in that sweet spot of comfort for many Information Workers
Useful: it is up to you to decide: what can Swivel do for you? Please give it a try, and post a comment below
Focused: it's a focused use case, which allows the UI to be specialized and avoids the need for coding
Simple: as a hosted service it is pretty easy for both IT and the business. Just write a check and away you go!

To summarize, I think Swivel is on the right track towards building a workable mashup toolset for the information worker.

Last Mile for Swivel

Swivel is currently in Preview mode, which means not all of the intended features are in place. With that said, the site is already quite usable and can be employed for general use within the enteprise. However, a couple of additional features will be needed to allow Swivel to really take hold within the enterprise. Fortunately, they are already in plan to be delivered for the official launch of the service.

Two key features for enabling Swivel within the enterprise:

1. Private data
I covered this above; sensitive enterprise data cannot be publicly available. The revenue model for Swivel depends on getting this feature in, so I think it's a safe bet to assume it's coming soon.

2. Data API
The current integration requires the user to manually publish graphs back to WLP. It would be nice for this process to be automatic, which would be possible if a data API is available. I have been in communication with the Swivel team, and it sounds like this API is forthcoming.

Resources:

dev2dev Code Share - the Swivel portlet for WLP will be posted on Code Share, I will update the comments below with the direct URL (as of 2009, CodeShare is offline an unavailable)
wlp.bea.com - WebLogic Portal Labs, demos of the latest thinking from the WLP engineering team
Swivel - spreadsheet driven data mashups with Swivel
My Articles and Blog - more writings on mashup tools and WebLogic Portal
Swivel blog - see what the Swivel guys are up to
James Dellow and Mark Bower blogged their opinion that Excel is in fact one of the earliest forms of a mashup tool

Technorati Tags: weblogicportal mashup

Comments from Original Post

The Swivel Portlet for WebLogic Portal has been submitted to dev2dev Code Share. It can be found at this location.

Posted by: plaird on June 26, 2007 at 11:32 AM

Monday, June 11, 2007

Google Maps Mashup Tool for Non-Technical People - schmapplets.com

If your mashup doesn't use Google Maps, do you really have a mashup? Of course, but Google Maps certainly is the accessory of choice for most mashup implementations. I wrote an article for dev2dev about how a programmer can easily create a Google Maps mashup using Ajax, JSON and the Google Maps API. A tool called Schmapplets has taken a different approach - it provides non-technical authors a rich tool to build a Google Maps mashup. This has no direct relevance to BEA, but I am blogging about it because it is an interesting approach to mashups from the consumer space. As we look at bringing mashups into the enterprise, we need to have tools to allow non-IT people to construct their own applications. What can be learned from Schmapplets?

NOTE: this blog entry was originally posted June 11th, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

Check your Programming Skills at the Door

If you are a programmer, building a mashup is a task that is well in reach. I have covered several approaches in my blogs and articles, and the dev2dev Tech Days seminars covered this topic in detail. Here are a few resources to get you up and running quickly:

Ajax-Powered Google Maps Mashup Tutorial - the Hello World mashup for a progammer, the place to start
Mashups with Microsoft Popfly - an IDE of sorts for assembling a mashup. Still requires technical skills.
dev2dev Tech Days "Mashup the Enterprise" - a traveling seminar that covers this topic in detail

But this blog entry is not about the programmer powered mashup. This blog entry looks at a tool that enables non-technical users to assemble a mashup.

The Mom Test for the Consumer Market

What do I mean by a non-technical user? I don't mean someone who has little experience with a computer and the web (that's the Grandma Test). We all have someone in the family who can use a computer fairly well, but are definately not programmers or techies. These people can read email, surf the web, install new software, upload digital pictures, and will use Windows OS. In my family, that's Mom. And so I will use the term Mom Test when evaluating how well tools intended for the consumer market address this skill set.

What are Schmapplets?

Schmapplets is a tool for creating a very specific mashup. It's so specific, it is debatable whether its an actual mashup or just a good integration with that universal mashup litmus test - does it use Google Maps? We won't dwell on that question because it isn't important, we will call it a mashup and take a good look at what they have done. The mashups that Schmapplets creates have these characteristics:

The mashup is centered around the Google Map for a single city
The mashup creator enters a set of addresses, perhaps indicating favorite coffee shops, stores or restaurants
For each address, the creator adds a description, photos, and a rating.
The Schmapplets team has already plotted numerous hotels, restaurants, shops and tourist attractions into the location list

In its implementation, Schmapplets has chosen to build a fat-client tool for the mashup creator. This tool is the data entry platform. With it, the mashup creator enters in a series of addresses, along with pictures and descriptions, which are plotted on the map. Once created, the mashup creator publishes the mashup which is uploaded by the tool to the Schmapplets web servers. Users of the mashup view the mashup in a browser, like most other mashup implementations.

View of the mashup creator tool:

View of the the finished mashup, rendering in a browser:

Follow these links to learn more about Schmapplets:

My Schmapplet for Denver - live link to the sample Schmapplet mashup shown above
Schmapplet creation video - video showing me building the Schmapplet mashup shown above
Schmapplet Home - links to existing Schmapplets, link to download the application
Schmap User Video - video demo of the user features of the Schmap application

Schmapplets Passes the Mom Test

While I haven't done a field test to prove it, I suspect that Schmapplets is fit for Mom. Er, with a few caveats:

Someone would need to explain what a mashup is, and why she should build one
Someone would need to point her to this solution - she would not find it on her own
Someone would need to convince her that she could do it.
I think the Beta user interface needs a few usability tweaks

Lessons about Mashup Construction Tools

Before trying out Schmapplets, I was working with Microsoft Popfly which is another visual mashup construction tool. Popfly is an interesting tool for developers, but I don't see Popfly as something to roll out to Mom. I feel that Popfly currently requires too much technical know-how to make a meaningful mashup.

So what did Schmapplets do right to pass the Mom Test? I think it is the following 4 characteristics, abbreviated as CUFS:

Comfortable: Google Maps and digital photos - those things are in the comfort zone of most consumer web users like Mom
Useful: one of the examples is plotting a wedding event. Mom could find value in doing something like that
Focused: Its a very focused use case, which allows the UI to be specialized and avoids the need for coding
Simple: no need to configure a Google Maps API key, no need to understand Geocoding, no need to understand web server deployment

Drop the Mom Test for the Enterprise Space

The Mom Test doesn't apply to the enterprise - we can assume our Information Workers have more technical skills. Beyond what mom can do, well trained Information Workers will be skillful with Excel, RSS, wikis, blogs, tagging, and perhaps some SQL. But the underlying principles of the CUFS acronym above still applies in the enterprise, only the user community changes. CUFS is relative to the intended user, and so a tool that fails in this regard in the consumer space may in fact be sufficient in the enterprise.

In future blog entries, I will look at more mashup tools aimed at non-technical users. As we look at other solutions, we will refer back to the 4 principles above. The ultimate goal is for IT to provide tools to Information Workers that allow them to build their own applications. Before this can be done, we must understand what makes these tools successful.

Monday, June 4, 2007

BEA WebLogic + Microsoft Popfly = Enterprise Mashups

In my last blog entry, I showed how to build a data driven Mashup using Microsoft Popfly. That exercise was a stepping stone to this blog entry: I will show how WebLogic Portal can surface enterprise data into a Popfly Mashup. In particular, I will show how images located in a WebLogic Portal Content Management repository can be brought into a Popfly project. This demo shows how BEA products can be the key ingredient in your enterprise mashup projects.

NOTE: this blog entry was originally posted June 4th, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

NOTE: as of 2009, Microsoft has announced that it is terminating support for Popfly.

Building a Mashup with Microsoft Popfly

My last blog entry covered Microsoft Popfly, which is a new mashup building tool from Microsoft. Instead of covering that topic again, I will simply point you to that blog entry if you need some more context.

Reference: previous blog entry

Enter BEA WebLogic Portal and Enterprise Mashups

We have been showing customers how WebLogic Portal can be a key component in your enterprise mashup project. You have hopefully seen how WLP can consume Google Gadgets and other external services to create portal based mashups. WLP can also work in reverse - WLP can produce its portlets onto other, non-portal web pages and mashups like iGoogle. For a quick recap of these capabilities please consult these resources:

Webinar: Enterprise Mashups with WebLogic Portal 9.2 - I presented an introduction to the Enterprise Mashup space
WebLogic Portlet Publishing - an article discussing how WLP portlets can be produced onto non-portal web pages
Add a WLP Portlet to iGoogle - WLP portlets can be exposed as Google Gadgets
WebLogic Portal Playground - home of the WLP engineering labs, showing some online demos of WLP+Google Gadgets

In this blog I will demo WLP participating in a Mashup in a different way: it will be a data feed producer for a mashup. Specifically, I will show you how you can surface images from the WLP Content Respository as a JSON data feed. That data feed will in turn be combined in a Popfly mashup using the technique I covered in the previous blog entry.

This helps to show how WebLogic Portal can be the gateway from the enterprise into the consumer mashup space. WLP provides not only a UI framework for producing portlets (as in the iGoogle case), but it is also a virtualization layer for enterprise data services. The diagram below shows this idea. Notice that WLP provides virtualization capabilities for User Profile and Content Management. Once virtualized, WLP can produce profile and content REST services with ease.

Before I show you how to build it, take a quick look at the WLP+Popfly live demo hosted here. You will be prompted to download the Microsoft Silverlight browser plugin - you must accept to see the demo. Notice how the Photosphere is animating with a set of pictures? Those pictures are being retrieved from the WLP instance at http://wlp.bea.com. Note: the Photosphere Block is computationally expensive, so your CPU usage will spike.

Curious to know how I built it? Follow the steps below to build your own enterprise mashup with WebLogic Portal and Microsoft Popfly!

Step 1: Provisioning the WLP Content Repository

The first thing to do is provision the WLP Content Repository with images. In an enterprise, this will be done by content authors who are using Content Management to store the artifacts that they create on a daily basis. In my case, I loaded a number of pictures I took at the Smithsonian Air and Space Museum in Washington DC into the content repository on wlp.bea.com. I made sure that anonymous users are entitled to see the content, and that all images are in 'Published' status.

Next, I modified the default image content type by adding a multivalued text property called "tag". Then, with each picture, I added a few tag words that describe the picture, like "plane", "propeller", "jet", "warplane", etc. This allows us to quickly query the images by keyword.

Step 2: Building a REST-style service for the WLP Content Repository

In the first blog post, I showed how to build a Popfly Data Block to consume a data service that returns JSON. We are going to use that same pattern in this mashup. To implement this, we need to deploy a REST service for WLP CM.

I turned to dev2dev CodeShare to quickly get my service up and running. There is a download available (that includes sample code for working with the WLP CM repository). I lifted the code from the Search sample to build my service. Also, Tejas Joshi coincidently just blogged about how to create a CM service just a few hours ago. My REST service does the following:

Queries the WLP CM system for all images
Optionally, the service accepts a tag to search for to scope the result set, like "plane"
The result set is composed as a JSON response, including the URL for each image
The client can then deserialize the JSON and extract the URLS for the images

The servlet I created for the service is implemented with a .JSP, not a .java file, to allow for easier distribution for others who want to use it. For a quick demo of the service, try the hosted service:

I won't post the code for the JSP here. But I will see about submitting it to CodeShare. (as of 2009, CodeShare is no longer available)

Step 3: Building the Popfly Data Block for Consuming the JSON Service

Now that we have our JSON service, we need the associated data block in Popfly. Since I have already covered how to build a JSON data block in the previous blog entry, I will just show the code and move on. The code snippets below show how I defined the JSON consumer block. I have shared my block on Popfly with the title Air and Space Museum Images, so you should see it in your Block palette.

Block Descriptor

<?xml version="1.0" encoding="utf-8"?>
<block class="AirAndSpaceMuseum">
  <providerName>AirAndSpaceMuseum</providerName>
  <providerUrl>
    http://ajaxmashup.googlepages.com/
  </providerUrl>
  <providerLogoUrl>
    http://wlp.bea.com/images/bea_logo.gif
  </providerLogoUrl>
  <blockIconUrl>
    http://wlp.bea.com/images/bea_logo.gif
  </blockIconUrl>
  <operations>
    <operation name="getImages">
      <description>
        Images from the Smithsonian Air and Space Museum, 
        hosted on BEA WebLogic Portal.
      </description>
      <inputs />
      <outputs>
        <output isArray="true" type="custom" object="Photo" />
      </outputs>
    </operation>
  </operations>
  <objects>
    <object name="Photo">
      <field name="url" type="imageUrl" 
                isArray="false" />
      <field name="thumbnailUrl" type="thumbnailImageUrl" 
                isArray="false" />
      <field name="id" type="id" 
                isArray="false" />
      <field name="owner" type="userName" 
                isArray="false" />
      <field name="title" type="title" 
                isArray="false" />
      <field name="longitude" type="longitude" 
                isArray="false" />
      <field name="latitude" type="latitude" 
                isArray="false" />
    </object>
  </objects>
</block>

Block Code Implementation

// create our namespace for the Block
function AirAndSpaceMuseum() { }

AirAndSpaceMuseum.prototype.getImages = function () {  
	// set the URL to our JSON service
	// hosted in CM on wlp.bea.com
	// tagged with the word 'plane' to scope it 
    url = "http://wlp.bea.com/dvt/"+
          "api/content/images.jsp?tag=plane";

	// get the JSON list of locations
    var resultJSON = environment.getText(url);
    var responseObject = eval("(" + resultJSON + ")");
    
    // convert to the Array type we want
    var resultsArray = new Array();
	for(i=0;i < responseObject.images.image.length; i++) {
		resultsArray[i] = new Photo(
			responseObject.images.image[i].id, 
			"Peter Laird", 
			responseObject.images.image[i].image_name,
			responseObject.images.image[i].url, 
			responseObject.images.image[i].url,
			0, 0);
	}		
    return  resultsArray;  
};

// Define the output payload, must match the XML 
function Photo(id, ownerName, title, url, mediumUrl, 
               lon, lat)
{
    this.id = id;
    this.owner = ownerName;
    this.title = title;
    this.url = mediumUrl;
    this.thumbnailUrl = url;
    this.longitude = lon;
    this.latitude = lat;

    this.toString = function()
    {
        return "<table><tr><td><a href='" + 
          environment.escapeQuotes( this.url ) + 
          "' target='_blank'><img src='" + 
          environment.escapeQuotes( this.thumbnailUrl ) + 
          "' title='" + 
          environment.escapeQuotes( this.title ) + 
          ", ID: " + 
          environment.escapeQuotes( this.id ) + 
          "'/></a></td><tr><td>" + 
          this.latitude + ", " + this.longitude + 
          "</td></tr></table>";
    }
}

Step 4: Assembling the Mashup

As we saw in the previous blog entry, assembling the mashup is easily done. In this case, we connect the output of our Data Block to the input of the Photosphere Block. Without any coding, the Mashup is complete. Click the following link to see it in action:

WebLogic Portal + Popfly Mashup

This image shows the linkage of the mashup in the Popfly workspace:

Taking a Step Back

Now that you have invested yourself in reading this tutorial, I will state without hesitation that this specific mashup has absolutely no value to the enterprise. Are your customers or employees really going to be entertained by images from your enterprise rotating in a Photosphere? Is it going to improve their productivity? Doubtful. To be a true mashup, shouldn't two services (like JSON+Flickr) be combined like in my previous blog? Probably.

So what was the point of building this mashup?!?

My intent was to show how easily it is to surface data from the enterprise into Popfly using WebLogic Portal. Mashups crave easily consumable data - which is exactly what WLP can provide. Finding a useful purpose for that data once it is in the Popfly environment is entirely up to you.

BEA WebLogic Portal + Microsoft Popfly = Enterprise Mashups

Happy mashing!

The simple CM REST service used above was submitted as a code sample here:

Simple CM REST CodeShare Project (as of 2009, CodeShare is no longer available)

Technorati Tags: mashup,weblogicportal,popfly,rest,json

Friday, June 1, 2007

Building a Data Driven Mashup with Microsoft Popfly

Regular readers of my blog know that I have been looking at Enterprise Mashups for a while now. Because I am an engineer on the WebLogic Portal product, I have focused on solutions involving that product. However, in this blog I will cover a new product called Popfly from Microsoft that takes an interesting approach to Mashups. I will walk you through how to build a Mashup in Popfly step by step, including the programming of a data service. The Mashup will display a slideshow that contains picture from Flickr of each of the US cities that dev2dev Tech Days visited.

NOTE: this blog entry was originally posted June 1st, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

The World of Enterprise Mashups

A common question people ask is: "what's a Mashup?". That question is usually followed by another question: "why would I want one in my enterprise?" We are covering this topic in detail with a traveling road show called dev2dev Tech Days. The topic of the seminar is "Mashup the Enterprise", and we answer these questions. In the last few days, you have seen others blogging about that event on dev2dev such as James Bayer and Sean Boiling.

For those unable to attend the seminar, I can offer a few pointers to help get you started.

AP News - my favorite sample mashup from the web; it uses an AP News wire feed, Yahoo Geocoding, and Google Maps
dev2dev Tech Days 2007 - Mashup the Enterprise - register for a FREE seminar from BEA, available in 45 cities
Webinar: Enterprise Mashups with WebLogic Portal 9.2 - I presented an introduction to the Enterprise Mashup space
Building a Simple Ajax Powered Google Maps Mashup - a tutorial to build a simple Hello World mashup example
Picasa and Google Maps Mashup - James Bayer shows how to use Workshop to build a Picasa Mashup

BEA and Enterprise Mashups

Before we head into Microsoft land, I need to quickly cover how BEA is addressing the Mashup space. BEA offers several approaches to building a Mashup. The first approach is to use a portal product like WebLogic Portal or AquaLogic Portal. A second approach is a new product that will launch soon called AquaLogic Ensemble that specifically targets the Mashup space.

WebLogic Portal Playground - home of the WLP engineering labs, showing some online demos of WLP+Google Gadgets
http://en.terpri.se - launch site of 3 new Web 2.0 products from BEA, including Ensemble

Enter Microsoft Popfly

Popfly is a new offering from Microsoft, currently just out the door as an Alpha release. The best way to get a feel for Popfly is to watch their 15 minute screencast here. Interesting eh? Let me summarize:

Popfly is a web based composition engine for assembling components into workable Mashups
The tooling intends to allow non-programmers to accomplish this
A Popfly Mashup is composed of "Blocks" which can either be data driven or presentation driven

It may seem odd for a BEA employee to cover a Microsoft product. But it all makes sense - Mashups are similar to Service Oriented Architectures (SOA) in that they demand interop between heterogeneous systems. While this particular blog entry will not show BEA WebLogic Portal integrating with Popfly, you should expect that I am heading in that direction. I have been showing how WLP portlets can be surfaced in iGoogle for nearly a year now, and customers have loved to see how WLP interops with other Mashups. Next, in a follow up blog I will show artifacts from WLP Content Management being surfaced in Popfly. Cool! But before we can show WLP+Popfly, I need to first introduce you to Popfly development which is what this blog entry is all about.

Your First Glimpse at a Popfly Mashup

To give you a better idea, see the iframe below which contains the completed demo that I will explain in this blog. I know it doesn't look like much, but I will show how this widget was assembled using 3 Popfly Blocks, including one that I built myself.

PJL revision:
Readers complained that just viewing my blog page required installing the Silverlight plugin because of the iframe. I am therefore just linking it here now. dev2dev Tech Days Popfly Mashup
/PJL revision:

Warning: the Flickr Block is presenting a random picture associated with each city. All apologies if a picture turns up that is not office-friendly. I just saw a picture of a guy in underwear. Yikes.

Here is what it is doing:

There already exists a hosted data feed (returning JSON) listing some of the locations that dev2dev Tech Days will visit this year
I built a Popfly Data Block that consumes that JSON feed, bringing the list of cities into my Popfly project
The dev2dev Data Block is wired into a pre-built Flickr Data Block, so that for each city, a single picture is retrieved using the city name as the image query
Finally, the Flickr Data Block is wired into a pre-built Slideshow Presentation Block to display the pictures

Starting with Popfly

Currently, Popfly is in a managed Alpha release. You must apply for a Popfly account and then wait. I applied last week and just got mine today, so perhaps it will go quickly for you.

Once you get your account, you will be able to start building Mashups. A developers kit is available for download once you have a login. There are a few hundred Blocks already available for your use, with more being added daily. You will also need to download a new browser plugin called Silverlight to use the toolset (you will be prompted). For this demo, you will also need to configure a Flickr API Key, available here.

Step 1: Creating the Data Service Block

As I mentioned previously, I assembled an Ajax powered Google Maps Mashup for my dev2dev article. The demo is hosted at http://ajaxmashup.googlepages.com, and includes a data feed that returns a list of dev2dev Tech Days cities. That data feed returns a format called JSON. If JSON is unfamiliar to you, I will add a link to my article that explains it as soon as it is posted.

To consume the JSON feed in Popfly, I first had to create a new Block project. A Block once again is a Mashup component that either works with data or generates presentation (a single Block can do both, but it shouldn't). My Block will retrieve the JSON, translate it into the appropriate Javascript object, and pass the object to the next Block. From what I have seen, Blocks are written in Javascript, although the intro screencast shows some interesting Visual Basic use cases.

Once a the Block project has been created, you must supply two files:

Block descriptor: an XML file that describes your block; this information is used to power the visual wireup capabilities
Javascript file: a single Javascript file that includes your Block implementation

The Block descriptor is self explanatory, and well documented. It enumerates the inputs and outputs of your Block so that a non-programmer can wireup the Block into a Mashup. Popfly defines a set of standard data types, including url, userName, title, etc. so that the tool can make intelligent default guesses on how to wire two Blocks together. The following is our Block descriptor. Notice how we have defined a single operation (method) on this Block: getLocations. This operation has no inputs, and its output is an array of Dev2DevLocation objects.

<?xml version="1.0" encoding="utf-8"?>
<block class="Dev2DevDays">
  <providerName>Dev2DevDays</providerName>
  <providerUrl>
    http://ajaxmashup.googlepages.com/
  </providerUrl>
  <providerLogoUrl>
    http://wlp.bea.com/images/bea_logo.gif
  </providerLogoUrl>
  <blockIconUrl>
    http://wlp.bea.com/images/bea_logo.gif
  </blockIconUrl>
  <operations>
    <operation name="getLocations">
      <description>
        Get the locations of BEA dev2dev Tech Days 
        2007, "Mashup the Enterprise".
      </description>
      <inputs />
      <outputs>
        <output isArray="true" type="custom" 
                   object="Dev2DevLocation" />
      </outputs>
    </operation>
  </operations>
  <objects>
    <object name="Dev2DevLocation">
    <field name="city" type="city" isArray="false"/>
    <field name="venue" type="name" isArray="false" />
    <field name="address" type="location" 
    	isArray="false"/>
    <field name="date" type="date" isArray="false"/>
    </object>
  </objects>
</block>

The second piece of our Block is the implementation itself. I implemented this in Javascript, and it is quite simple. It is essentially a function call that marshals the JSON feed into an object array that can be used in Popfly.

// create our namespace for the Block
function Dev2DevDays() { }

Dev2DevDays.prototype.getLocations = function () {  

// set the URL to our JSON service, hosted on googlepages.com
url = "http://ajaxmashup.googlepages.com/getD2DSites.html";

// get the JSON list of locations
var resultJSON = environment.getText(url);
var responseObject = eval("(" + resultJSON + ")");

// convert to the Array type we want
var resultsArray = new Array();
for(i=0;i < responseObject.locations.location.length; i++) {
	resultsArray[i] = new Dev2DevLocation(
		responseObject.locations.location[i].city, 
		responseObject.locations.location[i].location,
		responseObject.locations.location[i].address, 
		responseObject.locations.location[i].date);
}		
return  resultsArray;  
};

// Define the output payload, must match the XML descriptor
function Dev2DevLocation(city, venue, address, date)
{
    this.city = city;
    this.venue = venue;
    this.address = address;
    this.date = date;
}
Dev2DevLocation.prototype.toString = function(){
    return this.venue+", "+this.address;
};

For those that have implemented Mashups in a browser, you are aware that browser security would normally restrict access to the googlepages.com JSON feed. The Single Origin Policy prevents browsers from crossing multiple network domains. Popfly provides a workaround: it includes a server side proxy accessible via the getText() or getXml() Javascript calls.

Wiring the dev2dev Block to the Flickr Block

The next step is to take the output of the dev2dev Data Block, which is an array of locations, and wire it into the Flickr Block. The Flickr block is provided out of the box by Popfly, so you do not need to build it. The Flickr block needs a query string to retrieve a photo. In this case, I wire that using the "city" field from the JSON feed. This can be done using the visual UI.

Wiring the Flickr Block to the Slideshow Block

The last step is to take the output of the Flickr Data Block, which is an array of Photo objects, and wire it into the Slideshow Block. This can be done visually by just connecting the two blocks, as the default wireup will work just fine. Now the Mashup is done, how is easy is that!

Workaround for Popfly Alpha

OK, so for now it's not quite as easy as I said. I found I need one workaround to make this all hang together. The Flickr Block and Slideshow Block have a slight mismatch for their data contract because of the way I built the Mashup. Ordinarily, the Flickr Block is given a single string (city, in our case) to query on. But in our case, we are handing it a list of strings.

In this case, the Flickr Block will output an array of arrays - the outer array having a cell for each city, and the contents of each cell being an array of Photo objects. The Slideshow Block unfortunately expects a single dimension array of Photo objects. Therefore, the visual wireup will not succeed in building a working Mashup. Fortunately, there is a way to manually intervene in the wireup by editing the Javascript code for the wireup.

This is done by selecting the "Advanced Settings" when configuring the source Block, which is the Flickr Block in this case. In this case, the code needs to be altered to create a single concatenated array instead of a multi-dimensional array.

The code below is the replacement code to use, which was derived from the default code provided by Popfly. Note the addition of the concat() function to assemble one contiguous array. Also, the call to call_slideshow() has been taken out of the loop and is now called just once using the single array.

data["flickr"] = new Array();
for (var i=0; i<data["d2d Demo Block"].length; i++)
{
     try
     {
       data["flickr"] = data["flickr"].concat(
         flickr.getPhotos(data["d2d Demo Block"][i].city, 1));

     }
     catch (ex)
     {
       environment.reportErrorInLoop(i, ex);
     }
}
call_slideshow();

Testing and Deploying the Mashup

Now that the Mashup is complete, it is a simple matter to put it to the test. By clicking on the Preview button, you can immediately deploy the Mashup and make sure it works.

After, you can navigate to the Project Manager and "Share" your dev2dev Block and Mashup. This allows other users to see your work.

Another Look at this Demo

I have recorded a cheesy movie that shows how the UI works for assembling this Mashup. I made the movie very late at night, so please excuse the tired voiceover. You will need to download the free FLV player to view them.

Part 1: Building the JSON Data Block

Part 2: Assembling the Mashup

The Road Ahead

While cool, this demo may not have clear relevance to BEA customers. But hopefully you can see that we have established the basic working principles for building a data driven Mashup with Popfly. The next step: exposing enterprise data from a WebLogic Portal application into Popfly. Stay tuned!

Technorati Tags: popfly,mashup,weblogicportal,json,REST

Laird OnDemand