BEA WebLogic Portal + Swivel.com + Excel Spreadsheet = Enterprise Data Mashups

IT works hard to build web applications and other infrastructure to support the needs of information workers within the enterprise. However, it is commonly found that significant portions of the business are managed using ad hoc processes based around email and spreadsheets. While we have been tackling the issue of collaboration via email for years, we should also be looking at the role of spreadsheets and the amount of data locked up inside of them. In this blog entry I will explore an approach that allows spreadsheet data to be managed by IT, and then visually mashed up with other data sets. This solution combines the Content Management capabilities of WebLogic Portal with the data mashup features of Swivel.com.

NOTE: this blog entry was originally posted June 26th, 2007 on my previous blogging system (dev2dev.bea.com).

Mashups in the Enterprise

While mashups started in the consumer space, mashups are beginning to migrate into the enterprise space as well, as noted by technologists such as Dion Hinchliffe. The promise of cheap, quick and easy applications that tie together existing assets is appealing to both IT and the business. Enterprise mashups don't intend to solve every problem, but they do deliver rapid results with enough functionality to draw in the information workers. In previous blog entries and a webinar, I have been showing how to build enterprise mashups using web technologies. This entry will diverge and show how a non-web technology, namely the spreadsheet, can power an enterprise data mashup.

The enterprise data mashup will be built using two products that will be combined to help IT manage spreadsheets within the enterprise. First, I will use the Content Management capabilities of WebLogic Portal to manage the workflow of spreadsheets. Second, I will show you how a new tool called Swivel can extract valuable information out of spreadsheets, visualize it, and then combine in with other data to produce an enterprise data mashup. Finally, the output of Swivel will be fed back into WebLogic Portal Content Management, completing the lifecycle.

Managing File Based Assets using WebLogic Portal's Content Management

As a developer, you can probably appreciate the peril in not using a source code control system. Some of us have worked at places where the central code repository for an application is some guy's machine. The integration environment has a name, like "Larry" or "Bob", and is powered by bagels and coffee. With the widespread use of source code control systems, hopefully you haven't seen this in quite some time.

The same should apply to the non-developer community as well. Instead of source code repositories, file based documents should be managed with a content management system. Without belaboring the point, here is a list of features that make a proper content management system a critical component to deploy in the enterprise:

• Versioning - roll back that bad checkin from 3 AM
• Locking - ever had to hand merge two spreadsheets? A checkout mechanism can do wonders
• Auditing - nothing is more fun than playing the blame game when the train wreck happens
• Workflow - because who can keep track of who needs to sign off on that press release
• Search - philosophically, if a document cannot be found, does it really exist?
• Security - spreadsheets with salary information have a habit of just turning up on public shared volumes; this is bad
• Virtualization - there will never be only one, so WLP serves as a virtualization layer across multiple repositories
• Reuse - documents stored on a worker's laptop are not available for other uses

Because this blog entry is ultimately about enterprise mashups, the last point it especially relevent. Before you can pursue an enterprise mashup project for the enterprise data in these documents, they must be accessible. A content management system is the natural tool for that job, and therefore the first step in this type of project.

Using Swivel to Mashup Enterprise and Public Data

Swivel (www.swivel.com) is a startup funded by Minor Ventures in the bay area. Swivel offers a hosted service that aims to be the YouTube of spreadsheets. It allows the general public to upload spreadsheets from all corners of life, and provides mashup capabilities to combine those data sets to produce meaningful graphs. The intent of this service is to, in their words:

Swivel's mission is to liberate the world's data and make it useful so new insights can be discovered and shared.

What lies underneath that high-level statement is a rich web interface to a powerful data mashup engine. To get right to the point, the workflow to have in mind when using Swivel is as follows:

• Upload one of your spreadsheets
• View the graph of the data from that spreadsheet
• Find a public data set that may correlate to your data (world temperatures, Dow Jones average, GDP by nation, etc)
• Mash them together to create a combined graph - which may show interesting correlations

To take a short tour of the service, visit the Swivel Tour. Or, just visit the site and start clicking around, it is fairly intuitive. Here is a screen shot of the tool in action:

And the result of a mashup - this graph is measuring the sales of Avitek Air Conditioners against world temperature fluctuations. There appears to be some correlation:

The graph above is a sample enterprise data mashup. It shows how sensitive Avitek sales are to weather, which could affect projections and planning for the company's future. This seems valuable to me. In the real world, perhaps you will find some interesting trends that have not yet been identified in your company's performance. It's hard to really say until you give it a try.

But I know what your first question will be, because it always the first question no matter what the topic is. What about security? Who in their right mind would publish sensitive enterprise data to a hosted service? First, ever hear of Salesforce.com? Second, Swivel will offer a private data service in the near future that will allow you to keep your uploads private.

The remainder of this blog entry will cover an integration that I built that combines the Content Management features of WebLogic Portal with the data mashup capabilities of Swivel.

A Look at the WebLogic Portal and Swivel Integration

Note: please don't get the wrong idea about the status of this integration. I built this integration as a working demonstration of the combination of two great products. The code will be made available on dev2dev Code Share, and is not faked in any way. The integration uses only public BEA APIs, and those APIs are supported. But the integration itself is not an officially supported BEA product or solution.

The integration consists of a portlet that is deployed into WebLogic Portal. This portlet is custom designed to manage your Swivel graphs. It also provides an easy user interface to publish new spreadsheets to Swivel in order to create new graphs.

The integration is based on the following workflow:

• A spreadsheet is created in Excel
• The spreadsheet is managed in WLP Content Management, with features such as versioning, workflow, and security
• The Swivel Portlet is used to locate and publish the spreadsheet to Swivel as a new data set
• The data is mashed together with other Swivel data to create a new interesting graph
• The graph is published back to WLP Content Management, to complete the lifecycle

Watch, Learn, and Swivel!

Sometimes the best path towards understanding is to see a demo, and I think this is one of those times. To support this, I have recorded a video of the integration, showing all of the important parts and how they hang together.

Instructional Videos:

To view the demonstration videos, navigate to the video page, or click on one of the links below:

• Part 1: adding the Swivel portlet to WebLogic Portal
• Part 2: navigating the library of spreadsheets and graphs in WebLogic Portal
• Part 3: publishing a spreadsheet from WLP Content Management to Swivel
• Part 4: using the Swivel user interface to create compelling enterprise data mashups from spreadsheets
• Part 5: completing the round trip, bringing the Swivel graph back into WLP Content Management

Hosted Demo:

To try the integration hands-on, follow these steps:

• Navigate to the WLP 10 Playground at wlp.bea.com
• Create a user account
• Add a Page by clicking on the Add Page tab
• Add the Swivel portlet by clicking on the Add Portlet link and dragging the Swivel portlet to your page
• Start Swivelling!

CUFS - Measuring the Potential of Swivel

Swivel aims to empower information workers to build data mashups, but is it on target? I have covered other mashup tools in previous blog entries. Some of the tools I have blogged about are developer focused, and I also covered Schmapplets which is a mashup tool targeted at non-technical users. With the Schmapplet blog entry, I proposed 4 metrics for determining if a non-technical user will find success with a particular tool. Those 4 metrics, abbreviated as CUFS, are as follows along with how Swivel measures up to each:

• Comfortable: spreadsheets are in that sweet spot of comfort for many Information Workers
• Useful: it is up to you to decide: what can Swivel do for you? Please give it a try, and post a comment below
• Focused: it's a focused use case, which allows the UI to be specialized and avoids the need for coding
• Simple: as a hosted service it is pretty easy for both IT and the business. Just write a check and away you go!

To summarize, I think Swivel is on the right track towards building a workable mashup toolset for the information worker.

Last Mile for Swivel

Swivel is currently in Preview mode, which means not all of the intended features are in place. With that said, the site is already quite usable and can be employed for general use within the enteprise. However, a couple of additional features will be needed to allow Swivel to really take hold within the enterprise. Fortunately, they are already in plan to be delivered for the official launch of the service.

Two key features for enabling Swivel within the enterprise:

1. Private data
I covered this above; sensitive enterprise data cannot be publicly available. The revenue model for Swivel depends on getting this feature in, so I think it's a safe bet to assume it's coming soon.

2. Data API
The current integration requires the user to manually publish graphs back to WLP. It would be nice for this process to be automatic, which would be possible if a data API is available. I have been in communication with the Swivel team, and it sounds like this API is forthcoming.

Resources:

• dev2dev Code Share - the Swivel portlet for WLP will be posted on Code Share, I will update the comments below with the direct URL (as of 2009, CodeShare  is offline an unavailable)
• wlp.bea.com - WebLogic Portal Labs, demos of the latest thinking from the WLP engineering team
• Swivel - spreadsheet driven data mashups with Swivel
• My Articles and Blog - more writings on mashup tools and WebLogic Portal
• Swivel blog - see what the Swivel guys are up to
• James Dellow and Mark Bower blogged their opinion that Excel is in fact one of the earliest forms of a mashup tool

Technorati Tags: weblogicportal mashup

Comments from Original Post

• The Swivel Portlet for WebLogic Portal has been submitted to dev2dev Code Share. It can be found at this location.

  Posted by: plaird on June 26, 2007 at 11:32 AM

Want short SaaS/Cloud news updates throughout your day? Subscribe to the ondemand Twitter microfeed. http://twitter.com/ondemand