Enterprise Greasemonkey: Solving the Greasemonkey Script Versioning Problem

As I have covered in previous blogs, Greasemonkey is a powerful client-side tool for augmenting web sites or creating mashups. It consists of a Firefox plugin coupled with scripts that are written and installed to modify web pages as they return from a web server. When this tool is brought into the enterprise, IT must sort out issues like security and deployment. I covered security in my last post, and in this installment I will look at how IT can solve deployment and versioning issues for Greasemonkey scripts.

NOTE: this blog entry was originally posted August 19th, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

This is part 5 of a series of blog entries on Greasemonkey mashups.

• Blog 1: More Mashups: Using Greasemonkey to Weave New Features into Web Sites
• Blog 2: Building a Greasemonkey Mashup Tutorial
• Blog 3: Greasemonkey in the Enterprise: When is GM the Right Tool for IT?
• Blog 4: Beware of Greasemonkey's Inverted Security Model
• Blog 5: Solving the Greasemonkey Script Versioning Problem

The Need for Greasemonkey Script Deployment and Versioning

For this blog entry, imagine a web application in the enterprise with these characteristics:

• It is a closed source packaged application, purchased from an external vendor.
• It is used by a modest sized group of users, all of which are comfortable with computers and web browsing
• There are a couple of missing features that dramatically hurt productivity
• Those features can be easily woven into the application using a Greasemonkey script

Now, any one of those affected users could write the Greasemonkey script without involving IT. This script could then be passed around via email or a shared drive amongst colleagues. But there are few problems with this ad hoc approach to Greasemonkey:

• As the script evolves, distribution and version management of the script will be difficult.
• As new users are introduced to the web application, there is no formal process for getting them going with the script.
• GM scripts are like .exe's - they can be host to malicious code. Unmanaged distribution could open the door for a hacker.
• It is IT's mission to be the solution provider - IT should provide the Greasemonkey script.

Clearly, while IT can let the user community fend for themselves on this project, it would be far better for IT to own the solution. However, as always, unless IT can provide a solution quickly the users are apt to go with the ad hoc approach. Hopefully this blog entry will help you rapidly develop a sustainable solution, and add Greasemonkey to the list of official solutions for your enteprise.

Solving the Deployment and Versioning Issues for Greasemonkey Scripts

The rest of this blog entry discusses a solution that can provide manageability around Greasemonkey in the enterprise. It can solve the issues listed above, particularly the issues around deployment and versioning of a Greasemonkey script. This solution is by no means the only way to address these issues, but hopefully you will find it satisfies your requirements. The following describes characteristics of the solution:

• It specifies a new landing page, or Gateway page, for the web application that users will hit before going into the app
• This page can contain whatever you want, but it also has elements to help determine if the GM script is installed and up to date
• The solution enforces the use of the Greasemonkey script (though it could be easily changed to make it only optional)
• It provides version checking
• It does not require any changes to the closed source web application
• It is not foolproof - if the user avoids the Gateway page using a bookmark the enforcement is not run

The graphic below shows how the Gateway page fronts the web application to make sure the user has the proper Greasemonkey install. Ideally, the Gatekeeper page would be integrated into the web application's login page so a user could not bookmark their way around the enforcement. If the browser is not properly configured, the Gateway page obscures the Enter the Site button until the problem is corrected. Also note that the proper version of the Greasemonkey script should be located on a web server defined and managed by IT so the Gatekeeper page can direct the user to it if necessary.

Versioning is implemented by including a div element on the Gateway page with the version number of the script that users should be using.

<div id="gm-script-ver-415" />

The Greasemonkey script looks for the div tag with it's version number. If it does not find it, it will not make the Enter the Site link visible.

 

The Solution Step by Step

Check 1: Is the browser Firefox? If not, a message discloses the problem.

Check 2: Is Greasemonkey and the script installed? If not, the page offers the download links for both.

Check 3: Is the Greasemonkey script the current version? If not, the page offers the download link for the latest version.

Success: The browser is properly configured. The user is allowed to continue if all checks pass.

Implementing the Solution

I didn't include the code in this blog but it is attached here. The code is fairly short, and so you should be able to repurpose it for your needs quickly. Also, for illustration, I have deployed an online demo.

Online Demo

I am hoping others will provide alternative solutions - please speak up if you have built one. The solution I built is passive - it requires the user to navigate across the Gateway page. It is easy to imagine a more active solution, one that would use an XmlHttpRequest to periodically phone home to see if an updated script has been published. That type of solution would need to cache the answer, otherwise it's impact to the network could be substantial. I would think that the GM preference store could be used as a cache.

Greasemonkey Versioning on the Web:

There are others who have talked about the versioning issue. Here are some links:

GMVC (Greasemonkey Version Control)








"The objective of this project is to create a database-driven repository of GreaseMonkey Userscripts which allows for the archiving of previous versions, the release of production versions and the limited released of alpha and beta test versions through the same site."

AllPeers Blog Entry








"A first step would be to give web authors at least the option of making their sites script-friendly. This would involve placing a special header on their webpages that can be tied to a specific version of a Greasemonkey script, like so:




<head> <meta greasemonkey:somescript.js="1.2"/> </head> "

Jeff Barr's Blog








"At some point I could see the need for what we could call "Enterprise Greasemonkey," a tightly packaged version of Greasemonkey with a number of scripts installed and locked in to place. This would give system administrators the ability to deploy specialized versions of Firefox to users who don't need to know how to install, customize, or disable Greasemonkey scripts."

Resources:

• Online Demo
  
  
  
  
  
  



• Code Download
  
  
  
  
  
  



• Greasemonkey Home
  
  
  
  
  
  



• Dive Into Greasemonkey - a great guide to GM
  
  
  

Technorati Tags: Greasemonkey,Enterprise

Want short SaaS/Cloud news updates throughout your day? Subscribe to the ondemand Twitter microfeed. http://twitter.com/ondemand

Enterprise Greasemonkey: Beware of Greasemonkey's Inverted Security Model

I have been blogging about the power and usefulness of a tool called Greasemonkey. I have also explained when Greasemonkey is a good tool for use within the enterprise. But before you rush off and use it in your next IT project, you must understand an important issue with the tool. That issue specifically is Greasemonkey's security model, which was designed to protect benevolent scripts from evil websites. But from IT's point of view, the Greasemonkey security model is inverted - it trusts the wrong people. Within the enterprise, the IT hosted websites are trusted. It is the Greasemonkey scripts that may be evil. This blog entry explains the issue, and discusses some coping strategies.

NOTE: this blog entry was originally posted August 18th, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

This is part 4 of a series of blog entries on Greasemonkey mashups.

• Blog 1: More Mashups: Using Greasemonkey to Weave New Features into Web Sites
• Blog 2: Building a Greasemonkey Mashup Tutorial
• Blog 3: Greasemonkey in the Enterprise: When is GM the Right Tool for IT?
• Blog 4: Beware of Greasemonkey's Inverted Security Model
• Blog 5: Solving the Greasemonkey Script Versioning Problem

The Power of Greasemonkey

A discussion of the Greasemonkey (GM) security model needs to start first with a discussion of the power of Greasemonkey. Greasemonkey scripts have enormous power, and so a security model is key to its design. In fact, the original version of GM got the model wrong, which led to urgent pleas to uninstall GM until the security model was fixed. The urgency of that issue was due to the danger of misuse of Greasemonkey. And that danger is directly related to GM's power and capabilities.

The first thing to understand about Greasemonkey is that it has full access to the web page. If a web page is displaying sensitive information like account numbers, Greasemonkey can see it. Therefore, a compromised Greasemonkey can do great damage by virtue of the information it can steal.

The second issue to understand is that Greasemonkey is powered by JavaScript and has access to features like event handlers. This can enable GM to snoop on anything the user is doing within the page. For example, a compromised Greasemonkey could capture user input, like the entry of a password.

Finally, Greasemonkey operates at a higher privilege level than normal JavaScript, giving it even more power. The most important capability is the ability for Greasemonkey to be immune to cross domain browser security. Normally, HTTP requests issued from JavaScript (XmlHttpRequest) can only target URLs in the same network domain (e.g. bea.com) as the parent page. This restriction was put in place to help limit (it does not totally prevent) cross site scripting (XSS) attacks. However Greasemonkey, as a plugin, is immune to this restriction. This is a most useful feature when building mashups, but it is does mean that Greasemonkey is operating with higher privileges. It can issue requests to whatever URL it wants, perhaps allowing it to send sensitive data to a malicious web server.

In summary, Greasemonkey is a powerful tool. But just as this power can be put to good use, it can also cause much damage when under the control of malicious parties. The rest of the blog describes how Greasemonkey provides protection, and why IT will have concerns about it in the enterprise.

The Greasemonkey Security Model - Trust the Scripts

Currently, Greasemonkey is a tool in use on the greater world wide web. I don't know the number of people using it, but let's speculate that at most its only a few percent of the users on the web. Let's also speculate that the people using it are fairly sophisticated computer users that know what they are doing. In this situation, we have a small community of users that probably don't attract a lot of attention from the black hats out there. Even if they did, the users have the ability to self-regulate and guard against attackers. Every script is human readable, and so users can look at every script before installing them to determine if the script is safe.

With this understanding of the use case, we can now look at the security model. The Greasemonkey security model operates with the following assumptions:

• Trust the Greasemonkey script; the user installed it and therefore it can be trusted not to abuse the powers described above.
• Do NOT trust the targeted web sites. Do not allow those sites any access to GM objects or APIs.

What this means is that the focus of the Greasemonkey security model is on malicious web sites. This came about because the first security model was vulnerable to such attacks. A well crafted web site could steal cookies and even files off the machine of the unsuspecting Greasemonkey user. The security model was revised in 2005, and great care has been taken to prevent web sites from hijaking the privileges of Greasemonkey to do harm. For a great article on how this was accomplished, read Mark Pilgrim's explanation of XPCNativeWrappers and how they protect GM.

While attacks from malicious web sites should no longer be a problem, Greasemonkey is still vulnerable from certain kinds of attacks. Specifically, the security model does NOT have facilities for detecting or preventing malicious user scripts. Therefore, the user must be careful to only install scripts from trusted sources. This model is in use for the public web, but is it the right model for the enterprise?

The Enterprise Greasemonkey Security Model - an Inversion of Trust

Moving into the Enterprise, the angels and demons flip sides. In the enterprise, IT primarily owns the web sites. Rogue IT employees aside, these sites can be considered trusted and safe. But in this case the GM users aren't as sophisticated, and there is greater potential for them picking up malicious scripts. These are the same users that have been clicking on .EXE email attachments for years. IT cannot trust that this user community will correctly protect themselves from evil scripts.

From IT's point of view, the Greasemonkey security model is inverted - it trusts the wrong people.

Whether this is a fatal flaw that prevents an enterprise from using Greasemonkey universally is a decision IT must own. Personally, I don't think Greasemonkey should be a part of an enterprise wide application roll-out. But I do still think it has it's place in niche solutions. The Greasemonkey Selection Framework introduced in my previous post should help guide when it is the right tool.

Admittedly, it is valid to view any sanctioned Greasemonkey use as reckless, because it could open a Pandora's box. Once users get a hold of it for authorized uses, it may lead to other unauthorized use within the enterprise. This is something for IT to bear in mind when considering if Greasemonkey is an appropriate tool. As a counterpoint, malicious attacks aren't limited to Greasemonkey scripts. .DOC and .PDF files both have been used in the past in attacks, but that has not prevented enterprise use of those technologies.

Malicious Greasemonkey - a Real World Example

To show that Greasemonkey scripts can be a real danger, take a look at this image below. It is a screen shot of the public Greasemonkey script repository, and it currently displays a big warning message. Apparently, a malicious script writer has been publishing evil scripts to the public at large.

One could argue that this proves that the community approach works. Community members discovered the malicious behavior and have taken action to remedy it. But within the enterprise, would the users be able to catch such a problem?

Enterprise Solutions for the Greasemonkey Security Model

Now that you are aware of the problem, let's talk about how IT can solve the problem of the inverted security model. Unfortunately, a server side solution is not attainable. By design, a content page cannot know about or have access to the user script. Therefore a server side solution cannot prevent a malicious script from running on the page. In fact, public web sites that generate their revenue through ads have feared this problem for years. There is no way for these sites to stop the ad blocking Greasemonkey scripts.

We must look to the client for the solution. Unfortunately, Greasemonkey does not currently have a technical solution to this problem. For instance, there isn't a feature that prevents non-IT approved Greasemonkey scripts from being installed. Once the plugin is installed, the user is free to install any script, and there is no way to detect if that script has been tampered with. So if a coworker sends out an email like "Check this out, I updated the script to add a new feature", unwitting users may pick up a malicious script.

Instead, IT must rely on some coping strategies:

• Train workers to treat Greasemonkey scripts with as much suspicion as an .EXE file
• IT must host a read-only web site for all scripts to be published where they cannot be tampered with; do not just throw them on a file system
• Beg the Greasemonkey developers to add a signing feature

Further Reading on the Greasemonkey Security Model

• Mark Pilgrim's explanation of XPCNativeWrappers and how they protect GM. A must read for understanding how it works.
• Original Greasemonkey Fatally Flawed - mail thread discussing the flaw with the original security model.
• Top 10 Firefox Extensions to Avoid - Greasemonkey makes the list because of the security model.
• Evil Scripts in the Public Repository - blog entry talking about how userscripts.org now must carry the warning about the evil scripts.

Final Greasemonkey Blog Post Coming Shortly...

The last post in this Greasemonkey series should be done soon...I will show how I implemented a deployment and versioning system for Greasemonkey scripts.

Technorati Tags: Greasemonkey,Enterprise Security

Want short SaaS/Cloud news updates throughout your day? Subscribe to the ondemand Twitter microfeed. http://twitter.com/ondemand

Greasemonkey in the Enterprise: When is GM the Right Tool for IT?

As I have covered in previous blogs, Greasemonkey is a powerful client-side tool for augmenting web sites or creating mashups. It consists of a Firefox plugin coupled with scripts that are written and installed to modify web pages as they return from a web server. Greasemonkey generally is a tool for techies who want to tweak their favorite sites around the web. But does Greasemonkey have a place in the toolbox of IT departments in the enterprise? My next few blog entries will work towards an answer to that question. In this installment I will introduce a framework for measuring whether Greasemonkey is a good solution for a specific enterprise project.

NOTE: this blog entry was originally posted August 17th, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

This is part 3 of a series of blog entries on Greasemonkey mashups.

• Blog 1: More Mashups: Using Greasemonkey to Weave New Features into Web Sites
• Blog 2: Building a Greasemonkey Mashup Tutorial
• Blog 3: Greasemonkey in the Enterprise: When is GM the Right Tool for IT?
• Blog 4: Beware of Greasemonkey's Inverted Security Model
• Blog 5: Solving the Greasemonkey Script Versioning Problem

 

An Enterprise Use Case for Greasemonkey

You probably did not learn about Greasemonkey (GM) in a Computer Science class at University. When drawing up an ideal solution for delivering a web application, Greasemonkey is probably no where in sight. The truth is Greasemonkey is a pragmatic solution for real world problems when other options aren't available. It is not the cleanest solution, but it makes up for that by being effective. It will always have detractors who will dismiss it as a hack, and an evil one at that. While its shortcomings cannot be denied, it is a mistake to dismiss it.

Greasemonkey has the potential of fast tracking IT projects that would otherwise never get implemented. It can also lower the bar for IT skill sets - a junior scripter can use it to modify the most complex of web applications. I feel that Greasemonkey deserves a place in your IT toolbox, and should be an option when it is an appropriate solution.

But I will be the first to say that Greasemonkey is not for everyone, and is not for every project. Greasemonkey definately has a sweet spot. Outside of that sweet spot, you are best off looking at another solution like other mashup tools or custom development.

For a quick example of good GM use case, imagine a web application in the enterprise with these characteristics:

• It is a closed source packaged application, purchased and installed from an external vendor by IT.
• It is used by a modest sized group of users, all of which are comfortable with computers and web browsing
• There are a couple of missing features that dramatically hurt productivity
• Those features can be easily woven into the application using a Greasemonkey script

This web application would make a good candidate for augmentation with Greasemonkey. A GM script could be quickly created that would add the missing features without needing to acquire the source code for the application. The deployment of the script would be relatively simple - the user population is modest in size and are technically competent. IT could be a hero by providing a quick solution to an expensive problem.

But what makes this use case a good fit for Greasemonkey? What other applications are good candidates for a Greasemonkey mashup/augmentation? What guidelines should you use when deciding if Greasemonkey is right for a given project? This blog will answer these questions.

Greasemonkey Selection Framework

This section provides a framework for evaluating whether Greasemonkey is a good fit for an enterprise project. Like any software selection process, you must consider many factors before choosing a best fit solution. You cannot base your decision on a single factor; you must sum the overall suitability against the project requirements. The list below enumerates a number of factors to look at before making your choice. For each factor, I explain when Greasemonkey is best suited.

  Factor: Do you have access to the source code of the web application?
Description: IT brings in many packaged applications from external vendors. In such cases, IT may not have access to any source code, or only a partial set (like the HTML files). For applications home grown within IT, this is not an issue as the source code is available.
Greasemonkey Fit: When the source code is available, it may be easier to just update the original application. Greasemonkey is indicated more when the source code is unavailable.

 Factor: The application is home grown and the source code is available. But updating the application is very risky?
Description: Even if the source code for an application is available, it may not be cost effective to update the application directly. Perhaps all of the original developers are gone, and there is little documentation on how the site was built. Or perhaps the current staff is unfamiliar with the technology that was used to implement the site. If the code base is complex, it may be risky to make any modification to the source code.
Greasemonkey Fit: Greasemonkey can be a safer solution for adding new features to web applications that are risky to touch. The Greasemonkey script is externalized from the web application, and so the new development is clearly separated from the legacy application.

 Factor: How critical is the feature to be added?
Description: Is the feature a productivity enhancement, or is the application as-is no longer valid without the update? An example of the latter would be a feature related to a compliance project. In order for the legacy application to update company financial data, a new "explanation" field needs to be added to a web form for auditing purposes. Users must not submit this form without the added field.
Greasemonkey Fit: It would be difficult to enforce that a user have the Greasemonkey installed and up to date (although the last blog entry in this series will offer a possible solution for this). Therefore, if an application is not valid without the new feature, Greasemonkey may not be the right solution for implementing that feature. For optional features such as productivity enhancements, Greasemonkey is a great solution.

 Factor: Is Firefox certified for use in the enterprise, and does it work with the target application?
Description: This factor is obvious, but be careful not to miss this in your software selection. Greasemonkey requires Firefox (there have been attempts to port to IE (gm4ie, GreasemonkIE), but are they complete/solid?). Also, some web applications may not work correctly with Firefox.
Greasemonkey Fit: If your enterprise does not support users on Firefox, Greasemonkey is not an available solution. Also, if the target web application does not work well with Firefox, Greasemonkey is not a great fit for the project.

 Factor: To what degree are the target users computer literate?
Description: Within the enterprise, there is a broad range of computers skills amongst the employees. Greasemonkey does require users to be able to follow directions on installing Firefox, the Greasemonkey plugin and the required scripts.
Greasemonkey Fit: The more computer literate the user base, the better for Greasemonkey. While installing plugins and scripts seems like a no-brainer for anyone that would be reading this blog, it is a true obstacle for many within the enterprise.

 Factor: What size is the user population that needs the new feature?
Description: The appeal of web applications is in their ease of deployment. IT need only manage the application on a few server machines in the data center. Updates are efficiently deployed. Client side solutions are generally more expensive to deploy, as each client needs to be updated with each new version.
Greasemonkey Fit: The cost of deploying a Greasemonkey script to a small user population is likely modest. On the other hand, deploying a new version of a Greasemonkey script to a huge user population is an expensive proposition, particularly when computer literacy is an issue (see above). Greasemonkey is best when the user population is small.

 Factor: Is the application exposed to the public internet?
Description: Many enterprise applications are available to a closed audience of employees and partners. But some applications are also exposed out to customers and the general public on the internet.
Greasemonkey Fit: Because Greasemonkey requires Firefox, the GM plugin, and the proper user scripts installed, Greasemonkey is a good solution when the user base is well known and constrained. Otherwise support costs and customer complaints will be high ("Hello, Avitek Industries, how may I help you?"; "Your website says I need Firefox. What is a Firefox?").

 Factor: How often is the page structure in the web application changing?
Description: Regardless of who is changing the web application (IT, or the ISV if it is a packaged application), every update has the potential of breaking existing Greasemonkey scripts. Minor changes to the content will probably be safe, but bigger changes to the page templates will almost certainly cause problems.
Greasemonkey Fit: Greasemonkey works best if the structure of the pages within the target web application is not changing substantially.

 Factor: Does the application manage or expose sensitive information?
Description: Some enterprise web applications are low security risks. The site that hosts a monthly employee newsletter probably is not much of a concern from a security point of view. A payroll system is quite the opposite; the consequence of a security breach is severe.
Greasemonkey Fit: My next blog entry in this series will discuss the Greasemonkey security model and why it is not a great fit for the enterprise. Based on that analysis, Greasemonkey is not a good choice for adding features to highly sensitive enterprise web applications. Further, some IT Managers may feel GM is too risky to allow at all. More to come in the next post...

 Factor: Does the new feature require many changes to the existing web application?
Description: Javascript can update the DOM of an HTML page quite easily. But for each update on a page, there is code involved. Writing raw HTML (in an .html file, a JSP/ASP file, etc) is far easier than a similar effort in JavaScript code.
Greasemonkey Fit: Greasemonkey is a great fit for features that can be implemented with a few snips to the target HTML. As the number of distinct changes to a page increases, the suitability of Greasemonkey decreases.

 Factor: Does the target web application already have JavaScript code that mutates the page?
Description: The web application may have JavaScript code that modifies the page structure. This may be in support of Ajax capabilities, or just the way the initial page load was implemented. A Greasemonkey script runs before any page defined JavaScript runs. This may or may not be a good thing.
Greasemonkey Fit: Implementing a feature with Greasemonkey may be difficult if the page is already mutating itself with JavaScript.

 Factor: Does the feature require communication with a server in a different network domain than the web application?
Description: Browser based programmatic requests initiated from JavaScript (XmlHttpRequests) are restricted in their reach - they must not cross a network domain (e.g. bea.com). This is browser security, and this is generally good. But if you are adding a mashup feature to the web application, this would normally prevent the feature from being implemented in the browser. However, Greasemonkey scripts are not subject to this limitation, and can issue an XmlHttpRequest to any network domain.
Greasemonkey Fit: Greasemonkey is a good fit when you wish to implement the feature in the browser but need to connect to a different network domain.

Favorable Enterprise Greasemonkey References

To see how others have used Greasemonkey on enterprise projects, or for opinions on how Greasemonkey can add value to the Enterprise, consult these links:

Greasemonkey Improves Productivity - this company uses Greasemonkey to speed up enterprise form entry

Enterprise Greasemonkey is Inevitable - "...sooner rather than later we're going to see GM within the enterprise - if only b/c they have the potential to take some of the load off of beleaguered, shortstaffed IT staffs."

Firefox in the Enterprise Working Group - focuses on a larger issue than just Greasemonkey, but this expert group is looking at some of the issues noted above.

Using Greasemonkey to Enhance Auditing - this script adds time logging capabilities to JIRA.

Anti Enterprise Greasemonkey References

And the opposing view. There are those who think IT should stay well away from Greasemonkey:

Greasemonkey Primes Firefox For Embarrassment - this is an article you have to pay good money to read. Here is an excerpt from the abstract: "But IT managers beware: Greasemonkey will cause you nothing but headaches..."

[Greasemonkey] is Evil - "What's evil is the idea you mention that we'll end up making sophisticated infrastructure like automated script updaters in an attempt to turn a quick-and-dirty hack into a robust solution."

The Story Continues...

My next blog entry will discuss the Greasemonkey security model, and why IT will see it as "inverted" from what you would want.

Technorati Tags: Greasemonkey,Enterprise

Want short SaaS/Cloud news updates throughout your day? Subscribe to the ondemand Twitter microfeed. http://twitter.com/ondemand

Want short SaaS/Cloud news updates throughout your day? Subscribe to the ondemand Twitter microfeed. http://twitter.com/ondemand

Building a Greasemonkey Mashup Tutorial

Ever wonder what a mashup between Tim O'Reilly (technology guru) and Alfred Chuang (BEA CEO) would look like? Well, that might not be pretty but a mashup between O'Reilly Safari and BEA dev2dev holds more promise. What's not to like: embed a world class online technical library (Safari) into a world class enterprise developer web site (dev2dev). Sounds like a winner. Let me show how this easily done with the browser based mashup tool called Greasemonkey.

NOTE: this blog entry was originally posted July 5th, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

This is part 2 of a series of blog entries on Greasemonkey mashups.

• Blog 1: More Mashups: Using Greasemonkey to Weave New Features into Web Sites
• Blog 2: Building a Greasemonkey Mashup Tutorial
• Blog 3: Greasemonkey in the Enterprise: When is GM the Right Tool for IT?
• Blog 4: Beware of Greasemonkey's Inverted Security Model
• Blog 5: Solving the Greasemonkey Script Versioning Problem

The Magic of Greasemonkey

As I demonstrated in my last blog entry, it is very easy to meddle with any web site you visit on the web. The tool that makes this possible is Greasemonkey, a project started by Aaron Boodman, which enables you to inject, modify or delete pieces of a web page. And all of this is done client-side in the Firefox browser. As an example, in my previous blog entry I showed how easy it is to add a new link to the dev2dev web site. For that case, the Greasemonkey script simply found the right injection point in the page, and then blasted in a new link into that spot.

If you read Mark Pilgrim's Dive Into Greasemonkey online manual or his Greasemonkey Hacks book, you will find all sorts of things to do with an HTML document. You can slice and dice a page anyway you want. For example, one scripter wrote the Ad Blocker user script that deletes ads magically from any page. Also, writing Greasemonkey scripts for GMail seems to have become a cottage industry.

If you stretch the term mashup, you can even call the results of these activities mashups. The target web site is being mashed up with the Greasemonkey user script. But how about creating a more definitive mashup - can the script inject elements from a second site into that target web site? Once again, this is easily done in Greasemonkey and this blog will show you how.

O'Reilly Safari + BEA dev2dev

As mentioned in the preamble, the mashup will target BEA's dev2dev website like in my previous blog entry. However, in this tutorial, I will show you how to inject dynamic content from another site: O'Reilly's book catalog. The idea is this: as you read blogs and articles on BEA dev2dev about web technologies (JSON, Ajax, JavaScript, Greasemonkey, etc), it would be nice to have a list of embedded links to O'Reilly's book catalog. This will be helpful if you wish to find a book to learn more about the technology you are seeing in the blog or article. Further, this list will include links to the O'Reilly Safari online library, giving you instant access to in-depth books and papers on that topic. Safari eliminates the need to order an O'Reilly book via mail - almost the entire O'Reilly catalog is conveniently available for reading online.

BEA dev2dev does not currently have this O'Reilly feature, but that is not a problem. I will build a mashup in this blog entry using Greasemonkey to create this feature.

Let's get started...

Building a Web Mashup with Greasemonkey

I will assume you have read my previous blog entry and that you understand:

• What Greasemonkey is, and how it performs it's magic in the Firefox browser without modifying the target site
• How to inject HTML into a location on a web page
• A basic familiarity with a programming language like Java, JavaScript, etc

Additionally, you need to understand in essence what the XmlHttpRequest is, which I will cover immediately. The XmlHttpRequest is the heart and soul behind that Ajax buzzword you probably have heard something about. It allows the browser to issue HTTP requests programmatically from JavaScript. The XmlHttpRequest quietly operates behind the scenes on a page to gather information from a web server. This feature is used along with code to modify the current page (called DOM manipulation) to update small pieces of the page without causing a full page refresh. This is Ajax. There are detailed articles on the subject like this one from David Teare on dev2dev if you would like to learn more.

To build the O'Reilly Safari + BEA dev2dev mashup, the XmlHttpRequest will be the cornerstone of the solution. When the user browses to a blog or article on dev2dev, the Greasemonkey script will issue an XmlHttpRequest to the O'Reilly web site to gather a list of books that are relevent. It will pass the title of the blog or article as the search query, and it will specify that only 3 books are wanted in the result set. Once the response is received, the Greasemonkey script will extract the summaries of those 3 books and inject that list into the blog or article.

As a starting point, look at this mashup tutorial article of mine on dev2dev: Mashup Article. Now, see the image below to see the end result of this mashup with the O'Reilly links injected:

What about the Single Origin Policy?

If you have read about mashups in general or the XmlHttpRequest specifically you will perhaps see a problem with the plan I outlined above. Browsers have universally implemented a security policy that affects the XmlHttpRequest. This policy is called the Single Origin Policy (SOP), and is designed to prevent Cross-Site-Scripting (XSS) attacks. The details of XSS aren't important here, but the SOP is. The SOP prevents an XmlHttpRequest from targeting a URI from a network domain different from the parent page. For example, if a user types http://dev2dev.bea.com in the address bar of his browser, an XmlHttpRequest triggered from that page cannot target http://hack.evil.com. The SOP limits the XmlHttpRequest to the bea.com domain. This is security and this is good.

So how about my plan to implement this mashup? If the user targets a blog or article on dev2dev.bea.com, how is it possible to send an XmlHttpRequest to an O'Reilly web property? The answer is that Greasemonkey scripts operate at a higher privilege level than ordinary page JavaScript. The SOP does not apply to XmlHttpRequests driven from Greasemonkey scripts. So we are clear for take off with this mashup implementation!

Building the Script

I will once again assume you have read my previous blog entry and understand how Greasemonkey can inject HTML into a page. What I will show here is how to issue an XmlHttpRequest to a website from a Greasemonkey script and how to process the results to extract the required information. That information will then be injected into the dev2dev blog or article using the technique already covered. The full script is publicly available on the Greasemonkey script repository here: dev2dev Oreilly Script. I encourage you to look at the full script, install it, and use it when visiting dev2dev in the future.

What follows is the partial script listing. It shows these elements:

• Constructs the URI - sets a parameter to constrain the query to 3 results, and sets the blog/article title as the book query
• Invokes the XmlHttpRequest to the URI, passing a callback function to invoke when the response is received
• The callback function parses the response and injects the list of books into the page

// SNIP - code removed for brevity.
//  The full script finds the right place in the DOM
//  to inject the O'Reilly results

if (commentsAnchor)
{
// build our not-quite-REST URL, not oreilly.com but
//  nevertheless it is the O'Reilly book query.
// Note: passing blog title as the query string
var oreillyUrl = 
  'http://search.atomz.com/search/?sp-a=sp1000a5a9'+
  '&sp-t=store&sp-x-1=cat&sp-x-2=cat2&sp-q-2=Books'+
  '&sp-c=3&sp-q='+
  document.title;
GM_log('URL: '+oreillyUrl);

var user_agent = 'Mozilla/4.0 Greasemonkey';

// go ask Tim what he would recommend based on the 
//   blog/article title
GM_XmlHttpRequest({
    method: 'GET',
    url: oreillyUrl,
    headers: {
     'User-agent': user_agent,
     'Accept': 'text/html',
  },
  onload: processOreillyResponse
  });
}
else {
  GM_log('  Error: did not find a comment anchor');
}

// callback from the XmlHttpRequest with response
function processOreillyResponse(responseDetails)
{
oreillyHTML = responseDetails.responseText;

// we want to find this HTML in the response since
//   it delineates the list of books:
/* 
 <!-- ResultListStart -->
	blah blah	
 <!-- ResultListEnd -->
*/
var start = oreillyHTML.indexOf(
	"<!-- ResultListStart");
var end = oreillyHTML.indexOf(
	"<!-- ResultListEnd -->", start);
GM_log("Clipping start: "+start+" end: "+end);
var result = oreillyHTML.substring(start, end-1);

if (result)
{
  oreillyHTML = result;

  // create the HTML element containing the list
  resultsElement = document.createElement(
	"placeholder");
  resultsElement.innerHTML = oreillyHTML;
  
  // inject the list into the page
  commentsAnchor.parentNode.insertBefore(
	resultsElement, commentsAnchor);
}
else {
  GM_log("Nothing was returned from the clipping");
}
}

Once this script is installed into Firefox, Greasemonkey will properly inject O'Reilly book results into dev2dev pages. Furthermore, the results returned from O'Reilly are conveniently marked up with links to the traditional book catalog, and also to Safari. So with a single click from the blog or article, you can navigate to Safari and dive into a full length book on the technology you just read about on dev2dev. Nice!

Next: Is Greasemonkey a Good Mashup Solution in the Enterprise

If you have been following my blog you will know that I like to evaluate how to use various mashup technologies in the enterprise. I do have an opinion in this regard with Greasemonkey, but it's more than a few paragraphs worth of material. I will delay diving into this topic until my next post. Actually I am debating how to cover it - I may stretch this discussion across the next 3 blog posts, or perhaps I will blast it all in just one. We need to look at various issues such as versioning, Greasemonkey's inverted security model, and Greasemonkey's mashup sweet spot. Subscribe to my blog feed and you'll be sure to catch the discussion however I package it.

Further Reading:

• dev2dev Oreilly Script - the finished O'Reilly Safari + BEA dev2dev Greasemonkey script



• Using Greasemonkey to weave new features into web sites - my previous blog entry



• Dive Into Greasemonkey - Mark Pilgrim's definitive online guide



• Greasemonkey Hacks - Mark Pilgrim's O'Reilly tome on Greasemonkey



• Ajax Introduction - by David Teare, on dev2dev



• O'Reilly Safari - the ultimate online warehouse of technical books and articles
  
  
  

Technorati Tags: Greasemonkey,mashup,dev2dev,O'Reilly,Safari

More Mashups: Using Greasemonkey to Weave New Features into Web Sites

With this blog entry I am continuing the theme of demonstrating tools to help you build mashups. In this case, I will show how a tool called Greasemonkey can be a powerful approach for building browser side mashups. Greasemonkey is a plugin to Firefox that allows a script developer to inject useful Javascript into any web page. This capability enables you to add new features to sites that you do not own. I will show in this blog how you can add a new feature to the dev2dev website, without having access to the dev2dev code. Later, a follow-up blog will show how this same technique can create a feature on dev2dev that includes data from a different web site, producing a true mashup.

NOTE: this blog entry was originally posted July 2nd, 2007 on my previous blogging system (dev2dev.bea.com). Comments on the old blog were not transferred.

This is part 1 of a series of blog entries on Greasemonkey mashups.

• Blog 1: More Mashups: Using Greasemonkey to Weave New Features into Web Sites
• Blog 2: Building a Greasemonkey Mashup Tutorial
• Blog 3: Greasemonkey in the Enterprise: When is GM the Right Tool for IT?
• Blog 4: Beware of Greasemonkey's Inverted Security Model
• Blog 5: Solving the Greasemonkey Script Versioning Problem

Client side Code Injection with Greasemonkey

Greasemonkey is just cool - and I have been having a lot of fun playing around with it. It operates on a simple principle, but delivers massive power from that simplicity.

Greasemonkey allows you, a Javascript developer, to write a script that gets included into web pages that a user visits with the Firefox browser. This script can do really anything, but usually it will inject/update HTML elements into the page to create entirely new features for that web site. The targeted web site need not have any idea that this is going on - the Greasemonkey script is executed on the browser after the original page comes back from the web server.

To be useful, a Greasemonkey script generally will be targeted towards a specific web site, though a general purpose script is possible. The developer specifies which URLs (with wildcards) are valid for the script. The Greasemonkey plug-in in Firefox will silently watch the user's URLs as they browser, and will inject a script when appropriate.

To be clear, Greasemonkey is not installed by default, nor are the scripts that you write. It is an opt-in activity for the user to both install Greasemonkey and then each script that they want to have. This means that Greasemonkey is not quite as accesible as other mashup solutions that I have written about - the mashup here is not a URL. The user must be skilled enough to install Greasemonkey and your script.

Greasemonkey Hello World

Installing Greasemonkey is straightforward: you will find the download link at the Greasemonkey Home Page. You will need to use Firefox of course, but otherwise it can't go wrong. After you have it installed, the next step is to install your first Greasemonkey script.

Unlike a lot of cutting edge projects out on the web, Greasemonkey has excellent documentation. Primarily, the best place to go for an introduction, tutorials, and documentation is Mark Pilgrim's online book, Dive Into Greasemonkey. The book is quite comprehensive and is geared towards getting you on your feet quickly.

To that point, the Hello World example for Greasemonkey that I will show is derived directly from Mark's book. The example below has been stripped of comments for brevity, but otherwise is identical to his original found here. This script is offered as a public file named helloworld.user.js. By the file extension, you know that this is nothing but JavaScript. And a closer look will show that it contains some meta-data within the comments, and then a single JavaScript function call to alert().

The meta-data is hopefully self-explanatory: it instructs Greasemonkey to execute it on any (*) site except for a couple of exclusions. If you install Greasemonkey, and then install this script helloworld.user.js by clicking on the link, you will see that it just pops up an alert box for every page view.

// Hello World! example user script
// version 0.1 BETA!
// 2005-04-22
// Copyright (c) 2005, Mark Pilgrim
// Released under the GPL license
// http://www.gnu.org/copyleft/gpl.html
// ==UserScript==
// @name          Hello World
// @namespace     http://diveintogreasemonkey.org/download/
// @description   example script to alert "Hello world!" 
// @include       *
// @exclude       http://diveintogreasemonkey.org/*
// @exclude       http://www.diveintogreasemonkey.org/*
// ==/UserScript==

alert('Hello world!');

Mr. Wong and BEA dev2dev

If you visit BEA's dev2dev site, you will notice that Jon has helpfully included links to popular tagging services on all articles and blogs. This allows you to quickly tag the article or blog using del.icio.us, Digg, DZone, Furl or Reddit. See the image below to see how the site works today:

However, what if that list is not sufficient for your needs? That is my problem - there is one tagging site that I use that is not listed. As an aside, I have been doing research into the spread of Web 2.0 in China. In doing that work, I have been tagging chinese articles that I find, and some english ones just to be helpful. I am not using an english language tagging site, I am using Mr. Wong, which is a Chinese language tagging site. I would like to have Mr. Wong as one of my tagging options when browsing dev2dev. But since I don't have access to the dev2dev code, I cannot add it.

Weaving Mr. Wong into BEA dev2dev Using Greasemonkey

Enter Greasemonkey.

The process for developing a Mr. Wong feature on dev2dev is as follows:

• Navigate to dev2dev, and View Source on the HTML for blogs and articles
  
  
  
  
  



• Note that the tagging links box is easily found in both by looking for the last div tag with class box_gray
  
  
  
  
  
  



• The div tag contains a simple HTML table, all we need to do is add a new row
  
  
  
  
  



• Formulating the link to Mr. Wong is easy: we just need the document title and url. Both are easily available in JavaScript
  
  
  
  
  



• Write the JavaScript!
  
  
  
  
  
  
  
  

Before I show the code for the script, here is the result:

As you can see, even though I did not have access to dev2dev code, I was able to easily add a brand new feature to it!

The Mr. Wong Script

The script used to add this feature to dev2dev is below. But if you would like to try it out, or view the live version, navigate to my script repository for Greasemonkey (or here if that link does not work). Install the script named dev2dv Mr Wong. The code roughly follows this path:

• Find the last box_gray div tag on the page, this is the tagging box
  
  
  
  
  
  



• Find the inner table in that div
  
  
  
  
  
  



• Add a new row to that table
  
  
  
  
  
  



• Populate the row with a link and image for Mr. Wong
  
  
  
  
  
  
  
  

// ==UserScript==
// @name          dev2dev Mr Wong
// @namespace     http://dev2dev.bea.com
// @description   Injects a link to Mr Wong on each blog/article link
// @include       http://dev2dev.bea.com/blog/*
// @include       http://dev2dev.bea.com/pub/*
// ==/UserScript==

var divTagsWithClass, taggingDiv;
GM_log('Running dev2dev Mr Wong script');

// get the existing tagging div box 
//   by looking for the class (box_gray)
divTagsWithClass = document.evaluate(
    "//div[@class='box_gray']",
    document,
    null,
    XPathResult.UNORDERED_NODE_SNAPSHOT_TYPE,
    null);
// the tagging div appears last
taggingDiv = divTagsWithClass.snapshotItem(divTagsWithClass.snapshotLength-1);

if (taggingDiv)
{
	// find the table
	tableTag = taggingDiv.getElementsByTagName('table')[0];
	if (tableTag)
	{
		// create the new row
		var lastIndex = tableTag.rows.length;
		newLinkTR = tableTag.insertRow(lastIndex);
		
		// create the td for image and set styles
		var newLinkTD = newLinkTR.insertCell(0);
		newLinkTD.valign = 'bottom';
		newLinkTD.width = '20';

		// build the HTML	
		newLinkTD.innerHTML = '<img src='+
			'"http://www.mister-wong.cn/favicon.ico"'+
			' alt="Mr. Wong" border="0" height="18" '+
			'hspace="8" width="18">';

		// create the td for the image
		var newLinkTD = newLinkTR.insertCell(1);

		// set the styles
		newLinkTD.nowrap = 'nowrap';
		newLinkTD.valign = 'bottom';

		// build the HTML	
		newLinkTD.innerHTML = 
			'<a href="http://www.mister-wong.cn/index.php?'+
			'action=addurl&v=1&bm_url='+
			window.location+
			'&bm_description='+
			document.title+
			'">'+
			'Mr. Wong</a>';
	}
	else {
		GM_log('  Error: did not find the tagging inner table');
	}
}
else {
	GM_log('  Error: did not find a tagging div');
}

Pros and Cons

Hopefully you can see the power and simplicity of the Greasemonkey plugin. In a follow up blog I will show how you can take this technique further by injecting data from other sites to create a true mashup. Here is what I see as the pros and cons of this tool:

• Pro: it is easy for a developer to install both the plugin and scripts
  
  
  
  
  
  



• Pro: a seasoned JavaScript developer can crank out features in little time
  
  
  
  
  
  



• Pro: you do not need access to the web site's code to add new features
  
  
  
  
  
  



• Pro: the documentation and developer community around Greasemonkey are superb
  
  
  
  
  
  



• Con: mashups created with Greasemonkey aren't exactly URL accessible, it requires install steps
  
  
  
  
  
  



• Con: Greasemonkey is not supported on IE or any browser other than Firefox
  
  
  
  
  
  



• Con: there is a significant security issue that I will cover in my next blog entry
  
  
  
  
  
  
  
  
  
  
  
  

References

• Greasemonkey Home - download GM here
  
  
  
  
  
  



• Dive Into Greasemonkey - Mark Pilgrim's excellent online guide to Greasemonkey
  
  
  
  
  
  



• My script repository for Greasemonkey, includes my dev2dev scripts
  
  
  
  
  
  



• Programmable Web tag for Greasemonkey - not much there so far...
  
  
  
  
  
  



• Blog about the game changing nature of Greasemonkey
  
  
  
  
  
  
  
  
  
  
  
  

Technorati Tags: Greasemonkey,Mashup

Want short SaaS/Cloud news updates throughout your day? Subscribe to the ondemand Twitter microfeed. http://twitter.com/ondemand

Want short SaaS/Cloud news updates throughout your day? Subscribe to the ondemand Twitter microfeed. http://twitter.com/ondemand