Bits By The Pound

While trying to figure out while my laptop, a Dell Inspiron 1720, has been freezing occasionally, I found out that you can use SysRq codes to communicate directly with the Linux kernel to get system operations done even while the machine is locked up. This is handy as it allows you to specify a sequence of actions to safely sync your disks prior to rebooting your computer. Then you don’t have to go through the trouble of running fsck on your filesystems.

The typical sequence that you are told to input is to hold down Alt+SysRq and then slowly type r e i s u b. This presents a slight problem on the Inspiron 1720 since the SysRq key is shared with F11. Accessing the SysRq key requires pressing the Fn key but you must not have the Fn key depressed while you type the subsequent letters or they are interpreted as something different. I have found the easiest way to perform this magical little finger dance is to use your right hand to press the right Alt key with your thumb and the SysRq with a finger. So the sequence is:

Left hand: Press and hold “Fn” key (between Ctrl and the Windows key)
Right hand: Press and hold “Alt” + “SysRq” keys (Alt+F11)
Left hand: Release “Fn” key
Left hand: Press and release “r” key. (Screenshot dialogs may start popping up. Ignore them)
Left hand: Press and release “e” key. (Your GUI should collapse to a tty, most processes terminated)
Left hand: Press and release “i” key. (Progress of key shown in the tty, most proceses killed)
Left hand: Press and release “s” key. (Progress of key shown in the tty, syncs filesystems)
Left hand: Press and release “u” key. (Progress of key shown in the tty, unmounts filesystems)
Left hand: Press and release “b” key. (Progress of key shown in the tty, starts reboot)
Right hand: Release all keys

This note relates to Mozilla Firefox 3.5.7 running on Ubuntu Karmic (9.10).

The problem is that Firefox appears to be freezing for a quick moment, every 4 to 5 seconds. This is annoying when editing text and watching a video on YouTube.

I came across this helpful post: http://support.mozilla.com/en-US/forum/1/524464

My approach to the fix is to give up on session restore, altogether. I do not like it, anyway. The post above might help you with tweaking better values if you wish to retain this feature.

1. Open firefox to this page: about:config
2. In the filter box, type: browser.sessionstore
3. Double click on “browser.sessionstore.interval” and enter a larger value of your choice (I used 100000, but the next step might make this irrelevant)
4. Double click on “browser.sessionstore.resume_from_crash”, resetting the value (i.e. make it “false”)

This fixed the annoying behaviour. I hope it works for you.

If you have built a Google Wave Robot that interacts with a gadget, you have probably come to a number of frustrating dead ends. As of this writing, the ability for a robot to observe changes in the state of a gadget is easily determined using the Wave Robot API. However, the difficulty comes in attempting to have the robot change the state of the gadget.

References:

• Bug Report : http://code.google.com/p/google-wave-resources/issues/detail?id=406
• Mail thread

Reading the library code for the Java Wave-Robot-API (http://wave-robot-java-client.googlecode.com/svn/trunk/), it becomes apparent that facilities available to a robot to perform changes on gadget states have a very coarse resolution. In that, I mean that there appears to be only the ability of changing a whole gadget state, not a single field.

When a robot receives a set of events from the Wave server, it has an opportunity to return a number of operations to be executed. This is the opportunity to modify content of the Wave. Techniques to observe the discussion between a Wave server and a robot are explained in another post.

When it comes to changing the content of the text, a robot can perform operation with fine resolution. This means that the operations deal with small changes. These operations are not likely to collide with operations of other users concurrently editing the same content, and if they do, the collision does not yield a large error. To understand better this phenomena, look at discussions on Operational Transforms, which are used internally by Wave to keep track of changes.

However, when it comes to managing elements (gadget are elements), there appears to be capabilities designed in the protocol to transform properties/fields of the element, however the library code is not making use of them. Therefore, at this time, it is unclear if these functions are available.

The problem is compounded by the fact that “whole-element” operations do not always work as expected. One such operation is “DOCUMENT_REPLACE_ELEMENT” which, according to the library code, should enable a robot to replace a gadget (or any element) with another copy. Using the facilities to replace an element fails and the Wave server returns an error.

This is an interesting side observation. Since the operations performed by a robot are returned as a reply to the incoming events, the Wave server contacts the robot again with incoming events containing the error. Currently, the Java Wave-Robot-API is not able to decode these events, so they show up in the logs as a critical error, where the following is observed:

If you see the above, it means that the Wave server was not able act upon a returned operation.

To come back to the whole replacement of an element, a robot must first perform a DOCUMENT_ELEMENT_DELETE operation, followed by a DOCUMENT_ELEMENT_INSERT operation. In Java, using the current Wave-Robot-API (published 2009-09-16), the following code would be needed:

What does this mean? Basically, if a robot and a user are attempting to use a gadget simultaneously, the changes that a robot makes will wipe out the changes from the user. Also, as the gadget is removed and added, there is a visual cue that this happened if the user is looking at the wave. This is far from an acceptable work-around.

Even when the bug above is fixed, the resolution of the changes will still be too coarse. The only long-term acceptable solution will be based on the DOCUMENT_ELEMENT_MODIFY_ATTRS operation, but it is not clear when this will happen.

Therefore, the really exciting Wave applications, where robots and gadgets interact to provide a truly immersed environment, will have to wait or suffer the visual glitches.

You have written a Google Wave Robot, uploaded it on AppEngine, and it does not seem to behave the way you expected?  You would not be searching the Internet if it was working, would you?

First, you are not alone. The robot API appears to be changing, and I have observed changes in the amount of information my robots receive, over time. The best way to figure out what is going on with the robot is to look at the raw data sent by the Wave server to the robot. These messages are remote procedure calls encoded in JSON (jsonrpc).

This post assumes that the Google Wave Robot is deployed on AppEngine.  At the time of writing this post, this is the only avenue for deploying Wave Robots.

To see the raw data transmitted to a Wave Robot, one must:

1. Enable the robot to record FINEST log messages
2. Capture the logs from the AppEngine web site
3. Decode the JSON message in human readable format

This post deals with 1 and 3.

Enable FINEST logging

In your robot project, there is a properties file used to configure the logger. In general, this file is located in …/WEB-INF/logging.properties  This location can change, since it is declared in a file called …/WEB-INF/appengine-web.xml. If you can not find the logging properties files, inspect the appengine-web.xml file. It should look like this:

If the system property called “java.util.logging.config.file” is not present, then you should add it and create the “logging.properties” file. If the property already exists, then you have a clue where the file can be found.

The “logging.properties” file should contain something as follows:

The trick is to change the property called “.level” to a value of “FINEST”. By default, most example give a value of “WARNING” to this property.

Once this change is done, upload the robot on AppEngine, again, for the new logging level to take effect.

Capturing the logs

Once the robot is uploaded on AppEngine, interacting with a wave that includes the robot should generate messages sent to it. Using the AppEngine administrative page for the robot, navigate to the “logs” page. Changing the filter to “INFO” should refresh the page with a number of “I” icons. Those are the information entries in the logs. Some of those should display the incoming events and outgoing operations.

Copy the large JSON object from the incoming events into the clip board.

Format JSON for a human

A number of online services will readily translate JSON to a more readable format. One of those is: http://jsonformatter.curiousconcept.com/ Pasting the content obtained in the previous step into the web form and formatting should give you the full picture of the information available to the robot.

This note is written based on:

• appengine-java-sdk-1.2.6
• Apache Maven 2.2.1
• Apache Ant 1.7.1

If you create an application destined to Google AppEngine using Maven, the use of the Google AppEngine SDK may not be optimal. This is because, in some instances, the generated artifact (a WAR file) is not what is expected from the SDK. In general, when generating a WAR file using Maven, an interim directory is created with all the files found inside the WAR archive. This interim directory is the expected input for the AppEngine SDK.

However, there are circumstances where an interim directory is not created. Using the Cargo Plug-in for Maven and generating an Uberwar is one example where the generated artifact does not naturally produce the needed interim directory. I find Cargo useful to build Google Wave Robots that include their own gadgets. Since Google Wave Robots must currently be hosted on Google AppEngine, I needed a script to load them automatically.

This note introduces an ANT script that unpacks the WAR file and upload the application on Google AppEngine. Two files are needed:

• build.xml This is the script that uploads the application
• build.properties This file includes user specific information

build.xml
Create a file called “build.xml” in the root directory of the Maven project. The content of the file should be as follows:

build.properties
Create a file called “build.properties” in the same directory as “build.xml”. The content of that file should look like:

The content of the file above should be adjusted to reflect the user’s specific SDK installation, appengine user name and password.

Running
After the files are configured correctly, uploading the application to AppEngine should be done with the following command, executed from the root directory of the project:

Over the past couple of days, I have seen a situation where I boot my laptop (Ubuntu Karmic) and the Network Manager applet insists that the wireless is disabled. Rebooting does not help. Examining the syslog showed that the OS found 2 kill switches and one of them was disabling the wireless. Examining the state showed that each kill switch state was affected by the single physical kill switch, but their states were the opposite of each other. So regardless of which position the kill switch is in, one of the kill switch devices would disable the wireless.

I turned the kill switch off and reboot the computer. When it came up, the kill switch drivers were now synchronized and turning the kill switch on caused the Network Manager to start looking for Wireless connections. Listing the kill switch states shows they are again synchronized.

This post at LinuxTrap shows that others are experiencing a similar condition.

A Google Wave Robot, in the simplest sense, is a servlet that answers to a set of RPC calls and which is hosted on Google App Engine (appspot.com). At the time of this writing, those appears to be the only restrictions. Mind you, hosting a servlet on Google App Engine brings about other restrictions, such as saving all data via JDO. However, those restrictions might not apply to Wave Robots if Google allows to have them hosted somewhere else in the future.

The RPC calls that must be supported by a robot are mostly implemented in a set of libraries available from Google. However, those libraries have not been pushed to the Maven repositories, so they must be first installed in the local Maven repository so that a robot can be built using Maven.

This article describes this process.

There are three libraries required. The first library, the Wave Robot API, is the open source code which is needed to implement the RPC calls. The other two libraries are dependencies needed by the Wave Robot API. The following steps show how to download those libraries and install them in a local Maven repository.

At this point, you can create a regular Maven project that yields a WAR application, using your favourite environment. If you must do it from the command line:

Next, you must install the libraries previously downloaded as dependencies of your new project. Edit pom.xml and add:

Also to be added in pom.xml should be directives to use Java 6:

That pretty much sets up the project. Now, the robot needs to answers to a number of important requests, which are covered below.

Capabilities
A robot must answer to ‘/_wave/capabilities.xml’ with an XML document describing its capabilities. The easiest way to do this is to create a document called ‘capabilities.xml’ in the directory …/src/main/webapp/_wave

with the following content:

The above content must be modified to reflect the events the robot is interested in receiving.

App Engine Configuration
Because the robot is deployed on AppEngine, a couple of files are necessary/useful. The file app-engine.xml contains information that deals with the application, in this case the robot, that is deployed on the service. The logging.properties file is useful to be able to track logs on the AppEngine service.
Create a app-engine.xml file at the appropriate location (…/src/main/webapp/WEB-INF)

and provide the following content:

The application name in the file above must be adjusted to the name given on AppEngine. Also, the version must change every time the ‘capabilities’ document is modified.

The logging file should be created as well

and given some content:

Other Wave Requests

Now, this is where one needs to write some Java code. A wave robot requires to handle the following requests:

• /_wave/robot/jsonrpc
• /_wave/robot/profile

First, create a web.xml file to dispatch those calls at the right place:

Then, adjust content of web.xml:

Obviously, the above XML file must be adjusted to reflect what classes are used in your project.

The RobotProfile class should look something like this:

The code above figures out what the real URL for the robot is and then uses it to serve out the robot’s home page and avatar. With the code above, create a HTML home page and save it into …/src/main/webapp/index.html Also, saving an avatar at …/src/main/webapp/images/icon.png will provide that image for displaying in the robot’s profile.

Finally, an example of the RobotServlet is given below. This article is not aimed at explaining the details of this code, but an example is provided to help readers along.

Finishing Touches

As discussed above, add a home page and an avatar for your robot. If you wish, the profile code above handles the case where the home page and avatar are included right in the WAR file. The home page should be located at …/src/main/webapp/index.html The avatar should be a 100×100 pixels image saved at …/src/main/webapp/images/icon.png

Generate WAR file

Just like any other WAR generated via maven:

This should produce a WAR file under …/target This WAR file is just like any other and can be deployed anywhere. However, to be able to use it as a robot, one must deploy it on AppEngine. Create a project on AppEngine for this robot and deploy using AppCfg (http://code.google.com/appengine/docs/java/tools/uploadinganapp.html

If using the command line tool for AppCfg:

Note in the above exmaple that AppCfg does not accept a WAR file, but a directory with the WAR content. This is produced naturally by Maven, so you can use the directory directly. More information on an automated script can be found here: Uploading a Maven generated application on Google AppEngine

One can test the robot in any servlet container. If the robot is working, the following should be observed:

1. Using a browser to access a page at ‘…/_wave/capabilities.xml’ should return the capabilities document
2. Using a browser to access a page at ‘…/_wave/robot/profile’ should return a JSON document with the profile information

Using Robot

Using the robot in a Wave is akin to adding a participant. The address for the robot is <robot-name>@appspot.com

With luck, you will get there without a hitch. Remember, luck beats brains anytime.

Today, we presented an introduction to Google Wave Gadgets at the Ottawa WaveCamp (Hackathon). This presentation covers the creation, deployment and use of Wave Gadgets.

The presentation slides and example source code is available on a Google project. More information available here: http://owh01-ig.googlecode.com/svn/trunk/web/index.html

When deploying a Google Wave Gadget that depends on external resources such as images, javascript libraries and CSS stylesheets, resources referred to by relative paths do not load correctly.

The problem is as follows:

• a gadget is deployed on a third party server
• a gadget is loaded as a plugin of the Wave framework
• to the client browser, the gadget appears to be loading from the Wave service URL
• relative paths attempt to resolve to the Wave service domain, while the resources are sitting on the third party server

A naive solution would be to translate all relative paths to absolute paths that refers to the “real” location of the gadget, mainly URLs based on the third party server domain. However, this is not a workable solution. The problem with absolute URLs has to do with browser security, where certain resources can not be loaded from domains other than the domain of the original page. Since the resources associated with the gadget are not located in the same domain as the Wave service, then some resources will not load, depending on type and browser security enforcement.

The working solution is to proxy all resource requests via the Wave service. In this manner, all resources associated with the gadget appear to be served from the same domain as the original page.

This solution is based on the “Gadget libraries” and is not restricted to Google Wave Gadgets, per se. It might be applicable to other social gadgets as well.

The described solution is to:

1. Insert “bootstrap” code in the gadget module XML file
2. Use bootstrap code to load external resources

The bootstrap code is shown below:

getModuleBase()

This function retrieves the “real” URL of the gadget and returns the absolute path to the directory containing the gadget. This function computes the absolute path by retrieving the Gadget parameter called ‘url’. This information is provided by the gadget framework, which provide all gadget parameters via an associative array obtained with a call to _args()

rebaseRelativeUrl(relPath, shouldProxy)

This function accepts a path (relPath) relative to the gadget location and returns an equivalent absolute URL. This function uses the getModuleBase() to make this computation. If the flag ’shouldProxy’ is set, then the returned URL is an equivalent URL, however it is proxied via the gadget framework. This conversion is possible because of the _IG_GetCachedUrl() available with the gadget libraries. Therefore, when the flag is set, a path relative to the location of the gadget is converted to an absolute URL proxied via the gadget framework, avoiding browser security issues.

addStylesheet() and addScript()

These utility functions add style sheets and external javascript by using the rebaseRelativeUrl() and adding the appropriate elements to the document. These functions are available at run-time and should probably not be used before the gadget is loaded.

gadgets.util.registerOnLoadHandler()

This function is often used in gadget development and generally well understood. It will not be covered here. The reason for inclusion in the post is that this is the location where the external resources should be loaded, as shown in the example above.

Example, please?

Sure. An example gadget is available here: http://owh01-ig.googlecode.com/svn/trunk/gadgets/counter2/counter2.xml

Log in to your Dev Preview Account and then follow this link:

https://www.google.com/a/wavesandbox.com/ChangePassword?service=mail&hl=en

This worked as of the end of November 2009.