Archive for the ‘Software development’ Category

Unit Testing Google Wave Robots

June 18, 2010

I am happy to report that I am now successfully running unit tests on my Wave robot. For this example, I will show how to configure a unit test for a Java robot that responds to the WAVELET_SELF_ADDED event, which is fired when the robot is first added to a wave. My sample robot is designed to do something very simple: when added to a wave, it records the root blip id in an AppEngine data store. Specifically, I’m using JDO to persist objects in the data store.

The configuration comprises:

  • Eclipse with the Google Plug-in. This is not strictly necessary, but you would be crazy to go without it.
  • All of the robot development libraries. The installation of these is covered in the Wave Robots API Java Tutorial. You will need to make sure that gson.jar, oauth.jar, wave-model.jar, and wave-robot-api.jar are on your build path. (I’m skipping the version numbers in the files for clarity.)
  • AppEngine libraries on the build path as described in the Local Unit Testing of the GAE documentation. Two of these were in my google plugin directory but not attached to the project: appengine-api-stubs.jar and appengine-testing.jar. These I put into a lib folder and added to the classpath. Other libraries may be in the war and not yet on the build path, such as appengine-api.jar and appengine-api-labs.jar.

My WaveletRecord class for this example is very simple:

public class WaveletRecord {
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;

    private String rootBlipId;

    public String rootBlipId() { return rootBlipId; }

Now, we’re ready to write the unit test. Once again, the GAE/J Local Unit Testing documentation will get us started. We need to have a LocalServiceTestHelper along with setUp and tearDown methods, just like in their example:

public class RobotTest {

    private final LocalServiceTestHelper helper =
        new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig());

    public void setUp() {

    public void tearDown() {

I want to test how my robot responds to onWaveletSelfAdded, so I need a WaveletSelfAddedEvent object. As described in the Debugging Wave Robots article, we can use AppEngine logs to extract the JSON messages that are actually passed between the Wave client and a Robot. In my case, I have just such a String from my logs:

{"events":[{"type":"WAVELET_SELF_ADDED","modifiedBy":"","timestamp":1276086048055,"properties":{"blipId":"b+X"}}],"wavelet":{"creationTime":1276086030870,"lastModifiedTime":1276086048055,"version":13,"participants":["","",""],"participantRoles":{"":"FULL","":"FULL","":"FULL"},"dataDocuments":{},"tags":[],"creator":"","rootBlipId":"b+X","title":"Bugs squashed","waveId":"!w+JkmJGit7A","waveletId":"!conv+root"},"blips":{"b+X":{"annotations":[{"name":"conv/title","value":"","range":{"start":0,"end":14}},{"name":"lang","value":"en","range":{"start":0,"end":14}},{"name":"lang","value":"en","range":{"start":15,"end":431}}],"elements":{"0":{"type":"LINE","properties":{}},"14":{"type":"LINE","properties":{}},"15":{"type":"LINE","properties":{}}},"blipId":"b+X","childBlipIds":[],"contributors":[""],"creator":"","content":"Nothing to see here, please move along.","lastModifiedTime":1276086031862,"parentBlipId":null,"version":8,"waveId":"!w+JkmJGit7A","waveletId":"!conv+root"}},"robotAddress":""}

Of course, we can’t just put that into our Java code directly: we have to escape all of the quotation marks to make it a valid String literal. However, once we do that, we can make a handy constant like this:

private static final String SELF_ADDED_EVENT = "{\"events\":[{\"type\":\"WAVELET_SELF_ADDED\",\"modifiedBy\":\"\",\"timestamp\":1276086048055,\"properties\":{\"blipId\":\"b+X\"}}],\"wavelet\":{\"creationTime\":1276086030870,\"lastModifiedTime\":1276086048055,\"version\":13,\"participants\":[\"\",\"\",\"\"],\"participantRoles\":{\"\":\"FULL\",\"\":\"FULL\",\"\":\"FULL\"},\"dataDocuments\":{},\"tags\":[],\"creator\":\"\",\"rootBlipId\":\"b+X\",\"title\":\"Bugs squashed\",\"waveId\":\"!w+JkmJGit7A\",\"waveletId\":\"!conv+root\"},\"blips\":{\"b+X\":{\"annotations\":[{\"name\":\"conv/title\",\"value\":\"\",\"range\":{\"start\":0,\"end\":14}},{\"name\":\"lang\",\"value\":\"en\",\"range\":{\"start\":0,\"end\":14}},{\"name\":\"lang\",\"value\":\"en\",\"range\":{\"start\":15,\"end\":431}}],\"elements\":{\"0\":{\"type\":\"LINE\",\"properties\":{}},\"14\":{\"type\":\"LINE\",\"properties\":{}},\"15\":{\"type\":\"LINE\",\"properties\":{}}},\"blipId\":\"b+X\",\"childBlipIds\":[],\"contributors\":[\"\"],\"creator\":\"\",\"content\":\"Nothing to see here, please move along.\",\"lastModifiedTime\":1276086031862,\"parentBlipId\":null,\"version\":8,\"waveId\":\"!w+JkmJGit7A\",\"waveletId\":\"!conv+root\"}},\"robotAddress\":\"\"}";

Yes, it’s ugly, but it allows us now to easily create a real WaveletSelfAddedEvent using Google’s gson library, which you have as part of the robot development libraries. We just need to make a GSON factory, parse the JSON, and pull out our event from the bundle. That is,

GsonFactory gsonFactory = new GsonFactory();
Gson gson = gsonFactory.create();
EventMessageBundle bundle = gson.fromJson(SELF_ADDED_EVENT, EventMessageBundle.class);
WaveletSelfAddedEvent event = (WaveletSelfAddedEvent) bundle.getEvents().get(0);

Now, unlike in the GAE/J Local Unit Testing approach, I am going to use JDO to check that my robot is behaving as expected. Will this work? Honestly, it surprised me, but it sure does: configuring the LocalServiceTestHelper was enough to allow us to get an appropriate PersistenceManager using the usual techniques.

The following code will create for us a query that will return all WaveletRecord objects:

PersistenceManager pm = PMF.get().getPersistenceManager();
Query query = pm.newQuery(WaveletRecord.class);

We will also have a sanity check to ensure that there is nothing there yet.

List records = (List) query.execute();
assertEquals("We should start out with nothing in the datastore.", 0, records.size());

Running the robot is easy as pie once we know what we’re doing. My robot is implemented as DataCollectorServlet, so I can poke it and see how it reacts to being added to a wave.

DataCollectorServlet robot = new DataCollectorServlet();

How do we know if this worked? If there is now a WaveletRecord in the datastore and its root blip ID matches the one in the event. Notice that I don’t have to hardcode that root blip ID anywhere: I’ll just pull it out of the event. The event is not a “mock object”: it’s a real event as Wave would generate, but it has all the nice lightweightness of a mock object.

records = (List) pm.newQuery(WaveletRecord.class).execute();
assertEquals("There should be exactly one record returned by the query.", 1, records.size());
WaveletRecord waveletRecord = records.get(0);
assertEquals("The rootblip id of the single wavelet record should match the one in the event.",
  event.getWavelet().getRootBlipId(), waveletRecord.rootBlipId());

That’s it! The unit test passes, and now test-driven development can commence.

(Turns out my robot actually writes a blip on being added, too, but testing that is left as an exercise for the reader.)


Writing, Scholarship, and Software

May 22, 2010

My Google I/O travelling companion was Brian McNely, professor of Rhetoric and Writing in the English Department at Ball State. He told me a bit about his personal daily scholarly regiment: he reads at least 50 pages a day and writes at least 750 words, the latter supported by I respect this endeavor, knowing how, as a professional academic in a university, there are other forces that can drain the time and focus that should be devoted to academic pursuits. Brian clearly enjoys his scholarship, even when he has to make sacrifices to meet his goals.

This made me think about my own scholarship and personal goals. There is a great contrast between my feelings between Fall and Spring of the last academic year. In Fall, I felt productive and I also felt that I was learning. In much of the Spring, I was frustrated. (Interestingly, as I look back at the past semester’s writing to pull out those links, there is more positive ideas there than I remembered there being. That’s encouraging. I suppose I need to spend more time reading my own blog.) One of the major differences between the two semesters was that in the Fall, I was inventing: working on Confluence was a great experience, as I’ve said before. I know that I love the process of inventing software systems, and so I have been actively seeking out opportunities to leverage this as both personal fulfillment and scholarship. This Summer, in fact, my time should be nicely split between some interesting Wave and game projects. I embrace Boyer’s model of scholarship, and pertinent here is the scholarship of application.

When I try to tie these ideas together with Brian’s 50/750 regiment, I hit a contradiction. Clearly, as a professional scholar, I could just adopt 50/750 myself, and this would undoubtedly improve my scholarship. However, I would also like to have time for invention, for creating software. Unfortunately, you cannot measure productivity in software, so how does one set personal goals for invention?

Last semester, I set aside time Friday afternoons for writing on this blog, and I am happy with the results. I feel like I have been able to crystallize several concepts by writing them, and it has given me an outlet to explore and ripen new ideas. There were only a few times in the semester that I was unable to set time aside. However, even in reflecting on this writing, it was done in relatively small spurts of about an hour. That is, it was done in what Paul Graham calls the manager’s schedule. I know, from years of development experience and many semesters of frustration, that if I want to successfully build a system, I need to do it on the maker’s schedule. (Incidentally, adopting inbox zero has significantly improved my ability to occasionally adopt the maker’s schedule during the academic year.)

I don’t have a plan, then, for how to deal with this contradiction of wanting measurable goals for something that brilliant designers call unmeasurable. I have developed some experience using Scrum and MSF in leading teams, and I could always adopt these for myself, except that I am not sure that “maintain development momentum” or “mitigate one high-impact risk” are goals that I can meet. One one hand, these managerial techniques were designed for teams, and on the other, neither really is a measurement of productivity. Of these two, the idea of measuring momentum via Scrum or similar techniques is somewhat appealing, partially because it could hone my capacity for estimating time requirements, and in the end, time may be the only invariant I can use to commit to invention.

No conclusions, but I’m open to ideas.

From personal interaction to online reputation

April 18, 2010

Yesterday morning, I decided to roll out a new version of Connect the Dots in hopes that it would fix a bug due to Motorola’s Droid OTA update. In case you haven’t followed the story, Motorola released an update about two weeks ago that made my game nigh unplayable. This only affected Droid, and I don’t have a Droid, and so that made it very tricky to debug. I tried recruiting help from a Droid-owning cousin and even some of the kind people who emailed me bug reports, but the truth is, most folks only know how to install apps from the Market. It’s not intrinsically hard to install from an APK that someone emails you, but the Android team didn’t go out of their way to make it easy either. Must not be a key scenario.

I pushed out version 0.2.2, took a shower, and then downloaded the app to my Nexus One to make sure it was still happy. Running the app, I realized that I inadvertently left some experimental code in one of the classes, and suffice it to say that it led to suboptimal user experience. By this point, it was around 8:30am, and I need to be on campus for a prospective student fair by 9:15. I booted my development machine into Linux — my preferred development OS — only to find my screen would not go to a resolution above 640×480. I have no idea why, but Win 7 did this to me the previous day too, and I did not have any interest in figuring it out then. Reboot into Win 7, where I realize I did not have the ADT plugin installed for Eclipse. I run the installation, and suddenly I can no longer browse my svn repository. No repository, no code. I come very close to throwing anything I can find, then rush off to campus, hoping that I have everything I need on my office workstation.

(Also, it’s my sixth wedding anniversary, I haven’t even said “Happy Anniversary” to my wife, and I’m heading into a day at work, plus my bike is in the shop. Let’s say I was not in a good mood.)

Fortunately, my office workstation is all set up, so I update my working copy of Connect the Dots and start investigating. Sure enough, I had made a mistake when pulling from an old tag into the trunk, and some of the experimental junk code in the trunk had not been overwritten. Fortunately, all I had to do was copy all the changes from tag 0.2.1 into the copy in the trunk — version control FTW. I go to check to make sure this version is working correctly, only to find that I don’t have a micro-USB in my office so I cannot hardware test 0.2.3. Par for the course. It can’t be worse, and the emulator shows that at least the experimental code is gone, so 0.2.3 gets published.

That’s all the backstory. I show up at Cardinal Preview Day along with Josh, an amicable undergraduate who agreed to help me talk about the department. He’s kind enough to listen to my story while we get our complimentary coffee. We stand by the Computer Science table as prospective Ball State students meander around, learning about different departments.

The second group to stop by consisted of a young man and who I assume to be his parents. He had just been to Purdue to talk to their CS department, and so he and his parents had some very pertinent questions. I have fielded these questions before, so I was ready to discuss some of the qualitative differences between BSU CS and Purdue CS. This group stayed to talk for a while, and I think I counted three other family groups that joined the periphery of the discussion. I tried to address the first student’s questions while sharing other information over his shoulder to these other prospective students.

He asked about some of the work that was shown on the display behind me, so I went through a litany of some of the interesting student projects we have going on.When I mentioned that I am teaching HCI through Android application development, he stopped and asked if we had anything on the market. He reaches into his pocket and pulls out … you guessed it … a Motorola Droid.


Actually, that’s what my brain said, but my mouth knew better. I explained that I had just uploaded an untested update that I had no idea whether it would fix a Droid-specific bug or not, and he was kind enough to oblige. I actually brushed off a real question or two while he went to download it. This was important. He opened the Market, did a search, downloaded the app, and presto! It’s working perfectly! Que alegria!

The rest of the day went on as expected, and mid-afternoon, I took a moment to check out the reviews on Connect the Dots to see if people had gotten the update. Recently, there had been a lot of comments on how it was broken on Droid. There were already four positive reviews thanking me for fixing the Droid bug, including this five-star review from “Eric”:

Met the developer at ball state, he’s a cs prof. This game is great!

Thanks, Eric, for brightening the day of a frustrated developer.

Postscript: Motorola’s update botched their handling of points as size specification. Points are defined as 1/72-inch, which is resolution-independent. It seems that after the update, my specification of the dots as 12pt radius was misinterpreted as 12-pixel (12px) radius. I changed the dimension specification to 24dp, where “dp” is “density-independent pixels”, which are pixels as if you are running at 160dpi. Note that both pt and dp are resolution and density independent. The change to dp did not change the behavior on my G1 running 1.6 or on my Nexus One running 2.1-update1, but it did fix the bug on Droid. Dear Motorola: it’s hard enough to support the variety of Android devices without your breaking spec.

Ontological argument for the existence of software

April 3, 2010

I gave a talk two weeks ago in a seminar on design thinking and innovation. As I was preparing for the talk—not knowing exactly what to talk about, but knowing that it should relate design thinking and software—I got to thinking about what software is. It is intangible but has behavior. It must exist, then, but it cannot be sensed. We can interact with bits and pieces of representations of it, but I contend that none of these are the software, not in toto.

This led to me to define the ontological argument for the existence of software, with apologies to Anselm of Canterbury:

Software designed is software created.

Put another way, software that has been designed with sufficient specificity so as to be implemented has already been implemented, modulo syntactic transformation. This is intentionally stated as a material conditional, implying that software can exist which has not been designed, but software designed cannot non-exist.

For example, consider the design of a software system. If this design cannot be implemented, then there is an omission, and therefore there exists a design which is more rigorously specified. For any hypothetical software system, then, there exists “a design than which a greater cannot be designed.” In creating the design, one has created the software: the executable model, the mathematics over time.

(What about the defects, you ask? They are baked right in. I never said that we are good at design!)

One of the seminar participants suggested that this makes software similar to an argument, which exists in the mind and can be represented on paper, but the representation is not the argument. This is a keen insight on the nature of intangibles, but it misses the point that software is an executable model. I was presenting primarily to young architects who are comfortable with static models, which clearly exist, even if one argues that the idea of the model is distinct from the physical manifestation of it. I understand that there is a shift in architects towards parametric models, and the relationship of these creations to the software development process is one that merits investigation.

Design Thinking and the Ephemerality of Software

March 16, 2010

I recently read Richard Buchanan’s Wicked Problems in Design Thinking (Design Issues, Vol 8, No 2, 1992, pp5-21) on the recommendation of my esteemed colleague Mahesh Senagala. The article frames design as a liberal art, partially drawing on design as an approach to wicked problems. The arguments for the inclusion of design thinking as an necessary liberal art are compelling, but I was already in that camp before reading the article. What I found more interesting about this paper from 1992 was the omission of software design as a design category.

Buchanan categorizes the areas explored by design into four areas, and the terms and examples are his:

  1. symbolic and visual communications: graphic design, typography, advertising, book and magazine production, scientific illustration, photography, film, television, and computer display
  2. material objects: form and visual appearance of everyday products as well as the “physical, psychological, social, and cultural relationships between products and human beings.”
  3. activities and organized services: logistics, decision making, strategic planning
  4. complex systems or environments for living: systems engineering, architecture, urban planning

The author uses the shortened phrase “signs, things, actions, and thought” to refer to these four areas impacted by design thinking. It is worth noting that the author acknowledges that these are not separate fields of study, and that any significant design exercise will incorporate all four.

In the article, there is no treatment of software design, despite the fact that it had been studied within the Computer Science and Software Engineering disciplines for decades prior to this article’s publication. I scribbled a few notes in the margins, wondering if the author meant to include software design as part of systems engineering, since he does not extrapolate. (It’s also worth noting that Buchanan addresses the inherent problem in the word “design,” that different people use it to mean different things. Here, I am talking about the “internal” design of software—how it works—not interaction or interface design.)

I posit that software design is significantly different from all other manifestations of design because when software has been fully designed, it exists. A significant corollary is that when software exists, it has been designed. Flaws in software implementation are evidence of errors (of commission or of omission) in its design. Recognizing the intangibility of software leads to its ephemerality, characteristic of modern continuous integration systems. An “instance” of software—a build—may only exist for milliseconds. Even if software design then is included in Buchanan’s characterization of system design, it is substantially different from architecture.

(I noticed that my argument echoes in software Anselm’s ontological argument for the existence of God. I don’t know if it’s a coincidence. Also, it reminds me again that “software architecture” is a misleading term, since software design is characteristically different than traditional architecture.)

I would be interested to know if the science of design literature contains subsequent work to Buchanan’s, and whether philosophies of design include “design of the intangible and ephemeral” within their ontologies.

Here Comes 222

March 3, 2010

In Fall 2010, my department is introducing a new course, CS222: Advanced Programming. The role of this course is to bridge the introductory programming sequence with development-heavy upper-level coursework. It is in the cards that I shall be teaching the inaugural offering of CS222. I’ve been keeping my eyes open for ideas to incorporate into this class.

I’m reading and thoroughly enjoying Ship It, from the Pragmatic Bookshelf. In fact, I’m contemplating using it, The Pragmatic Programmer, and Effective Java as textbooks for the course, though I need to spend more time planning out how that might work. Ship It spends a chapter on Tracer Bullet Development, and I have already shared an overview of this methodology on the CS345/545 course blog. In writing that post, I came across an interesting article on Tracer Bullet Development vs. Extreme Programming by Martin Ankerl. The two approaches are significantly different, with TBD encouraging some upfront design (though accepting the agile principle that interfaces will change) and XP encouraging evolutionary design (primarily user story and test-driven).

Either of these are good candidates for CS222, in my opinion. An advantage of TBD, pointed out by Ankerl, is that it facilitates parallelization. I certainly want CS222 students to learn how to work in teams, and so having a methodology that makes it easy to break pieces off for different developers is very important. However, TBD also requires some savvy with design, and the students will be coming fresh out of CS2. Software design—in the software engineering sense—is not a major topic of CS222. Learning-by-doing is important, but there is a simple elegance to scenario-driven design in XP. Unit testing and test-driven development are elements that I would like to explicitly include in 222, and so that makes me lean towards XP.

Fall 2010 looks like a lifetime away from here, but I have started keeping more notes on what I’d like to do with 222. These include: effective use of the IDE; interactive debugging; test-driven development; scenarios and user stories; how to read technical books; and the differences between CS1/2-style programming and writing real software. I had thought about trying to get a GA to make this a real qualitative research study, but I don’t think I’ll have the time to get that together.

I’m happy to hear your thoughts in the comments below, kind reader. For BSU-affiliated folks, I am planning on soliciting feedback from current students (e.g. ask current juniors and seniors what they wish they had learned in their sophomore year regarding advanced programming topics) as well as alumni. I’ll probably use our nifty new Facebook group for that.

Orders of Magnitude

January 19, 2010

As loyal readers know, I published an application on the Android market on January 3 called “Connect the Dots” (blog, project page). Today, January 18, I just checked the stats on the Developer Console: there have been 7489 installations of Connect the Dots, and it has 3.5 of 5 stars.

This means that more people have used Connect the Dots than have used any other software I have written by two orders of magnitude. When I first noticed this, it absolutely floored me. For all of my work on JIVE, in education, and on information visualization, it’s this silly one-week project that most people have used. It’s very hard to measure my impact on students in higher education in any kind of quantitative terms, but here, I can see that Connect the Dots has brought happiness to thousands of people. It may be fleeting happiness, but it’s happiness nonetheless.

Part of the inspiration may be the classic Powers of Ten video. Watch it if you haven’t. All my other work is a picnic blanket: Connect the Dots is a park. It borders on ludicrous.

Connect the Dots

January 4, 2010

As we get ready to start the Spring 2010 semester, I should get working on my resolution to write more reflective pieces! This is inspired by a desire to “practice what I preach”—specifically, being a reflective practitioner. In my case, I see that as being relevant to higher education and software development, among other things.

Yesterday, I published an application on the Android Market called “Connect The Dots”. It’s a simple dot-to-dot application that is briefly explained over at (By the way, you are encouraged to add the “La la la la!” after singing the title of the application.)

One reason for making the application is that my son, who is nearly three, seems to enjoy dot-to-dot puzzles, as I remember enjoying when I was young. This application was a way to make something that I could share with him. My wife is quite crafty and does all kinds of neat artsy things with him, but it’s harder to share my love of software creation. This was a personally fulfilling use of my time.

More to the point of this blog, by developing and publishing a mobile phone application, I have gained experience in the processes that I will expect my students in CS345/545 to follow next semester. One of the most important parts of this process is reflection. We’re studying human-computer interaction, after all: this cannot be done without adopting some qualitative research methods, at least stopping to think about what we’ve observed.

I feel like I should mention that I’m happy with the Android documentation. I appreciate that it goes beyond the engineering-technical into the design-technical, dealing with best practices of application design from both the performance and UI points of view. I think this will make it a valuable teaching tool, and I found it quite readable. I am eager to see how students with much less programming and UI experience take it, but I suspect most will enjoy it more than a traditional textbook.

The evolution of the “dot” design is interesting. From the beginning, I knew that the dots had to be large so that they could be easily touched. Also, the area that is sensitive to touch is actually larger than the dot, since fingers are not very accurate on touchscreens. The initial design looked like a conventional dot-to-dot, with the numbers outside of the dots.

early prototype showing numbers to the right and slightly below each dot

Connect the Dots: An Early Prototype

It’s connect-the-dots, right? How hard can it be? (Incidentally, I did not originally take screenshots during development, but through the magic of version control software and intelligent commit comments, I was easily able to roll back to this version and grab a screen capture. Subversion for the win!)

As I continued development and my team (i.e. wife) started making puzzles, I realized that I needed to move the numbers around. In a conventional dot-to-dot, the numbers are manually placed so that lines don’t cross over them. Even in the silly test image above, you can see that the line between 3 and 4 crosses over the “4” slightly. I started sketching some ideas of either computationally guessing where to put the numbers or incorporating another field into the puzzle description language (more on that later).

Then, after discussions with my wife and watching my son play the game, I realized that I had failed to question my assumptions. Context matters. In a conventional dot-to-dot, one uses a pencil with a very small tip to connect the points, so the points can be very small. The numbers go outside the dots, because the pencil marks through the dots, not the numbers. On a touch screen, if I need bigger dots, then I should just put the numbers inside the dots. It’s hardly genius, and in retrospect it seems obvious. However, I think it’s a great example of failure to question assumptions. Initially, I was just thinking “take paper dot-to-dot and make it electronic”. With this mindset, I had missed an obvious place for positive adaptation to a different environment. Now, in the version on the Market, we have two-state (pre-click and post-click) dots with numbers within, as shown below.

screenshot of Connect the Dots showing numbers in the dots

Better Dots

It was very exciting to me when I first gave Alex this puzzle. He went through it patiently, and as soon as the image drew, he smiled and shouted “a car!” Aside from being glad that my son was pleased, it meant that this actually is fun. It’s easy to lose sight of that when knee-deep in an API.

Later posts will address some other aspects of the application design and development process that I found interesting. Thanks for reading!