How Bazaar

6 months on Unity

2011-10-30T10:37:00.001+13:00

We have just finished another design sprint prior to UDS-P.

While talking with some others I realise that I have worked on Unity for six months, and not changed a single pixel on the output. No graphical changes, no moving widgets, no changing colours.

So what have I been doing?

First step was getting some new coding standards accepted by the team, which was much easier than I was expecting.

I added some property classes to nux, and did some general clean up in the code of nux and unity.

Refactored the indicator internals for the panel service which started off the shared unity core library for sharing code between the 2D and 3D code-bases.

Then I focused primarily on fixing memory leaks and crashes.

Once we hit final freeze, I did a little more refactoring internally, and now we are on to Precise Pangolin.

Properties in C++

2011-07-11T14:46:00.001+12:00

Once you have done any development in a language that natively supports properties, like Python or C#, going back to C++ and not having them often feels like a real pain. I've just proposed my second attempt at C++ properties for the nux library.

This change leans heavily on a paper written by Lois Goldthwaite: SC22/WG21/N1615 - C++ Properties -- a Library Solution. I added change notifications using sigc++. I found that using sigc::slot was nicer than templating the properties on the class and member function pointer. This also meant that I could provide a way for a simple property to get its own custom setter method while still having a sensible default.

Compiling C++ templates still gives absolutely horrendous error message that can take a while to mentally parse. I guess one advantage of having done a lot of template programming in the past is that I don't get too phased by copious quantities of error messages, especially for templates, as for one example today, I had just forgotten to change a template arg in a test function, and got way too many lines to sensibly look at. One benefit of that was it caused me to look at what I was doing, and I ended up simplifying my tests a little more.

Thank you Lois for the time you spent writing up the C++ properties proposal, it was a fantastic starting point for me.

Getting back into C++

2011-05-23T10:28:00.001+12:00

I have to admit that unit testing in C++, even with google-test, is so much more of a PITA than in python. Especially when checking string output. Simple string matching using split and regular expressions has really spoiled me.

Another thing that I've noticed is that I spend more time thinking about object design, and what should that object really be able to do, and what should it allow others to do to it.

It is an interesting time as I realise how much I still have to learn for our current domain. Most of my previous C++ experience has been on server side processing. Drawing stuff on the screen, real end user stuff, is still relatively new for me.

DX Sprint

2011-05-08T03:02:00.001+12:00

What a week. I've spent the last week in Budapest sprinting with the rest of the Desktop Experience (DX) team. This week was also my first official week with the DX team as I have moved now from the Launchpad team to the DX team. This was a good week. I had met some of the DX team before at other company get togethers, but not really talked to them much. A really important part of any new job is meeting the people that you are working with. This is always ends up happening when you work in the same office with them, but for a distributed company like Canonical, you can end up working on the same team with people that you don't get to meet for months.

It was great to meet different sub-teams of DX, especially those that I'll be working closely with. I'll be hanging out in the #ayatana irc channel now, but I'll also still be in #launchpad and #launchpad-dev. There are some very interesting plans for oneiric, and it will be interesting to see how much we can end up getting done. In the normal way we have "too much to do" and the gauntlet has been thrown.

So... I'll be hacking on the unity stack. Please don't ask me to fix any particular bugs yet as it'll no doubt take me time to find my way through the code :-)

Now for the 40+ hour journey home.

Launchpad and stacked branches

2011-04-21T13:38:00.001+12:00

As I'm sure most of you are aware, Launchpad hosts Bazaar branches. One early design decision that we had on Launchpad was that branches should be able to be renamed, moved between people and projects without limitation. This is one reason why each branch that was pushed to Launchpad had a complete history. We wanted to make sure that there weren't any problems where one person was blocking another pushing branches, or that people weren't able to get at revisions that they shouldn't be able to.

The Bazaar library, bzrlib, gives you awesome power to manipulate the internals giving you access to the repository of revisions through the branch. This can be a blessing and a curse, as if you have private revisions, then they can't be in a public repository.

Having a complete copy of the data for every branch became a severe limitation, especially for larger projects, of which Launchpad itself is one. A solution to this was a change in Bazaar itself that allowed a fallback repository which contained some of the revisions. This is what we call stacked branches. The repository for the branch on Launchpad has a fallback to another repository, which is linked to a different Launchpad branch. We ideally wanted all of this to be entirely transparent to the users of Launchpad. What it means is that when you are pushing a new branch to Launchpad, the bzr client asks for a stacked location. If there is a development focus branch specified for the project, this is then offered back to the client. The new branch then only adds revisions to its repository that don't exist in the development focus branch's repository. This makes for faster pushes, and smaller server side repositories.

The problem though was what do we specify the stacked on location to be? When we created the feature, we used absolute paths from the transport root. What the mean was that we stored the path aspect of the branch. For example, lp:wikkid gets translated to bzr+ssh://bazaar.launchpad.net/~wikkid/wikkid/trunk or http://bazaar.launchpad.net/~wikkid/wikkid/trunk depending on whether the bzr client knows your Launchpad id. The absolute path stored would be /~wikkid/wikkid/trunk. This information was then stored in the branch data on the file system.

The problem however was that the web interface allows you to rename branches. The actual branch itself on disk is referred to using a database id, which is hidden from the user using a virtual file system which has rewrite rules for http and at the bazaar transport level. However since the stacked on location refers to a full branch path, changing any part of that, whether it is the branch owner, branch name, or the project or package that the branch is for, would cause any branches stacked on that changed branch to break, bug 377519.

In order to fix this we had to change the location that the branch is stacked on to be independent of the branch path. The best solution here is to use the database id. I really didn't want to expose the user to this opaque id, but one opaque id is as good as another. Now when pushing branches to Launchpad, when it is creating a stacked branch you'll see a message like:

Created new stacked branch referring to /+branch-id/317141.

Existing branches still have their old branch paths saved for now. We'll run a migration script early next week to fix all these up, and hopefully we'll have seen the last of this bug.

Blueprint magic

2011-03-11T11:17:00.001+13:00

Just landed in qastaging is some itch-scratching work I did adding AJAX widgets to the main blueprint page. This has passed QA and will end up in production with the next no-downtime rollout (which should be real soon now).

This work was adding a bunch of the lazr-js wrapped widgets. Now we can update the following without reloading the primary page:

title - the H1 heading
summary
whiteboard
assignee
drafter
approver
priority
implementation status
definition status

Using the new custom events that the page raises when the context object changes (using YUI magic and API PATCH requests), when you change the title of the blueprint, the document title (title bar) and the breadcrumbs also change. When the implementation status is updated, the overall status updates, and the "started by" and "completed by" are shown or hidden as appropriate.

This is work that I've wanted to see done for almost a year, and recent other changes I've done adding more widget wrappers and javascript goodness have made this possible without adding copious amounts of custom javascript.

A side-effect of these changes is that there are now more fields exported over the API for blueprints.

Announcing sloecode

2011-03-11T10:19:00.001+13:00

Sloecode is a simple Bazaar hosting project.

Last year I tried to set up my home server to offer a place for three people to have shared, private access to a bazaar repository for a project. I found it really ackward. I felt that there had to be a simpler way.

Launchpad is an awesome place to host Bazaar branches. However Launchpad is for open source projects and personal branches are public.

I was approached towards the end of last year by Thomi who suggested we create a simple Bazaar hosting project for Otago Polytechnic to provide a place for the students to host their senior year projects. Since it was something I also cared about, and hadn't found a reasonable solution elsewhere, I agreed to help.

Our initial requirements went something like this:

Users defined in a database and no local login needed on the hosting server
Private repositories for the students to host personal branches
Lecturers should be able to see the repositories of the students
Projects have private repositories only visible to the members of the project team, and the Lecturers
Simple URLs for getting access to the branches
Scalability isn't a priority

In the end we went with the new pyramid libraries for the web application. We tried briefly with django, but I found the framework blocked me whenever I tried to do something. I had worked a lot with zope, and repoze.bfg was something we looked at. When repoze.bfg and turbogears merged into pyramid we felt that we had found a good match for us.

Installing and running the sloecode server is still a bit messy. We'd love to get it to the stage we you can just run an installer and magic happens. But we are not there yet.

The application server runs on the same machine as the filesystem hosting the branches and repositories. Shared repositories are created for each user as they are added. When a project is created so is its shared repository, and trunk branch which is set to append only.

Users log in and add their public SSH key. Users should also install the bzr-sloecode client plugin. Initially the plugin had hard coded site names for the Otago Polytechnic. But looking forward I decided that we should just use an environment variable. This allows access to the sloecode server using a short hand url.

sc:my-project - gets access to the trunk branch of the project called my-project
sc:my-project/trunk - also gets access to the trunk branch
sc:my-project/some-branch - gets access to the branch some-branch of my-project
sc:~myid/personal-branch - gets access to the branch personal-branch on my repository

The sc translation is done client-side, like the lp expansion for Launchpad.

We run a custom twisted SSH server on the hosting machine. This does the SSH key lookup against our user database, and doesn't allow password login. It also restricts the commands that can be executed on the server side to what it expects Bazaar to ask for. No shells are given. The server then launches a subprocess that uses a wrapped smart server that has a virtual filesystem to translate the requested paths to the underlying filesystem. This operates in a similar way to Launchpad, but much more trivially. The repository hosting configuration just needs two paths: one for the project repositories, and one for the personal repositories. The smart server code also handles the privacy aspect, not allowing unauthorized users access to repositories they shouldn't see.

The future

We'd like to have some form of code browsing functionality added. Whether we use loggerhead, or something else is still up in the air. We'd also like to integrate wikis for the projects. Wikkid would be a good fit (another of my personal projects) and could spur on some of the stalled development.

Ease of installation and configuration is another aspect we'd like to tidy up.

If you are interested in getting involved join the sloecode-dev team on Launchpad as it handles the development mailing list.

More responsive recipe builds

2011-02-15T18:01:00.001+13:00

SteveK has recently landed a change that at the same time makes our admins happy, and should give our recipe users more responsive builds.

Daily builds were previously kicked off at 23:00 UTC. A rather arbitrary time that put quite a load on the build farm as recipes became more popular.

The new change has our script to kick of daily recipe builds run much more often, and uses a cunning mix of magic and smarts to run the jobs (ok, not so much magic).

When new changes are pushed to Launchpad for branches that are used in source package recipes, the recipe is marked as dirty. Dirty recipes are candidates for daily builds. If a recipe has not been built into the daily build PPA within the last 24 hours a new recipe build job is created very soon after Launchpad notices the new changes. If there has been a build, then the new build job isn't created until 24 hours has past since the last build. Manual builds into other PPAs do not affect the daily build time check.

What this means is that if you have a daily build recipe, and tend to change the branch less often than every day, then when you do change it, the package is built much more quickly.

Refactoring Launchpad's lazrjs widgets

2011-01-31T14:57:00.001+13:00

Just landed (r12285 on devel) is some refactoring I've been doing on the lazr-js widget wrappers. After hacking on the picker at Dallas, I felt the other editor widgets needed some attention as well. The primary documentation can be found in the source tree at lib/lp/app/doc/lazr-js-widgets.txt, in fact this is probably the best thing to read.

This new changes did several things:

fixed the multi-line editor so you don't need extra HTML tags surrounding the widget in the page template
changed the widgets to actually use page templates to render the content instead of fiddling with string substitution
refactored the class inheritance - the multi-line editor no longer inherits from the single line editor, but instead they both inherit from a common base
it changed the way the editable attribute is defined - from being the name of the attribute, to being the field from the interface
the initial text for the multi-line editor is now determined from the object itself, not a parameter passed through to the constructor
the permission checks were unified, which fixed a problem with the text editors using mutator functions exposed through lazr.restful
you now have to be explicit about the id used for the HTML tag surrounding the editor - this wasn't much of an ask as all call sites were already doing this

All in all I'm pretty happy with this change.

Code Blue at the Thunderdome

2011-01-23T05:39:00.001+13:00

I am sitting in the hotel at the end of the Launchpad Thunderdome. Really this was just a sprint for all of the Launchpad developers, but someone decided it needed a neat name, and Thunderdome stuck.

It is at this sprint that the Launchpad team is transitioning away from the old application based teams to "squads". There are five squads, each given a colour for now: Blue, Green, Orange, Yellow, and Red. So my "Code" team has been split up and I have a new squad "Blue". Before we even switched to the squads, on of my new members resigned to move to something new, so I have an opening. Actually right now we have two openings. One squad is primarily based in Australasia (with one in New York) and the other has people from North America and Europe.

The majority of our time has been spent getting to know our new squads. Three of the squads are working on features, and two are on maintenance. The Blue squad (called "Code Blue" from now) is finishing of the recipe feature. This allows packages to be built automatically directly from a source package assembled from one or more branches on Launchpad. This feature was started over a year ago and needs to be finished off. Most of what we are doing now is just polishing the user experience and interface. The feature has been available since around the middle of the year, and we have almost 250 recipes that are configured to build packages automatically. Once we finish this off Code Blue will move on to maintenance until some other feature teams finish up.

We have done some pretty impressive work on the webservice this week. Primarily due to trying to make nice AJAX popups for changing the owner and PPA of a recipe. This resulted in much yak shaving, and refactoring. The work we did make it trivially easy to add an AJAX popup to any place on Launchpad where a single value is chosen from a list of options. We also fixed a number of bugs in lazr.restful and very soon there will be a release that will add a 'web_link' property to the entries in launchpadlib that will refer to the actual object on Launchpad.

Coworking

2010-11-03T13:27:00.001+13:00

I've been working for Canonical for around four years now, and for the vast majority of that time I've worked out of a room in my house. You need to have a certain style of discipline to work effectively from home and I like to think that I have that.

More recently I've been feeling a little down about work in general. I think it is because I am insulated and tend to get a lot of information from blogs, a number of feeds, twitter, identi.ca, facebook and email lists. A lot of the information that I then get exposed to is often from the vocal minority, and this can be a little draining.

Last Friday I started spending some of my work day at the distiller lab. I have to say it has been great for my general state of mind. Today is my forth day in a row spending some time here and so far, so good.

Wikkid Wiki

2010-05-26T21:35:00.002+12:00

As with any creation being released, I'm writing this with some trepidation. I'd like to announce the first release, 0.1, of Wikkid Wiki.

What is it?

Wikkid is a wiki that uses the Bazaar DVCS as an underlying storage model. Wiki pages are text files in a branch.

Why another wiki, surely there are enough already?

There is the obvious reason, because I felt like it. But this is not the primary reason. Since I started working for Canonical I've come to appreciate the whole culture of free, libre, and open source software more. During the day I work on Launchpad. Primarily in the area of integrating the Bazaar DVCS. Launchpad doesn't have a wiki integrated, and it is my plan to see wikkid be the wiki that is integrated.

I have to admit to having very strong opinions myself on what I wanted for wikis in Launchpad. I've tried to encapsulate that in the vision below. Since no one else was looking at it, I took it upon myself. Discussions started last year between a small group of Launchpad developers, but there was no traction. At the start of March I started writing the Bazaar backed wiki. It needed a name though. Thankfully after trying several I got the perfect name from Aaron Bentley - wikkid.

The wikkid vision

Any wikkid wiki can be branched locally for off line editing.

Any branch can be viewed using wikkid - not limited to branches created through wikkid

A local wikkid server can be run using a Bazaar plugin

Local commits use the local users Bazaar identity

Wikkid can be configured to operate in a stand alone, public facing mode where it has a database of users

Wikkid can be integrated as a library into other python applications

Wikkid uses standard wiki markup languages - not inventing its own

What does Wikkid Wiki offer?

Right now, wikkid offers basic page editing, rendering and browsing.

ReST is the default wiki format

Creole 1.0 is also supported (as long as the first line is "# creole")

source files are syntax highlighted using pygments

you get to see your gravatar for your local Bazaar identity

no page locks are used, but instead a three way merge

conflicts due to concurrent editing are shown for the user to resolve

Where to from here?

Well here is just the beginning. The TODO file is quite long already, and
that is just a simple brain dump.

Things that I want to have done for 0.2 include:

Change the underlying server from twisted.web to WSGI

Change the generated URL format

Add the stand-alone user database code, along with sessions and logins and email validation

Add a view to show changes for a page

Allow the reverting of any historical change

Daily build of trunk into the wikkid developer's PPA

Ideally for the 0.2 release I'd like to provide everything that is needed for
wikkid to be deployed as a stand-alone, public facing, wiki.

Wow, how can I help?

Wikkid uses Launchpad for colaboration and project tracking -
https://launchpad.net/wikkid.

You can get a copy of trunk using 'bzr branch lp:wikkid'

File bugs for any issues you find playing with it

Join the development and discussion mailing list

If you feel so inclined, you could implement a feature or fix a bug, push the branch to Launchpad and propose the merge

Come and chat in #wikkid on freenode, nothing fires developers up like having encouraging users

Launchpad code update

2010-05-06T12:00:00.002+12:00

We've been very busy over the last couple of months with lots of changes that most people will never notice.

Reduced latency

Branches pushed to Launchpad are now immediately available over http to anonymous readers of the branch, which includes the loggerhead code browser.

Code review email changes

When proposing a branch for review the initial emails and subsequent comments will now come in order. Previously if someone commented before the script that generated the diff was run, the comment would be emailed out first, now it isn't.

Teams requested to review now get email

Everyone in the team that is requested to review will get email now. This is a blessing for those that want it, and almost a curse for those that aren't interested. Launchpad adds a number of email headers to help users with filtering of email. Here is an example from an email I received:

X-Launchpad-Message-Rationale: Reviewer @drizzle-developers X-Launchpad-Notification-Type: code-review X-Launchpad-Project: drizzle

Since it was a team that was requested to review, there is the @drizzle-developers added to the X-Launchpad-Message-Rationale. If I was personally asked to review, the header would just say Reviewer.

Build branch to archive

This was the original name of the feature, but it is more about recipes now. A recipe is a collection of instructions on how to build a source package. We are still testing this internally, but I'm hoping to get this enabled on edge very soon. This will be extended to add daily builds.

What does this really mean?

Lets say you want to have a daily build of a project, like gwibber. You would then create a recipe that uses trunk as a base branch, merge in the packaging info, and say "Please build this every day into my PPA". And Launchpad will.

Trivial bugs

2010-02-22T20:30:00.004+13:00

This is just a quick note really. One thing I've been trying to do more and more is to address simple bugs in a more timely manner.

I use the tag "trivial" to indicate to me that the bug is very simple to fix. By this I mean that I should be able to have the fix and the test all written in under an hour, and normally under 30 minutes.

Personally I'm (hopefully) fixing one trivial bug a day in addition to other work. This way the simple bugs get some attention, and I get the feeling of accomplishing something when other things are in the pipeline that take longer to get completed.

My scheduling of trivial bugs is somewhat arbitrary. Often the most recently commented on trivial bug will get my attention.

Don't wait for perfection

2009-11-13T21:32:00.002+13:00

Way back in July I was thinking of writing a post about the new branch listings in Launchpad. I was working on making branch listings for distributions and distroseries work, for example Package branches for Ubuntu and Packages branches for Ubuntu Lucid. But the code wasn't entirely optimised. Then as things happen, the optimisation got pushed back... and back. And finally when I did get the optimisation in, I didn't feel it was worthy of talking about.

I guess the thing to remember is: don't wait for perfection. Sure it wasn't perfect, but if more people were accessing the pages, the optimisation may have happened sooner.

One thing going on at the moment is more integration of the lazr-js widgets. The main merge proposal page now has in in-page multi-line editor for the commit message. Sure, it needs tweaking, but the main functionality is there. More ajaxy goodness is finding its way into Launchpad.

One of the things that I'm thinking about right now is splitting the concepts of code reviews and merge proposals. At the moment we almost use the term interchangeably, which does cause some confusion. I'd like to have the merge proposal reflect the meta-data and information around the intent to have work from one branch be landed or merged into another branch (normally the trunk branch), and the code review the conversation that goes on around the merge proposal. Merge proposals may have an associated code review, but right now, a code review must be associated with a merge proposal.

Associated with this, I'd like to extract some state information. Currently merge proposals have only one status, which really reflects two things. I'd like to break this out into two: review status; and merge status. Review status would be one of: work in progress; needs review; approved; rejected; superseded; and maybe withdrawn. Merge status would be one of: proposed; merged; queued; and merge failed. Queued relates to the merge queues which are currently partially implemented in the Launchpad codebase, and merge failed is the state that a proposal would be set to when a landing robot like PQM or Tarmac attempt to land the branch but it fails due to either merge conflicts or test failures.

My goal for the next six months it to write more often, talk about ideas more, and not wait for perfection.

Breaking up work for review

2009-07-05T22:59:00.002+12:00

It was Friday morning after three days of working on one feature. Last thing Thurday I counted the size of the change and it was over 1100 lines and I wasn't quite finished. I found myself in the situation that turns up periodically that I wanted to break up my work into cohesive reviewable chunks. Now it isn't a matter of taking commits x through y as chunk one, and so on, as the size grew organically as I changed what needed to be changed, and wrote what needed to be written without really stopping to think about the size of the change until it was done. However now it was done, I wanted to break it up.

Last time I did this, I used looms, but Aaron told me we could do it easily using his new Bazaar pipeline plugin. So I spent some time talking through with Aaron on how to do it, promising to write it up if it worked well. I must say that it was good. During the process we identified a number of enhancements to the plug in to make it even easier.

I'm going to show the progression we made, along with our thoughts. I have trimmed some of the output when I've decided that it doesn't add value.

The first thing I had to do was to get the pipeline plugin.

$ bzr branch lp:bzr-pipeline ~/.bazaar/plugins/pipeline

Unfortunately this seemed to clash slightly with the QBzr plugin. The were both trying to redefine merge. Personally I don't use QBzr and had probably just installed it to take a look, so I removed that plugin.

Caution: the pipeline plugin relies on switch so works with lightweight checkouts. This is how I work anyway, so I didn't have anything to do here, but if you work differently, YMMV.

The pipeline plugin is designed around having a set of branches one after (individual pipes) the other that perform a pipeline, clever eh? When you have the pipeline plugin, any branch is also considered a pipeline of one.

$ bzr show-pipeline
*  nice-distribution-code-listing

What I was wanting to do was to break up this work into a number of distinct change sets, each that could be reviewed independently. We decided that the way to do this was to create a pipe before the current one, and bring changes in. This is done with the command add-pipe.

$ bzr add-pipe factory-tweaks --before nice-distribution-code-listing
$ bzr show-pipeline
   factory-tweaks
*  nice-distribution-code-listing

Right here we decided that there should be an easier way to add a pipe before the current pipe, as right now it needs a pipe name. A bug was filed to track this.

You can see from the show-pipeline command that the new pipe is before the current one. The pipeline plugin addes a number of branch aliases:

:first - the first pipe in the pipeline

:prev - the pipe before the current pipe

:next - the pipe after the current pipe

:last - the last pipe in the pipeline

Now to make the switch to the first pipe. Both :prev and :first refer to the same branch here, and I could have used either.

$ bzr switch :prev
  ... changed files shown
All changes applied successfully.
Now on revision 8747.

Now this pipe was added from the pipe after it, so it starts off with the same head revision. Not exactly the starting point I wanted, so we replaced the head of this branch with the last revision of the trunk branch that we had merged in.

$ bzr pull --overwrite -r submit:

The submit: alias refers to the submit branch. This is often trunk, and is in my project layout (specified using submit_branch in .bazaar/locations.conf).

Now the lower pipe was a copy of trunk. A good place to start adding changes I think. The next problem was how to get the changes from the following pipe into this one. Our first attempt was to merge in the following branch, shelve what we didn't want, throw away the actual merge, but keep the changed text, and commit.

$ bzr merge :next
$ bzr shelve
$ bzr stat
modified:
  lib/lp/testing/factory.py
pending merge tips: (use -v to see all merge revisions)
  Tim Penhey 2009-07-02 New view added.
$ bzr revert --forget-merges
$ bzr stat
modified:
  lib/lp/testing/factory.py
$ bzr commit -m "More default args to factory methods and whitespace cleanup."

Now this seemed very convoluted. Why merge and then forget the merge? I seemed kinda icky, but it worked. The next thing to do is to merge these changes down the pipeline. This is done through another command pump.

$ bzr pump

This merges and commits the changes down the pipeline. If there are conflicts, it stops and leaves you in the conflicted pipe. This didn't occur here, nor did it occur for any of my other ones. Here you can see the commit message that pump used:

$ bzr switch :next
$ bzr log --line -r -1
8714: Tim Penhey 2009-07-03 [merge] Merged factory-tweaks into nice-distribution-code-listing.
$ bzr switch :prev

Now it was time to add the next pipe.

$ bzr add-pipe code-test-helpers
$ bzr show-pipeline
*  factory-tweaks
   code-test-helpers
   nice-distribution-code-listing
$ bzr switch :next

This time, instead of merging in the changes, we shelved them in. The shelve command. The shelve command can apply changes from arbitrary revisions, and it also knows about files. The change that I wanted in this branch was a single added file, so I could tell shelve about that file.

$ bzr shelve -r branch::next lib/lp/code/tests/helpers.py
Selected changes:
+   lib/lp/code/tests/helpers.py
Changes shelved with id "2".

However the big problem with this is it all looks backwards. We are shelving from the future not the past. This really did my head in. Shelve would say "remove this file?" and by shelving it, it would add it in. It worked but made my head fuzzy. We filed a bug about this too. By adding a better way to take the changes, the command could do the reversal for you and provide you with a nicer question.

More of the same happened for the next few pipes, and I won't bore you with repeated commands.

On the whole, the pipeline plugin worked really well. I was able to break my work up into five hunks which could be reviewed easily. In the end I kept working on the branch that was my original, so all my original history remained intact. It would have been just as easy to add another pipe and take the remaining changes. This would have left me with five branches, each with one commit. This works well for the way we work as we have reviews based on branches. Each pipe could be pushed to Launchpad and a review initiated for it. With some more UI polish, I think pipelines will be even more awesome than I think they are now.

You're doing it wrong!

2009-06-18T20:42:00.004+12:00

Just yesterday I found a missing feature in one of the apps I just started using. My thought processes were something along the lines of “hey, I could add this feature and it would be good”. So I went to the project's website, found their source code repository, and got blown away by the comment that was with it:

Please note that code you get from this repository is not intended for productive use (unless it's tagged as a released version, of course, in which case the usual alpha/beta disclaimers apply ;-)). We like to break our codebase, config files, database schemas and all kinds of stuff. We sometimes commit non-compiling revisions to facilitate collaborative development. Running such an unstable version might trash your settings, your backlog and maybe your computer. You have been warned!

Eh? OK, I get the first sentence. It is even a good disclaimer. Tagged releases are more stable. People regularly commit code that is unpolished. Sometimes even with some known bugs or issues.

The second sentence has me going “NO!?! What are you doing?”

The third sentence just blew my mind. This project is using a DVCS. Not my DVCS of choice, but really that doesn't matter. All DVCSs are made to have good merging and sharing of code between developers. Saying “We sometimes commit non-compiling revisions to facilitate collaborative development” is just a lack of understanding of how to use the tools. You are using a DVCS to facilitate collaborative development! This is centralised version control thinking.

Try this for a code to work by:

Trunk should always at least compile, run, and pass all the tests.

This hasn't stopped me wanting to work on the code, but it has raised my caution levels.

kiwipycon

2009-06-16T21:45:00.003+12:00

Today NZPUG held yet another organisation meeting for the first kiwipycon. Organising conferences takes a lot of effort by many dedicated people. The Christchurch python user group has volunteered to host the first PyCon in New Zealand. Personally I suggest things from time to time, but a big thanks goes out to those guys for the hard work that has gone on even before the call for papers.

The dates have been set as Saturday the 7th and Sunday the 8th of November 2009. A weekend was chosen to allow those working or studying who can't get leave to attend. As I understand it they are still working on pricing. The call for papers will probably go out next month some time.

Should be interesting...

launchpadlib updates for branches

2009-06-15T17:04:00.003+12:00

Just a quick note to yet you know of a few changes to launchpadlib for branches. Mainly because I've removed a method that I know someone is using.

You used to be able to get to the branches for a project by saying my_project.branches, but I've removed this. It would have been nicer to deprecate it but we don't have a nice deprecation method right now for launchpadlib, and since it is still in beta, I didn't feel too bad.

The branches of a project was an attribute, now we have a getBranches method. The old attribute would give you all the branches of the project, including the merged and abandoned ones. The method defaults to give you the active branches, and allows you to pass in the statuses that you'd like to get.

Also with this change you can now get the branches for a project group, or the branches owned by a person using the same getBranches method call.

Project groups also grew the method getMergeProposals in the same way that the method was already available for people and projects.

Please file any bugs on the launchpad-code project on Launchpad.

lp:mad

2009-03-11T21:19:00.002+13:00

One of the things that I have spent quite a lot of time on recently is the code review stuff in Launchpad. Recently, as of the 2.2.2 release, new merge proposals get a review diff created for them automagically. This review diff is based on the changes that have been done in the branch relative to the least common ancestor (LCA) of the target branch. Since the review diff only has changes that have been added, there is no way for this diff to ever have conflicts.

There is another diff that is useful to see however. This is the diff of what changes would happen if the source branch was merged into the target branch right now. Sometimes this might conflict. Sometimes this might be a smaller diff as some other dependent functionality has landed. This diff isn't generated automatically by Launchpad. However this is something that you can run to add it.

The Merge Analysis Daemon

Alright, it isn't exactly a daemon (yet), but the name was cool.

What this script does, using the launchpadlib API, is to get all of the current merge proposals for a project and works out the diff that would be — what we call the preview diff.

What do you need

Firstly you need branches of wadllib, launchpadlib, and mad.

$ bzr branch lp:wadllib
$ bzr branch lp:launchpadlib
$ bzr branch lp:mad

Inside the mad directory, there is the LICENSE file (GPL v3), and the script.

The script has many parameters.

$ ./mad.py --help
Usage: mad.py [options]

Options:
  -h, --help            show this help message and exit
  -v, --verbose         Display extra information
  -q, --quiet           Display less information
  -p PROJECT, --project=PROJECT
                        The name of the Launchpad project.
  -r DIR, --repo=DIR    The location of a local repository to use.
  --dry-run             Don't upload the diffs
  --force               Force an update of the diff.
  --staging             Update the proposals on staging.launchpad.net.
  -c FILE, --credentials=FILE
                        The credentials file. Defaults to ~/.launchpad/mad
  --cachedir=DIR        The location of the cache directory. Defaults to
                        ~/.launchpadlib/cache.
  --no-op               Don't get the proposals for the project.
  --new-credentials     Get a new OAuth token and save in the credentials
                        file.

I have the following in the server's crontab listing:

20 * * * * PYTHONPATH=/home/tim/launchpadlib:/home/tim/wadllib /home/tim/mad/mad.py -p storm -r /home/tim/sandbox/mad-playground -c /home/tim/.launchpad/mad -v >> /home/tim/mad-storm.log 2>&1

Basically this says:

At 20 minutes past every hour

Run the mad.py script using a PYTHONPATH that knows about wadllib and launchpadlib.

Use the credentials file ~/.launchpad/mad

Use the respository at ~/sandbox/mad-playground

Be verbose

Make all output go to a log file.

If the specified repository directory does not exist, a new shared repository with no working trees is created. If there is an existing repository, it will use that.

Each of the source and target branches are pulled into the repository. MAD won't create branches for them, it just grabs all the necessary revisions. MAD then calculates the diff that would be if the source was merged into the target, and sends that to Launchpad to have it annotate the proposals. As an example, see the storm ones.

You will also need to have permission to edit the proposals that you are wanting to update. If you are the person that is running the project, and are in the team that owns the target branches, then you should be able to update them.

There is a --staging option to test the script against what is in staging.

The script also walks you through the necessary OAuth token acquisition the first time you run the script.

Report bugs on Launchpad.

Shallow branches or history horizons

2009-01-21T20:53:00.002+13:00

There is an idea floating around and I'm curious to see if it is an idea that has merit and worth putting effort into. This idea is in the DVCS space and is called "shallow branches" or "history horizons".

The concept itself is pretty simple. When using a DVCS with a project with a long history, each and every user has a copy of this history. Now much of this history may be ancient (for some definition of ancient, 6 months, 6 years, whatever). Most developers will never have a need to go into the ancient history of a project, and so a truncated history is fine as long as their branches that they create are still mergable with the main repository.

Here's how it could play out:

Bob wants to work on the fooix project to fix a minor bug, this is Bob's first look at the fooix source. The fooix project has been around for eons and has a huge history. Bob doesn't care about the history, he just wants to do his simple fix (think a typo).

Bob grabs the fooix trunk branch but only gets enough history to create the working files.

Bob makes his fix, and publishes his branch for the fooix developers to grab.

The advantage here is that when Bob grabs his branch, he is only getting just enough history to work, and so his resulting repository is smaller and faster.

Commands that worked by inspecting the history would stop at the repository's horizon and say something like "and that's all I've got". Obviously there'd need to be a way to say "go and get me another 4 months of history" or even "ok, now I'm really interested, get me the complete history".

This is conceptually different from a lazy loading or stacked repository as there is an explicit horizon where normal history commands stop.

So lazyweb, the question I have is this: "Is this a worthwhile feature in a DVCS tool?"

Bazaar has the model right

2008-07-11T23:14:00.003+12:00

Some people in the GNOME community have suggested that if Bazaar has nice usability, then GNOME can just use Git on the back-end, and Bazaar lovers can just use the Git back-end via Bazaar. It's true that Bazaar could support this — an experimental plug-in exists to do this right now. But this suggestion betrays several wrong assumptions.

People assume Git and Bazaar are the same. They're not. People assume that if Git and Bazaar have technical differences, then Git must have it right.

The problem with these assumptions is that usability begins at the ground level. Bazaar started with a focus on usability. Git began with a focus on speed. The data models of both Bazaar and Git reflect their initial focus. But Bazaar's model can also be fast. In fact, the Bazaar developers are currently optimising a number of key operations for speed.

Data retrieval

Git and Bazaar are both key/value mapping systems. When bytes are needed, they are requested with that key.

The big difference is that Git's keys are also the hashes of the bytes. This is why it's called a content-addressable file system. This allows git to offer a guarantee that if the value hashes to the key, it has not been modified, whether deliberately or by accident. The Bazaar team considered adopting this approach, but decided it was too constricting. Bazaar uses UUIDs instead.

Authenticating revisions

For detecting malicious modification of revisions, Git uses its cryptographic hashes.

Bazaar uses revision-signing. All revisions can be PGP-signed. No signed revision can be forged. And the hashed representation can easily be generated and passed around to ensure that exactly the same content is used.

If SHA-1 is broken, both Bazaar and Git will lose their ability to detect malicious modification. But since Bazaar uses UUIDs to identify revisions, users can re-sign their old revisions with whatever method proves to be secure. Changing the hash used by Git would make it incompatible with all existing repositories.

Data Integrity and Serialization formats

Bazaar stores hashes of every value, so it equally capable of detecting accidental modification. It can be useful to have different representations of a tree in different repositories. For example, when Git lists files, it divides this data by directory. This is a good approach, but not necessarily the best approach. An alternative approach would be to use a radix tree. This would ensure that Git performed quickly even if users put unreasonable numbers of files in a single directory. But Git's keys are hashes, upgrading Git's format to use radix trees would change the keys, which means that people could not use the commit-id from one repository to refer to the same tree in an other form.

Bazaar doesn't assume it has the perfect format. It provides an upgrade path, and does't change the commit-id of a revision if you change your format. What's more, Bazaar can even reference data it has never seen. This allows partial imports from other VCSes to be fully compatible with more complete imports. And if a VCS provides UUIDs (content hashes certainly qualify as UUIDs), Bazaar can refer to those UUIDs directly.

File and directory representation

Git refers to files by path. It makes no attempt to track renames in its data store.

Bazaar has an inode abstraction; files and directories both have ids. When a file is renamed, its id stays the same. Bazaar's core code refers to files by their id, so merging a renamed file requires no special effort.

Git's approach means that users are warned not to rename files while changing their content. But when files are renamed, those files that refer to the renamed files must have their contents changed as well. For example, if you rename foo.h and foo.c to bar.h and bar.c, you should update the contents of bar.c, or else you will break the build. With Bazaar, users can do whatever they want, and the VCS just works. While Git must always use heuristics to deduce renames, Bazaar does not have to. Of course, it can if it wants to. This is an example of why it is important to design a model for usability from the beginning.

Bazaar can import rename data losslessly from foreign VCSes. Some other VCSes support file-ids, and Bazaar can reuse those without change. For VCSes that support renames, but not file-ids, Bazaar's representation is also non-lossy. When data imports are deterministic and non-lossy, it's easy to export them back to their source VCS. Bazaar's Subversion integration is a great example of how this can work.

Choose the back-end with the right model

In any situation it makes sense to use a back-end that stores the richer dataset. It makes more sense to have a front end client that doesn't use all the functionality or data representation of the back-end than it does to have a richer client that isn't able to store the required information as the back-end is not able to represent it.

If a single back-end storage is going to be used, it makes more sense to use a Bazaar back-end as Bazaar is able to represent everything that Git does, but the reverse is not true.

Conclusion

The Bazaar developers focused on usability, which requires having a model that supports usability. Bazaar has improved its model to increase the usability of the system. We believe that Bazaar has the right model.

co-written by Aaron Bentley and Tim Penhey

Re: Interesting (rough) statistics at the GNOME distributed RCS BOF

2008-07-10T22:14:00.002+12:00

J5 mentioned in his post his interpretation of the number of users for GIT, Bazaar and Hg (Mercurial). He also finishes with "Converse amongst yourselves".

I guess I should first point out that I am a Bazaar user, and that I work for Canonical. I felt somewhat enraged at the post from J5, and have spent some time trying to work out some response.

John Carr mentioned that 83% of statistics are made up on the spot, and that cannot be more true here. I had been waiting for someone else to post the numbers that they saw at the BOF, but so far I have not seen one.

Here is my take on it.

Yes there were more GIT users than Bazaar users at the BOF, but the numbers were more like 50% of the audience were GIT users, and about 40% were Bazaar users. Someone piped up and said "What about Mercurial?" and so the question was asked, and there were about five or six people. There was an overlap of the GIT and Bazaar groups, and there was by far the larger majority of the audience that had not used any DVCS.

What conclusions can we draw from this? Not much. Many people attending the pre-conference work for larger companies, like Red Hat, Novell, and Nokia, and many of those people work on some hard core linux stuff, many of which have chosen GIT. Many have chosen GIT because that is what the linux kernel is using. Is that a good reason to chose a DVCS? I don't feel that we can really answer that question as I am sure there are strong advocates for both sides.

An interesting question is "Which DVCS is easier for the casual contributor to use?" Surely one of the reasons that a project chooses a DVCS is to allow for more community contributions in an easy to merge way that has a clear contribution history. Bazaar just works. It works for the hard-core developers, but is also easy for those soft-core (?).

From the people I talk to, and I've tried to talk to many here, is that of those that use Bazaar it just works. Bazaar doesn't get in your way of developing the software that you are working on. It is just a tool that works.

One final point. The questions were "Who uses <insert DVCS>?", not "Who likes/loves using <insert DVCS>?".

Shelving looms

2008-06-28T22:06:00.002+12:00

For a feature that I'm currently working on, I decided to try out the loom plugin. Looms have been around for a little while, but I just hadn't gotten around to trying them out.

We have code reviews of all work that is to be merged. Part of this process is to try to limit changes to 800 lines of unified diff. We have found that when the branches have more changes than this the time to review the branch increases non-linearly with the increase in line count. In the past in order to break up a "chunky" branch I would branch from trunk for the first part:

$ bzr cbranch trunk feature-part-1

(I use cbranch as I don't have my working trees in my repository. This is another story to write about. cbranch is found in bzrtools.)

Once this part was complete, I would branch from that for part-2:

$ bzr cbranch feature-part-1 feature-part-2

Complications come in when I want to bring an updated trunk into my branch for part-2, as it makes getting a diff of changes much more difficult as I can no longer generate a diff simply. This problem propagates if I need three or four parts to implement the feature.

Enter looms. Looms provide a new branch format for Bazaar. To convert your branch to the new format, you use the command loomify. You can then create threads of your loom. Each thread is like another branch.

So, the process goes something like this:

$ bzr cbranch trunk my-new-feature
$ cd my-new-feature
$ bzr loomify
$ hack, commit, hack, commit, hack, commit
$ bzr create-thread next-part-of-feature

Creating a thread is like creating a new branch which has the same revisions as the last thread.

$ hack, commit, hack, commit, et al

However what happened after several hours of hacking away, and several diversions in the code that needed fixing, I checked out the size of the branch.

$ bzr diff -r thread:

(The loom plugin adds a revision specifier to easily allow things like this to see all the changes that were introduced by the current thread.)

Oh, damn. The resulting size was about 1100 lines. Now while our 800 line limit is set in stone, it is considered a bit rude if you could have broken it up but didn't. Stepping through the diff I identified three distinct chunks of work that could be broken out for review. My question was "now that I have all this in a loom, and I have a vested interest in keeping the loom as the current work is based on an earlier thread, how the hell am I going to break this work up?" Shelve to the rescue. Shelve is also found in bzrtools. I also wanted to have the threads named reasonably, and unfortunately there is no easy way to rename a thread right now. I wanted to have the threads named alpha, bravo and charlie (well, not really, but you get the picture). The first step is to create alpha and get rid of the current thread.

$ bzr create-thread alpha
$ bzr down-thread

create-thread takes you to the new thread too, so using down-thread to go back to the thread I was working on before.

$ bzr combine-thread

This effectively discards the current thread. The assumption is that the changes from this thread had been merged into the lower thread through merging another branch. This hasn't happened in this case, and discarding is what we want here. So now I'm at the state where I was before except my thread is now called "alpha". Now to break out the changes for alpha.

$ bzr create-thread bravo
$ bzr down-thread

I created a thread bravo. This is also at the state where all three parts are there and working. Next I went back to "alpha" thread. Now we use shelve using the revision specifier that looms introduce. Shelve by default will just allow you to shelve (or put to one side) the uncommitted changes.

$ bzr shelve -r thread:

Now I get lots of questions. Do I want to shelve each of the chunks. I shelve all changes that are unrelated to the alpha feature. What I'm left with after this command is a working tree as it would be if I had just written the alpha feature on a clean thread. I checked the results with:

$ bzr diff -r thread:

Looks good, so commit.

$ bzr commit -m "Shelved changes unrelated to alpha."

Now for the magic.

$ bzr up-thread

This takes me back to "bravo". However up-thread also merges in the changes from the thread below. Now my tree is showing that all the changes relating to bravo and charlie have been removed. The actual merge magic is done with this command:

$ bzr revert .

Take a special note of the dot. Without the dot the revert would revert the entire merge. I don't want this. I just want to revert the changes to the tree. I need bzr to remember that I have merged the changes from the thread below. This is exactly what the "revert ." does. The changes to the three are reverted, but the merge isn't. Next you need to commit.

$ bzr commit -m "Merge from alpha while splitting up the changes."

Now I have the alpha thread with just the changes needed to implement alpha, and a bravo thread that appears to introduced the bravo and charlie features. I also have a .shelf directory (created by shelve). Since I have no intention of unshelving these changes (as they are already there), I delete this directory. I'm not sure if this is strictly necessary, but I like to run a clean shop.

To break apart the bravo and charlie features I repeated the process. The end result was three separate threads that each appear to introduce a single feature.

Phew.

One point of caution. Sometimes in the breaking apart, you don't always get a clean break. In these situations you need to keep more than you need (i.e. don't shelve that change), and once the revert and commit is done, then go back to the earlier thread and clean up. If you try to do it earlier, the changes will be thrown away in the revert dot command, and then it just gets messier.

All in all I'm really enjoying working with looms. I currently have about eight threads, and will probably need another four or five to finish the feature, but this way it is dead easy to keep the changes small and distinct and simple to review.

bzr alias

2008-06-08T00:22:00.002+12:00

I now officially have some of my code in an open source project where the work was actually done entirely on my own time. Despite being involved with professional software development for almost twenty years, I've normally kept my programming to work hours, or private projects.

This time though, it was different. This time it was truly scratching an itch. I've become somewhat of a Bazaar convert. Bazaar has for a long time allowed you to define command aliases in your bazaar.conf configuration file. I have always used bash aliases for commands that I do really often, so it seemed natural to me to define aliases for bazaar for commands that I used often as well.

Next came the internal conflict. I am an inherently lazy person. This is why I like aliases, less typing. One of the things that bugged me was having to actually edit the configuration file any time I wanted to add or modify an alias. This bugged me. It bugged me so much it caused me to actually do something about it. Luckily for me Bazaar is written primarily in python, and this just happens to be my current favourite language.

The code is now committed to trunk, and should be available generally in bzr 1.6.

bzr alias — lists your current aliases

bzr alias commit="commit --strict" — sets the alias for commit

bzr alias commit — print out the current alias for commit

bzr alias --remove commit — removes the commit alias

Obviously if you use alaises as much as I do, one of the first things you'll do is set the alias for unalias.

bzr alias unalias="alias --remove"

My current aliases (that I'll tell you about):

tim@slacko:~/src/bzr/alias-command$ ./bzr alias
bzr alias cbranch="cbranch --lightweight --hardlink"
bzr alias col="checkout --lightweight"
bzr alias commit="commit --strict"
bzr alias lastdiff="diff -r-2..-1"
bzr alias lastlog="log -r-2..-1"
bzr alias ll="log --line -r-10..-1"
bzr alias my-missing="missing --mine-only"
bzr alias unalias="alias --remove"

Code in Launchpad

2008-05-17T22:55:00.001+12:00

Launchpad offers many things to developers, and open source software developers in particular. One of these things is the ability to host Bazaar branches. For those that have looked a little deeper, they will have noticed that there are four types of branches in Launchpad: Hosted; Mirrored; Remote; and Imported. Hmm, this isn't really what I was intending to talk about at all, but I'm going to go with the flow.

Hosted branches are those where Launchpad is the primary public location of the branch. Hosted branches are normally created by pushing a branch directly to Launchpad. Before you do that though, you need to have registered on Launchpad, and supplied an SSH key. This is how Launchpad knows who you are. There are two ways you can push a branch to Launchpad: one is via SFTP; and the other using the Bazaar smart server (bzr+ssh).

As an example I'm going to use my alias-command bzr branch. The complete SFTP location would be sftp://thumper@bazaar.launchpad.net/~thumper/bzr/alias-command, and the smart server one bzr+ssh://thumper@bazaar.launchpad.net/~thumper/bzr/alias-command. These are a bit unwieldy, so we extended the lp type urls for bzr to support writing if the launchpad plug-in knows who you are. In order for you to do this you use the lp-login command. bzr lp-login will tell you the username that is currently set. If you have not done this yet, you'll see a message like "No Launchpad user ID configured." I set mine by saying bzr lp-login thumper. This stores thumper as the launchpad_username in the bazaar.conf file. This also means I can use bzr push lp:~thumper/bzr/alias-command to push to my hosted Launchpad branch.

Mirrored branches allow you to have your branches stored publicly in some location that you control, and you let Launchpad know where this is. Launchpad will then update its copy of your branch every six hours. This is handy if you don't have an SSH key, or you have a slow network connection, or you just like having your branches available on your own server.

Remote branches are a bit different. Remote branches were sort of created out of necessity. Some people were registering mirrored branches with unreachable locations. Some of these were possibly by mistake, but quite a few were obviously inaccessible. But more strange is that those branches were linked to bugs or blueprints. There was obviously a desire to have branch meta-data there, but not actually allow Launchpad to get access to the branches. So we have remote branches. You cannot get a copy of a remote branch from Launchpad as Launchpad does not have a copy of it.

Imported branches are those branches where Launchpad get the code from either CVS or Subversion, and puts it into a Bazaar branch. I was really wanting to talk about this as I saw two projects recently where we are importing code that I didn't know about. One is my favourite music player, Amarok, and the other was MPlayer. Just out of curiosity I looked at both of these branches on Launchpad. The Amarok one has 12195 revisions as I'm writing this, and the last revision was 11 hours old, and MPlayer had even more revisions, at 26761. However that isn't even the cool bit. What is really nifty is you can go bzr branch lp:amarok or bzr branch lp:mplayer to get the code. Just to check I did just that, and got a copy of the amarok source. It was the first bit of C++ I had looked at in a long time (it used to be all I did).

Anyway, that was what I really wanted to say. Oh yeah, and bzr rocks.

The Launchpad branch directory service

2008-05-10T23:38:00.002+12:00

Recently Bazaar grew a branch directory service. This allows plug-in developers to define custom "protocols" that resolve the branch names into some other branch location.

The Launchpad plug-in defines a protocol "lp". Launchpad uses the other parts of the URL to relate to projects, series or individual's branches. The shortest valid URL for Launchpad is something like lp:do. You can also use an empty authority (or site part of a URL), so lp:///do is exactly the same, just longer. Personally I prefer the one without all the slashes.

The do part of the branch location relates to the GNOME Do project on Launchpad. There is a little magic (a.k.a. configuration) that is needed to make this work. Projects in Launchpad get an initial development focus series created for them. This is intended to relate to the branch of development that is where current or new work goes. In order to have the code available through the Launchpad directory service, the code has to be available through Launchpad as a normal branch.

Once a branch has been either hosted, mirrored or imported for the project, one of the people responsible for the project in Launchpad can relate the branch with the series. Once this is done the branch is easily accessible. People that have permission to make these links will be shown a link on the main code tab for their project (we don't taunt people who can't make the links with an invalid option).

If there is another series, say 1.0 for our project fooix, and we have branches associated with them, then we could get the 1.0 branch using lp:fooix/1.0. Normal Launchpad branches are also accessible using the lp protocol using lp:~username/project/branch-name.

The December push-up challenge

2007-12-21T16:20:00.000+13:00

Yay, finished early. We had a challenge running at Canonical to do 2000 push-ups during December. I've just done my 2000th push-up for the month. It has actually been quite good. I've found that the most important bit is just to do a set regularly, and really try for four or five sets a day.

Also I've finished work for the year. Another yay! It has been a good year, with much good stuff getting done. Even more interesting stuff is due to come out in the first six months of next year.

With another year heading past, the girls are a year older. Jessie is starting school in February next year at the start of term, and Maia is now old enough to go to playcentre by herself (and she has done so at least once).

It seems time to start thinking about new year resolutions...

Cambridge, MA

2007-11-01T15:50:00.000+13:00

Well, my body is now almost completely adjusted to US east coast time. I do find myself waking each night between two and three in the morning. I'm thinking that this might be due to me expecting to hear Jessie or Caitlin waking, but here it is only Joey.

The rest of the Launchpadders arrived tonight. I ended up meeting so many of them it isn't really worth me writing them down for fear of forgetting someone. We have the agile training tomorrow, which should be quite interesting.

Cambridge, or the small bits of it that I have seen, reminds me of London. But perhaps that is just because they are digging up the roads all over the place, and the pavement is made up of concrete slabs and bricks that have been laid for so long that they are no longer flat.

I thought the USA was supposed to be cheap, but so far the meals out in the evening have been about the same that I'd pay back in NZ (converting the dollars). However generally books and DVDs are cheaper. I've put through an order to amazon.com and getting it delivered to the hotel.

Trivialities

2007-10-10T20:25:00.000+13:00

Well the laptop has been running mostly well since I upgraded it. Just a few times when it really had problems. Most of those seem to have been solved by making KDE start a fresh session every time rather than trying to use what I had running last time. It means a bit of manual tweaking each time I log in, but I tend to just leave it on most of the time.

I have Jono and James here this week working with me and we are attacking both features and usability issues with Launchpad and Bazaar. I feel that things are coming along nicely.

I've just joined a squash club, but yet to organise my first game. I know that I'll be really stiff after the first smack around.

Going gutsy

2007-09-26T14:47:00.000+12:00

I upgraded my desktop machine to Gutsy (Gutsy Gibbon, Ubuntu 7.10) several weeks ago, and it has been going reasonably well, so it is time to upgrade the laptop. The distribution upgrader application happily told me that I needed to download around 1.4 gig of updates that would take around 2 days 7 hours using a 56K modem. I'm happy that I have a decent ADSL connection, although it is still saying that it has got between 4 and 7 hours remaining depending on the current download speed.

The official Gutsy release candidate is in about a week or two. If you feel so inclined, you too can upgrade and help report bugs.

I just wish I could figure out how to use my favourite window picker that beryl has that compiz doesn't seem to (or at least I haven't found it yet).

Rediscovering stuff

2007-08-22T22:35:00.000+12:00

When we left for the UK we only took a couple of suitcases. After a number of years away we sold our house and put much stuff in storage. This stuff has just been delivered to our house.

There is much cool stuff there. As I was helping the guys get the stuff out of the van, I grabbed a box that said "Glass, Fragile" and it was somewhat squashed and at the bottom of a pile. Yes there was a broken glass vase in there, but I haven't gone through everything yet.

I did find a copy of "Ferdinand" that was given to me back in 1975. It is in quite good nick given its age.

More unpacking to happen this weekend I guess.

Crap at blogging

2007-08-22T22:31:00.000+12:00

I can't believe that it has been almost a month since my last post.

I'm sure it is because I don't really think that much I've what I'm doing is really interesting to anyone :-)

CVu time again

2007-07-25T09:11:00.000+12:00

Every couple of months I end up with a busy time for ACCU. This is the time when all the articles need to be with the production editor, and the period just before making sure that they all get there on time (or not too late anyway), and the period after when reviewing all the proofs.

ACCU is great in that it gets developers writing articles for other developers on all sorts of topics, and this month is no exception.

Facebook

2007-07-19T21:42:00.000+12:00

I've been sucked into the world that is facebook. Invited by one of the guys that I used to play rugby with in the UK. I had been staying away from the social networks (apart from Linked In). I had heard they were addictive, and I didn't really see the point.

However when I received the invite I thought "Why not?", so I registered. The interesting part was when I let facebook check my gmail box for people that I know. There were quite a few people that I knew already using facebook. Yay what fun.

Another week

2007-07-15T21:49:00.000+12:00

Well the school holidays are now almost over and tomorrow morning I have to start getting up early again. Not looking forward to that.

Last week was another chaotic work week with the branches for the next release needed to to be ready for review a week ago, and then reviewed and merged by the end of the week just been. By the start of Friday there were around 15ish branches in the queue. By the end of the day over 20. It took some time and a couple of tries but finally my branches made it through.

I really do love bzr. The tool allowed me to merge my three branches into the first. When that failed due to a branch that landed just prior, I merged that first branch (now three branches) into the second (which is one of the three) and fixed the problems. That one failed too but for a different reason. I then merged the second branch into the third and fixed the next. At this stage I also included another branch of a colleague and resolved all the conflicts and errors. That one made it. Yay! Bzr rocks!

Back home again

2007-07-04T11:45:00.000+12:00

For my nine elapsed days away, a full three of those days were spent traveling. That really does suck.

Arrived back in NZ very early Monday morning. Quickly through immigration to spend ages waiting at the baggage carousal. There ended up being about 20 people still waiting for luggage. My guess is that the baggage handlers missed a pile and eventually found it. Anyway after finally getting my bag I headed through customs to the domestic transfer desk. The line was huge, and I only had an hour before my flight left. Last time we waited at the domestic transfer desk we got told that there wasn't enough time to get our luggage transferred and we had to make our own way to the domestic terminal. Well, this time I didn't feel like waiting for half an hour to be told to go away, so I decided just to walk to the domestic terminal.

Rachel and the girls were waiting at the airport and it was wonderful to see them again.

Yesterday afternoon we finally got the central heating working. Oh boy it feels good! This morning it did feel too hot, so we went around turning the radiators down. There is another cold snap on the way, so I'm really happy.

I seem to have moved completely back into NZ time already with just feeling a little more tired in the evening than usual.

Dinner

2007-06-29T10:27:00.000+12:00

I have just got back from the team dinner. Me and my roomie both grabbed laptops and are sitting on our respective beds "surfing" the internet. Gee, what geeks!

Some of us left work slightly early today, which really means about 6:30pm rather than 7. We went to a close pub for a drink before dinner. When the crowd arrived, they settled in playing a "pub quiz" game. Funny, but when there were "world" quiz questions, they related directly to either England or if you were lucky, Europe.

The interesting topics of conversation over dinner were games. We played two games over dinner:

What movie - given a quote or two from frequently obscure movies

Canonical trivia - like "who had their laptops stolen at the first sprint"

All in all a reasonably nice evening. The problem with the movie trivia game, is that you think you've watched heaps of movies, but when it comes to thinking of a quote that isn't immediately obvious as to which movie it comes from, your mind goes blank. Well mine did.

The "Air Blade"

2007-06-28T10:28:00.000+12:00

The Canonical offices are in a much nicer state now than when I first saw them at the start of April. Firstly the construction has finished, all the meeting rooms are finished, and the kitchen now has a fridge. There is even an espresso machine, although the coffee grinder is currently broken and they only have medium grind beans for the filter machine. Nice office though.

One really cool device that they have though can be found in the toilets. Not normally a place where you expect to find gadgets, but I include this device in that category. It is the Dyson airblade. This thing is a hand dryer like you haven't seen before. Surprisingly fast at drying your hands.

We seem to be spending very long hours talking about work stuff. In the office at 9 and not leaving until after 7 in the evening. On Monday it wasn't until almost 8. I did manage to pick up some CDs to pass around back home, and some ubuntu stickers.

Lots of planning going on, nothing that I want to bore y'all with.

Hotel Wifi

2007-06-25T07:44:00.000+12:00

It is now 8:44pm London time and I am starting to get a fuzzy head. Internet access at the hotel is £10/day, but it was pointed out to me by my roomie that there was an open wifi access point. So I'm taking advantage of someone else's internet connection right now. They haven't even set an administrator password on the router! This is a first for me. I've heard about it, but this is the first time I've been tempted to use it.

The flight from Auckland to London was long and uncomfortable. A number of times I awoke due to my arms going to sleep from having pressure in odd places from the really uncomfortable arms on the chairs. I did however manage to get some sleep, and have kept awake until 9pm so far. I'm guessing that I'll be completely out by 9:30.

Breakfast is at 7, work at 8(ish).

Snow?

2007-06-25T07:42:00.000+12:00

Here I am sitting in Auckland International Airport slightly irritated that there isn't a Telecom wireless hotspot. There is an Auckland Airport hotspot, but of course they want their pound of flesh to use it.

I very nearly didn't make it this far today.

It snowed yesterday evening. As it was snowing I was thinking "well there it goes, should be gone by morning". I realised that there may be problems when I was up at 2am to see to Jessie, who had just had a bad dream, and I saw it snowing outside again. When I woke up this morning there was a good 5-7cm of snow on the ground. I know that doesn't sound like much to those of you used to a lot of snow, but in Dunedin everything stops.

As soon as I was up, I got on the web to see if I could get any information about closures for roads or the airport. I found the phone number for the airport, but no answer there. Any how, I had to get packed. These days I can pack for international travel in about 20 minutes. That done I thought it best to check with my airport shuttle. I called the shuttle number and talked to the lady at the other end. Before I got much out she said "If you are chasing a taxi, forget it." After a brief explination she said that there would be no chance of me getting picked up, but if I got to their depo, they would get me to the airport. My next thought was "who do I know with a four wheel drive vehicle?" Unfortunately the list was slightly shorter than one.

Walking it was. Dragging my suitcase, I started down the hill to the taxi depo. I only arsed up once going from Highgate onto Stuart Street. It was very icy there. I decided to walk down Stuart Street rather than some of the more direct routes down the hill on the off chance of hitching a lift. I was about 100m down the hill when I saw a van start down. As I was looking at it I decided "I'd rather walk than get in that on this ice." As they went past, a guy hung out the window and snapped a photo of me dragging the suitcase through the snow. The van went about another 20m and then stopped. It tried to reverse a little up the hill, but no chance, it just slid sideways. As I came up next to it they asked if I wanted a lift, and before I thought any more about it, "sure" came out of my mouth. They were just three guys out for a drive to look at other people in trouble. They had reduced the pressure in the van tires to 10 PSI and the driver seemed to know how to handle driving in the snow really well. Since they were just off for a drive, they dropped me off right outside the taxi depo. Here I was, nice an early.

The airport at this stage was closed due to ice on the runway. I had managed to talk to someone at the airport, and they said they were currently doing 360s on the runway, but he expected it to be OK within a few hours. My international flight wasn't until 9:45pm, so I though I had better get myself there.

When we finally got to the airport it was about 40 minutes before my scheduled departure of 11:20am. The queue was HUGE. London huge. Earlier flights had been cancelled, and as I stood there in the queue they let us know that my flight was cancelled too.

When I finally got to see someone on the desk, he looked at my tickets and said "Sorry, there is no way I can get you to Auckland today. You have to call 0800 737 000 to talk to Air NZ reservations. I can't help you since you have an international leg on the flight."

Dragging my bag I head up to the cafe to make the call. "We are experiencing heavier than normal call volumes. The current wait time is in excess of ten minutes." No shit sherlock. Anyway, I hold and finally talk to someone. "All the flights to Auckland are entirely full" she says. I ask about tomorrow. There was one space on a flight to Auckland tomorrow morning, but the international flight was full unless I wanted to upgrade to business class. "How much is that?" I ask. "£6000" comes the reply. Otherwise the soonest I could get a flight to London was Wednesday. Bugger. "The only option" she says "is to go and see if you can fly standby to Auckland somehow and hope you get there before you flight tonight." This did not fill me with excitement. However, I felt that I must do my best to attempt to catch the flights or I couldn't honestly say to the powers that be that I did my best.

I head back to the check-in desks. The queue is still really long. I completely ignore the queue and walk to the check-in desk for the Koru club members. I tell the guy there the situation, and that I had talked to Air NZ reservations and what they had told me. Clack, clack, clack, "this must be your lucky day" he says. Bullshit I thought. He managed to get me the last seat on a flight to Auckland via Christchurch. So here I am, just downed my coffee, and saving this in a text file to upload to the blog when I don't have to pay for it.

Ramping it up

2007-06-19T20:35:00.000+12:00

Following the frustration at the end of 1.1.6, I'm ramping up for 1.1.7.

The major thing to think about here is not to commit myself to too much again. Hopefully I haven't done that this time.

Heading over to London on Saturday for a week of planning meetings. The only silver lining to 40 hours of travel to London is meeting some of the people that I work with. Chances are I'm going to spend most of the time completely jet lagged, and then get on a plane the following Saturday and fly back to little old NZ.

Missed it

2007-06-17T22:23:00.000+12:00

Well the deadline came and went, and my major feature addition missed it.

And here I thought I had it all under control. The penultimate branch didn't get reviewed until Friday, and it brought up a couple of issues. One was insufficient test coverage, and while increasing the coverage, another ugly problem raised its head. It was at that stage I realised that I really had a problem. I agreed with the proposed changes that the reviewer came back with and that caused a problem. Here it was, around 11pm on Friday night and I needed to make some quite severe changes to the implementation.

While talking to the team lead I decided that there was no point landing the code as it was, and that the changes needed to be made. So I started writing them. At around 1am I realised that it was not going to be as quick as I hoped, and still had about another hour of work to fix the code, then I had to fix all the tests that I would have broken, then I would have had to go through the code and fix all the prose in the doc strings that would no have been out of date due to the changes. So I decided to head to bed.

Saturday morning and I called the project manager. It was still Friday afternoon for him. I talked him through the situation, and we decided to let the feature slide until 1.1.7.

I decided not to touch any code at all for the weekend, and step away from it all for a few days. I had been going hard out to get the whole thing written, and was quite gutted when it all sort of fell apart on the last day.

Pressure is mounting

2007-06-12T11:34:00.000+12:00

With the deadline for having code landed looming this Friday, there is pressure to get the features code complete and reviewed ASAP.

I feel that I am almost code complete. Now to just get the code reviewed and landed in time.

Ability lost

2007-06-11T15:49:00.000+12:00

Google has seemed to have screwed up my ability to edit in need of caffeine so I've started a new one.