A Contributor's Guide to Launchpad.net - Part 2 - Code Management

Refer A Contributor's Guide to Launchpad.net - Part 2 - Code Management

 

In this second part of my Launchpad guidebook series, I'll be covering the code management and repository features of Launchpad.net. If you missed the first part of my series, go check it out and get established on Launchpad.net. Then pop back to this article to dive into the magic of http://code.launchpad.net. In this article, we'll cover the following aspects of the code management pieces of Launchpad:

• The Structure of Project Source Code on Launchpad.net
• Pulling Code into a Local Repository
• Creating a Local Working Branch for Bug Fixing
• Pushing Code to Launchpad
• Notifying a Merge Captain of Your Code Pushes
• Keeping Trunk Up To Date
• Merging Local Code with a Trunk Branch

For the following article, we'll be acting as if you are contributing to the MySQL Server project and wish to create a patch to fix a bug in the MySQL server. We'll be working through all the steps to do so. If you are looking to contribute to a different project, or your own project, simply replace the names and URLs in the article with ones for your particular project.

The Structure of Project Source Code on Launchpad.net

Projects hosted on Launchpad.net are organized using the terminology branches, repositories and series. A branch is simply a Bazaar branch of the project's source code. A repository is a collection of a project's Bazaar branches. A series is a named branch which represents something special for the project — usually a tagged release or a development branch.

Because Launchpad uses Bazaar, a distributed version control system, sometimes it takes a little while to get used to the fact that there is not a "central" source tree to which you check code into and out of. Instead, what you do is "pull" a named branch from a Bazaar server to your local workstation and work on that local branch in peace and quiet until you want to "push" your code into another branch. The branch to which you "push" may be the "trunk", or "active development" branch, but normally you will not push to the trunk branch. Instead, you'll push your local branch containing your source code changes to a personal branch on the Launchpad.net server and propose that your branch be "merged into trunk". The image to the right, from the Bazaar user guide, show the general flow of code using this type of process, called a "decentralized system with a human gatekeeper".

This pull, code, push and merge process is the recommended way to manage code changes for a project. It allows a core set of "merge captains" to review and check your code before merging your code into the active development branch. It is this process which I will be demonstrating in this article.

Pulling Code into a Local Repository

When you work on a project in Launchpad, you work on code in a local branch of the project. To get rolling, you will first want to set up a local repository if you haven't already done so. To do so, we use the bzr init-repo command:

NOTE: You will need Bazaar installed to do so. Don't have Bazaar installed? See my previous article on Getting a C/C++ Development Environment Established. Not coding in C/C++? Don't worry, just read the section of that article on installing Bazaar.

cd ~/repos # Change this to the folder in which you plan to have bzr repositories...
bzr init-repo mysql-server-5.1

At this point, a shared repository will be created. What the heck is a shared repository? Well, it's basically a special folder that Bazaar knows contains information about source code branches. It facilitates speedier branching and merging and is especially useful for larger source trees and changeset histories like the MySQL server. You can verify that Bazaar knows something about your newly created repository by checking for a .bzr hidden folder in your repository folder:

[504][jpipes@serialcoder: /home/jpipes/repos]$ ls -la mysql-server-5.1/
total 12
drwxr-xr-x 3 jpipes jpipes 4096 2008-08-26 15:00 .
drwxr-xr-x 7 jpipes jpipes 4096 2008-08-26 15:00 ..
drwxr-xr-x 4 jpipes jpipes 4096 2008-08-26 15:00 .bzr

The next step is pulling the active development series of your particular project. In our example, we'll pull the active development branch of the "5.1" series of the MySQL Server. To do so, we use the bzr branch command:

cd mysql-server-5.1/
bzr branch lp:mysql-server/5.1 trunk

WARNING: When doing the initial pull of the first branch in a shared repository, the branch command can take quite some time to execute, especially when pulling a branch of a project like the MySQL Server which has a huge history of changesets to it. Be prepared to wait a while, and if Bazaar looks like it's stuck doing stuff, just leave it alone. It typically will take about 80-90 minutes to complete the first time!

In the bzr branch command above, lp: designates we are looking for a branch residing on the Launchpad.net Bazaar servers. The colon is followed by the name of the project, in this case mysql-server, followed by a slash and the name of the series, in this case 5.1. You can always check the names of a project's series by going to the main code area of a project. For MySQL Server, that address would be http://code.launchpad.net/mysql-server

When finished the initial branch, you'll see something like the following, shown with the time command to illustrate the amount of time you should expect for the MySQL Server initial branch:

[511][jpipes@serialcoder: /home/jpipes/repos/mysql-server-5.1]$ time bzr branch lp:mysql-server/5.1 trunk 
Server is too old for streaming pull, reconnecting.  (Upgrade the server to Bazaar 1.2 to avoid this)
Branched 2719 revision(s).

real	91m30.337s
user	14m6.825s
sys	6m7.355s

The total amount of "stuff" that is downloaded for the 5.1 server is around 600MB, so it shouldn't be surprising that it takes some time to do the initial branch...

You can ignore the message about upgrading the server to Bazaar 1.2; it's because I'm not using the very recent Bazaar 1.6 client. The performance of this first branch is currently being investigated by John Arbash Meinel, one of Bazaar's developers, whom I spent some "quality time" with on IRC today. He mentions that the developers are working on something called "shallow trees" which should significantly speed up initial branching for large projects.

Creating a Local Working Branch for Bug Fixing

OK, if you've gotten this far, then you will have a local shared repository that contains a single branch which contains the source code and changeset history for the 5.1 series of the MySQL Server. What we want to do now is fix a bug in the MySQL Server 5.1 locally on our workstation. This will be a fictitious bug called Bug#99999 - "authors.h file doesn't contain MY NAME!"

Certainly, we could just start hacking up the code in the trunk branch we just pulled. But, that's not the most practical way of doing structured development on a local workstation. Instead, you should create a branch from trunk which will house only the changes specific to what you are working on: in this case, our fictitious bug#99999. Why is this a better practice than simply making the changes in the trunk branch? Well, a couple reasons:

• It keeps the trunk branch free of any of your changes. This is very important if you want to keep trunk up-to-date with changes from the rest of the project's developers and not have to constantly resolve merge conflicts (more on that later). Having a clean trunk branch local to your workstation means you can quickly and easily make local working branches from trunk for anything you are working on.
• It creates an easy-to-understand naming structure for your local repository. After a while, you might have a repository with the following structure:

  ~/repos
    /mysql-server-5.1
      /trunk # perhaps a cron job keeps this in synch every morning...?
      /bugXXXX-fix-invalid-pointer-in-binlog
      /bugXXXX-wrong-error-message-on-create-table
      /my-fancy-storage-engine
  

OK, so hopefully I've convinced you to follow the advice of creating separate local working branches for actually changing source code. Let's create a local working branch to do the fix for our Bug#99999. We use the bzr branch command, as before, but this time we'll be branching from our local trunk, and not the Launchpad trunk branch for the 5.1 series. You will notice that the branching will be significantly faster than before. This is due both to doing things locally, and to the fact that you now have a shared repository set up. Enter the following:

bzr branch trunk bug99999-fix-authors-file

Once completed, you should see something like the following:

[571][jpipes@serialcoder: /home/jpipes/repos/mysql-server-5.1]$ bzr branch trunk bug99999-fix-authors-file
Branched 2719 revision(s).

We're now ready to start fixing our bug. Hop into the newly created working branch and open up the sql/authors.h file in your editor of choice (here, I'll use Vim)

cd bug99999-fix-authors-file
vim sql/authors.h

The "fix" for this bug is simply adding your name to the sql/authors.h file of course. Go ahead and add your name to the list in that file and save and close the file. Now, that has to be the easiest bug fix ever. No wait, I take that back.

Before we commit anything, let's first check to see what changes we have made in the local branch. To do so, we use the bzr status command, like so:

bzr status

If you've done everything up until now, you should see something very similar to the below:

[503][jpipes@serialcoder: /home/jpipes/repos/mysql-server-5.1/bug99999-fix-authors-file]$ bzr status
modified:
  sql/authors.h

The next thing we'll need to do is commit our changes to the local branch. For those of you used to CVS or Subversion, this step will look familiar, however remember that with Bazaar you are committing to the local branch, not a central repository. (This isn't always the case, but for now, assume it is...)

Like in other source control systems, we'll use the commit command. There are a number of command options that you can use with the commit command, and I will outline two of them here. The most important is the -m option, which allows you to enter a string which will be the comment for the set of changes in this commit. This is an extremely useful option for smaller changesets. For larger ones, leave off the -m option and your environment's editor will pop up after hitting enter to allow you to enter in larger comments.

TIP: Remember, a best practice whenever you commit source code to a revision control system is to make the changeset comments as descriptive as possible, so other developers can clearly tell what you were doing.

The second option I'll tell you about is a nifty one which integrates Bazaar with the Launchpad.net bug tracking system. If you have a Bug report that is managed by Launchpad.net, you can supply the bug number to the --fixes option and automatically close a bug report with your bzr commit. Pretty cool, eh?

Below, I show how to use these two options with the bzr commit command:

bzr commit -m "Add my name to authors.h" --fixes lp:99999

You should see something like the following appear:

Committing to: /home/jpipes/repos/mysql-server/bug99999-fix-authors-file/
modified sql/authors.h
Committed revision 2720.

Did you know that Bazaar supports more bug tracking systems than just Launchpad.net? Check out the Bazaar user guide section on Bug Tracking integration for more information.

Pushing Code to Launchpad

Now that you've made your code changes, it's time to push those changes up to a branch on Launchpad.net. Why do we want to push the local branch changes to Launchpad.net, instead of sending the changes to another team member (using the bzr send or bzr export command)? Well, first of all, pushing the changes to Launchpad.net allows anyone to see and review your code changes, making the Launchpad.net website an easy and centralized place to do that. Secondly, having the branch on Launchpad.net allows you to get more out of the Launchpad.net platform and integrate your branch and code with other features of the platform, such as Bug Tracking and the Blueprints task management and milestone system.

To get your local branch to Launchpad.net, you use the bzr push command. The push command takes as an argument the address of the branch to which you wish to push your local changes. If you are pushing changes to a new branch on Launchpad.net, the system will create that new branch for you automatically. If you are pushing to an existing branch, the system simply uploads your changesets and applies them to that branch.

On Launchpad, there are a number of locations where we can push Bazaar branches. Each Launchpad user gets their own http://code.launchpad.net/~username area in which to put branches. Whenever you are a member of a project team, you also can push code into the code.launchpad.net/~username/projectname/ location. Also, each Launchpad user has a "Junk" area (code.launchpad.net/~username/+junk/ that they can post any old branch to. After either of these locations, you put the name of the branch you are pushing to (or creating).

To see an example, let's use our "Junk" area for right now, and push our local bug99999-fix-authors-file branch to Launchpad.net. We want to push our local bug-fix branch to a general branch in our junk folder which will contain all of our bug fixing efforts. Why? Well, there's no need to create separate branches for each bug fix on Launchpad.net just so people can see the code in the single bug fix. We can push all our code changes to a general branch and then point reviewers to the specific revision we worked on. This saves a whole lot of time when pushing branches. So, here is how we push:

bzr push lp:~jaypipes/+junk/mysql-server # Of course, replace jaypipes with your own username!

After a while, you'll see the following:

Created new branch.

You can now go view your branch and it's associated code changes by visiting the branch's code URL, which will be http://code.launchpad.net/~yourusername/+junk/mysql-server if you've been following the steps in this article.

Belonging to a Project Team

Having a branch in your junk folder is fine, but Launchpad is all about belonging to a community of developers! When you belong to a project team, you are automatically able to push your branches to the project's code area. Importantly, if you belong to a project team, then you can use Launchpad's ability to propose a branch merge — something you cannot do if you push to your "Junk" folder. To push one of your local branches to a project, you would do:

cd ~/repos/projectname/branchname>
bzr push lp:~yourusername/projectname/branchname

Once pushed, the branch will be visible to anyone when they look at the project's code branch listing, which is always at http://code.launchpad.net/projectname. If you click on your branch, you will go to the branch's main page. You'll notice links for a number of actions that you can take, including one called "Propose for merging into another branch". I'll be talking a lot about these options in future articles in this series, since many of them relate to the other parts of the Launchpad platform.

Notifying a Merge Captain of Your Code Pushes

OK, now that your code is up in a branch, the next step is to ask for a review of your code to be merged into the development branch of the project. In this case, I'm pretty sure our Bug#99999 fix isn't going be passing any code reviews to get into the MySQL Server, but I'll explain the process anyway for reference.

A merge captain is someone who chaperons a project's main development branch by being a gatekeeper for new code being merged into it. This is a critical role that some folks think is a fun or coveted job. It's not. As a merge captain for the Drizzle project, I can tell you that in fact it's mainly grunt work comprising tedious pulling, merging, building the code, and running of test suites. So, be nice to your merge captains! The best way to be nice to your merge captain is to follow these two simple rules:

1. Always make comments for your commits that are descriptive and explain clearly what your code is doing and what tasks or bugs the work addresses
2. Follow a standard, agreed upon process for notifying a merge captain of merge requests

In the case of the second rule, the standard process uses the Launchpad.net platform to notify the merge captain of a request — instead of, for instance, bugging the crap out of the captain on IRC to merge your branch.

To notify the merge captain about your branch, navigate to the main code page of the branch you pushed. Remember that to do this step, you must be a member of the project team! On the branch's main page, look for the link "Propose merging into another branch" and click it. You will be taken to the Propose Branch for Merging page. Select the target branch. The default will be the active development branch for the project in question. You may also provide a branch name. In the "whiteboard" text area, you can provide a brief description of the changes contained in your branch. Typically, you will want to keep the option "Needs Review" selected, and then click the Register button.

At this point, an email will be fired off to any subscribers for the project's trunk branch commits. The merge captain will, of course, be one of the ones which receives this email and will initiate the review of your code and the merging of it into trunk (or whichever target you specified).

Keeping Trunk Up to Date

The final two parts of this article looks at how to keep your own local branches up to date with the development branch of your project and also shows you how to merge someone else's branch with your own.

To pull all the latest changes from a remote branch into a local branch, the bzr pull command is used. You navigate to the local branch you wish to pull changes for and then issue the pull command.

This will bring in all the merges and changesets of the branch you originally branched from into the local branch. In our case, the local "trunk" branch was branched originally from the branch at lp:mysql-server/5.1 and so if we do:

cd ~/repos/mysql-server-5.1/trunk
bzr pull

We will update our local trunk branch with the changes in the active development branch on Launchpad.

Merging Local Code with a Trunk Branch

A merge, as alluded to above, is simply when you want to combine the code changes of one branch with the code in another. We use the bzr merge command to do so. The merge command takes a single argument: the location of the branch you wish to merge into the current one.

To demonstrate how the merge command is used, I'll just paste the work I just did for the Drizzle project in merging in a contributor's fixes into trunk:

[509][jpipes@serialcoder: /home/jpipes/repos/drizzle]$ cd trunk/
[510][jpipes@serialcoder: /home/jpipes/repos/drizzle/trunk]$ bzr merge ../grant-bug261687/
 M  drizzled/sql_derived.cc                                            
All changes applied successfully.
[511][jpipes@serialcoder: /home/jpipes/repos/drizzle/trunk]$ bzr commit -m /
"Merged Grant's fixes for sql_derived.  Fixes bug#261687" --fixes lp:261687
Committing to: /home/jpipes/repos/drizzle/trunk/
modified drizzled/sql_derived.cc
Committed revision 373.

The above was executed after a pulled Grant's Launchpad.net branch called bug261687, built the code, and ran the test suite to ensure no failures. I change to my local branch of the drizzle project development branch (called "trunk" locally). I then merge Grant's branch into trunk locally with the bzr merge ../grant-bug261687 command. After merging, I must commit the trunk code. I do so, making a comment that this is a merge of Grant's work, and noting the bug # which the changeset fixes. Once I do this, I am free to bzr push lp:drizzle and push the local merge to Launchpad...

Conclusion

As you can see, Launchpad and Bazaar is a feature-full code management system with a lot of bells and whistles. Hopefully, this article can get you started in your adventures in contributing to projects hosted on Bazaar/Launchpad.net. See you on Launchpad! Up next in this series: how to do task management with Launchpad's Blueprints system.