Currently showing posts tagged: hacking

Why and how to correctly amend GitHub pull requests

By , March 24, 2015 3:00 pm

Like many F/OSS developers, I’m a heavy user of GitHub, collaborating on many projects which use the typical “fork & pull” workflow based on pull requests. The GitHub documentation on pull requests covers this workflow fairly comprehensively, but there seems to be one area which is significantly lacking in detail: why and how to amend existing pull requests. The article simply says:

After your pull request is sent, any new commits pushed to your branch will automatically be added to the pull request. This is especially useful if you need to make more changes.

The problem is that this completely ignores the fact that there are often very good reasons for amending existing commits within the pull request, not just for adding new commits it.

Why amend an existing pull request?

A peer review cycle can potentially reveal many issues which make the pull request unready for merging, e.g.

  • typos
  • bugs in the proposed code changes
  • missing features in the proposed code changes
  • incomplete test coverage
  • incomplete documentation changes
  • style inconsistencies (including whitespace issues)
  • incorrect or incomplete commit messages
  • the commits violate the rule of one logical change per commit
  • some changes are outside the scope of the pull request

This is of course what makes peer review of pull requests so valuable: the problems can be addressed even before they hit the master branch, which helps maintain high quality in the development trunk. But then how do we address the issues?

Continue reading 'Why and how to correctly amend GitHub pull requests'»


Announcing git-deps: commit dependency analysis / visualization tool

By , January 19, 2015 12:15 am

I’m happy to announce a new tool called git-deps which performs automatic analysis and visualization of dependencies between commits in a git repository. Here’s a screencast demonstration!

Back in 2013 I blogged about some tools I wrote which harness the notes feature of git to help with the process of porting commits from one branch to another. These are mostly useful in the cases where porting is more complex than just cherry-picking a small number of commits.

However, even in the case where there are a small number of desired commits, sometimes those commits have hidden dependencies on other commits which you didn’t particularly want, but need to pull in anyway, e.g. in order to avoid conflicts during cherry-picking. Of course those secondary commits may in turn require other commits, and before you know it, you’re in dependency hell, which is only supposed to happen if you’re trying to install Linux packages and it’s still 1998 … but in fact that’s exactly what happened to me at SUSEcon 2013, when I attempted to help a colleague backport a bugfix in OpenStack Nova from the master branch to a stable release branch.

At first sight it looked like it would only require a trivial git cherry-pick, but that immediately revealed conflicts due to related code having changed in master since the release was made. I manually found the underlying commit which the bugfix required by using git blame, and tried another cherry-pick. The same thing happened again. Very soon I found myself in a quagmire of dependencies between commits, with no idea whether the end was in sight.

So wouldn’t it be nice if you could see the dependency tree ahead of time, rather than spending a whole bunch of time resolving unexpected conflicts due to missing dependencies, only to realise that the tree’s way deeper than you expected, and that actually a totally different approach is needed? Well, I thought it would, and so git-deps was born!

In coffee breaks during the ensuing openSUSE conference at the same venue, I feverishly hacked together a prototype and it seemed to work. Then normal life intervened, and no progress was made for another year.

However thanks to SUSE’s generous Hack Week policy, I have had the luxury of being able to spending some of early January 2015 working to bring this tool to the next level. I submitted a Hack Week project page, announced my intentions on the git mailing list, started hacking, missed quite a bit of sleep, and finally recorded the above screencast.

The tool is available here:

Please give it a go and let me know what you think! I’m particularly interested in hearing ideas for use cases I didn’t think of yet, and proposals for integration with other git web front-ends.


managing your github notifications inbox with mutt

By , October 5, 2014 1:59 pm

Like many F/OSS developers, I’m a heavy user of GitHub. This means I interact with other developers via GitHub multiple times a day. GitHub has a very nice notifications system which lets me know when there has been some activity on a project I’m collaborating on.

I’m a fan of David Allen’s GTD (“Getting Things Done”) system, and in my experience I get the best results by minimising the number of inboxes I have to look at every day. So I use another great feature of GitHub, which is the ability to have notification emails delivered directly to your email inbox. This means I don’t have to keep checking in addition to my email inbox.

However, this means that I receive GitHub notifications in two places. Wouldn’t it be nice if when I read them in my email inbox, GitHub could somehow realise and mark them read at too, so that when I look there, I don’t end up getting reminded about notifications I’ve already seen in my inbox? Happily the folks at GitHub already thought of this too, and come up with a solution:

If you read a notification email, it’ll automatically be marked as read in the Notifications section. An invisible image is embedded in each mail message to enable this, which means that you must allow viewing images from in order for this feature to work.

But there’s a catch! Like many Linux geeks, I use mutt for reading and writing email. In fact, I’ve been using it since 1997 and I’m still waiting for another MUA to appear which is more powerful and lets me crunch through email faster. However mutt is primarily text-based, which means by default it doesn’t download images when displaying HTML-based email. Of course, it can. But do I want it to automatically open a new tab in my browser every time I encounter an HTML attachment? No! That would slow me down horribly. Even launching a terminal-based HTML viewer such as w3m or links or lynx would be too slow.

So I figured out a better solution. mutt has a nice message-hook feature where you can configure it to automatically execute mutt functions for any message matching specific criteria just before it displays the message. So we can use that to pipe the whole email to a script whenever a message is being read for the first time:

message-hook "(~N|~O) ~f" "push '<pipe-message>read-github-notification\n'"

(~N|~O) matches mails which have the N flag (meaning new unread email) or O (meaning old unread email) set.

The read-github-notifications script reads the email on STDIN, extracts the URL of the 1-pixel read notification beacon <img> embedded in the HTML attachment, and sends an HTTP request for that image, so that github knows the notification has been read.

This means an extra delay of 0.5 seconds or so when viewing a notification email, but for me it’s a worthwhile sacrifice.

If you want to try it, simply download the script and stick it somewhere on your $PATH, and then add the above line to your ~/.muttrc file.


more uses for git notes, and hidden treasures in Gerrit

By , October 2, 2013 2:05 pm

I recently blogged about some tools I wrote which harness the notes feature of git to help with the process of porting commits from one branch to another. Since then I’ve discovered a couple more consumers of this functionality which are pretty interesting: palaver, and Gerrit.

Continue reading 'more uses for git notes, and hidden treasures in Gerrit'»


Easier upstreaming / back-porting of patch series with git

By , September 19, 2013 9:22 pm

Have you ever needed to port a selection of commits from one git branch to another, but without doing a full merge? This is a common challenge, e.g.

  • forward-porting / upstreaming bugfixes from a stable release branch to a development branch, or
  • back-porting features from a development branch to a stable release branch.

Of course, git already goes quite some way to making this possible:

  • git cherry-pick can port individual commits, or even a range of commits (since git 1.7.2) from anywhere, into the current branch.
  • git cherry can compare a branch with its upstream branch and find which commits have been upstreamed and which haven’t. This command is particularly clever because, thanks to git patch-id, it can correctly spot when a commit has been upstreamed, even when the upstreaming process resulted in changes to the commit message, line numbers, or whitespace.
  • git rebase --onto can transplant a contiguous series of commits onto another branch.

It’s not always that easy …

However, on the occasions when you need to sift through a larger number of commits on one branch, and port them to another branch, complications can arise:

  • If cherry-picking a commit results in changes to its patch context, git patch-id will return a different SHA-1, and subsequent invocations of git cherry will incorrectly tell you that you haven’t yet ported that commit.
  • If you mess something up in the middle of a git rebase, recovery can be awkward, and git rebase --abort will land you back at square one, undoing a lot of your hard work.
  • If the porting process is big enough, it could take days or even weeks, so you need some way of reliably tracking which commits have already been ported and which still need porting. In this case you may well want to adopt a divide-and-conquer approach by sharing out the porting workload between team-mates.
  • The more the two branches have diverged, the more likely it is that conflicts will be encountered during cherry-picking.
  • There may be commits within the range you are looking at which after reviewing, you decide should be excluded from the port, or at least porting them needs to be postponed to a later point.

It could be argued that all of these problems can be avoided with the right branch and release management workflows, and I don’t want to debate that in this post. However, this is the real world, and sometimes it just happens that you have to deal with a porting task which is less than trivial. Well, that happened to me and my team not so long ago, so I’m here to tell you that I have written and published some tools to solve these problems. If that’s of interest, then read on!

Continue reading 'Easier upstreaming / back-porting of patch series with git'»


Panorama Theme by Themocracy