more uses for git notes, and hidden treasures in Gerrit

By , October 2, 2013 2:05 pm

I recently blogged about some tools I wrote which harness the notes feature of git to help with the process of porting commits from one branch to another. Since then I’ve discovered a couple more consumers of this functionality which are pretty interesting: palaver, and Gerrit.

Continue reading 'more uses for git notes, and hidden treasures in Gerrit'»

Share

Easier upstreaming / back-porting of patch series with git

By , September 19, 2013 9:22 pm

Have you ever needed to port a selection of commits from one git branch to another, but without doing a full merge? This is a common challenge, e.g.

  • forward-porting / upstreaming bugfixes from a stable release branch to a development branch, or
  • back-porting features from a development branch to a stable release branch.

Of course, git already goes quite some way to making this possible:

  • git cherry-pick can port individual commits, or even a range of commits (since git 1.7.2) from anywhere, into the current branch.
  • git cherry can compare a branch with its upstream branch and find which commits have been upstreamed and which haven’t. This command is particularly clever because, thanks to git patch-id, it can correctly spot when a commit has been upstreamed, even when the upstreaming process resulted in changes to the commit message, line numbers, or whitespace.
  • git rebase --onto can transplant a contiguous series of commits onto another branch.

It’s not always that easy …

However, on the occasions when you need to sift through a larger number of commits on one branch, and port them to another branch, complications can arise:

  • If cherry-picking a commit results in changes to its patch context, git patch-id will return a different SHA-1, and subsequent invocations of git cherry will incorrectly tell you that you haven’t yet ported that commit.
  • If you mess something up in the middle of a git rebase, recovery can be awkward, and git rebase --abort will land you back at square one, undoing a lot of your hard work.
  • If the porting process is big enough, it could take days or even weeks, so you need some way of reliably tracking which commits have already been ported and which still need porting. In this case you may well want to adopt a divide-and-conquer approach by sharing out the porting workload between team-mates.
  • The more the two branches have diverged, the more likely it is that conflicts will be encountered during cherry-picking.
  • There may be commits within the range you are looking at which after reviewing, you decide should be excluded from the port, or at least porting them needs to be postponed to a later point.

It could be argued that all of these problems can be avoided with the right branch and release management workflows, and I don’t want to debate that in this post. However, this is the real world, and sometimes it just happens that you have to deal with a porting task which is less than trivial. Well, that happened to me and my team not so long ago, so I’m here to tell you that I have written and published some tools to solve these problems. If that’s of interest, then read on!

Continue reading 'Easier upstreaming / back-porting of patch series with git'»

Share

in partial defense of GitHub’s review system

By , May 12, 2013 12:47 pm

Julien Danjou recently posted a thought-provoking rant about GitHub’s pull request workflow implementation. His main point is essentially that Gerrit provides a more sophisticated review system. Of course I’m not going to disagree with that ;-)

I am generally a big fan of Julien’s work and I’m very excited for the future of OpenStack Ceilometer of which he is the current PTL. However, in this case I think his views are understandably biased towards review workflows on very large projects like OpenStack, and I found the length of the rant slightly disproportionate to the actual substance of the points made within it. Somewhat ironically, AFAICS his blog’s own review process could use some improvement due to comments currently being disabled ;-) So here are my thoughts.

Disclaimer: currently I use GitHub for reviews more regularly than Gerrit, so my views are likely to be biased at least as much as Julien’s, but in the opposite direction ;-)

Continue reading 'in partial defense of GitHub’s review system'»

Share

7 principles for contributing patches to software projects

By , November 10, 2012 3:28 pm

In the Free and Open Source software worlds, it seems that a lot of people with good intentions don’t understand how to contribute efficiently and effectively to upstream software projects. I say this based on personal experience: I sometimes receive patches to projects I maintain, where despite good intentions and a perfectly valid goal (e.g. a bug-fix or feature implementation), the patches are fundamentally flawed in a variety of ways. So unfortunately I cannot accept them without negatively impacting the quality or maintainability of the project.  And I don’t even maintain any wildly popular projects.

It is possible to spend a lot of money on training to learn how to avoid these mistakes, and also many well-organized projects already provide good documentation on how to contribute, such as the Linux kernel and git. However, these documents typically mix up generic advice with project-specific advice. In this post I would like to present a list of generic advice on how to write patches which have the maximum chance of being accepted quickly and with gratitude rather than resentment!

Continue reading '7 principles for contributing patches to software projects'»

Share

port redirection from kvm host to guest

By , January 23, 2012 2:58 am

I’ve just started using kvm in earnest, and immediately ran into the challenge of how to access my guest via ssh. My first instinct was to configure the guest in bridged mode, but this doesn’t work well (or at all) with wireless interfaces.

So plan B was to set up port redirection from the host to the guest, e.g. so that ssh’ing to localhost port 2222 would redirect to the guest’s port 22.

After a quick google, some fiddling with iptables, and a glance at the libvirt Networking wiki page, I was still having no luck. Then it hit me – my guest was using user-mode networking, and rather than getting its DHCP-allocated IP from the libvirtd-launched dnsmasq instance on the host, was receiving a hardcoded allocation of 10.0.2.15 from the host which is on 10.0.2.2. This can be extremely puzzling at first, because no network commands run on the host (such as ifconfig, iptables, brctl, route) will reveal this magic address, yet the host is still accessible from the guest via it.

After a lot more googling, I stumbled across a technique for configuring host to guest port redirection on a running VM. This sounded very promising, but virt-manager refused to accept the magic Control-Alt-2 key combination to switch to QEMU monitor mode. It turns out that this is no accident. However, since libvirt 0.8.8, the QEMU monitor can be accessed via virsh.
Note that the --hmp option is required, otherwise the monitor expects the command in JSON format, so omitting it leads to errors like error: internal error cannot parse json ... lexical error: invalid char in json text.

The final hurdle was figuring out the correct monitor command. The host_net_redir command as mentioned in the above article is no longer recognized. Luckily the QEMU monitor interface helped me out here – I spotted an encouraging sounding command hostfwd_add:

# virsh qemu-monitor-command --hmp sles11 'help hostfwd_add'
hostfwd_add [vlan_id name] [tcp|udp]:[hostaddr]:hostport-[guestaddr]:guestport -- redirect TCP or UDP connections from host to guest (requires -net user)

and google confirmed that the latter had superceded the former.

So finally we have the complete solution:

# virsh qemu-monitor-command --hmp sles11 'hostfwd_add ::2222-:22'
# ssh -p 2222 localhost
Password:
Last login: Mon Jan 23 00:37:44 2012
linux-mnsh:~ #

Hooray!

UPDATE: just found another very simple solution – add a new NIC to the VM which doesn’t use user-mode networking. Then it will get a IP (on 192.168.100.0/24 by default) which is still NAT’d but also routable via virbr0 on the host, meaning no redirection is necessary; just ssh directly to the guest’s IP from the host. A minor disadvantage of this is that the guest won’t be directly reachable from outside the host, but that’s unlikely to be an issue in most scenarios.

Share

Panorama Theme by Themocracy