Currently showing posts tagged: Free Software

git branch auto-magic: git-splice, git-transplant, git-deps, and announcing git-explode!

By , June 14, 2018 11:00 pm

For the last few years I’ve been enjoying the luxury of SUSE’s generous HackWeek policy to work on building four tools supporting higher-level workflows on top of git. I’d already (quietly) announced three of them: git-splice, git-transplant, git-deps (more details below). But I’m now excited to announce that I’ve just released the fourth: git-explode !

git-explode automatically explodes a large topic branch into a set of smaller, independent topic branches. It does this by harnessing git-deps to automatically detect inter-dependencies between commits in the large source branch and using that dependency tree to construct the smaller topic branches.

I recently presented all four tools at a Meetup of the Git London User Group, and thanks to the awesome services of the host venue Skills Matter, I’m delighted to announce that the talk is now online:

video of my talk on git auto-magic at the Git London User Group Meetup

If you don’t have time to watch the whole thing, you can look at the slides, or just keep on reading to see which ones you might be interested in. I’ll list them first, and then talk about the motivation for writing them.

Continue reading 'git branch auto-magic: git-splice, git-transplant, git-deps, and announcing git-explode!'»


Credify – Reputation as a Service

By , May 4, 2018 7:30 pm

I went to a blockchain event this week out of curiosity, and heard a talk which really impressed me in that it could solve many unrelated problems with online sites in a single stroke. But before I tell you what it was, let’s set the scene with some example contexts in which I could see it helping in the future.

Some websites such as StackOverflow and Reddit have achieved a fairly impressive and high profile degree of success by harnessing gamification, i.e. a system based on a reputation metric (sometimes labelled “karma” or “gold”), together with a set of rules and rewards engineered in a way which incentivizes the desired behaviour, namely the contribution of high quality content. These generally work pretty well, but their systems can still be abused / manipulated in undesirable ways, and also typically there is no mechanism by which a user who has acquired a high reputation score on one site can benefit by transferring that reputation to another domain. (Although reputation earned within one community within the Stack Exchange network does give you an initial boost when joining other communities within that network.)

A different problem I’ve been mulling over for years is how to create a reliable marketplace for placing bounties on FLOSS engineering problems. Companies such as BountySource are already having some success doing this, but it feels to me that we’re still a long way from fully realising the potential for a freelance FLOSS development industry. One of the big problems in this space is figuring out how to measure good behaviour and reward it appropriately, whilst discouraging bad behaviour. Sound familiar?

A third area I’ve been considering is that of bug/issue trackers for public FLOSS projects (and even commercial products). Unfortunately it’s not uncommon for these systems to get spammed to hell and back by idiots. For instance GitHub is so scared of this problem that they’ve chosen not to open this particular Pandora’s box. However a karma-based approach could drastically improve the signal-to-noise ratio on these systems.

After spotting the common theme between these use cases, I concluded a few years ago that building a Reputation as a Service (RaaS?) platform to track reputation in a reliable manner, and allow federation of reputation between different communities, could make a lot of sense. That way each service which relies on reputation metrics doesn’t need to reinvent the wheel.

Fast forwarding several years, blockchain is a proven technology, and its transparency, immutability, and global decentralized nature all suggest that it’s a near perfect technology on which to build my (probably unoriginal) RaaS idea. And the good news is that there is already an organisation apparently well on the way to solving this: Credify. (No need to google it – the links are below!)

One significant hurdle to building RaaS (which needs to support thousands or even ultimately hundreds of thousands of transactions per second in order to work on a truly global scale) on a blockchain is the poor scalability of the blockchains offered by the likes of Bitcoin and Ethereum. However Credify has neatly side-stepped this by building on top of EOS, and they’ve taken advantage of the blockchain to design a much more powerful system which rewards not only good behaviour but crucially also good recommendations and punishes bad ones.

One of their obvious initial targets is the online product/service review industry, since sites like Yelp, TripAdvisor, Amazon etc. are plagued with fake reviews, and that translates to a huge financial impact within those markets. However I’m very interested to see whether this kind of approach will extend to less commercial areas like the ones I highlighted above.

If you want to play around with Credify, here’s a referral link. Or if you just want to learn more, check out their very nifty home page. If this thing takes off like I suspect it will, I suspect there might be some advantages to becoming an early adopter.


Cloud rearrangement for fun and profit

By , May 17, 2015 4:42 am

In a populated compute cloud, there are several scenarios in which it’s beneficial to be able to rearrange VM guest instances into a different placement across the hypervisor hosts via migration (live or otherwise). These use cases typically fall into three categories:

  1. Rebalancing – spread the VMs evenly across as many physical VM host machines as possible (conceptually similar to vSphere DRS). Example use cases:
  2. Consolidation – condense VMs onto fewer physical VM host machines (conceptually similar to vSphere DPM). Typically involves some degree of defragmentation. Example use cases:
  3. Evacuation – free up physical servers:

Whilst one-shot manual or semi-automatic rearrangement can bring immediate benefits, the biggest wins often come when continual rearrangement is automated. The approaches can also be combined, e.g. first evacuate and/or consolidate, then rebalance on the remaining physical servers.

Other custom rearrangements may be required according to other IT- or business-driven policies, e.g. only rearrange VM instances relating to a specific workload, in order to increase locality of reference, reduce latency, respect availability zones, or facilitate other out-of-band workflows or policies (such as data privacy or other legalities).

In the rest of this post I will expand this topic in the context of OpenStack, talk about the computer science behind it, propose a possible way forward, and offer a working prototype in Python.

If you’re in Vancouver for the OpenStack summit which starts this Monday and you find this post interesting, ping me for a face-to-face chat!

Continue reading 'Cloud rearrangement for fun and profit'»


Announcing git-deps: commit dependency analysis / visualization tool

By , January 19, 2015 12:15 am

I’m happy to announce a new tool called git-deps which performs automatic analysis and visualization of dependencies between commits in a git repository. Here’s a screencast demonstration!

Back in 2013 I blogged about some tools I wrote which harness the notes feature of git to help with the process of porting commits from one branch to another. These are mostly useful in the cases where porting is more complex than just cherry-picking a small number of commits.

However, even in the case where there are a small number of desired commits, sometimes those commits have hidden dependencies on other commits which you didn’t particularly want, but need to pull in anyway, e.g. in order to avoid conflicts during cherry-picking. Of course those secondary commits may in turn require other commits, and before you know it, you’re in dependency hell, which is only supposed to happen if you’re trying to install Linux packages and it’s still 1998 … but in fact that’s exactly what happened to me at SUSEcon 2013, when I attempted to help a colleague backport a bugfix in OpenStack Nova from the master branch to a stable release branch.

At first sight it looked like it would only require a trivial git cherry-pick, but that immediately revealed conflicts due to related code having changed in master since the release was made. I manually found the underlying commit which the bugfix required by using git blame, and tried another cherry-pick. The same thing happened again. Very soon I found myself in a quagmire of dependencies between commits, with no idea whether the end was in sight.

So wouldn’t it be nice if you could see the dependency tree ahead of time, rather than spending a whole bunch of time resolving unexpected conflicts due to missing dependencies, only to realise that the tree’s way deeper than you expected, and that actually a totally different approach is needed? Well, I thought it would, and so git-deps was born!

In coffee breaks during the ensuing openSUSE conference at the same venue, I feverishly hacked together a prototype and it seemed to work. Then normal life intervened, and no progress was made for another year.

However thanks to SUSE’s generous Hack Week policy, I have had the luxury of being able to spending some of early January 2015 working to bring this tool to the next level. I submitted a Hack Week project page, announced my intentions on the git mailing list, started hacking, missed quite a bit of sleep, and finally recorded the above screencast.

The tool is available here:

Please give it a go and let me know what you think! I’m particularly interested in hearing ideas for use cases I didn’t think of yet, and proposals for integration with other git web front-ends.


How to build an OpenStack cloud from SUSEcon’s free USB stick handouts

By , December 11, 2014 3:28 pm

Once again, SUSEcon was a blast! Thanks to everyone who helped make it such a great success, especially all our customers and partners who attended.

If you attended the final Thursday keynote, you should have been given a free USB stick preloaded with a bootable SUSE Cloud appliance. And if you missed out or couldn’t attend, download a copy here! This makes it possible for anyone to build an OpenStack cloud from scratch extremely quickly and easily. (In fact, it’s almost identical to the appliance we used a few weeks ago to win the “Ruler of the Stack” competition at the OpenStack summit in Paris.)

Erin explained on stage at a high-level what this appliance does, but below are some more specific technical details which may help in case you haven’t yet tried it out.

The appliance can be booted on any physical or virtual 64-bit x86 machine … but before we start! – if you would like try running the appliance in a VM using either KVM or VirtualBox, then there is an even easier alternative which uses Vagrant to reduce the whole setup to a one-line command. If you like the sound of that, stop reading and go here instead. However if you want to try it on bare metal or with a different hypervisor such as VMware or HyperV, read on!

Continue reading 'How to build an OpenStack cloud from SUSEcon’s free USB stick handouts'»


Panorama Theme by Themocracy