Currently showing posts tagged: development

Credify – Reputation as a Service

By , May 4, 2018 7:30 pm

I went to a blockchain event this week out of curiosity, and heard a talk which really impressed me in that it could solve many unrelated problems with online sites in a single stroke. But before I tell you what it was, let’s set the scene with some example contexts in which I could see it helping in the future.

Some websites such as StackOverflow and Reddit have achieved a fairly impressive and high profile degree of success by harnessing gamification, i.e. a system based on a reputation metric (sometimes labelled “karma” or “gold”), together with a set of rules and rewards engineered in a way which incentivizes the desired behaviour, namely the contribution of high quality content. These generally work pretty well, but their systems can still be abused / manipulated in undesirable ways, and also typically there is no mechanism by which a user who has acquired a high reputation score on one site can benefit by transferring that reputation to another domain. (Although reputation earned within one community within the Stack Exchange network does give you an initial boost when joining other communities within that network.)

A different problem I’ve been mulling over for years is how to create a reliable marketplace for placing bounties on FLOSS engineering problems. Companies such as BountySource are already having some success doing this, but it feels to me that we’re still a long way from fully realising the potential for a freelance FLOSS development industry. One of the big problems in this space is figuring out how to measure good behaviour and reward it appropriately, whilst discouraging bad behaviour. Sound familiar?

A third area I’ve been considering is that of bug/issue trackers for public FLOSS projects (and even commercial products). Unfortunately it’s not uncommon for these systems to get spammed to hell and back by idiots. For instance GitHub is so scared of this problem that they’ve chosen not to open this particular Pandora’s box. However a karma-based approach could drastically improve the signal-to-noise ratio on these systems.

After spotting the common theme between these use cases, I concluded a few years ago that building a Reputation as a Service (RaaS?) platform to track reputation in a reliable manner, and allow federation of reputation between different communities, could make a lot of sense. That way each service which relies on reputation metrics doesn’t need to reinvent the wheel.

Fast forwarding several years, blockchain is a proven technology, and its transparency, immutability, and global decentralized nature all suggest that it’s a near perfect technology on which to build my (probably unoriginal) RaaS idea. And the good news is that there is already an organisation apparently well on the way to solving this: Credify. (No need to google it – the links are below!)

One significant hurdle to building RaaS (which needs to support thousands or even ultimately hundreds of thousands of transactions per second in order to work on a truly global scale) on a blockchain is the poor scalability of the blockchains offered by the likes of Bitcoin and Ethereum. However Credify has neatly side-stepped this by building on top of EOS, and they’ve taken advantage of the blockchain to design a much more powerful system which rewards not only good behaviour but crucially also good recommendations and punishes bad ones.

One of their obvious initial targets is the online product/service review industry, since sites like Yelp, TripAdvisor, Amazon etc. are plagued with fake reviews, and that translates to a huge financial impact within those markets. However I’m very interested to see whether this kind of approach will extend to less commercial areas like the ones I highlighted above.

If you want to play around with Credify, here’s a referral link. Or if you just want to learn more, check out their very nifty home page. If this thing takes off like I suspect it will, I suspect there might be some advantages to becoming an early adopter.

Share

Report from the OpenStack PTG in Dublin

By , March 9, 2018 7:30 pm

Last week I attended OpenStack’s PTG (Project Teams Gathering) in Dublin. This event happens every 6 months in a different city, and is a fantastic opportunity for OpenStack developers and upstream contributors to get together and turbo-charge the next phase of collaboration.

I wrote a private report for my SUSE colleagues summarising my experience, but then Colleen posted her report publicly, which made me realise that it would be far more in keeping with OpenStack’s Four Opens to publish mine online. So here it is!

Continue reading 'Report from the OpenStack PTG in Dublin'»

Share

Squash-merging and other problems with GitHub

By , August 16, 2017 6:45 pm

(Thanks to Ben North, Colleen Murphy, and Nicolas Bock for reviewing earlier drafts of this post.)

In April 2016, GitHub announced a new feature supporting squashing of multiple commits in a PR at merge-time (announced on April 1st, but it was actually bona-fide 😉 ).

I appreciate that there was high demand for this feature (and similarly on GitLab), and apparently that many projects have a “squash before submitting a PR” policy, but I’d like to contend that this is a poor-man’s workaround for the lack of a real solution to the underlying problems.

Why squash-merge?

So what are the underlying problems which made this such a frequently requested feature? From reading the various links above, it seems that by far the biggest motivator is that people frequently submit pull requests (or merge requests, in GitLab-speak) which contain multiple commits, and these commits are seen as too “noisy” / fine-grained. In other words there is a desire to not pollute the target/trunk branch (e.g. master) with these fine-grained commits, and instead only have larger, less fine-grained commits merged.

But where does this desire come from? Well, if the fine-grained commits which accumulate on a PR branch are frequently amendments to earlier commits in the same PR (like “oops, fix typo I just made” or “oops, fix bug I just introduced”) then this desire is entirely understandable, because noone wants to see that kind of mess on master. However the real problem here is that that kind of mess should have never made it onto GitHub in the first place – not even onto a PR branch! It should have instead been fixed in the developer’s local repository. That is why there is a whole section in the “Pro Git” book dedicated to explaining how to rewrite local history, and why git-commit(1) and git-rebase(1) have native support for creating and squashing “fixup” commits into commits which they fix.

Use the force-push, Luke

If an existing PR needs to be amended, make the change and then rewrite local history so that it’s clean. The new version of the branch can then be force-pushed to GitHub via git push -f, which is an operation GitHub understands and in many situations handles reasonably gracefully. I have previously blogged about why this way is better, but one way of quickly summarising it is: don’t wash your dirty linen in public any more than you have to.

Continue reading 'Squash-merging and other problems with GitHub'»

Share

Why and how to correctly amend GitHub pull requests

By , March 24, 2015 3:00 pm

Like many F/OSS developers, I’m a heavy user of GitHub, collaborating on many projects which use the typical “fork & pull” workflow based on pull requests. The GitHub documentation on pull requests covers this workflow fairly comprehensively, but there seems to be one area which is significantly lacking in detail: why and how to amend existing pull requests. The article simply says:

After your pull request is sent, any new commits pushed to your branch will automatically be added to the pull request. This is especially useful if you need to make more changes.

The problem is that this completely ignores the fact that there are often very good reasons for amending existing commits within the pull request, not just for adding new commits it.

Why amend an existing pull request?

A peer review cycle can potentially reveal many issues which make the pull request unready for merging, e.g.

  • typos
  • bugs in the proposed code changes
  • missing features in the proposed code changes
  • incomplete test coverage
  • incomplete documentation changes
  • style inconsistencies (including whitespace issues)
  • incorrect or incomplete commit messages
  • the commits violate the rule of one logical change per commit
  • some changes are outside the scope of the pull request

This is of course what makes peer review of pull requests so valuable: the problems can be addressed even before they hit the master branch, which helps maintain high quality in the development trunk. But then how do we address the issues?

Continue reading 'Why and how to correctly amend GitHub pull requests'»

Share

Announcing git-deps: commit dependency analysis / visualization tool

By , January 19, 2015 12:15 am

I’m happy to announce a new tool called git-deps which performs automatic analysis and visualization of dependencies between commits in a git repository. Here’s a screencast demonstration!

Back in 2013 I blogged about some tools I wrote which harness the notes feature of git to help with the process of porting commits from one branch to another. These are mostly useful in the cases where porting is more complex than just cherry-picking a small number of commits.

However, even in the case where there are a small number of desired commits, sometimes those commits have hidden dependencies on other commits which you didn’t particularly want, but need to pull in anyway, e.g. in order to avoid conflicts during cherry-picking. Of course those secondary commits may in turn require other commits, and before you know it, you’re in dependency hell, which is only supposed to happen if you’re trying to install Linux packages and it’s still 1998 … but in fact that’s exactly what happened to me at SUSEcon 2013, when I attempted to help a colleague backport a bugfix in OpenStack Nova from the master branch to a stable release branch.

At first sight it looked like it would only require a trivial git cherry-pick, but that immediately revealed conflicts due to related code having changed in master since the release was made. I manually found the underlying commit which the bugfix required by using git blame, and tried another cherry-pick. The same thing happened again. Very soon I found myself in a quagmire of dependencies between commits, with no idea whether the end was in sight.

So wouldn’t it be nice if you could see the dependency tree ahead of time, rather than spending a whole bunch of time resolving unexpected conflicts due to missing dependencies, only to realise that the tree’s way deeper than you expected, and that actually a totally different approach is needed? Well, I thought it would, and so git-deps was born!

In coffee breaks during the ensuing openSUSE conference at the same venue, I feverishly hacked together a prototype and it seemed to work. Then normal life intervened, and no progress was made for another year.

However thanks to SUSE’s generous Hack Week policy, I have had the luxury of being able to spending some of early January 2015 working to bring this tool to the next level. I submitted a Hack Week project page, announced my intentions on the git mailing list, started hacking, missed quite a bit of sleep, and finally recorded the above screencast.

The tool is available here: https://github.com/aspiers/git-deps

Please give it a go and let me know what you think! I’m particularly interested in hearing ideas for use cases I didn’t think of yet, and proposals for integration with other git web front-ends.

Share

Panorama Theme by Themocracy