Structured Procrastination geek Archives – Structured Procrastination

Improving trust in the cloud with OpenStack and AMD SEV

By Adam, September 13, 2019 1:00 pm

This post contains an exciting announcement, but first I need to provide some context!

Ever heard that joke “the cloud is just someone else’s computer”?

Of course it’s a gross over-simplification, but there’s more than a grain of truth in it. And that raises the question: if your applications are running in someone else’s data-centre, how can you trust that they’re not being snooped upon, or worse, invasively tampered with?

Until recently, the answer was “you can’t”. Well, that’s another over-simplification. You could design your workload to be tamperproof; for example even if individual mining nodes in Bitcoin or Ethereum are compromised, the blockchain as a whole will resist the attack just fine. But there’s still the snooping problem.

Hardware to the rescue?

However, there’s some good news on this front. Intel and AMD realised this was a problem, and have both introduced new hardware capabilities to help improve the level to which cloud users can trust the environment in which their workloads are executed, e.g.:

AMD SEV (Secure Encrypted Virtualization) which can encrypt the memory of a running VM with a key which is only accessible to the owner of that VM. This is done on-chip so that even if you have physical access to the machine, it makes it a lot harder to snoop in on the running VM¹.
It can also provide the guest owner with an attestation which cryptographically proves that the memory was encrypted correctly and can only be decrypted by the owner.
Intel MKTME (Multi-Key Total Memory Encryption) which is a similar approach.

But even with that hardware support, there is the question to what degree anyone can trust public clouds run on proprietary technology. There is a growing awareness that Free (Libre) / Open Source Software tends to be inherently more secure and trustworthy, since its transparency enables unlimited peer review, and its openness allows anyone to contribute improvements.

And these days, OpenStack is pretty much the undisputed king of the Open Source cloud infrastructure world.

An exciting announcement

So I’m delighted to be able to announce a significant step forward in trustworthy cloud computing: as of this week, OpenStack is now able to launch VMs with SEV enabled! (Given the appropriate AMD hardware, of course.)

The core functionality is all merged and will be in the imminent Train release. You can read the documentation, and you will also find it mentioned in the Nova Release Notes.

While this is “only” an MVP and far from the end of the journey (see below), it’s an important milestone in a strong partnership between my employer SUSE and AMD. We started work on adding SEV support into OpenStack around a year ago:

This resulted in one of the most in-depth technical specification documentations I’ve ever had to write, plus many months of intense collaboration on the code and several changes in design along the way.

I’d like to thank not only my colleagues at SUSE and AMD for all their work so far, but also many members of the upstream OpenStack community, especially the Nova team. In particular I enjoyed fantastic support from the PTL (Project Technical Lead) Eric Fried, and several developers at Red Hat, which I think speaks volumes to how well the “coopetition” model works in the Open Source world.

The rest of this post gives a quick tour of the implementation via screenshots and brief explanations, and then concludes with what’s planned next.

Continue reading 'Improving trust in the cloud with OpenStack and AMD SEV'»

front page, geek, work | AMD, architecture, cloud, coopetition, OpenStack, SEV

What does negative harmony sound like? Here’s the answer!

Comments (4)

By Adam, June 23, 2019 11:00 am

In the last year or so there’s been quite a buzz in the music theory world about the concept of negative harmony, mainly thanks to a few YouTube interviews with Jacob Collier which have gone viral, especially the ones by June Lee.

But while this has been great for introducing the idea to many people, still most people don’t really know what music based on negative harmony actually sounds like! And as Jacob mentioned in some of these interviews, clever theoretical tricks are rather pointless unless you can actually make some great music from them. Most of the videos just focus on a few chords, which is a great start but far from the full picture.

Introducing the SHIMANator negative harmony app!

So I’ve built an app called “the SHIMANator” which can convert any music into its negative harmony equivalent, and I’m very excited to finally announce it to the world! Check out the video:

OK, but what’s the point?

I mainly wrote this app because a) Jacob asked me to, b) it sounded like a fun challenge, and c) the thought of being able to instantly hear the negative harmony equivalent of any music was very appealing.

But in the process of getting it working, it became apparent that this could actually be a useful tool for generating fresh new musical ideas and sounds. For example, I’ve talked to film / TV composers who got excited about using it to quickly generate music which is coherent with and relating to their existing material. For example if a musical motif in a major key represented a character in a film, flipping it about an axis would give you the negative version, which would sound in a minor key and could be used to represent the “dark side” of that character’s personality.

Can I try it out?

Not yet, but I’m aiming to offer early beta access to a select group of people at some point soon. Please show your interest by taking the following actions!

2. Let me know what you think and win a free copy!

Do you find this app interesting? Could you imagine using it yourself? If so, would you use it for composition, or as an extra effect in live improvisation, or maybe even for something else?

So I’d love to hear what you think – and the person who gives the best feedback will win a free copy of the software when it’s fully productised and ready to publish!

Please leave comments on the Facebook page or the YouTube video.

How does it work?

At the simplest level, it takes MIDI events in, does some magic to convert to negative harmony, and then outputs the same events with modified pitches. So it should work with pretty much any piece of MIDI-compatible software or hardware under the sun.

The actual algorithm for the conversion is very complicated, so I’ll save the explanation for another time. But you can see from the video how melodic motion is inverted, as is motion around the circle of fifths.

How did this all come about?

Here’s the history, in case you’re curious.

I was introduced to negative harmony a few years ago by Barak Schmool, who later taught Jacob in his role as professor of jazz and world rhythms at the Royal Academy of Music. (Barak’s awareness of the technique was heavily influenced by his friend Steve Coleman, an incredible jazz saxophonist who has been using negative harmony in his music for many decades.)

At the time I built a really dumb prototype for fun, and fed Mozart’s 40th Symphony through it. The results were promising, but also sounded amusingly terrible due to every high note being converted to a very low note and vice-versa.

It was clear that without some magic octave transposition algorithm, notes would stray too far from their original register and completely screw up the voicing “texture” which the composer had intended. So shortly afterwards, Barak and I basically forgot about it, and instead got distracted experimenting on Giant Steps with some brilliant ideas he had regarding unequally tempered systems of intonation (which later inspired some of Jacob’s great work with microtonal voice-leading).

A year or two later, Barak told Jacob about my prototype. He was interested enough to get in touch, and you already know the rest of the story.

By the way, there is already some great music out there made entirely using negative harmony. For example see Steve Cruickshank’s fantastic YouTube channel, which is full of negative harmony covers of famous music. And I’m not even the first to come up with a negative version of Beethoven’s Moonlight Sonata. But I’m not yet aware of any other software which does what mine does. Hopefully you find its potential interesting.

Have you made any other music software?

Funny you should ask! Actually yes: in an earlier collaboration with Barak I built the Scale Matcher – a free app (web page) for finding which scales match a given chord. Check it out 🙂

If you read this far, congratulations – you are a most excellent and dedicated music theory nerd. Let’s have a pint some time. But until then, don’t forget to subscribe and let me know what you think!

front page, geek, music | Barak Schmool, composition, harmony, Jacob Collier, jazz, MIDI, music theory, negative harmony, SHIMANator, SuperCollider

git branch auto-magic: git-splice, git-transplant, git-deps, and announcing git-explode!

Comments (9)

By Adam, June 14, 2018 11:00 pm

For the last few years I’ve been enjoying the luxury of SUSE’s generous HackWeek policy to work on building four tools supporting higher-level workflows on top of git. I’d already (quietly) announced three of them: git-splice, git-transplant, git-deps (more details below). But I’m now excited to announce that I’ve just released the fourth: git-explode !

git-explode automatically explodes a large topic branch into a set of smaller, independent topic branches. It does this by harnessing git-deps to automatically detect inter-dependencies between commits in the large source branch and using that dependency tree to construct the smaller topic branches.

I recently presented all four tools at a Meetup of the Git London User Group, and thanks to the awesome services of the host venue Skills Matter, I’m delighted to announce that the talk is now online:

If you don’t have time to watch the whole thing, you can look at the slides, or just keep on reading to see which ones you might be interested in. I’ll list them first, and then talk about the motivation for writing them.

Continue reading 'git branch auto-magic: git-splice, git-transplant, git-deps, and announcing git-explode!'»

front page, geek, hacks, work | backporting, branches, development, forward-porting, Free Software, git, hacking, software, upstreaming, version control, workflow

Credify – Reputation as a Service

Comments (0)

By Adam, May 4, 2018 7:30 pm

I went to a blockchain event this week out of curiosity, and heard a talk which really impressed me in that it could solve many unrelated problems with online sites in a single stroke. But before I tell you what it was, let’s set the scene with some example contexts in which I could see it helping in the future.

Some websites such as StackOverflow and Reddit have achieved a fairly impressive and high profile degree of success by harnessing gamification, i.e. a system based on a reputation metric (sometimes labelled “karma” or “gold”), together with a set of rules and rewards engineered in a way which incentivizes the desired behaviour, namely the contribution of high quality content. These generally work pretty well, but their systems can still be abused / manipulated in undesirable ways, and also typically there is no mechanism by which a user who has acquired a high reputation score on one site can benefit by transferring that reputation to another domain. (Although reputation earned within one community within the Stack Exchange network does give you an initial boost when joining other communities within that network.)

A different problem I’ve been mulling over for years is how to create a reliable marketplace for placing bounties on FLOSS engineering problems. Companies such as BountySource are already having some success doing this, but it feels to me that we’re still a long way from fully realising the potential for a freelance FLOSS development industry. One of the big problems in this space is figuring out how to measure good behaviour and reward it appropriately, whilst discouraging bad behaviour. Sound familiar?

A third area I’ve been considering is that of bug/issue trackers for public FLOSS projects (and even commercial products). Unfortunately it’s not uncommon for these systems to get spammed to hell and back by idiots. For instance GitHub is so scared of this problem that they’ve chosen not to open this particular Pandora’s box. However a karma-based approach could drastically improve the signal-to-noise ratio on these systems.

After spotting the common theme between these use cases, I concluded a few years ago that building a Reputation as a Service (RaaS?) platform to track reputation in a reliable manner, and allow federation of reputation between different communities, could make a lot of sense. That way each service which relies on reputation metrics doesn’t need to reinvent the wheel.

Fast forwarding several years, blockchain is a proven technology, and its transparency, immutability, and global decentralized nature all suggest that it’s a near perfect technology on which to build my (probably unoriginal) RaaS idea. And the good news is that there is already an organisation apparently well on the way to solving this: Credify. (No need to google it – the links are below!)

One significant hurdle to building RaaS (which needs to support thousands or even ultimately hundreds of thousands of transactions per second in order to work on a truly global scale) on a blockchain is the poor scalability of the blockchains offered by the likes of Bitcoin and Ethereum. However Credify has neatly side-stepped this by building on top of EOS, and they’ve taken advantage of the blockchain to design a much more powerful system which rewards not only good behaviour but crucially also good recommendations and punishes bad ones.

One of their obvious initial targets is the online product/service review industry, since sites like Yelp, TripAdvisor, Amazon etc. are plagued with fake reviews, and that translates to a huge financial impact within those markets. However I’m very interested to see whether this kind of approach will extend to less commercial areas like the ones I highlighted above.

If you want to play around with Credify, here’s a referral link. Or if you just want to learn more, check out their very nifty home page. If this thing takes off like I suspect it will, I suspect there might be some advantages to becoming an early adopter.

front page, geek | blockchain, Credify, development, Free Software, gamification, karma, Reddit, reputation, software, StackOverflow

Report from the OpenStack PTG in Dublin

Comments (0)

By Adam, March 9, 2018 7:30 pm

Last week I attended OpenStack’s PTG (Project Teams Gathering) in Dublin. This event happens every 6 months in a different city, and is a fantastic opportunity for OpenStack developers and upstream contributors to get together and turbo-charge the next phase of collaboration.

I wrote a private report for my SUSE colleagues summarising my experience, but then Colleen posted her report publicly, which made me realise that it would be far more in keeping with OpenStack’s Four Opens to publish mine online. So here it is!

Continue reading 'Report from the OpenStack PTG in Dublin'»

front page, geek, travel, work | architecture, cloud, development, OpenStack