So what does Continuous Delivery have to do with the foremost compilation of cheesy pop music. Well, short of the obvious 'CD' parallel...not a lot. But I needed some way to frame this blog and I like music so here we are.

Stay tuned pop pickers, let's take it from the top. Ah-1-2-3-4..

Freddie Mecury

One Vision - Queen

It’s a funny story all-in-all.

Our CTO, Chris Adams, had a dream. A dream where we could deploy code changes to our sites quickly, easily and, in some magical cases, without even bringing the site down for maintenance. This was met with looks of disbelief, comments of, "It can never be done", "You're crazy, man" and "What are we, wizards?!" and there was one shaky moment where we phoned the men with the butterfly nets but hung up through awkwardness.

But then, like being caressed by the rosy fingers of a new dawn, the idea of Continuous Delivery came to light.

This introduced Madgex to a brave new world of aligning our Agile methodologies (developing our software in short cycles) with the tooling to support those cycles and releasing the finished code more quickly and easily.

The theory behind this being it will have the dual impact of:

  • Reducing the time and risk in the development team of delivering code changes.
  • Adding benefit to clients by allowing faster and more incremental changes to their live sites.

That's a long winded way of saying:

  • We can develop, test and release our Platform to live whenever we want on the click of a button, without worry.
  • We can release a code change to any site, at any time.

Amazed by the idea, we all bought in. But that opened the door to...


Some Kind Of Monster - Metallica

It was at this point; I was brought on board.

Our trusty band of misfits; Chris Adams, Platform developer Martin Digon and myself started to sit down and scope out what all this meant in a Madgex context. It was here we realised just how fundamental, technologically and culturally, a change like this would be for us. The big hitters being:

  • Making huge changes to our development process and company-wide mindset.
  • Rolling-out new internal software and provide training and onboarding to the entire Madgex development team.
  • Introducing a new “Staging” environment - a test environment that is a true representation of live. As well as this, we needed to isolate our UAT environment from the Live environment, meaning a radical change in our existing deployment flow.
  • Removing all of our trusted, but custom, installer code from the deployment process.
  • Redesigning crucial, but legacy, Platform code to support all of this.

All of that and migrate hundreds of websites to use the new system. Piece of cake, right?

Suffice to say we did start to wonder what we'd got ourselves into. But we knew, despite all of this, that if we kept our eyes on the prize and deliver this project to its initial goals, it would be a total game changer. 

We weren't wrong...

David Bowie

Better Future - David Bowie

Skip ahead 9 months.

Martin and I are rattling through the plan like nobody's business. New infrastructure is in place, teams are trained on the new tools and processes and clients are being rolled out. It really is an exciting time. But this wasn't without its niggles.

A running joke had developed where Martin would wander over to my desk and open with, "Funny story...". This was usually followed by his latest tale of why the seemingly simple idea we had concocted turned out to be a nightmare, the horrible bug he’d uncovered couldn't be worked around or how suddenly the ground had moved beneath our feet and made a few day's work obsolete. All to be expected with working on a live system but definitely not, as I kept reminding him, funny.

However, despite the pains, the system was out in the wild and we could really feel the rumblings of change beginning to echo around Madgex Towers. I started to look back at where we'd come from to where we already were, even though the project still rolling on. It then really began to sink in what a difference we'd made.

We'd already seen:

  • The old build and release process be put out to pasture. Including the idea of deployment ‘blocks’ (unapproved changes preventing product updates arriving on client sites for weeks, sometimes months). No more would clients have to wait for their approved code changes.
  • Happier developers, using an updated toolset.
  • Support ticket "time-to-live" metrics falling off a cliff (that's a good thing, despite the imagery).
  • Clients starting to report back that they love the new system.
    (More on those last two points later)

With the good ship CD setting sail, we were not only seeing the gains we had expected to see being met but being exceeded. We really did feel like we were building a better future for not only our clients, but for ourselves.

Being at the epicentre of this sort of innovation is the reason why I work in the technology sector. Madgex has this ethos stitched into its very fabric and the whole dev team here understand that. Which is why we put our development team at the prow of all our innovation and this project was no different.

We wanted to radicalise our deployments, sure. But the by-product of this was allowing our developers to...

Zach De La Rocha

Take the Power Back - Rage Against the Machine

Agile and Continuous Delivery go hand-in-hand. Both these methodologies foster a sense of ownership in a development team. This ownership piece is something that the wider developer community, not just Madgex, is trying to build upon.
All our developers now deploy all their own code changes to the destination sites.
Right from making the code change on their machine all the way up to the live servers, this 'end-to-end' philosophy has given our team that sense of ownership that comes from Agile and CD by allowing them to take every change they make to its conclusion. It's great to be on the bleeding edge of such initiatives.
With the developers owning everything all the way to live, and no more passing on changes to other people to deploy, this has led to an exponential increase in...

Chali 2Na

Quality Control - Jurassic 5

I wear two hats at Madgex. One is the Internal Projects Manager, hence the CD PM role (among other acronyms). The other is the Team Lead for the Support development team. Being a member of the Support team means that my team and I are on the sharp end of customer service if an issue is found on a live site. 
Since rolling out CD we have seen a marked, almost dramatic, reduction in the number of issues coming out of other teams. With the introduction of the Staging environment (a nightly snapshot of the client site configuration and data), coupled with all the devs owning their own code changes, it has meant that any developer that has made a change to a site (pending internal code review and QA processes) can check their work against a true representation of that live site.

This is huge.

No longer do we hear devs mutter "It works on my machine!?" as they no longer have to test using placeholder data such as lorem ipsum or “test 1, 2, 3” to work with. Staging is as close to the real environment as they’re going to get for a test system so it instills confidence in the developer knowing that their code is sound.
Even with a change in requirements after a deployment, our new forwards-not-backwards deployment mindset and toolset allows us to address change far faster than before. Everyone is now a…

Michael Jackson

Speed Demon - Michael Jackson

We used to be bound pretty tightly by our old deployment system. CD changed everything.

A long running problem we had with our method was that if a site had on-going work, we couldn't deploy it - the ‘blocks’ mentioned before. Pretty frustrating if you’re a customer. With the change in deployment flow (isolating UAT from Live) and updating our process to remove these impediments it meant that we could progress our Support tickets that much faster.

There is a statistic we use here called "Full Resolution Time". This is the total time from a ticket being logged to it being closed down. From February 2016, just before we started migrating sites to CD, to August 2016 we decreased our Full Resolution time by 71%.

One more time. By 71%.

It didn't take long for our clients to notice...


Shiny, Happy People - R.E.M

As with any software release, we Beta tested the new process. We rolled out the first iteration of the changes onto a selected few of our clients’ sites. After these sites went live, I did a follow up call with all of them a few weeks later to see how they were getting on. I expected the odd throwaway compliment but the response I received was overwhelming.

Among them, things like this:

"Taken the stress out of the whole deployment process"
"The fluidity of it is brilliant"
"Massively benefited us"

And this is just in the Beta phase!

We used the positive and negative feedback (of which there was very little) to help design our released version and we haven't looked back since. Comments come in regularly about the improvement in turnaround and delivery time, the upswing in quality and the fact that all the old blockers have been removed.

It really has been heart-warming to hear. Especially as so much time and effort has gone into this project by the team and a number of other people in the business. It helps you to get over those 'funny stories'.

But if there's one thing we don't do here at Madgex, it's rest on our laurels. So, it's time for the..


Next Episode - Dr. Dre and Snoop Dogg

[Pretentious quote alert]

Socrates said, "The secret to change is to focus all of your energy, not on fighting the old, but building the new". 

I genuinely believe that Madgex does think in this way. We've always tried to put innovation above stagnation. Whether that's with our technology, our culture, our process, anything.

We already have a Roadmap for CD, with new features or ways things can be done better that’s as long as your arm. Stakeholders are invested in this process internally and we have 'champions' in each team to own and drive this initiative forward.

CD is here to stay. But in its current guise, knowing us… probably not for long.


So that brings me to the end of my little mixtape, I do hope you've enjoyed listening. If you're working in Tech and thinking of moving your teams over to CD, don't hesitate, just do it.

If you're client and you're reading this, I hope you like Continuous Delivery as much as I do. If you don't then please let us know.

Just don't open with, "Funny story..."

A special thank you to the heroes at @MinimalPosters for supplying the amazing images for this blog.

Other articles you may like...