Nick Clegg: ‘Michael Gove went from charm personified to hiding in the toilet'

In this exclusive extract from his new book, the former deputy prime minister talks about how difficult it is to get things done in Whitehall. Especially when you're up against an ideologue education secretary with a team of 'unhinged' advisers

• Part one: 'No wonder there was a Nick Clegg looking sad website'

• Making change is not easy: it is an art, not a science. As junior partners in a coalition and having been out of power for decades, the Liberal Democrats had to learn quickly how to get things done in a system that can resist change at every turn. The coalition government was, of course, a unique experiment in British politics with some unusual features, not least the need to secure agreement across the leadership of two parties. Unsurprisingly much of the significant change that came about during the coalition involved trade-offs and compromises.

  Sometimes that involved a straight policy swap. On other occasions, it involved forensic haggling about the substance of contentious new policies. When we wanted a tax on plastic bags, for instance, and the Conservatives resisted, we compromised by allowing a number of exemptions from the tax for smaller chains, shops at airports and, somewhat bizarrely, "ultra-biodegradable" plastic bags that degrade within days (a type of bag that Owen Paterson insisted be exempted, despite the fact that it doesn't exist).

  While this toing and froing sounds a little unseemly, most of the time it was conducted in good humour. The main disadvantage was when disagreement on a particular policy led to gridlock and a failure to take a decision. But the need to strike compromises also had important advantages: having to justify one's ideas in the face of scrutiny from a sceptical coalition partner forced both parties to be more thoughtful and creative; and the challenge of finding agreement between two parties ensured that extreme or excessively partisan policies rarely survived. Moderation invariably triumphed.

  Perhaps the best example that came about through a straightforward policy swap was the introduction of universal free school meals for infants. It is a policy whose inception was intimately bound up with one of the most infuriating, yet creative characters in the government, Michael Gove, who moved from ambivalence to hostility to acceptance of a policy for which he was both an unwitting co-author and a serious obstacle. Gove was a prominent early public advocate for coalition. When we went for a meal near my home in Putney very early on in the parliament, I found him to be charm personified. He was generous, witty, collegiate and very smart. Over dinner, I tried to persuade him that the A-level system provided too narrow an education, which was neither good for pupils nor beneficial for the economy. He didn't agree - but he disagreed with amusing verve and flair, over a couple of bottles of wine.

• A little over three years later, our relationship had soured to the point that he banned Lib Dem special advisers from physically entering the Department for Education, hid on one occasion in the toilet to avoid speaking to [schools minister] David Laws, and let loose his somewhat unhinged advisers to brief against me, and even against [my wife] Miriam, in the press. On one of the last occasions I spoke to him in government, I asked him to come and see me in my office. He was keen to be accompanied by his assistant, but I insisted we meet alone, as I wanted to deliver a blunt message. Once we were alone, I asked him how he would react if someone on my behalf wilfully lied about his wife in the newspapers? (Miriam had, ludicrously, been accused in the Mail on Sunday by one of Gove's advisers of seeking a Whitehall contract for a children's book charity - the contract had in fact been decided by No 10.) He mumbled that he thought he "knew what had happened". I was livid.

  Shortly afterwards, I told David Cameron that I wasn't prepared to waste my time working with Gove, and that Laws would do so on my behalf (hence Gove's concealment in the toilet). Given how much No 10 clearly loathed Gove's principal adviser at the time and were as exasperated as I was by his occasional public outbursts, this didn't seem to come as much of a surprise to Cameron.

  But even before the personal relations soured, policy disagreements had started to accumulate. At the beginning of the government I was encouraged by what I thought might be an exciting confluence of Lib Dem and Conservative priorities in education: Gove's support for an extension of Tony Blair's emphasis on greater school autonomy; and our emphasis on promoting social mobility through the implementation of radical funding reform such as the pupil premium.

  At first there seemed to be genuine enthusiasm from Gove, too, to forge a new cross-party approach. Yet, almost exactly two years into the government, as I was leading the government delegation to the UN sustainable development summit in Rio in June 2012, Gove announced out of the blue that he wanted to return to a traditional two-tier O-level system. Laws called me very early in the morning in my hotel room in Rio to discuss our response. We decided it should be immediate, so I did a couple of TV interviews making it clear that such a regressive turning back of the clock would not take place on my watch. From bonhomie in a Putney restaurant to policy disputes across the Atlantic - the contrast couldn't have been greater.

• There ensued numerous other policy disagreements, from an attempt by the Conservatives to increase childcare ratios in nurseries - the number of children each trained member of staff can look after at a time - to Gove's unceasing effort to divert money away from needy mainstream schools to free schools. Given this fractious context, it would require a lot of effort to ensure that the DfE that Gove controlled would successfully implement the provision of free school meals for all infants. Unlike the raising of the personal tax allowance, for example, it had not been discussed at the start of government, or enshrined in the coalition agreement. In opposition, however, Laws had strongly advocated the policy. It seemed common sense that children would perform better in class in the afternoon if they weren't hungry. Research also showed that as many as four in 10 children officially categorised as living in poverty were still not eligible to receive free school meals - and that a universal provision would be disproportionately beneficial to those on lower incomes.

• Ironically, its origin as government policy can be traced to a holiday that Gove took in Marrakech, where he stumbled across Henry Dimbleby, the co-founder of the Leon restaurant chain and a healthy-food campaigner. As a result of the encounter, Gove commissioned Dimbleby and his business partner, John Vincent, to produce a report on how to raise the standard of school food. This became the School Food Plan, which was welcomed heartily by the education secretary. Buried in it was a recommendation to introduce free school meals for primary-school children.

  Later that summer, as the conference season approached, Oliver Letwin and Laws began the annual process of agreeing which government policies each party would announce that autumn. As ever, any spending commitments would have to be agreed as part of the autumn budget statement, and George Osborne had a rabbit he was determined to pull from his hat: a tax cut for married couples, where one of the spouses stayed at home. In reality, the amount a couple would save was relatively puny, but Osborne saw it as a symbolic gesture to warm the hearts of traditional Tory voters as well as the editorial teams at the Telegraph and the Mail. Knowing this was an unpalatable policy for us - it was a sop to married couples where one stayed at home and the other worked, while doing nothing for working couples, unmarried couples, single people or widows - he proposed a deal: the policy's price tag was £600m; if we would agree to it, he would find a similar amount for a policy of our choosing.

  Laws suggested that we should take a look at the free school-meal proposals and work out what we might be able to do with £600m. Given the spats with Gove, and certain as we were that he would object to a policy that imposed more obligations on free schools and academies, the proposal was worked up without his knowledge. We took the proposal back to Cameron, Osborne and Letwin and they agreed to the trade-off - and also agreed to keep the move to themselves, including keeping it from Gove. Laws eventually informed Gove at a later stage. To his credit, Gove said he was happy for officials to continue to work on the policy in private and he also agreed to keep it confidential, even from his own team of political advisers, so that it wouldn't leak.

  Those advisers, predictably enough, took this rather badly. Not only had their authority been undermined and their permission circumvented, but their own department and party leadership had worked against them, while the policy itself challenged the very basis of their free school project, requiring, as it did, supposedly autonomous schools to comply with the legislation. Soon the papers were peppered with specious stories planted by Gove's office about the diversion of resources from other areas of the education budget, implementation difficulties and budget overruns - all of which were nonsense. The implementation was remarkably smooth, not least because Laws oversaw it in minute detail. By the time the policy came into effect at the start of the next school year in September 2014, he could practically recite the lunch menus for each of the 17,000-odd state primary schools in England. And today over a million more small children eat a healthy meal at lunchtime every day at school.

• While it is fashionable for politicians to bemoan the inherent caution of Whitehall civil servants - Gove provided regular entertainment at cabinet meetings, with his florid, if unreasonable condemnations of the civil service - their insulation from the political madhouse of Westminster serves as a crucial counterweight in our democratic system. The vast majority of civil servants I encountered in my five years in office were hard-working, thoughtful and driven by a quiet, if fiercely held belief in public service. The flaws of Whitehall (like any great bureaucracy, it has plenty of them) are systemic, not personal: the tendency for bright, young and wholly inexperienced officials to hold great sway over key areas of public policy for short periods of time, before being moved up to the next rung of the ladder, creates discontinuity and immaturity in policy-making. The over-centralisation of power in the Treasury leads to daft policy gyrations: from the failure to regulate the banks, to micromanaging them; from lax fiscal policy, to overzealous contraction; from the underfunding of capital projects, to a bevy of white elephants.

  Above all, perhaps, it is the Balkanisation of authority across Whitehall that is responsible for dysfunctional decision-making. Some departments, notably the Treasury and the FCO, are regarded as the Oxbridge colleges of Whitehall. Others, notably the Department for Culture, Media and Sport or the Department for Communities and Local Government, are snobbily derided as red-brick departments. Each department develops a strong identity of its own - lofty at the top, chippy at the bottom - in the Whitehall pecking order. Understandably enough, discussions between Whitehall departments are as much a trial of strength for the civil servants who are involved as they are about sensible collective decision-making. In the paperwork I read late at night at home, officials would regularly describe arguments about a policy decision as if they were a chess game between different departments.

  At the same time, both Cameron and I would frequently bemoan how difficult it was to get a particular department to produce recommendations or enact a decision, even when demanded by both sides of the coalition. This reluctance was exacerbated if the secretary of state in charge was particularly difficult, if there were tensions between them and the prime minister or me, or if the secretary of state had been in post for such a long time that they were convinced they knew best. The effect of this could be debilitating in the face of the 24-hour media circus. While the stately pace of decision-making in government is, in many ways, a useful antidote to the fickle daily demands made of politicians in the press, it can also come across as being out of touch with the reasonable public expectation that action should be taken rapidly.

  In late 2012 and early 2013, for example, the Times printed a number of compelling accounts of Afghan interpreters who had assisted British forces in Helmand province and who were worried that they would be subject to retribution from the Taliban, once British forces had left the country. It was obvious that the government should move swiftly to provide reassurance that these interpreters would be treated fairly, if need be by relocating them and their families to the UK. Given that we were talking about just a few hundred people, the numbers were minuscule in the context of overall migration flows. I duly demanded an early discussion in the National Security Council and asked the Home Office and the national security adviser to prepare proposals as swiftly as possible. Instead, it took weeks and weeks of haggling to drag a grudging Home Office to come up with a plan that, in the end, was so convoluted it barely helped anyone.

• Another instance of sloth-like decision-making in government related to careers advice and guidance. Shortly after the coalition came to power, a decision was made to scrap the network of Connexions offices that had been established by the previous government, because, by and large, they were not providing high-quality careers advice and guidance to youngsters across the country. While this was harsh on those centres that were doing a better job than most, it was not an unreasonable measure as long as something more effective was swiftly installed as a replacement. Little did I realise at the time that Gove had a personal bee in his bonnet against any publicly organised careers advice and guidance of any description. So once the Connexions service was scrapped, nothing happened. Hundreds of thousands of youngsters were left in limbo without any properly organised support - other than that provided by individual schools - to make excruciatingly difficult decisions about their future educational, vocational and professional futures. Apparently, according to Gove's orthodoxy at the time, any requirement imposed upon schools to provide meaningful careers advice and guidance was regarded as a breach of the sacrosanct autonomy of schools. The push to spread "Academy" autonomy to schools had become a weird dogma, which risked depriving youngsters of exactly the kind of help they needed to make the difficult journey from the world of education into the world of work.

  Strikingly, it was not just my opinion that this was wrong; it was also the opinion of No 10 and, most especially, David Willetts, the Conservative minister in the Department for Business, Innovation and Skills at the time who shared some responsibility for educational outcomes for school leavers. Yet the non-cooperation of just one department in Whitehall prevented the government from doing what was patently needed. It wasn't until the very fag-end of the coalition that Gove's successor, Nicky Morgan, finally introduced a new approach to careers advice and guidance, which went some way towards filling the vacuum that had opened up in the previous four years.

  Needless to say, the principal fault in both cases did not lie with officials - the failure of No 10 to assert its will over two obstreperous secretaries of state was a far greater problem - but they both illustrate the mismatch between the need for swift action and the reality of constipated decision-making within Whitehall. I regularly found myself squeezed uncomfortably between the wish to react rapidly to reasonable public demands for action and the reality of cumbersome decision-making in government, stuck between the politics of a digital age and the analogue arrangements of Whitehall.

  • This is an edited extract from Politics: Between the Extremes by Nick Clegg. To order a copy for £16.40 (Bodley Head, RRP £20) go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min p&p of £1.99.