It is now forty years since the formation of the Social Democratic Party (SDP), by Roy Jenkins, Shirley Williams, William Rodgers and David Owen was supposed to have heralded a 'historic re-alignment of the left'. Why did it fail in this objective ? A rather post facto justification, used by some of its founding members, was that the advent of New Labour was the eventual successful outcome of the SDP. However, if that really was their ultimate aim, splitting from the Labour Party was a rather illogical way to achieve it. Why did they not stay to 'fight, fight and fight again', to quote the words of their political hero, Hugh Gaitskell ?
There are, however, reasons to believe that most of them had severe reservations about New Labour. Although Roy Jenkins often gave paternal advice to Tony Blair (most publicly in the form of the Jenkins report on electoral reform in 1997) this advice was often ignored, of which the Jenkins report itself is a case in point. When the former Labour MEP Hugh Kerr asked the late Shirley Williams whether she was going to re-join the Labour Party, she replied that, under Tony Blair, it was much too rightwing for her. Indeed, most of the former SDP (with the exception of David Owen and his followers, a number of whom, such as Danny Finkelstein, eventually joined the Conservatives) joined the Liberal Democrats, who became increasiingly anatagonistic to New Labour. (A few prominent SDP members, such as David Sainsbury, Andrew Adonis and Roger Liddle, re-joined Labour, but most stayed with the Liberal Democrats). This in itself points to an important truth: Blair's New Labour project was essentially neo-liberal, and not social democratic, and therefore had nothing to do with the SDP's original aims.
The SDP committed three monumental errors of political analysis at the outset, both of which originated in two fundamental misreadings of British political history. This is odd given that its first leader, Roy Jenkins, was a published historian and a distinguished author of political biographies, and that the SDP numbered among its early adherents several eminent historians of modern Britain, including Peter Clarke, David Marquand, Alan Bullock and John Cannon. The first error was to assume that the British electoral system has much room for third parties. A glance at the history of the Liberal Party since 1922 ought to have put paid to that idea. Parliamentary representation for the Liberal Party declined during the 1920s at a faster rate than its overall support in the country, because its support was spread too evenly. This is lethal for the prospects of third parties in an electoral system which lacks proportionality. The SDP experienced precisely the same problem. There was no reason to think that it would not do so. Second, the founders of the SDP assumed that it would quickly overtake the Labour Party in support and representation. Again, political history casts doubt on this assumption. The first Labour MPs entered Parliament in 1906. Not until 1945 did it become in any way a national party. From the Liberal Party's electoral eclipse in 1929, it took over sixty years for it to achieve any respectable Parliamentary representation. First-past-the-post compels parties to construct mass support in concentrated geographical areas, which inevitably takes time, and entails setbacks.
Last, and perhaps most important of all, was the belief, which the founders often eloquently and persuasively stated, that the Labour Party had, during the turbulent years of the 1970s, so irrevocably changed its nature and purpose that the only alternative for those who disagreed with the course which it had set itself was to leave the party. Many people agreed with this rationale and joined the SDP. However, that was based on a misapprehension about the nature of the Labour Party, At periodic intervals in its history - the 1930s, 1950s, 1980s, and the period after 2010 - the membership of the Labour Party has rebelled against its leadership. This has been caused by inevitable disagreements and tensions caused by the Labour Party's (perceived or real) shortcomings in government. It is inevitable that left-wing parties will always be disappointed with their own performance in government, because, for many activists, no programme of social and economic change, however successfully executed, will ever be enough. (The fact that the Labour governments of the 1960s and 1970s were buffeted by severe economic storms only deepened activists' sense of betrayal. However, it is worth bearing in mind that, even after the most successful Labour government in history, that of Clement Attlee from 1945-51, members were evidently discontented, which led to the 'Bevanite' insurgency in constituency Labour parties in the 1950s).
After such rebellions, the Labour Party always swings back to the Right. Therefore, the constitutional changes in the structure of the Labour Party, in the selection of MPs and in the election of its leaders, would not, in the end, usher in the dominance of the Left. At no stage did the Left become dominant in the Parliamentary Labour Party (PLP) or in the trade union movement. Factional dominance in the National Executive Committee (NEC) does not entail dominance of the entire Party, as the Labour politician Peter Shore has pointed out in 'Leading the left' (1993).
The founders of the SDP often pointed to the policy direction of the Labour Party in the late 1970s and early 1980s as evidence that the party had changed fundamentally and had ceased to be 'moderate.' However, I would argue that this argument was borne out by only one policy: that of the adoption of unilateral nuclear disarmament as Labour Party policy. This was, indeed, a radical departure from the Atlanticist foreign policy orientation of the Labour Party since the 1940s. However, it did not entail a wholesale adoption of a pacifist stance, as Michael Foot's (and most of the PLP's) support of the Falklands War in 1982 shows. The adoption of a policy of withdrawal from the EU (or Common Market, as it was then popularly described) was far from unexpected, as many leading figures on the Right of the Party (such as Ernest Bevin, Hugh Gaitskell and James Callaghan) had vehemently opposed the Common Market, so that in itself is not evidence that the Party was going leftwards. The SDP often cited the leftward trend in economic policy in the party, particularly the adoption of the Alternative Economic Strategy (AES) as evidence for the Party's growing 'extremism'. However, this criticism is disingenuous, as the founders of the SDP were members of the Wilson and Callaghan governments which, by today's standards, were very interventionist. They nationalised industries, subsidised sectors of the economy, controlled prices and incomes, and supervised a regime of exchange controls. They were also, let it not be forgotten, elected in 1974 with a manifesto commitment to 'shift irrevocably' the distribution of wealth and power in favour of working people.
The founders of the SDP no doubt felt a sense of deep disquiet that their hold on the Labour Party was being, however temporarily, disputed. When power shifts toward the Left in the Labour Party, as during the leaderships of George Lansbury, Michael Foot and Jeremy Corbyn, the Right of the Party have always expressed outrage. What remains surprising is that so many on the Labour Right in 1981 believed that they had decisively lost, and that the Party had no conceivable future.