Return to: Search Page or to: Table of Contents Vol. 16, issue 4

Elias Dinas and Kostas Gemenis, "Measuring Parties' Ideological Positions With Manifesto Data: A Critical Evaluation of the Competing Methods," Party Politics, 16 (July, 2010), 427-450. [Available at http://ppq.sagepub.com/content/vol16/issue4/ ]

First paragraph:
Positioning political actors (and political parties in particular) along the Left-Right (L-R) continuum and other policy dimensions has been an important feature of recent empirical research in comparative politics. Based on a variety of theories and methods, political scientists are now 'able to operationalize a wide range of models within what has become an important sub-discipline of political science' (Laver, 2001a: 6). Three main approaches have been proposed for the study of party location: (a) expert surveys, (b) opinion poll data and (c) content analysis of party manifestos.1 Although there is ongoing discussion about the strengths and weaknesses of each approach (see Budge, 2000; Kleinnijnhuis and Pennings, 2001; Mair, 2001; McDonald et al., 2007; Steenbergen and Marks, 2007; Volkens, 2007), the last-mentioned has nevertheless become the most popular for two reasons: first, data from party manifestos attain a greater degree of impartiality. Expert surveys and opinion poll data give us the picture of the party as perceived by political analysts and voters, respectively. Manifestos, on the other hand, provide a more accurate and representative picture of where the parties stand in the policy space, without our requiring further knowledge about their policy record. Second, the Manifesto Research Group (MRG, now renamed Comparative Manifestos Project, CMP) has produced a rich time-series of data (see Budge et al., 2001; Klingemann et al., 2007) that is unrivalled by any other method. Consequently, the MRG/CMP approach has emerged as the prima facie method by which to estimate parties' policy positions against alternative data sources and also among alternative methods of coding party manifestos.

Figures and Tables:
Figure 1. Examining the scalability of the items forming the right-wing indicators according to the 'standard' CMP method. Note: all items have been recoded, ranging from 0 to 10
Figure 2. Examining the scalability of the items forming the left-wing indicators according to the 'standard' CMP method.
Table 1. Reliability of the estimates according to the Heise test-retest method
Figure 3. Greek parties' positions according to the 'standard' CMP method (Laver and Budge, 1992).
Figure 4. Greek parties' positions according to the 'vanilla' method (Gabel and Huber, 2000)
Figure 5. Greek parties' positions according to the 'two-stage factor-analysis' method (Laver and Budge, 1992)
Figure 6. Greek parties' positions according to the 'domestic' method (Klingemann, 1995)
Figure 7. Greek parties' positions according to the 'inductive' method (Laver and Budge, 1992)
Figure 8. Greek parties' positions according to the 'regression' method (Franzmann and Kaiser, 2006);
Table 2. Correlations among CMP estimates, expert surveys and voters' perceptions

Last Paragraph:
(First paragraph of conclusion) If there is an answer to this question, then it is probably not the standard CMP method for the measurement of parties' positions in general (Budge and Klingemann, 2001), or, in the case of Greece, more specifically (Konstantinidis, 2004). Beyond that, to argue that either the 'a-theoretical' 'vanilla method' or the 'regression' method (which similarly relaxes the 'valence issue' assumption of CMP coding) performs better, becomes a rather subjective and probably not very useful enterprise. What is most important, however, is that by analysing most of the employed methods in terms of their reliability and validity, we find substantial divergence between the methods. This finding questions the robustness of results based on only one of the measures. Given that there is no particular method that clearly outperforms all others, it seems that it is pivotal for studies which employ the CMP data to subject their analysis to sensitivity testing.

Last updated July 2010