Return to: Search Page or to: Table of Contents Vol. 16, issue 3

Luigi Curini, "Experts' Political Preferences and Their Impact on Ideological Bias: An Unfolding Analysis based on a Benoit-Laver Expert Survey," Party Politics, 16 (May 2010), 299-321. [Available at http://ppq.sagepub.com/content/vol16/issue3/ ]

First paragraph:
Since Castles and Mair (1984), expert surveys have become increasingly popular as a way of identifying the policy preferences of political parties. In such cases, a survey of country specialists is directly conducted, whereby they are asked to locate political parties in their own countries on a set of predefined policy dimensions: from a general left-right scale, to a variety of more specific policy dimensions. Several factors help to explain the success of expert surveys (Mair, 2001). First of all, such surveys provide information on party policy positions in a common, standardized format across a wide range of countries. Second, the fact that they reflect the judgement of experts - who are presumably well-informed - gives them weight and legitimacy. Finally, expert surveys are quickly and easily compared to other forms of analysis (such as the analysis of the content of party electoral programs, or legislative behavioral studies). In the present work we are going to analyze the expert survey conducted by Benoit and Laver (2006), which is probably the most comprehensive and ambitious of its kind. We will be focusing in particular on the Italian case, given that in this case we can avail ourselves of two expert surveys, the one conducted soon after the other (2003 and 2006), using the same methodology. This fact can, inter alia, allow us to check for the robustness of our results. However, instead of using experts' responses to identify the policy preferences of parties, we will use these answers to directly map the ideal points of the respondents. Our interest goes beyond the merely descriptive. One of the problems of using surveys of any kind to estimate party positions is that respondents may be influenced by their own subjective political views. As a consequence, there is the risk that experts can give biased responses and that such (ideological) bias affects certain parties more than others. By using the estimated ideal points of the said respondents, we are going to show that there is an ideological bias in the Benoit and Laver survey, particularly in terms of the left-right dimension. In this case, approximately 16 percent of those parties analyzed appear to be significantly affected by it. As a result, the expert survey is incapable of directly producing a valid position for such parties. Thus, we shall be proposing two different ways of generating less biased scores. We shall then go on to discuss the importance of these findings for empirical research.

Figures and Tables:
Table 1. Summary data from the Italian expert survey (2006)
Figure 1. Relationship between the left-right score of a party and its average level of sympathy (expert survey: Italy 2003 and 2006)
Figure 2. The Kernel density distribution of the ideal points of the respondents along the left-right dimension (Italy 2003 and 2006)
Figure 3. The mean expert position in the second scenario (Italy 2003 and 2006)
Table 2. Cross-national comparison of the ideal points of the respondents along the Left-Right dimension
Figure 4. The mean expert position in the second scenario (all countries considered).
Table 3. Parties for whom the difference between actual left-right placement and 'corrected' left-right placement is significantly different from zero, using sympathy scores
Table 4. Parties for which the difference between actual left-right placement and ' Unbiased' left-right placement is significantly different from zero, using experts' ideal points
Table 5. (Re)Logit estimates for parties' probabilities of being biased (s.e. clustered over countries)

Last Paragraph:
Expert surveys have become increasingly popular among political scientists for a variety of good reasons. However, even this method, designed to identify the policy preferences of parties, is not without its flaws. In this paper, we have used the latest expert survey by Benoit and Laver (2006) to unfold the ideal points of the said survey's respondents. The estimated ideal points have been employed to show that in almost 16 percent of the cases in question, there is evidence of ideological bias in the experts' placement of parties along the left-right dimension, especially among right-wing parties (albeit not necessarily extreme right-wing parties). Moreover, the left-right position of a party appears to be less important than the particular pattern of respondents' policy preferences when it comes to explaining the presence of the said bias. We have also examined two ways of producing less biased estimates. The first method is directly based on a regression technique, while the second is based on the negligibility of ideological bias in the experts' answers to more specific policy questions (such as those regarding Economic or Social policies). Thus, while our analysis would seem to suggest that any precise assessment of the positions of parties along the left-right scale should be treated with caution, especially in certain given cases, it is nevertheless reassuring with regard to the use (and the usefulness) of experts' party scores along other, more specific, policy dimensions.

Last updated May 2010