Methodological Challenges in Cross-comparative Surveys

The Case of Understanding Attitudes towards Democracy in Hungary

Abstract

Our paper examines the validity of the rotating questionnaire block about perceptions about and attitudes towards democracy included in the sixth round of the European Social Survey (ESS). The preliminary assumptions that inspired our analysis were that respondents’ understanding of the questions formulated in such an internationally comparative survey may be challenged due to diverging theoretical constructions and narratives that feed historically developed notions of ‘democracy.’ Moreover, even within the same country people with a different socioeconomic, ethnic, and educational background may have different perceptions about the same questionnaire ‘items.’ We applied a multi-method approach to analyze the above metho-dological puzzle: a complex statistical analysis of the Hungarian ESS data served to help examine the consistency of answers to individual items and the entirety of the questionnaire block, while supplementary focus group research helped us apprehend the variety of interpretations of and perceptions about the individual items, as well as problems with understanding various terms included in the questions that assessed attitudes towards democracy. Our findings support the initial hypothesis: respondents had obvious difficulties understanding some of the items designed to assess attitudes towards democracy, while many others had differing interpretations. We conclude that even though the ESS is one of the most refined, well-prepared and validated comparative surveys in Europe, the related data cannot be analyzed without careful consideration of what the individual questions might mean in different contexts.