ABSTRACT
A prerequisite for intercultural comparisons is that theoretical concepts are measured in an identical way in different countries. Using semantically identical questions is neither a sufficient nor a necessary condition to achieve equivalence. What is important is that items present comparable stimuli to respondents, i.e. that they are interpreted in an identical way. In this chapter, it is argued that different cultural and socio-economic backgrounds in individual countries may lead to different interpretations of items, even if there are no translation errors. These are discussed in terms of framing effects which arise when items are interpreted before the background of different societal contexts. Two subtypes – ambiguity- and schema-based framing effects – are distinguished and demonstrated using the gender-role battery of the ISSP. Ambiguity-based framing effects result from a lack of information an item provides to respondents. Respondents in different countries fill in these gaps with country-specific knowledge, which then gives the item different meanings across countries. Schema-based framing effects emerge when some components of the item generate associations which are not intended to be evoked by an item and are country-specific. For ISSP as a replicative survey program, designed to measure social change across countries, the inclusion of new countries turns out to be a particular challenge. This is shown by means of an item included in the 1994 survey but not replicated in 2002 because it did not work in the intended way in former socialist countries.
Internationally comparative research is seen as an excellent possibility to promote theoretical and methodological progress in the social sciences (e.g. Kohn 1987). However, comparative survey research has continuously to answer the question whether differences between countries are real or measurement artifacts. A prerequisite for intercultural comparisons is that theoretical concepts are measured in an identical way in different countries. Using semantically identical questions is neither a sufficient nor a necessary condition to achieve equivalence. What is important is that items present comparable stimuli to respondents, i.e. that they are interpreted in an identical way. If 396a common interpretation of items is not guaranteed in different countries, items are not functionally equivalent. Direct comparisons between countries on the basis of problematic items, e.g. regarding means but also other statistical measures, are not warranted under these circumstances. Moreover, comparison of means is admissible only if scalar equivalence of the measurements in different countries is given. The potential pitfalls in comparing very dissimilar countries are well known (e.g. Jowell 1998; Küchler 1987, 1998). However, as argued here, similarities with regard to culture or to survey tradition do not prevent severe methodological difficulties. Even comparisons of relatively similar countries might be impossible on the basis of at least some of the items designed to measure a concept.
In this chapter, it is argued that different cultural and socio-economic backgrounds in individual countries may lead to different interpretations of items, even if there are no translation errors. Expectations regarding inequivalences are derived from a general theoretical approach which consists of several components: cognitive and motivational processes influencing response behavior are assumed to be largely invariant across cultures. Intercultural differences in the interpretation of questions are then generated by an interaction of characteristics of societies and items. These are discussed in terms of framing effects which arise when items are interpreted before the background of different societal contexts.
In the following, I shall first present the domain of gender roles as an example used throughout this chapter. I shall then discuss cognitive processes which have an impact on the interpretation of survey questions by respondents and on finding the appropriate response. Different kinds of framing effects will be described together with potentially problematic characteristics of items which give respondents some leeway in interpreting a question and selecting their answers. After a description of the data and methods used, the results will be presented. The conclusions will proffer some recommendations for future research.
