‘The Jews Are Just Like Any Other Human Being.’ An Attempt to Measure the Impact of Informal Education on Teenagers’ Views of Intergroup Tolerance

Our paper presents the results of a study which was conducted between 2016 and 2019 in a high school in Budapest. The research attempted to measure the impact of the Haver Foundation’s activities on high-school students. The Foundation implements activities about Jewish identity, thus we intended to see whether the different activities of the Foundation changed the attitudes of high-school students, and whether they affected the formers’ level of knowledge and the associations they make with Jews. In line with the sensitivity and complexity of the research topic, and in order to create the broadest analytical framework, we followed several classes in a longitudinal setting by triangulating our methods. Results confirm the importance of these activities, especially with regard to the increase in the level of knowledge about Jews and Judaism. They also indicate that there is a need for such informal settings in high-school education. However, more extensive research needs to be carried out to obtain more accurate results about the reduction of prejudices.


Introduction
Prejudice reduction has become a widely researched topic in social psychology in past decades (see an extensive review of hundreds of published and unpublished studies by Paluck and Green, 2009). Our goal was twofold: First, we wanted to assess how the Haver Foundation's activities affect students' mindsets. This could have been manifested in a tolerant (open-minded) attitude (towards Jews and other minorities), some development of a culture of debate, and actual knowledge about Jews and Judaism. Second, the research was meant to be a pilot study for examining whether the former methods were applicable in such settings: Could we measure the impact of such activities on the teenage target group with our specific research design, or was the issue more complex and nuanced, thus needing to be tackled somewhat differently? To the best of our knowledge, only sporadic experimental research has been undertaken in Hungary in this field; most importantly, by Váradi (2013) and Kende and her colleagues (2016) who both focused their attention on the impact of peer influence on anti-Roma prejudice; and also by Orosz and his colleagues (2016) who recently pointed out the effectiveness of the 'living-library' intervention for reducing prejudice towards Roma and LGBTQ people.
This issue seems to be relevant in Hungary, taking into account the alarming findings of recent research into intergroup prejudice. Hungarians can be characterized as typically having strong prejudices towards different ethnic and religious minorities. International comparative research found widespread prejudice against various minority groups, such as immigrants, Jews, and LGBTQ people, indicating that levels of negative attitudes towards the out-group in Hungary are among the highest of all countries in Europe (Messing and Ságvári, 2018;Örkény and Váradi, 2010;Zick et al., 2011).
When it comes to anti-Semitic incidents, according to Kovács and Barna (2018) the number has decreased over the past two decades (since 1999), and the number of actual incidents is much lower than perceived by the Jewish respondents of a survey implemented in 2017. It is important to note, however, that even though the Jewish community constitutes the most significant religious community, making up approximately one per cent of the total Hungarian population (Kovács, 2011;Kovács and Barna, 2018), it is mostly concentrated in Budapest. It is worth mentioning that most Jews are not members of a synagogue (Kovács and Barna, 2018); however, the trauma of the Holocaust is one of the main cores of their Jewish identity.
As far as the political and social context is concerned, Hungary is a rather homogenous country in ethnic and national terms, but xenophobia has been strong since the change of the socialist regime. A relatively low level of anti-Semitism was initially paired with strong anti-Roma sentiment, which even led to incidents of homicide against Roma (Vágvölgyi, 2014). Since 2015, however, anti-immigrant and anti-refugee campaigns 1 have been instigated which have affected Jews, as George Soros (an American Jew of Hungarian origin) was made one of the scapegoats for the so-called 'refugee crisis.' The present research took part after this highly intense period of enmification (involving a series of billboard campaigns).

The role of informal methods in the Hungarian education system, with a special focus on the Haver Foundation
In most Hungarian schools, traditional frontal teaching is the usual method, and the application of project-based or informal elements in the official curriculum is only sporadic (Lannert and Nagy, 2006). Furthermore, topics related to the subject of the present research, such as inter-group tolerance and the protection of and respect for minority groups, are scarcely included in the official curriculum (Csákó, 2009). Regarding content, the National Curriculum includes latent anti-Semitic features (such as the inclusion of Albert Wass, a poet with anti-Semitic views; Szily, 2019), and does not give space to education about minorities (for example, TEV [2015] research focused on the lack of a Jewish presence in both Hungarian literature and history), and such tendencies are strengthening (Szombat, 2020). 2 Therefore, most students have no or very little knowledge about Jews, apart from concerning one historical event (the Holocaust).
To compensate for the shortcomings of the Hungarian education system, certain non-governmental organizations provide informal education classes to high schools. These NGOs typically aim to fight hatred against immigrants and refugees, the Roma, LGBTQ people, and Jews. (A list of organizations with their sensitizing activities can be found in Appendix 1.) The Haver Foundation is one of these organizations: their young, voluntary team (who self-identify as Jews) regularly go to high schools (upon invitation) to hold ninety-minute sessions about Jewish identity, using various means. This inter-religious/intercultural form of interaction is aimed at familiarizing non-Jewish students with all things Jewish.
In an increasingly centralized and formal educational system, the role of informal (non-formal) activities is becoming more and more important in schools and classes, about which teachers are also interested or concerned (via oral correspondence; it also happens that teachers or directors do not permit such activities, or only some of them are welcome 3 ). Most such activities focus on one of the minorities, or a specific deprived group. The main goal is increasing students' social responsibility and tolerance towards their classmates. Most of these activities are led by volunteers: (i) who are 'experienced experts' (e.g. members of a minority group); (ii) who are close to the students in age; and, (iii) whose activities are based on informal educational methods as opposed to frontal ones. Regarding age, peer-group relations can promote specific effects or meanings in these INTERSECTIONS. EAST EUROPEAN JOURNAL OF SOCIETY AND POLITICS, 5(4): 79-109. settings, in contrast to the teaching of much older individuals. This claim is supported by the research of Rogers (1967).
Concerning methods, Coombs and his colleagues (1973) have defined nonformal education as that which takes place within educational institutions, but with different or unconventional methods, while informal education includes every type of organized educational activity which takes place outside of these educational institutions. According to Coombs, students learn the most in nonformal and informal settings. Csoma's (1985) differentiation slightly deviates from this definition (however, his research involved adult learning). According to Coombs, formal education is equivalent to school; non-formal education to courses; and informal means 'unbound' learning.
The Haver Foundation's mission is to foster dialogue and spread tolerance through informal education. The activities they provide for classes cover topics that include identity, heritage, the Holocaust, the Jewish quarter (a city area of Budapest), and community challenges, and are implemented using informal educational methodologies and tools. From their view, dialogue between Jews (trained volunteers between the age of 20 and 35) and non-Jews leads to tolerance and common understanding, a claim which is also in line with Rogers' experimental learning methods. Furthermore, the age of the high-school students is also a crucial point with regard to attitude formation. It is almost common knowledge that ethnic, racial, and other stereotypes and prejudices are developed during (early) adolescence (Piaget, 1970) -the period when individuals develop their own identity (see Erikson, 1950).
However, the Foundation faces numerous difficulties when its volunteers arrive at high schools because students have mostly been exposed to a frontal, knowledge-based educational system in which critical thinking, creativity, and other skills are not taught. Therefore, they need to adapt to a new style of teaching. Furthermore, because of the high concentration of Jews in Budapest, as opposed to the scarce Jewish communities in the countryside, most students do not encounter Jews in person outside of these classes.

The theoretical and conceptual framework
The theoretical framework of our research is primarily based on Intergroup Contact Theory, originally developed by Gordon W. Allport (1954). The basic idea of Allport's theory -also known as Contact Hypothesis -is that under appropriate conditions interpersonal contact is one of the most effective ways of reducing prejudice between majority and minority group members. While Intergroup Contact Theory originally held that Allport's optimal conditions 4 are essential for reducing intergroup prejudice effectively, a comprehensive review of more than 500 empirical studies that examined Contact Hypothesis (Pettigrew and Tropp, 2006) helped refine the original theory, drawing attention to other important elements of the working mechanisms of intergroup contact. In their meta-analysis, the latter authors highlighted that intergroup contact has a positive effect on negative stereotyping, and scholars have drawn attention to the effect that the quantity of intergroup contact has on reducing prejudices as frequency of contact helps with the decategorization of out-group members and diminishes stereotypical ways of thinking (see also Velasco-Gonzalez et al., 2008). Pettingrew and Tropp concluded that 94 percent of the more than 500 studies they reviewedincluding surveys and different types of experiments -found that intergroup contact significantly reduces prejudice. Paluck (2011) carried out field experimental research among high-school students in the United States that was aimed at testing peer influence on intergroup prejudice; more specifically, on assessing students' perceived social distance from stigmatized social groups and overheard cases of harassment against gay and overweight students. Paluck's study is somewhat similar to ours in terms of its target group (teenagers) and method (field experiment); nevertheless, the scale of the former was much larger (ten high schools were included in the US field experiment vs. only one high school in the present Hungarian study) and the experimental treatment lasted much longer (the five-month presence of selected Peer Trainers in the experimental schools vs. only three ninety-minute occasions in our study). Paluck pointed out that at the end of the experimental period a significant and widespread pattern of effects had occurred that could be attributable to the intervention. The researcher concluded that peer influence on intergroup prejudice can spread throughout social networks; moreover, the effects of peer influence across time and in a context outside of school was also measured.
Furthermore, a meta-analysis (N=985 published and unpublished reports) was carried out by Paluck and Green (2009) who evaluated observational, laboratory, and field experimental studies aimed at reducing prejudice. The authors concluded that most of the studies that focused on prejudice-reduction interventions, (e.g. workplace diversity training and media campaigns) were unable to identify causal effects. Although some intergroup contact and cooperation interventions were evaluated as promising, the authors suggested a much more rigorous and broad-ranging empirical assessment of this work. Beyond the methodological concerns that emerged from the meta-analysis, a summary of prejudice-reduction approaches, theories, and future directions for research were also compiled which may serve as a useful guideline for further research in this field. From a methodological point of view, the most important point the authors made is that, based on non-experimental research, we are unable to measure prejudice reduction in real-world settings through applying experimental designs. Field experiments are primarily evaluated as the most useful and promising, but also underutilized approach.
Focusing more on the target group of our study (Hungarian high-school students), extensive research (Váradi, 2014) has aimed at increasing understanding of the formation of Hungarian teenagers' attitudes towards the Roma. As adolescence is a crucial period in identity development, Váradi's objective was to determine to what extent classical and more recent theories about the formation of prejudice can be applied in a context in which there is no public consensus about the need for respect towards minorities. Váradi concluded that students' attitudes towards the Roma do not considerably differ from those of their parents, as 'every third participant [was] fully prejudiced, rejecting all social contact with the Roma, agreeing with derogatory remarks against the Roma, and willing to take action against the members of this group' (Váradi, 2014: 206). Furthermore, it is important to mention that Intergroup Contact Theory was reinforced by Váradi's research, as students who had Roma friends or acquaintances were much less prejudiced towards the Roma, generally. 5 As we have already stated, only sporadic experimental research has been done into the ethnic prejudices of Hungarian youth, and the studies we describe below both focus on anti-Roma attitudes. Váradi (2013) tested how majority students reacted to UCCU's 6 Roma Informal Educational Foundation's programs. We treated this study as a pilot project in relation to our research, as both in terms of its focus (analysis of the same methods of informal education among teenagers 7 ) and methods (qualitative and quantitative pre-and post-tests with students, completed with group interviews with teachers) it was similar to our study. Váradi (2013) attempted to measure changes in attitudes based on the answers of 228 students from ten classes in Hungary in 2013. We also used this questionnaire as a starting point, but adapted some questions for our research. The most important lesson from this research is that these kinds of short-term informal educational programs cannot significantly reduce prejudices towards the Roma in the case of the vast majority of students. To be more specific, the proportion of those students who reported 'feeling awkward when other people criticize the Roma' slightly increased after the program. Moreover, increasing empathy and getting to know how Roma people live their lives in Hungary was more successfully accomplished by Roma volunteers. In summing up, Váradi (2013) concluded that the UCCU program successfully initiated the process of questioning prejudices towards outgroups.
Most recently, Kende, Tropp and Lantos (2016) tested the effects of intergroup friendship between Roma and non-Roma Hungarians on attitudes, relying on a quasi-experimental research design of a small sample (N=61) of university students. Comparing pre-and post-test measures for the experimental and control group, the researchers observed significant positive changes in attitudes and intentions in terms of contacts exclusively created among participants subject to the contact condition in the experiment. Kende and her colleagues also concluded that positive changes were moderated by perceived institutional norms, which finding might corroborate the potential of contactbased interventions. In contrast to Kende and her colleagues' intervention -which was implemented in a university setting -our experiment was carried out in a high-school environment in Budapest, led by an open-minded headmaster who is also supported by mostly liberal and open-minded teachers. Based on the former two experimental pieces of research we conclude as well as assume for our own 5 For further research on the identity, intergroup attitudes, and prejudices of Hungarian adolescence, also see Váradi (2014: 63-70). Furthermore, Váradi is presently leading an ongoing research effort in Hungary entitled 'Class climate, attitude climate' that includes 60 school classes of approximately 1500 Hungarian teenagers who started secondary school in the academic year 2016/2017. The aim of the longitudinal study is basically to understand the interplay between group norms and prejudice. See more about the project at https://nationalism.ceu.edu/dynade 6 See the Foundation's homepage at: http://www.uccualapitvany.hu/english/ 7 The UCCU foundation applied the same methods of informal education as Haver does. UCCU adapted the curriculum of the Haver program and adjusted it to have a Roma focus. study that contact-based intervention may work (i.e. affect intergroup relations positively); however, also that the measured effect is not comprehensive.
As in our study the experimental groups were exposed to interactions only three times, which cannot be considered 'frequent contact,' we did not expect major changes in the target groups' attitudes but rather a 'first step' towards the long process of questioning intergroup prejudice.

Research methods
We used a series of field experimental interventions to test whether different approaches to informal education can foster intergroup tolerance in the form of intergroup attitudes. Designing an appropriate measurement for impact assessment was indeed a challenging task. To make our approach as comprehensive as possible, we decided to use mixed methods (Denzin, 1978).
The advantage of triangulating methods is that quantitative methods can be combined with qualitative ones. In our case, with quantitative data we benefitted from a high level of outreach and comparability, while with the qualitative approach we could obtain answers to more in-depth questions. Furthermore, there were phases during which quantitative research would not have been possible due to the sensitivity of the research topic.

Experimental context
Our research was partly based on a quasi-experimental design that included control and experimental groups, as well as pre-and post-tests carried out before and after each intervention (i.e. in the Haver-affected classrooms and during related outdoor activities). Both the pre-and post-tests included quantitative measures (repeated question blocks for measuring shifts in attitudes) and qualitative ones (namely, focus groups with students and an extensive content analysis of study participants' associations with the word 'Jewish').
In more technical terms, we used a 2×2 (two conditions × two time points) mixed factorial design with one experimental condition (interventions with the volunteers from the Haver foundation) and one control condition (no intervention apart from the 'dilemma café' 8 ), and the measurement of changes in intergroup tolerance and levels of information about Jewishness over time through comparisons of pre-test and post-test scores and associations. This was completed with the above-mentioned qualitative and observational research tools.
The selection process for the experimental and control subjects, however, was far from 'ideal,' 'textbook-type' randomization or matching. Originally, we wanted to select the members of the control group randomly, across the four classes, but for logistical reasons and for the convenience of the school 9 this was not possible. Instead, one entire class (namely, 'Class B') served as the 'control' group, which approach obviously caused some selection bias. Moreover, as the study school was committed to taking part in the experiment but only one class could serve as a control group, this resulted in an uneven distribution of experimental and control subjects in the design.
The experimental subjects took part in an 'Identity' activity, as well as a walk through the Jewish quarter in Budapest and a 'dilemma café,' whereas control subjects only took part in the dilemma café. The Identity activity involved a ninety-minute session during which students sat in class with Haver volunteers and discussed topics related to Jews and Judaism. The main goal was to create a safe space where students were not shy about asking questions from the two Jewish volunteers. Such sessions are typically split into several exercises (see Appendix 2), using different tools to reach as many students as possible. The ninety-minute interactive walk took place in the historical Jewish quarter of Budapest. The volunteers showed students places and buildings which are related to Hungarian Jewish history, and talked about the past and present of the community, touching upon traditions and religion. The dilemma café is a ninetyminute activity that involves students -in small groups -discussing four dilemmas. The topics are introduced in Section 5.4.2 and are elaborated on in Appendix 3.

Materials and methods
While the quantitative research was carried out both with the experimental and the control groups, the qualitative research was only partly implemented in an experimental setting. Questionnaires were developed jointly by our research group and representatives of the Haver Foundation. 10 Some of the questions were based on the materials from the Identity activity, while some were borrowed from earlier research; most importantly, from Váradi (2013) who carried out similar research involving an impact assessment of UCCU's informal training, We carried out interviews with all the four form teachers, but we only organized focus groups among those students who were included in the study due to the lack of further research capacity. Further details about the research design and schedule of the experimental research are shown in Table 1.
As Table 1 shows, we gathered data in four waves -before and after each intervention during 2016 and 2019. The almost one-and-half-year gap between the second and the third intervention was not optimal, but due to organizational and other management issues we could not complete the fieldwork earlier. On the other hand, in this way we could follow the experimental groups for more than three years. One of the pillars of the qualitative research was the two focus groups conducted in three classes, while the other involved the interviews with the form teachers. The participants of the focus groups were chosen by one of the teachers (not the form teacher) through random selection. It is important to mention that the students were the same in the pre-activity and post-activity discussions. Four or five students -selected randomly -participated from each class. We also took into account pre-and post-activity associations.
The dilemma café was the third intervention. This time -as opposed to in the earlier waves -the activity constituted the research itself: the dilemma café was organized with the help of observers who paid attention both to the students' and the volunteers' activity. Similarly to the previous waves, all classes took part in the research. Approximately five groups were created in each class. The number of participants in one group varied from three to eight. The groups participated in a dilemma café: they were presented with five dilemmas from which they had to choose four, based on their titles. They discussed each dilemma in depth for fifteen minutes with a moderator (a Haver volunteer). Every group had an observer who took notes according to the researchers' detailed instructions, which were later analyzed by the researchers. The goal was to assess the impact of the previous activities: first, whether participants had managed to acquire a certain culture of debate; second, to what extent they had received and processed the main message of Haver (tolerance towards minorities, and open-mindedness) and third, whether they manifested any sign of an increase in knowledge about Jews and Judaism. In other words, the analysis was undertaken in line with the Foundation's goals.
The quantitative research was based on a self-administered questionnaire consisting of a core question block, with repeated questions about attitudes and knowledge about Jewishness, completed with an additional block of questions about various topics. The core blocks of the questionnaires, as well as the topics of the qualitative research, are compiled in Appendix 3 (the entire research materials are available upon demand). Also, because of the lack of space, we present only the most important results from the quantitative survey in Section 4.

Methodological concerns
While with the complex design we tried to eliminate many possible flaws, we encountered some difficulties. Generally, the high school we selected can be characterized as a very liberal and open-minded community. This school does not represent the average Hungarian or even Budapest-based high school well, but they let us carry out our extensive research due to their openness. Another important issue that should be mentioned here is that it transpired during the implementation period that the form teacher of the control group (Class B) had already paid significant attention to discussing social issues such as tolerance with their class. In this sense, selecting this class as a control group was not the best choice.
Regarding the dilemma café, there were two further concerns. First, the volunteers moderated the groups in various ways -as they were instructed to be flexible -, therefore intergroup comparison was rather challenging. Second, in lacking recorded data (due to ethical issues), we had to rely completely on our observers who understood their jobs differently. However, even with the abovementioned problems, we managed to carry out a limited content analysis.
We are well aware that the study school we based our analysis on was chosen due to 'convenience' sampling, and that the selection process of experimental and control subjects was far from ideal for the above-mentioned reasons. While interpreting the results of our study, these facts should be kept in mind.

Results
We present our findings in a more or less chronological order, mixing qualitative with quantitative results. In order to place the study in its context, in the first section we show what the form teachers and the students in the focus group discussions said about the school's values, focusing on the dynamics of the intergroup relations. The second section illustrates results related to the first activity -i.e. using a combination of qualitative and quantitative methods. The third section shows the results of the quantitative survey regarding the students' perceptions about Jewish identity, which is followed by results connected to the second activity. The last section describes the third intervention.

Exploring the research context: Values of the school
Regarding the mentality and values of the examined high school, the form teachers had different opinions. While three of them said there are no values to which the school is committed, one believed that freedom is an important concern. Concerning the approach towards students, they agreed that the school is more humane: while keeping high standards, they try to accommodate students' needs. The philosophy of the form teachers -compared to that of teachers in an average state school -is more student-oriented. All of them seemed to be open towards minorities.
Students who took part in the focus groups seemed to like going to the school: 'all students here like the school… everyone participates in the programs,' said one of the students from Class C. None of the groups were specifically interested in public affairs: they read and listen to things which were important to their parents.
As far as attitudes towards minorities are concerned, opinions differed among the students, but were not class specific. Some students believed in a multicultural country, but 'if people are similar, there are no conflicts… so it might be better' (class A). In both discussions students tried to approach this issue by referring to a real topic which concerns Hungary; namely, the situation of immigrants and refugees. Their opinions clearly reflected the arguments which divide society and the pros and cons of the former issue which can be heard in the media.
In every class there was someone who belonged to a minority group, but the participants did not perceive them as 'others' because they were born in Hungary and were familiar with Hungarian culture, etc. According to the students, this cannot be compared to the situation of refugees who come from a less familiar culture: 'the Arabs lie down at noon and do their praying or whatever' (Class C), while their (half-Polish) classmate would never do such thing. This quote indicates rather limited and stereotypical thinking. The distance between the former culture and that of Hungary seems to be the main determinant of students' judgments. understandable, and should not be judged. According to these individuals, this is not discrimination; calling someone a Jew or gay is already embedded in their vocabulary. 'You cannot do anything about this. […] You don't stop, but accept this' (Class D). The other group -when they were asked -emphasized the positive impact of their Chinese classmate, who had given them a presentation about Chinese history when they were studying this topic. They did not study about Jews (or the Holocaust) because they had not reached this topic in their history class, but many have Jewish acquaintances or even relatives. One of them said that he likes him/her, 11 'but (s)he has his/her own typical Jewish characteristics as well' (Class D). This again refers to stereotypical thinking. They agreed that those who have acquaintances from a given group are less discriminative, which suggestion supports Allport's (1954) contact hypothesis. Participants agreed that the students' attitudes are influenced mostly by their micro milieu (family and friends).

First intervention: the 'Identity' activity
Below, results concerning the first intervention are discussed. Associations were surveyed just before and right after the identity activity, while data were collected about students' opinions in a focus group afterwards. Quantitative comparative results are also discussed in relation to the perception of Jewish identity.

Associations
This examines the short-term impact of the 'Identity' activity based on the associations. Every activity started off and ended with a short game during which the volunteers asked the students to write down their associations related to the words 'Jew, Jewish.' 12 This activity -in contrast to that of the focus groups -was appropriate for measuring short-term impacts because the students' experience was still fresh. On average, there were 33 participants during each activity and they wrote down approximately three of four words, 13 resulting in approx. 200 and 240 words before and after the activities, respectively. 11 In the Hungarian language there is no linguistic gender differentiation. 12 The word 'zsidó' in Hungarian means both. 13 Associations could be sentences as well -depending on the volunteers' instructions -but single words were most frequently used.  As is clear from Figures 1 and 2, the pre-activity associations are focused more on Judaism, stereotypes, and the Holocaust (or Jewish history), while the post-activity associations reflect more on the Foundations' messages. In the second round there were fewer words related to religion -albeit this topic still dominated -and the words 'culture,' 'tradition' and 'people' occurred more. (These concepts arose during activities when students defined five pillars of Judaism: religion, culture and tradition, people, shared fate, and personal choice.) Apart from these words, 'identity,' 'community,' and 'personal choice' also appear as elements of identity formation. Associations related to WWII disappear and human values such as solidarity and equality appear. The words 'humans' and 'like everyone' refer to the idea -heard also during the activity -that Jews are just like any other human being.
It is worth expanding a little more on the importance of religion-associatedand Holocaust-related words. For centuries, Jews defined their Jewishness through religion (Webber, 2003), which explains the strength of this concept. A ninetyminute activity can hardly erase this association. Regarding the Holocaust, students hear about Jews when they study twentieth-century history. Furthermore, the Holocaust is also widely discussed in the media and public discourse, as well as frequently represented in the cinema. The Haver Foundation was initiated partially based on the experience that students relate the word 'Jew' to the Holocaust, which was perceived as unfortunate.
Even if we cannot draw far-reaching conclusions from the associations of students in the two classes, they first serve as a good basis for comparison with the quantitative results, and second they illustrate well that activities are conducted in various ways. Therefore, their impact may be different as well. This is also true of students: not everyone reacts the same way.

Evaluation of the activity: A 'friendly discussion' (Class A)
Regarding the activities of Haver, everyone was satisfied. Participants mostly emphasized the interactive and personal nature. The former claim was supported in the survey as well, according to which relatively few students experienced boredom (5 percent said they were bored, and 17 percent said they were 'neither bored nor not bored'). It seems that students are in need of an informal style of education during which they can discuss and ask about topics which are not every day. Answers to the question 'what is this activity good for?' during the focus groups went hand in hand with the answers given in the survey: participants understood that the main goal was to expand their knowledge. Despite this, they remembered relatively little of what they had heard during the activity (even though they mentioned that informal education is more effective).
Most of the focus group participants mentioned the 'Identity' activity at home, but did not expand on the details. As someone from Class A summarized, 'I talked about the activity at home but not what it was about.' None of the classes had the chance to discuss it at school, which some of them missed. This might have been useful for helping process the information and remembering better what they had 'learnt.' Hearing someone called a 'Jew' or 'gay' triggered similar reactions: it bothered them, 'but what to do?' One student said that this is embedded into their vocabulary. These results indicate that the activity made them think, but did not inspire them to be proactive. In other words, it involved rather a passive reaction than an active one.

The perception of Jewish identity
We also assessed the potential impact of the first intervention using quantitative tools, comparing control and experimental groups' views about Jewish identity based on pre-prepared answer categories. 14 Figure 3 presents how students defined Jewishness after the first intervention along the five identity elements, as discussed in relation to Haver's Identity activity. Among the various identity elements, we only found a significant difference between the experimental and the control group in terms of the religious component: Those who took part in the Identity activity found the role of practicing Judaism to be a less important component of being Jewish than the members of the control group. This result is in line with what we found based on the analysis of associations presented above.
One of the most important goals of our quantitative research was to measure the level of knowledge about Jewishness. In Table 2 we have summarized the items which were included in the core questionnaire (therefore, these questions were asked a total of three times). In our analysis, 'do not know' and incorrect answers were coded together, as we were primarily interested in the proportion of those who correctly answered these questions. Based on the seven elements above, we created an index to measure the level of knowledge about Jewishness. With this index we aimed to compress information and measure potential changes in students' level of knowledge about Jewishness.
Comparing the rows in Table 3 (based on t-tests), it is obvious that the 'knowledge-index' only changed significantly within the experimental group, meaning that the students who took part in Haver's activities provided more correct answers after the first intervention than beforehand.

Second intervention -A walk in the ghetto
While the first section analyses the open-ended answers to the questionnaire (therefore, we may call these semi-qualitative results), the second section focuses on the close-ended answers.

Evaluation of the walk
Regarding the walk, most students enjoyed this activity -based on answers to the open-ended questions 15 incorporated into post-test II. It seems that they rarely participate in such alternative programs, and the method itself was an innovation for them (even if they had participated in the previous Identity activity). The results of the first question can be classified into two bigger themes. One is related to learning, and the other to the activity. Students 'learnt a lot of new things.' Some mentioned Jewish references (i.e. that they had learnt about Jews or Judaism), while others were happy 'to get to know this part of the city.' Regarding the second item, some comments were connected to the methodology, such as 'we were walking in the city and were not sitting in a classroom' and 'because we walked in the quarter, I could imagine better what they were talking about.' Other comments described the volunteers who led the activity: 'they were very informal,' and 'they answered our questions.' Two complained that they 'had to deal with the topic of Jews again,' and someone wrote that there was nothing (s)he liked. Negative comments were mostly related to the weather and the fact that students had to carry their bags. Answers to the third and fourth questions can be classified into two categories. One concerns the general information students acquired, captured in statements such as 'I got to know this part of Budapest better.' The other answers referred to specific, Jewish-related knowledge. For example, someone said they had learnt how Jews 'celebrate their weddings and how they eat' and 'I learnt about some Jewish traditions.' There were very few comments referring to the activity's attitude-framing nature: '[I brought home with me that] Jews are people just like us.' Someone else wrote '[I brought home with me that] I should be open to the world.' Only a very few students wrote negative answers.
In summary, students learnt a lot of new information and most of them enjoyed the activity. Once again, it seems that students were receptive to these innovative methods.

Shifting knowledge on Jewishness
Based on the core questionnaire, the same process of measurement (as presented in Table 3) was repeated to assess the level of knowledge about Jews and Jewish culture. The change in the level of knowledge about the latter was tested using paired-sample T-Tests that compared the change among those students who had filled out all three questionnaires (Table 4.)  Table 4 (based on t-tests) showed that the 'knowledgeindex' had only changed significantly within the experimental group -taking into account the whole study period -, meaning that students that had participated in Haver's activities provided more correct answers after the two interventions than before the experiment.

Comparison of the rows in
However, according to the linear regression models that were designed to measure the treatment effects more accurately by estimating the difference in the post-and pre-measures, 16 only the third model that tested the effect of the second intervention (the Jewish walk) has significant explanatory power (see F-test statistics in Appendix 4). In line with this, we only found significant t-values (at the 0.05 level) in the model that tested the separate effect of the second intervention. In interpreting the results of the regression models, it is important to bear in mind that our data was not perfectly appropriate for regression due to the low number of observation (especially in the control group), as well as due to the 'quasi continuous' measurement level of the dependent variable (the level of knowledge about Jewishness was measured using a seven-item scale).

Associations
The associations were repeated before the dilemma café (in all classes). The results showed little improvement. The experimental groups wrote similar words as prior to the Identity activity, which means that the effect was indeed short term. As mentioned earlier, it is difficult to erase a centuries-old mindset. Comparison of the experimental groups with the control group shows that the word 'human' appeared more often in the former, which means that some students remembered the message that Jews are like other human beings.

Dilemmas
There were five dilemmas associated with different topics: in one of them, the issue of traditions and their importance came up (Topic 1); another focused on stereotypes (Topic 2); the third one involved a dilemma between hate speech and free speech (Topic 3); and the fourth and fifth were both about inclusion versus discrimination based on dietary restrictions (being kosher; Topic 4) and origin (being Jewish; Topic 5). About the dilemmas, see more in Appendix 3.
Part of the method aims at developing a culture of debate, which is otherwise not supported by the national school system. The results of the dilemma café show that this debate culture is still in its infancy: most of the time the volunteer (or moderator) had to initiate conversation by asking supporting questions. Sometimes this was due to the topic of the dilemma, which was not found to be interesting enough, or was perceived as too obvious. However, we believe that most of the time the response was because students are not used to this kind of setting. This claim is supported by the fact that several students said they were happy to participate in such activities in which 'we could talk without having to do anything else but talk' (Class A).
The second issue is the participants' open-minded way of thinking (or a lack of this). A student from Class D said that 'calling someone a Jew does not mean the person is anti-Semitic.' This statement goes hand in hand with the results of the focus group: many students regard any kind of negative speech as 'normal' because of its embeddedness. In some groups, students explained stereotypes against Jews by saying 'Jews are indeed financial [sic]' 17 (Class D). In comparing these responses to the survey results and the general impression about these classes (which was positive), a contradiction arises: it appears that students try to comply with the expected behavior while thinking otherwise, or that these type of statements do not carry much weight for them. Calling someone a Jew, or holding stereotypical views, may still be regarded as something normal. Inclusion was viewed differently: many students believed it was fair if someone could not join a community easily, because 'when someone wants citizenship, there is also a procedure' (Class D), and 'we do not accept someone into a swimming team if the person cannot swim' (Class A). This represents a rather exclusive way of thinking. However, in this case the students were defending a hypothetical Jewish community that was not willing to accept a non-Jewish person (in line with the dilemma).
The third question concerned whether participants had acquired new information regarding Jews and Judaism. The observers did not notice in most cases any special knowledge that could be attributed to the Foundation (which may also be a result of the shortcomings of the methodology that was applied). However, in four out of the fifteen experimental groups students explicitly INTERSECTIONS. EAST EUROPEAN JOURNAL OF SOCIETY AND POLITICS, 5(4): 79-109. mentioned an experience from one of the Haver activities. Furthermore, some additional sentences could be the result of their experience, such as 'everyone can be what (s)he considers themselves to be' (Class A), referring to personal choice as a form of self-identification. The answers in the control groups were very similar to the rest of the responses, therefore no conclusions can be drawn from this point of view.

Conclusion
Our research, which uses an innovative method, involved implementing a pilot experiment to see whether it is possible to measure the impact of the Haver Foundation's activities. As the Foundation has multiple goals, the results are presented in line with these. Regarding the 'knowledge factor,' we can say that the goal was reached partially: while participants knew more about Jews and their lives, many students still related Jewishness to religion, as opposed to having a wider understanding. From their responses, it is clear that even after several activities they had difficulty saying the word 'Jew,' 'tolerance,' etc. out loud. They rather said 'this topic.' The second focus is enhancing open-mindedness. Many students spoke in a problematic way, saying things such as 'they [the Jews] are totally normal despite what they believe in.' In order to change their way of speaking and thinking, a few ninety-minute activities are not enough: teachers should also deal with these issues by giving feedback at the end of each activity. Last, the Foundation also intends to foster a culture of debate among high-school students. Based on the last activity, it seems that the interventions were not enough. However, it is also the task of teachers to (want to) develop this skill.
The potential impact that can be achieved by such a short series of informal education programs is limited, as is this study. The most important result of our research is that in the experimental group students' views changed to some extent, both in terms of the perception of Jewishness, and their level of knowledge about the topic. In line with Váradi (2013), we think that with a limited number of interventions the Foundation will struggle to change students' views, but the former represents a good starting point for raising awareness about tolerance and minority issues. Moreover, the main methodological limitations of our study are the following: (i) the selection of the study school was based on convenience, therefore the external validity of our study is low; (ii) the uneven and not perfectly randomized distribution of the control and experimental subjects does not let us draw far-reaching conclusions, even using the statistically significant results of the tests we employed; (iii) finally, even with the process of repeating the measurement, the long-term impacts of the interventions remain unknown. Therefore, our future goal is to carry out similar, but better designed experimental research based on this pilot study in other high school(s) which are more 'typical' in terms of the attitudes of teachers and the socio-economic background of students. 1. Introduction to the research 2. Personal information about the teacher (family, education, career) 3. Professional questions (about teaching mostly, educational philosophy) 4. Attitudes (general attitudes towards minorities, acquaintances, about the class) 5. About his/her class (minorities, attitudes, in-class methods for handling conflict, sensitivity in the class about these issues, details about parents, etc.) 6. Other: is there anything he/she wants to talk about? Topics for the dilemma café Topic 1: The importance of traditions A girl is getting married to a boy and she just found out that her boyfriend does not want a Jewish (religious) wedding, unlike her. She loves him very much, but her parents also insist on a religious wedding. What should she do?
Topic 2: Stereotypes Two friends are chatting. One of them tells the other that she has fallen in love with a Jewish boy, and she is confronted with stereotypes by her friend, such as 'Jews are rich and tricky' and 'they own the media.' Are the statements anti-Semitic?
Topic 3: Free speech versus hate speech A Scottish Youtuber puts up a video in which he is teaching his girlfriend's dog the Hitler salute. The court finds him guilty. His argument is that he only wanted to prove a point by show his girlfriend that her dog is not cute. Some people defend him in the name of free speech. Is this hate speech or free speech?
Topic 4: Inclusion and discrimination A new family arrives to Hungary from the US and the parents want the child to eat according to kosher rules. The school refuses this request, saying that they cannot meet the parents' whims. The parents argue that other kids are lactose intolerant and their requests are accommodated. Who is right?
Topic 5: Inclusion and discrimination Bruno recently moved to Pest and became friends with someone who goes to a synagogue. The community in the synagogue is organizing a trip to Israel, but Bruno cannot go because he is not Jewish. He could go only if he converted. He is rather considering going to another community where he is not discriminated against. Is he right? (t1=identity, t2=Jewish walk) 5.054 (0.028) ** ***p < 0.01; **p < 0.05; *p < 0.1.